Is LMDeploy the Ultimate Solution? Why It Outshines VLLM, TRT-LLM, TGI, and MLC

by helloericsfon 6/20/2024, 3:48:34 PM with 4 comments
  • by sshengon 6/20/2024, 4:04:35 PM

    How does Exllama rank among these? Heard good things about it.

  • by ShawnBasquiaton 6/20/2024, 4:05:07 PM

    Why aren't there more of these benchmark studies? How did TGI make the cut?

  • by timliu9on 6/20/2024, 3:50:28 PM

    Why was onnx not part of the tested runtimes? Seems like an oversight