Hacker News

Petals runs Llama 2 (70B) from Colab at 5 tokens/sec

by borzunovon 7/19/2023, 8:52:20 PM with 2 comments

by borzunovon 7/19/2023, 8:52:20 PM
Chat web app: http://chat.petals.ml
Colab: https://colab.research.google.com/drive/1uCphNY7gfAUkdDrTx21...
Project description: https://github.com/bigscience-workshop/petals
by amrbon 7/19/2023, 10:20:40 PM
Great project and I'm happy to see it expand to more models!