AskHN: What's a real world production use case for Ollama?

  • It’s not for production. Ollama’s documentation explicitly warns you not to use the API in production. It’s for messing around and experimenting with LLMs and different models.

    If you don’t see the use of it, I recommend you use something else that suits your tastes more.

  • Not everything must be about commercial use. ollama is a great way to get from "I have no idea what this LLM stuff is" to "cool, I‘m already talking to one?".

  • We use it in our research group to run the largest LLMs quickly on a descent server. However, the people with the latest MacBooks don’t bother and prefer to use their laptops as they are fast enough.

    We also throw money at Microsoft and OpenAI when we evaluate ChatGPT. But open LLMs are cheaper to play with if you already have the hardware. And more aligned to our research values.

  • Didn't know Ollama was used in production. I use Ollama + the Ollama Raycast plugin as it's probably the quickest way to run an LLM locally