Ask HN: Hardware for 1k RPS?

  • Depends on your model size and how many of it can fit in memory. Multiply the size by 1k and divide by the memory capacity of the hardware for a rough ballpark.

  • https://venice.ai claim to offer uncensored models (I’ve not tested that claim)