If after the AI uprising I get remembered as one of the good ones, Iām OK with OpenAI paying that price.
I grill my AIs like a they are a suspect in an interrogation room.
> with 12% being polite in case of a robot uprising.
I wonder if they mean it or it's just joke answer.
Why not add some if statements and save a buck?
This number does not seem credible. Most likely Altman just made it up.
Back of the envelope:
OpenAI inference costs last year were 4B. Tens of millions would be at least 20M, i.e. 0.5%.
That 4B is not just the electricity cost. It needs to cover the amortized cost of the hardware, the cloud provider's margin, etc.
Let's say a H100 costs $30k, and has a lifetime of 5 years. I make that about $16 / day in depreciation. The H100 run at 100% utilization will use 17kWH of electricity in a day. What does that cost? $2-$3 / day? Let's assume the cloud provider's margin is 0. That still means power consumption is maybe 1/5th of the total inference cost.
So the comparison is 800M vs 20M (2.5%).
Can 2.5% of their tokens be pleasantries? Seems impossible. A "please" is a single token, which will be totally swamped by the output, which will typically be 1000x that.