Yeah, and its a waste. Nvidia runs the A100s in a relatively inefficient power band (300W or 400W), and tons of power is burned on the interconnect huge LLMs need to fit in memory.
And the servers cost a fortune, with a huge profit margin.
Its not sustainable... in fact, its probably less efficient than the crypto mining boom, as miners were downclocking GPUs (and building simple ASICs) to run at more efficient voltages.
Yeah, and its a waste. Nvidia runs the A100s in a relatively inefficient power band (300W or 400W), and tons of power is burned on the interconnect huge LLMs need to fit in memory.
And the servers cost a fortune, with a huge profit margin.
Its not sustainable... in fact, its probably less efficient than the crypto mining boom, as miners were downclocking GPUs (and building simple ASICs) to run at more efficient voltages.