The cost of "AI"

Emp_Fab

Sporky said:

Also 23,000 households is about 0.015% of the total in the US. Residential use is about 38% of the overall total.

So AI is using 0.006% of US total consumption.

Less than 1% of 1%.

Yeah, well I flew in an airplane recently and got accused of producing hundreds of tons of CO2, so count yourself lucky. 

If they'd been sat in the seat behind me they would have known it was at least double that after my feta and spinach pie at the airport.

Bowks

darthed1981 said:

I believe one of the main focuses of AI development is reducing the amount of computing power needed to make it work.

Yes, I know that Intel and AMD will be producing processors with a dedicated AI chip so the prompts can be handled in the box.

digitalscream

Bowks said:

darthed1981 said:

I believe one of the main focuses of AI development is reducing the amount of computing power needed to make it work.

Yes, I know that Intel and AMD will be producing processors with a dedicated AI chip so the prompts can be handled in the box.

Don't get too hung up on that - those things are not especially useful for LLMs (although they can be used effectively for smaller models and tasks). The primary reason that GPUs are great for LLMs is the insane amount of memory bandwidth available - we're talking a couple of orders of magnitude higher on an average GPU than is available to the highest-end mainstream CPU, regardless of any integrated acceleration. Even then, the power used by one of those CPUs will on the order of 200W at full boost, when the "average GPU" will be sucking back 250W to do the same job at least 10 times faster.

crosstownvamp

So how many watts per lasagne is it?

Sporky

Less than to cook it. 

snowblind

@digitalscream If you get nerdy about such things take a look at the IBM power10 block schematics.

It is probably more of a concern where all the energy used for AI ends up being spent eg generating fake animations of politicians, although I do find some of Scared Ketchup's stuff quite amusing. Not sure it's worth wrecking the atmosphere for though. In the grand scheme of things the power used specifically for AI work likely pales into insignificance compared to your average mega-cruise ship.

TimmyO

Is anyone else generally disappointed that "AI" has come to mean "generative AI" which is basically generating word-soup and seeing what sticks, rather than there being great progress in 'actual AI' or whatever the proper name for that is? 

Stratavarious

Sporky said:

Also 23,000 households is about 0.015% of the total in the US. Residential use is about 38% of the overall total.

So AI is using 0.006% of US total consumption.

Less than 1% of 1%.

They were talking about one chatbot today.

Globally 2-3% of electrical energy currently goes to data centres and is predicted to go to 4-5% by end of the decade,  AI may double that as it’s embedded into more things.  So it’s not an unreasonable estimate.


https://iea.blob.core.windows.net/assets/6b2fd954-2017-408e-bf08-952fdd62118a/Electricity2024-Analysisandforecastto2026.pdf

https://www.datacenterdynamics.com/en/news/global-data-center-electricity-use-to-double-by-2026-report/

In the US, which the report said is home to 33 percent of the world’s data centers, consumption is expected to rise from 200TWh in 2022 to 260TWh in 2026, some six percent of all power use across the country.

The situation is more acute in Ireland where, by 2026, data centers could account for 32 percent of all power consumption due to a high number of new builds planned. This compares to 17 percent in 2022. 

It’s a big cost for generic, superficial pictures and very ropey text copy.

digitalscream

Stratavarious said:

Sporky said:

Also 23,000 households is about 0.015% of the total in the US. Residential use is about 38% of the overall total.

So AI is using 0.006% of US total consumption.

Less than 1% of 1%.

They were talking about one chatbot today.

Globally 3% of energy currently goes to data centres and is predicted to go to 4-5% by end of the decade,  AI may double that as it’s embedded into more things.  So it’s not an unreasonable estimate.


https://iea.blob.core.windows.net/assets/6b2fd954-2017-408e-bf08-952fdd62118a/Electricity2024-Analysisandforecastto2026.pdf

https://www.datacenterdynamics.com/en/news/global-data-center-electricity-use-to-double-by-2026-report/

In the US, which the report said is home to 33 percent of the world’s data centers, consumption is expected to rise from 200TWh in 2022 to 260TWh in 2026, some six percent of all power use across the country.

The situation is more acute in Ireland where, by 2026, data centers could account for 32 percent of all power consumption due to a high number of new builds planned. This compares to 17 percent in 2022. 

It's worth bearing in mind that most datacentre energy use is nothing to do with AI services, and everything to do with data storage and cooling. This is what I discovered when I built the 220TB archiving server - the main cost wasn't the hardware or the CPU/GPU (~180W), it was the drives themselves (~400W), the hardware to run them (~30-50W) and keeping them cool (another 40W). And that was using an old, horrifically inefficient CPU to do the job.

Ultimately, AI is highly unlikely to make a significant difference to that on a global scale. On the other hand, we're on the verge of much higher-density spinning rust storage coming to the market (think in terms of 50-100TB on a single drive, so a single disk can replace 4-5 existing units), as well as high-capacity SSDs that use a fraction of the power and require almost no cooling (comparitively-speaking).

Sporky

Stratavarious said:

It’s a big cost for generic, superficial pictures and very ropey text copy.

If that was all AI is used for I'd agree, but look at the links DS posted for starters. That's very much not what AI is being used for overall. 

LionAquaLooper

Watch those figures quadruple in the run up to the US presidential elections.

rze99

Paul_C said:

where's that from in terms of source?

AntonHunter

I thought this was going to be about them using cheap labour to train the models...

“So-called AI systems are fueled by millions of underpaid workers around the world, performing repetitive tasks under precarious labor conditions.” https://www.noemamag.com/the-exploited-labor-behind-artificial-intelligence/

https://time.com/6247678/openai-chatgpt-kenya-workers/

https://arstechnica.com/gadgets/2024/04/amazon-ends-ai-powered-store-checkout-which-needed-1000-video-reviewers/

sev112

Can you recycle AI outputs in the same way we can throw paper in the recycling bin ?

digitalscream

AntonHunter said:

I thought this was going to be about them using cheap labour to train the models...

“So-called AI systems are fueled by millions of underpaid workers around the world, performing repetitive tasks under precarious labor conditions.” https://www.noemamag.com/the-exploited-labor-behind-artificial-intelligence/

https://time.com/6247678/openai-chatgpt-kenya-workers/

https://arstechnica.com/gadgets/2024/04/amazon-ends-ai-powered-store-checkout-which-needed-1000-video-reviewers/

Cheap labour is nothing new - the horrifying hourly rates in those articles are actually more than the average wage in those countries. 

Sadly, without exploiting the vastly different economic circumstances around the world, we wouldn't have any tech at a scale that matters at all.

As for the last one...that's a classic case of tech not working on a schedule that appeases the salesdroids, so...they just built a Mechanical Turk to cover over the cracks.

Adey

digitalscream said:

Adey said:

Absolutely. I think AI is the worst thing we have invented. What could possibly go wrong with it? Other than making people more stupid...

Yes. It's absolutely, totally, utterly terrible and will obviously be the destruction of us all.

Absolutely. Yes. It's terrible.

It is a bit naive to see just the obvious "it can solve this problem" thing that it can do, 

but the opportunity for hostile.countries to cause havoc are massive.

The camera used to never lie. Now, the camera (or picture/video) always lies. Deep fakes of your mates, or family, calling you up and conning you aren't far away.

AI will be deciding for us what is right or wrong and "human error" will just become "an error in decision making" and the human is always responsible for it and wrong.

AI is inevitable. And inevitably going to be a bad thing for Homo Sapiens

digitalscream

Adey said:

And inevitably going to be a bad thing for Homo Sapiens

It's actually at least as likely to be the means by which we solve climate change and actually survive.

As for the rest of your post...that's just alarmist stuff, along the same lines as "Photoshop will destroy the world" was 20 years ago, and ignores the limitations on the technology and the true pinch points.

For example, it's insanely costly to run the most reliable and accurate models, which are still subject to hallucinations. To run them privately - which any of the uses you're scared of would require - would incur costs in the millions just to buy the hardware, much less run it.

Those models are also hitting limits in terms of source data, because they're already trained on almost every piece of data of verifiable quality that's available to humanity. There is no more that actually exists with which to build more complex models.

ewal

Is anyone waiting for the day when excited leaders at your organisation realise that GenAI basically amplifies the shite in, shite out maxim? Chatbots giving the wrong advice, extremely rapidly generated error strewn corporate bollocks, etc. Last week I was being told how quickly CoPilot could index and ingest screeds and screeds of information, none of which has even the most basic of quality controls in place. Suppose they'll learn.

snowblind

Interesting how much of the AI effort is being put into financial work when inflationary economics is itself an unbalanced equation and therefore unstable. Have an irrational computer programmed by even more irrational humans work on a fundamentally chaotic system. Impossible to see anything other than good outcomes here.

dano

Can they cook the lasagna in the data centre ? Just thinking, but the more sttupid questions people ask AI, the crisper the lasagna gets.