AI futures, meet Net Zero futures
March 21, 2024 11:33 AM Subscribe
The IPCC, the world authority on climate science, advises we need to cut greenhouse gas emissions by more than half by 2030, and get emissions down to net zero by 2050, if we want a chance of limiting average temperature rise to 1.5 degrees. Actually, we've already crossed that threshhold, kind of.
Information Technology itself may contribute as much as 5% to global greenhouse gas emissions. Internationally recognised methods and standards for assessing the environmental impacts of AI don't yet exist, although they will.
Are AI revolution futures compatible with net zero futures? Are science and technology still on the same team?
AI is already contributing to net zero by optimising resource allocation in energy, transport, agriculture, and other fields -- but we're not really sure by how much. A lot of the research making these claims doesn't really bring receipts.
AI itself takes energy and other resources to train and deploy, and companies like OpenAI, Meta, Google and Amazon are very cagey about exactly how much. The digital is material. The cloud may sound wispy, ethereal, and poetic, but the reality is massive data centres. These massive data centres consume electricity, water, and have socioeconomic impacts that are fascinating for some anthropologists. Their electricity may be powered by renewable energy, and/or offset by carbon removal projects. But even so, there is almost always an opportunity cost: renewables that are powering data centres could have been powering something else, forests that are offsetting data centres could have been offsetting something else.
That 1.5 degrees figure refers to 1.5 degrees above pre-industrial levels. It was agreed back in 2015, the Paris Agreement. 1.5 degrees may not sound like much, especially to Americans. But the difference between 1.5 degrees and 2.0 degrees of warming could be huge.
The major public cloud providers Google, Microsoft, and (to some extent) Amazon all have bold decarbonisation strategies. As you might expect from the tech sector, these all rely pretty heavily on the expectation of future innovations, especially in the scaling up of carbon removal technologies, to balance out the carbon emissions. The risk that this might lead to silly delays in reducing the emissions themselves is called mitigation deterrence.
Google and Microsoft have both pledged that by 2030 all their energy consumption will be matched with renewable energy, generated locally. But there are other factors like embodied emissions (the greenhouse gas emitted during the manufacture of the hardware). According to Corporate Climate Responsibility Monitor, Google and Microsoft's pledges would still only mean a 37% and 38% emission reductions across their value chains compared to 2019 levels.
Meanwhile, ESG, the ratings framework that tries to price Environmental, Social, and Governance factors into the value of corporations, has become slightly embroiled in culture wars. This could be a little ironic, since both pro and anti ESG camps fundamentally agree on maximising long term shareholder value, just doing it in different ways. ESG has always been about identifying material risks and opportunities that investors might want to know about, not striving for positive impacts for their own sake.
Apocalyptic messages may do more harm than good, and there is plenty of hopeful news out there too. Global greenhouse gas emissions are starting to plateau, with growth in renewable energy a key driver. We'll be hearing a lot of blame put on China and India. In interpreting the data, it's important to bear in mind per capita emissions (China and India have huge populations) as well as historic emissions (the excess greenhouse gases emitted by industrialising countries in the 19th and 20th century are still up there heating up the planet).
This article, about Red AI and Green AI, doesn't make any sense to me. Maybe it was written by AI?
AI is already contributing to net zero by optimising resource allocation in energy, transport, agriculture, and other fields -- but we're not really sure by how much. A lot of the research making these claims doesn't really bring receipts.
AI itself takes energy and other resources to train and deploy, and companies like OpenAI, Meta, Google and Amazon are very cagey about exactly how much. The digital is material. The cloud may sound wispy, ethereal, and poetic, but the reality is massive data centres. These massive data centres consume electricity, water, and have socioeconomic impacts that are fascinating for some anthropologists. Their electricity may be powered by renewable energy, and/or offset by carbon removal projects. But even so, there is almost always an opportunity cost: renewables that are powering data centres could have been powering something else, forests that are offsetting data centres could have been offsetting something else.
That 1.5 degrees figure refers to 1.5 degrees above pre-industrial levels. It was agreed back in 2015, the Paris Agreement. 1.5 degrees may not sound like much, especially to Americans. But the difference between 1.5 degrees and 2.0 degrees of warming could be huge.
The major public cloud providers Google, Microsoft, and (to some extent) Amazon all have bold decarbonisation strategies. As you might expect from the tech sector, these all rely pretty heavily on the expectation of future innovations, especially in the scaling up of carbon removal technologies, to balance out the carbon emissions. The risk that this might lead to silly delays in reducing the emissions themselves is called mitigation deterrence.
Google and Microsoft have both pledged that by 2030 all their energy consumption will be matched with renewable energy, generated locally. But there are other factors like embodied emissions (the greenhouse gas emitted during the manufacture of the hardware). According to Corporate Climate Responsibility Monitor, Google and Microsoft's pledges would still only mean a 37% and 38% emission reductions across their value chains compared to 2019 levels.
Meanwhile, ESG, the ratings framework that tries to price Environmental, Social, and Governance factors into the value of corporations, has become slightly embroiled in culture wars. This could be a little ironic, since both pro and anti ESG camps fundamentally agree on maximising long term shareholder value, just doing it in different ways. ESG has always been about identifying material risks and opportunities that investors might want to know about, not striving for positive impacts for their own sake.
Apocalyptic messages may do more harm than good, and there is plenty of hopeful news out there too. Global greenhouse gas emissions are starting to plateau, with growth in renewable energy a key driver. We'll be hearing a lot of blame put on China and India. In interpreting the data, it's important to bear in mind per capita emissions (China and India have huge populations) as well as historic emissions (the excess greenhouse gases emitted by industrialising countries in the 19th and 20th century are still up there heating up the planet).
This article, about Red AI and Green AI, doesn't make any sense to me. Maybe it was written by AI?
what the salesmen call "AI" these days
Is it though? Because most of the recent all-encompassing push has been focused specifically on generative AI, largely LLMs, largely owned by huge companies and called by APIs. I think we can quite easily let scientists run a climate model AND be against that shit, honestly.
posted by Artw at 12:13 PM on March 21 [12 favorites]
Is it though? Because most of the recent all-encompassing push has been focused specifically on generative AI, largely LLMs, largely owned by huge companies and called by APIs. I think we can quite easily let scientists run a climate model AND be against that shit, honestly.
posted by Artw at 12:13 PM on March 21 [12 favorites]
Science and tech are on the same team, but capitalism is its own team and only has 1 team member and 8 billion servants being abused to support it.
posted by GoblinHoney at 12:35 PM on March 21 [8 favorites]
posted by GoblinHoney at 12:35 PM on March 21 [8 favorites]
>>what the salesmen call "AI" these days
> Is it though?
Yes. I'm a senior deep learning researcher, and basically anything that uses backprop is called "AI" these days. The phrase "AI" is marketing speak; not words with any deeper or technological meaning.
> the recent all-encompassing push has been focused specifically on generative AI
Maybe that's what you pay attention to, but that's not what's driving the research or industry. As an example, I work in robotics and learned scene understanding, task planning, etc. is huge. It's a multi-billion dollar industry that dwarfs generative art tools. Generative tools are cool, and make pictures or text and people respond to that. But look at proceedings of ICLR or NeurIPS and you'll see it's only a small percentage of work being done.
posted by riotnrrd at 1:16 PM on March 21 [18 favorites]
> Is it though?
Yes. I'm a senior deep learning researcher, and basically anything that uses backprop is called "AI" these days. The phrase "AI" is marketing speak; not words with any deeper or technological meaning.
> the recent all-encompassing push has been focused specifically on generative AI
Maybe that's what you pay attention to, but that's not what's driving the research or industry. As an example, I work in robotics and learned scene understanding, task planning, etc. is huge. It's a multi-billion dollar industry that dwarfs generative art tools. Generative tools are cool, and make pictures or text and people respond to that. But look at proceedings of ICLR or NeurIPS and you'll see it's only a small percentage of work being done.
posted by riotnrrd at 1:16 PM on March 21 [18 favorites]
I’ve made pretty lengthy comments listing the numbers on this topic a couple times recently so I’m just going to link to one of them.
TL;DR: ML is a tiny fraction of crypto’s power utilization and the open source ML community has made multiple orders of magnitude efficiency improvements to equivalent-capability models. Which will likely continue because - unlike OpenAI / Google / Meta - open source ML is bounded by household current and gaming PCs. Any team that abandons that will lose their audience and be replaced by the next group that succeeds.
nVidia claims their just-announced Blackwell-based enterprise GPU improves power efficiency by about 3x over A100/H100s, but we’ll see what the real world numbers are soon.
That said, OpenAI’s Step-By-Step Verification and Self-Taught Reasoning (STaR) papers point to a future where major corporate players potentially abandon being low-impact. Altmann’s recently hinted we likely won’t see a full implementation of these papers (Q*, collectively and colloquially) in the upcoming post-election release, so it seems likely the next of their roughly annual releases will feature it.
In short, ML wouldn’t be a major cause for concern if current methods were simply continued forward. Diminishing returns on scale with standard transformers recently lead to widespread adoption of Mixture of Experts (GPT-4, Gemini, Mixtral 8x7B, etc). That’s all well and good since their attempt at keeping memory utilization low is also good for reducing power consumption, but the “different approach” of Q* a couple years down the road reads like the most incredibly wasteful brute force approach to reasoning possible. Basically beam-searching each problem space with umpteen reinforcement learning instances (progressively culled), if I’m reading it right.
Sidenote: I’m going to try to be better about using ML rather than AI in these threads, to prevent muddying the water. Neural networks indisputably learn, but anything like meaningful “intelligence” is some ways off.
posted by Ryvar at 1:39 PM on March 21 [9 favorites]
TL;DR: ML is a tiny fraction of crypto’s power utilization and the open source ML community has made multiple orders of magnitude efficiency improvements to equivalent-capability models. Which will likely continue because - unlike OpenAI / Google / Meta - open source ML is bounded by household current and gaming PCs. Any team that abandons that will lose their audience and be replaced by the next group that succeeds.
nVidia claims their just-announced Blackwell-based enterprise GPU improves power efficiency by about 3x over A100/H100s, but we’ll see what the real world numbers are soon.
That said, OpenAI’s Step-By-Step Verification and Self-Taught Reasoning (STaR) papers point to a future where major corporate players potentially abandon being low-impact. Altmann’s recently hinted we likely won’t see a full implementation of these papers (Q*, collectively and colloquially) in the upcoming post-election release, so it seems likely the next of their roughly annual releases will feature it.
In short, ML wouldn’t be a major cause for concern if current methods were simply continued forward. Diminishing returns on scale with standard transformers recently lead to widespread adoption of Mixture of Experts (GPT-4, Gemini, Mixtral 8x7B, etc). That’s all well and good since their attempt at keeping memory utilization low is also good for reducing power consumption, but the “different approach” of Q* a couple years down the road reads like the most incredibly wasteful brute force approach to reasoning possible. Basically beam-searching each problem space with umpteen reinforcement learning instances (progressively culled), if I’m reading it right.
Sidenote: I’m going to try to be better about using ML rather than AI in these threads, to prevent muddying the water. Neural networks indisputably learn, but anything like meaningful “intelligence” is some ways off.
posted by Ryvar at 1:39 PM on March 21 [9 favorites]
So, there's no choice but to use deep learning [to speed up the pace of modeling for climate mitigation]. He acknowledged the tension here, but you have to use the best resources available if you want to advance science.
It may be politic to acknowledge tension there, but it's stupid tension. Like the "tension" around Al Gore flying around to promote An Inconvenient Truth.
The problem isn't the minuscule amount of emissions which are generated while trying to fight climate change. The problem is the vast majority of emissions, which aren't.
posted by gurple at 1:42 PM on March 21 [7 favorites]
It may be politic to acknowledge tension there, but it's stupid tension. Like the "tension" around Al Gore flying around to promote An Inconvenient Truth.
The problem isn't the minuscule amount of emissions which are generated while trying to fight climate change. The problem is the vast majority of emissions, which aren't.
posted by gurple at 1:42 PM on March 21 [7 favorites]
Oh people would definitely respond to robots and self-driving cars. It's rather a matter of public accessibility, as anyone today can go online and play with AI art.
It terrifies me how the public (my boomer family members) unknowingly equate ChatGPT and the like with AI. I constantly remind them that deep net technology is nascent, that it is the result of very brute force computation on big data, that they keep making mistakes (hallucinations), and if the science can't figure it out it could even plateau or fizzle like a fad (though on some days this out seems less and less likely to me). And yet all this recent news about NVidia's stocks is reifying the belief that AI has arrived, when we don't have said self-driving cars available to actually buy. "Wake me when we actually have a cancer cure / fusion power", but with 100x more capital-driven speculation.
posted by polymodus at 1:46 PM on March 21 [5 favorites]
It terrifies me how the public (my boomer family members) unknowingly equate ChatGPT and the like with AI. I constantly remind them that deep net technology is nascent, that it is the result of very brute force computation on big data, that they keep making mistakes (hallucinations), and if the science can't figure it out it could even plateau or fizzle like a fad (though on some days this out seems less and less likely to me). And yet all this recent news about NVidia's stocks is reifying the belief that AI has arrived, when we don't have said self-driving cars available to actually buy. "Wake me when we actually have a cancer cure / fusion power", but with 100x more capital-driven speculation.
posted by polymodus at 1:46 PM on March 21 [5 favorites]
(Also even if we did have some sort of expert systems that lived up to the current hype, we are absolutely not prepared to use that responsibly, not as a society or even a species.)
posted by Ryvar at 1:49 PM on March 21 [2 favorites]
posted by Ryvar at 1:49 PM on March 21 [2 favorites]
Thanks for this post, lots of interesting links to explore.
One surprising omission though is Apple*, since they have pledged to go beyond most of their peers and plan to have net zero operations throughout their manufacturing supply chain by 2030.
*(they aren't branded as an "AI" company, but virtually all of their products utilize ML in one way or another)
posted by gwint at 1:54 PM on March 21 [3 favorites]
One surprising omission though is Apple*, since they have pledged to go beyond most of their peers and plan to have net zero operations throughout their manufacturing supply chain by 2030.
*(they aren't branded as an "AI" company, but virtually all of their products utilize ML in one way or another)
posted by gwint at 1:54 PM on March 21 [3 favorites]
We're not going to "cure" cancer, but we have and will continue to improve cancer detection and treatment, which has saved or extended millions of lives. Similarly, we're not going to "stop" climate change, but the actions that have been taken will moderate the harm.
It's not contradictory to say past actions have been beneficial but remain insufficient. In fact, that's the only reason to continue to push for more: If they weren't beneficial or were sufficient, we could stop.
posted by Mr.Know-it-some at 1:58 PM on March 21 [1 favorite]
It's not contradictory to say past actions have been beneficial but remain insufficient. In fact, that's the only reason to continue to push for more: If they weren't beneficial or were sufficient, we could stop.
posted by Mr.Know-it-some at 1:58 PM on March 21 [1 favorite]
Yeah but the mistake is in calling a treatment for cancer the cure for cancer, which is what the public is starting to do, calling stochastic parrots and super-autocompletes the AI. The danger with this mistake is in uncritical acceptance and usage of these programs when they hallucinate and confabulate (their results literally manipulate people the way bullshit does), and financial/economic consequences when the public and governments gold-rush to invest in questionable businesses and industries. There is no such social behavior happening in the area of oncology, in contrast.
posted by polymodus at 2:05 PM on March 21 [7 favorites]
posted by polymodus at 2:05 PM on March 21 [7 favorites]
Does anyone have a simple understanding (or can you point me at one of the links above to understand) of how "AI" accounts for a large and rapidly growing proportion of electricity use and yet Google and Microsoft are pledging reductions in emissions by 2030 compared with 2019 levels?
If a messy but accurate answer is "energy decarbonization that really actually doesn't mean the same amount of dirty energy being used by someone else", then I'd be delighted, because, if that's the case, energy decarbonization is going way faster and better than I could've imagined.
posted by gurple at 2:18 PM on March 21
If a messy but accurate answer is "energy decarbonization that really actually doesn't mean the same amount of dirty energy being used by someone else", then I'd be delighted, because, if that's the case, energy decarbonization is going way faster and better than I could've imagined.
posted by gurple at 2:18 PM on March 21
It's because there's a lots of misreporting around energy usage in data centers.
"Digital computer stuff" accounts for ~3-4% of US CO2 emissions, but data centers are only 0.2% of carbon emissions. This is because the folks designing and deploying the largest data centers have been pushing for renewable power for them for over a decade (in many cases leading to new renewable infrastructure being built out, which otherwise wouldn't exist), and have strong financial incentives to minimize energy use generally.
There's also a long history of extrapolating from the present, when in fact things tend to get far more efficient over time, especially for new technologies like LLMs. (In one example I worked on directly in audio compression, the original research model took something like 2 hours of server CPU time to process one second of audio... and just a couple-few years later we could run the new+improved version of the model at >2x real-time on a $50 phone.)
posted by kaibutsu at 2:40 PM on March 21 [6 favorites]
"Digital computer stuff" accounts for ~3-4% of US CO2 emissions, but data centers are only 0.2% of carbon emissions. This is because the folks designing and deploying the largest data centers have been pushing for renewable power for them for over a decade (in many cases leading to new renewable infrastructure being built out, which otherwise wouldn't exist), and have strong financial incentives to minimize energy use generally.
There's also a long history of extrapolating from the present, when in fact things tend to get far more efficient over time, especially for new technologies like LLMs. (In one example I worked on directly in audio compression, the original research model took something like 2 hours of server CPU time to process one second of audio... and just a couple-few years later we could run the new+improved version of the model at >2x real-time on a $50 phone.)
posted by kaibutsu at 2:40 PM on March 21 [6 favorites]
Huh. Just occurred to me: y’know, training doesn’t have to be a 24/7 full-throttle process. We could push energy cost incentives into the pricing of cloud GPU compute rental. LambdaLabs currently charges $27.92/hour for a standard 8xH100 SXM. What if it was $22.50 during peak hours for renewable generation and $32.50 off-peak?
OpenAI will just continue flinging Microsoft’s money at it - they’ve always been the poster child for scale-at-any-cost. But there’s an entire raft of mid-level players and an ocean of small teams that might actually look at that and opt into throttling.
If nothing else, you could use it to solve the core dilemma of this thread: how to make large compute tasks for ecological studies less ecologically harmful. One can almost envision extremely large latency-tolerant workloads migrating around the globe’s datacenters daily to track with the sun.
(Apologies if this idea is obvious and already being done, I’ve just always heard of training specified as a continuous X GPUs * Y hours deal).
posted by Ryvar at 3:00 PM on March 21 [2 favorites]
OpenAI will just continue flinging Microsoft’s money at it - they’ve always been the poster child for scale-at-any-cost. But there’s an entire raft of mid-level players and an ocean of small teams that might actually look at that and opt into throttling.
If nothing else, you could use it to solve the core dilemma of this thread: how to make large compute tasks for ecological studies less ecologically harmful. One can almost envision extremely large latency-tolerant workloads migrating around the globe’s datacenters daily to track with the sun.
(Apologies if this idea is obvious and already being done, I’ve just always heard of training specified as a continuous X GPUs * Y hours deal).
posted by Ryvar at 3:00 PM on March 21 [2 favorites]
We could push energy cost incentives into the pricing of cloud GPU compute rental.
...
If nothing else, you could use it to solve the core dilemma of this thread: how to make large compute tasks for ecological studies less ecologically harmful.
I'm on the same page, Ryvar. Just take my comment from a few months ago and replace "federated" with "distributed":
...
If nothing else, you could use it to solve the core dilemma of this thread: how to make large compute tasks for ecological studies less ecologically harmful.
I'm on the same page, Ryvar. Just take my comment from a few months ago and replace "federated" with "distributed":
While federated AI currently faces certain inefficiencies, if a federated network could span the globe, that would enable the most important efficiency: training the models when electricity is cheap. It's always night somewhere.posted by a faded photo of their beloved at 3:08 PM on March 21 [1 favorite]
AWS (and I’m sure others) offer discounts for using off-peak compute capacity. It’s in their interest to have their datacenters at a continuous load rather than having to overbuild and have wasted capacity for several hours a day.
posted by jedicus at 3:13 PM on March 21
posted by jedicus at 3:13 PM on March 21
This is because the folks designing and deploying the largest data centers have been pushing for renewable power for them for over a decade (in many cases leading to new renewable infrastructure being built out, which otherwise wouldn't exist), and have strong financial incentives to minimize energy use generally.
Maybe where you live. Where I live, Georgia Power is using energy demand for data centers as justification to build new fossil fuel plants in the year 20-fucking-24.
posted by hydropsyche at 3:35 PM on March 21 [9 favorites]
Maybe where you live. Where I live, Georgia Power is using energy demand for data centers as justification to build new fossil fuel plants in the year 20-fucking-24.
posted by hydropsyche at 3:35 PM on March 21 [9 favorites]
AI/ML may actually have some utility. Crypto, meanwhile, is nothing but a techbro ponzi scheme carefully distributed among innumerable criminal gangs.
If we gotta stop things, then kill the latter, that the former may yet live.
posted by aramaic at 3:41 PM on March 21 [4 favorites]
If we gotta stop things, then kill the latter, that the former may yet live.
posted by aramaic at 3:41 PM on March 21 [4 favorites]
The nvidia Blackwell announcement included the DGX GB200 NVL72, a 120kW 1.4 exaflop rack that can be treated as one giant gpu and that can be combined into an eight rack cluster that draws an entire megawatt of power.
posted by autopilot at 5:05 PM on March 21 [1 favorite]
posted by autopilot at 5:05 PM on March 21 [1 favorite]
nVidia’s claim is that 2,304 Blackwell GB200 chips (four of those 1 MW, 576 GB200 8-rack clusters) can do with 4 MW what 8,000 A100s can do with 15 MW. I know that reductions in process size (non-techies who somehow wandered in here: the size of a processor’s transistors, smaller = less resistance, less waste heat) can lead to startling gains in efficiency, and that memory size/bandwidth is often the bottleneck for training. But that kind of efficiency jump is definitely one I’d like to see verified as more than technically accurate for one highly specific case. Hence the phrasing in my comment above.
posted by Ryvar at 6:19 PM on March 21
posted by Ryvar at 6:19 PM on March 21
Why is everybody is focused on efficiency as if it will reduce emissions? When lightbulbs got efficient and cheap, power usage went up in the aggregate because they became economically viable in more applications. I don't see why that wouldn't be the case here as well.
Anyways, for a spot of good news, in 2023 China added more green power generation capacity than their energy demand grew, which suggests their emissions have already peaked.
posted by ndr at 8:38 PM on March 21 [4 favorites]
Anyways, for a spot of good news, in 2023 China added more green power generation capacity than their energy demand grew, which suggests their emissions have already peaked.
posted by ndr at 8:38 PM on March 21 [4 favorites]
Can't we just have a carbon tax and let the market sort out what is simultaneously economically and ecologically efficient?
posted by Reverend John at 8:56 PM on March 21 [2 favorites]
posted by Reverend John at 8:56 PM on March 21 [2 favorites]
Cap, auction, rebate per capita.
posted by clew at 9:41 PM on March 21 [1 favorite]
posted by clew at 9:41 PM on March 21 [1 favorite]
Why is everybody is focused on efficiency as if it will reduce emissions?
Because this isn’t crypto. Cryptocurrency has no natural completion point, no endgame except monetizing the conversion of Earth into Venus 2.0 as quickly as possible. Any efficiency gains get immediately swallowed by first adopters and simply shift the pricing matrix one column over. There is no outcome except “MOAR.”
ML applications are trying to solve a set thing - an M3 Macbook Pro running Mixtral 8x7B is probably sufficient to answer most homework questions before college. If nVidia announces a new GPU that can do so twice as efficiently? Then the carbon footprint for completing that task is now halved.
It’s completely normal to miss this because a) everything in ML is still ratcheting up scale while shifting methods and tackling new tasks so the fixed-cost aspect can be obscured and b) a whole lot of cryptobros with suddenly worthless mining GPUs heard this new “AI” thing used those and they were desperate to claw back their buildout costs. They brought their shitty exponential hashing growth + line goes up assumptions with them and it’ll take time for the smell of burning rainforests to dissipate.
posted by Ryvar at 12:05 AM on March 22 [1 favorite]
Because this isn’t crypto. Cryptocurrency has no natural completion point, no endgame except monetizing the conversion of Earth into Venus 2.0 as quickly as possible. Any efficiency gains get immediately swallowed by first adopters and simply shift the pricing matrix one column over. There is no outcome except “MOAR.”
ML applications are trying to solve a set thing - an M3 Macbook Pro running Mixtral 8x7B is probably sufficient to answer most homework questions before college. If nVidia announces a new GPU that can do so twice as efficiently? Then the carbon footprint for completing that task is now halved.
It’s completely normal to miss this because a) everything in ML is still ratcheting up scale while shifting methods and tackling new tasks so the fixed-cost aspect can be obscured and b) a whole lot of cryptobros with suddenly worthless mining GPUs heard this new “AI” thing used those and they were desperate to claw back their buildout costs. They brought their shitty exponential hashing growth + line goes up assumptions with them and it’ll take time for the smell of burning rainforests to dissipate.
posted by Ryvar at 12:05 AM on March 22 [1 favorite]
suddenly worthless mining GPUs
didn't become worthless because the price of Bitcoin collapsed (as it would have done in any just world; in this one, it's currently trading at very close to a recent all time high - make of that what you will) but because GPUs got thoroughly outcompeted on price by dedicated Bitcoin mining ASICs.
Given the speed at which GPU capabilities have been improving, I would be surprised to learn that a fleet of used ex-mining GPUs would actually be price competitive with a newer fleet, especially if that newer fleet is still owned by its manufacturer rather than ever needing to have been sold retail.
So I'm not sure it's crypto bros pumping the current ML wave so much as nVidia investors, except insofar as there's overlap between those communities.
posted by flabdablet at 12:20 AM on March 22
didn't become worthless because the price of Bitcoin collapsed (as it would have done in any just world; in this one, it's currently trading at very close to a recent all time high - make of that what you will) but because GPUs got thoroughly outcompeted on price by dedicated Bitcoin mining ASICs.
Given the speed at which GPU capabilities have been improving, I would be surprised to learn that a fleet of used ex-mining GPUs would actually be price competitive with a newer fleet, especially if that newer fleet is still owned by its manufacturer rather than ever needing to have been sold retail.
So I'm not sure it's crypto bros pumping the current ML wave so much as nVidia investors, except insofar as there's overlap between those communities.
posted by flabdablet at 12:20 AM on March 22
I thought Ethereum was still largely GPU-based until the proof-of-stake conversion which almost perfectly coincided with the explosion in LLM advancement? The Bitcoin part I remember because I did my 4-day deepdive on it just as the first ASICs were hitting (2016?). After which: “nah, I’m good.”
posted by Ryvar at 12:36 AM on March 22
posted by Ryvar at 12:36 AM on March 22
Fair point. I'd forgotten about Ethereum because its profligate energy burn has always been dwarfed by Bitcoin's, but I think you're right about it having been run mostly on GPUs until the PoS cutover and now your timing analysis makes perfect sense to me.
Anybody know what the current capex and opex are for used high-end pre-PoS Ethereum-mining GPUs, in dollars per teraflop per second, compared to today's best cards bought new?
posted by flabdablet at 12:46 AM on March 22
Anybody know what the current capex and opex are for used high-end pre-PoS Ethereum-mining GPUs, in dollars per teraflop per second, compared to today's best cards bought new?
posted by flabdablet at 12:46 AM on March 22
Best available consumer would’ve been RTX 3080 & 3090 Ti, which use the same Ampere architecture as the A100 and launched well prior to PoS Ethereum. A100 80GB was launched a year prior to the former and 7 months before the latter.
Basic stats:
3080 Ti was $1.2K USD, 350W, and 36 TFlops. Only 12 GB VRAM, though. 16 months prior to PoS.
3090 Ti was $2K USD, 450W, and 40 TFlops. Far more importantly: 24 GB VRAM. 9 months prior.
A100 80GB was $19.2K USD, 300W, 312 TFlops. 80GB and 28 months prior.
GPT-4 was trained on 25,000 of the latter for three months according to the “leak” I linked in the previous thread. The new model’s training is widely rumored to still be be using A100s, supposedly 100,000 also for 3 months (which is a scary leap until you realize this is the big multi-modality push).
So yeah, I think that narrative probably holds some water. It’s certainly not original to me, but I’m not being sarcastic when I say I appreciate you pushing me to fact check feasibility.
posted by Ryvar at 2:04 AM on March 22
Basic stats:
3080 Ti was $1.2K USD, 350W, and 36 TFlops. Only 12 GB VRAM, though. 16 months prior to PoS.
3090 Ti was $2K USD, 450W, and 40 TFlops. Far more importantly: 24 GB VRAM. 9 months prior.
A100 80GB was $19.2K USD, 300W, 312 TFlops. 80GB and 28 months prior.
GPT-4 was trained on 25,000 of the latter for three months according to the “leak” I linked in the previous thread. The new model’s training is widely rumored to still be be using A100s, supposedly 100,000 also for 3 months (which is a scary leap until you realize this is the big multi-modality push).
So yeah, I think that narrative probably holds some water. It’s certainly not original to me, but I’m not being sarcastic when I say I appreciate you pushing me to fact check feasibility.
posted by Ryvar at 2:04 AM on March 22
Surely the Jevons Paradox is likely to be in full force here, reducing the impact of any energy use efficiency gains - as companies will need to feed their ever-hungry shareholders, or am I missing something?
posted by aeshnid at 5:17 AM on March 22
posted by aeshnid at 5:17 AM on March 22
The Jevons paradox is bad if your energy comes from coal, but is it that terrible if it comes from renewables? (One could even imagine--maybe--a good outcome: My company demands so much energy from renewables that my city has to double its solar and wind output. Then I go bankrupt--but the energy infrastructure sticks around for everyone else's use.)
posted by mittens at 6:00 AM on March 22
posted by mittens at 6:00 AM on March 22
Why is everybody is focused on efficiency as if it will reduce emissions? When lightbulbs got efficient and cheap, power usage went up in the aggregate because they became economically viable in more applicationsDo you have any pointers on that? It sounds fascinating since I’ve only seen the opposite - my home power usage in 1998 was considerably higher than a decade later after switching to CFLs, LED displays, etc.
posted by adamsc at 7:30 AM on March 22
Can't we just have a carbon tax and let the market sort out what is simultaneously economically and ecologically efficient?
you'd think we could do that
posted by elkevelvet at 9:42 AM on March 22 [2 favorites]
you'd think we could do that
posted by elkevelvet at 9:42 AM on March 22 [2 favorites]
adamsc, that would be a case of the Jevons paradox. For lighting we’d probably see it not in the energy to light a specific house, but an increase in use of and dependency on artificial light “because it’s really cheap now".
I mean, I have neighbors with huge strings of decorative lights outside running 24/7/365 and I’m pretty sure they wouldn’t have in incandescent days.
posted by clew at 10:35 AM on March 22
I mean, I have neighbors with huge strings of decorative lights outside running 24/7/365 and I’m pretty sure they wouldn’t have in incandescent days.
posted by clew at 10:35 AM on March 22
The Jevons thing is not so much a paradox as just a bit of a backlash, usually; efficiency improvements almost always do end up as a net win despite it.
LED lighting has a confounding factor that's only rarely brought up, which is that if a dwelling is fitted with resistive electric heating - fan heaters, say, or radiators - then replacing its incandescent lighting with LEDs cuts its total power consumption much less than would naively be expected. Incandescent lighting is radiative resistive electric heating, and the main thing that removing it does in that case is make the dedicated heaters need to work harder to replace all the "waste" heat from lights that was never actually being wasted.
LEDs for lighting plus heat pumps for home heating and hot water, though, is an unambiguous energy consumption win even if Jevons dictates that the heat pumps now get used for summer cooling that was previously not provided. Heating a building requires far more energy than anything else inside it unless the building is both small and extremely well insulated, but there's no benefit to be had from applying more heat to it than it actually needs. A heat pump that can do the same job with a quarter of the energy is a net win even if it ends up being run for the other half of the year as well.
posted by flabdablet at 11:06 AM on March 22 [1 favorite]
LED lighting has a confounding factor that's only rarely brought up, which is that if a dwelling is fitted with resistive electric heating - fan heaters, say, or radiators - then replacing its incandescent lighting with LEDs cuts its total power consumption much less than would naively be expected. Incandescent lighting is radiative resistive electric heating, and the main thing that removing it does in that case is make the dedicated heaters need to work harder to replace all the "waste" heat from lights that was never actually being wasted.
LEDs for lighting plus heat pumps for home heating and hot water, though, is an unambiguous energy consumption win even if Jevons dictates that the heat pumps now get used for summer cooling that was previously not provided. Heating a building requires far more energy than anything else inside it unless the building is both small and extremely well insulated, but there's no benefit to be had from applying more heat to it than it actually needs. A heat pump that can do the same job with a quarter of the energy is a net win even if it ends up being run for the other half of the year as well.
posted by flabdablet at 11:06 AM on March 22 [1 favorite]
The Jevons thing is not so much a paradox as just a bit of a backlash, usually; efficiency improvements almost always do end up as a net win despite it.
If that were true we wouldn’t increase energy use year over year for centuries. We observably, disastrously, understandably do.
posted by clew at 11:46 AM on March 22
If that were true we wouldn’t increase energy use year over year for centuries. We observably, disastrously, understandably do.
posted by clew at 11:46 AM on March 22
Total US electricity generation has been surprisingly flat since 2007, despite about a 20% rise in generative capacity. That's not what you'd expect Jevon's paradox to look like.
(I also tend to think that we shouldn't actually care about total electricity use in an all-renewables world: once you remove the externalities that make it morally wrong to eat lots of power, you have market forces to sort out who uses how much of the available energy.)
posted by kaibutsu at 11:50 AM on March 22
(I also tend to think that we shouldn't actually care about total electricity use in an all-renewables world: once you remove the externalities that make it morally wrong to eat lots of power, you have market forces to sort out who uses how much of the available energy.)
posted by kaibutsu at 11:50 AM on March 22
Also I think all of the current discussion is a distraction from what actually matters on the carbon front. My bet is that electricity usage will go dramatically up in the US over the next ten years, but mainly because everyone is switching to electric vehicles -nothing to do with ai or data centers. Transportation is our single largest energy sector, and right now it's mostly fossil fuels. Switching to electric cars is great, but only so long as we continue decarbonizing electricity production. (And abandoning
cars is better than switching to an electric car.)
posted by kaibutsu at 11:59 AM on March 22 [2 favorites]
cars is better than switching to an electric car.)
posted by kaibutsu at 11:59 AM on March 22 [2 favorites]
Hmm, was there some major event in 2007 that altered the US economic and financial landscape? ;)
The cost and usage of bandwidth, compute, storage, etc has dropped dramatically, but I don't think the total spend has decreased. It's possible this is still a net energy gain if as suggested by the OP it results in savings in other sectors... but it could also result in the ML equivalent of people streaming the same album on Spotify over and over again as opposed to just downloading it once. I'm not making any judgments on whether it's offset by improved quality of life, etc, just pointing out a drive towards efficiency may not result in reduced power usage. It's a fair point that we shouldn't care as much in an all-renewables world, but we're not quite there yet.
posted by ndr at 1:48 PM on March 22 [1 favorite]
The cost and usage of bandwidth, compute, storage, etc has dropped dramatically, but I don't think the total spend has decreased. It's possible this is still a net energy gain if as suggested by the OP it results in savings in other sectors... but it could also result in the ML equivalent of people streaming the same album on Spotify over and over again as opposed to just downloading it once. I'm not making any judgments on whether it's offset by improved quality of life, etc, just pointing out a drive towards efficiency may not result in reduced power usage. It's a fair point that we shouldn't care as much in an all-renewables world, but we're not quite there yet.
posted by ndr at 1:48 PM on March 22 [1 favorite]
Hmm, was there some major event in 2007 that altered the US economic and financial landscape? ;)
Not one that would cause flat electricity use for 17 years? The US GDP had a small decrease in 2008, and had gone steadily upward since. And it looks like total energy consumption (including fossil fuels for transportation) is also flat over that time period.
So there's a decoupling of energy use and GDP. Where previous economic eras would have seen, say, more steel production causing perpetually more energy use, in the current she we see GDP increasing without a commensurate increase in energy usage. That's interesting, imo.
posted by kaibutsu at 9:48 PM on March 22 [1 favorite]
Not one that would cause flat electricity use for 17 years? The US GDP had a small decrease in 2008, and had gone steadily upward since. And it looks like total energy consumption (including fossil fuels for transportation) is also flat over that time period.
So there's a decoupling of energy use and GDP. Where previous economic eras would have seen, say, more steel production causing perpetually more energy use, in the current she we see GDP increasing without a commensurate increase in energy usage. That's interesting, imo.
posted by kaibutsu at 9:48 PM on March 22 [1 favorite]
There is no decoupling of energy use and GWP ala Lotka's wheel and the long arm of history by Keen, Garrett, and Grasselli. Yes, local GDP increases remain possible now, thanks to trade exploitation.
Also significant decoupling looks impossible long-term. Itn fact, decoupling maybe impossible short-term too, excpet you can "redefine" GDP in extremely subtle ways, becuase it excludes so much.
Jevons definitely covers AI. If cheaper then VCs would train more silly models that do useless things better, definitely similar to crypto-currencies.
We definitely do useful things using AIs too, but capitalism must continue, meaning ever more exploitation. At the investor side, an ever higher precentage of companies shall more & more closely resemble Theranos. At the worker side, more jobs can be "replaced by AI", in the sense that an AI does something shitty, and nobody cares, often because the job was bullshit in the first place, but often because consumers have little choice. Yes, there shall be jobs who improve productivity by human workers using AI too, but this is no sea change, and does not justify nVidia have a market cap larger than the who energy sector. lol
posted by jeffburdges at 7:03 AM on March 23 [1 favorite]
Also significant decoupling looks impossible long-term. Itn fact, decoupling maybe impossible short-term too, excpet you can "redefine" GDP in extremely subtle ways, becuase it excludes so much.
Jevons definitely covers AI. If cheaper then VCs would train more silly models that do useless things better, definitely similar to crypto-currencies.
We definitely do useful things using AIs too, but capitalism must continue, meaning ever more exploitation. At the investor side, an ever higher precentage of companies shall more & more closely resemble Theranos. At the worker side, more jobs can be "replaced by AI", in the sense that an AI does something shitty, and nobody cares, often because the job was bullshit in the first place, but often because consumers have little choice. Yes, there shall be jobs who improve productivity by human workers using AI too, but this is no sea change, and does not justify nVidia have a market cap larger than the who energy sector. lol
posted by jeffburdges at 7:03 AM on March 23 [1 favorite]
« Older Hotel Fred's back - but Luna's not well. | An Anarchist’s Guide To Dune Newer »
This thread has been archived and is closed to new comments
posted by riotnrrd at 11:55 AM on March 21 [3 favorites]