Well, that depends. Very inefficient code tends to only be used when absolutely needed. If an LLM becomes ten times faster at answering simple prompts, it may very well be used a hundred times more as a result, in which case electricity use will go up, not down. Efficiency gains commonly result in doing way more with more, not more with less.

Correct. This is also known as a rebound effect [1], or, specifically with regard to technological improvements, as the Jevons paradox [2].

[1]: https://en.wikipedia.org/wiki/Rebound_effect_(conservation)

[2]: https://en.wikipedia.org/wiki/Jevons_paradox

Indeed, that is a common occurrence that called Jevons Paradox.

Energy used for lighting didn't decrease when the world moved to LED lights which use much less energy - instead we just used more lighting everywhere, and now cities are white instead of yellow.

I know what you mean, but do you have a citation for that? LEDs are so much more efficient that I wonder if its true.

Very true, but in recent years feature development has taken precedence over efficiency. VP of whatever says hardware is cheap, software engineers are not.

Unless your code is running on a large number of machines across data centers that energy is about 2-3 figures a month in total utilization.

So if we use cost as a proxy for environment impact it’s not saving much at all.

I think this is a meme to help a different audience care about performance.