Yep, watt-hours are correct. Think of it this way: Power supplies and laptop chargers are rated in watts and that represents how much energy they can pull from the wall at a point in time. If you wanted to know how much energy a task takes on your computer and you know it maxes out your 300 watt desktop [0] for an entire hour, that would take 300 watt-hours.
[0] To my knowledge, no desktops are built to max out their power supply for an extended time; this is a simplification for illustration purposes.
Watt-hours seem good right now, and rebut the current arguments that LLMs are wasteful. They might soon stop being the right lens, if your agents can go off and (in parallel) expend some arbitrary and borderline-unknowable budget of energy on a single request.
At some point you’ll need to measure in terms of $ earned per W-H, or just $out/$in.
(average) Watt-hours would still be the right unit to measure that, if the question is about energy efficiency / environmental impact. You still care how much energy those agents consume to achieve a given task, and the time it takes for tasks is still variable, so you need to multiply by time.
> People are often curious about how much energy a ChatGPT query uses; the average query uses about 0.34 watt-hours
My point was twofold, “the average query” becomes less meaningful as the variance increases. Sure one can _in principle_ report W-h spent on your account or query, but I think it will get more opaque and hard to predict. Average becomes less useful when agents can do an unpredictable amount of work with increasingly large bounds.
Second, and this was perhaps worthy of expanding in my post - I think in the radical abundance world that Altman is describing, energy and efficiency stops being something that people talk about. Fusion, space based solar farms, whatever, it’s easy to imagine solving this stuff. And I think you can even imagine this happening sooner if for example one provider stamps a “100% renewable datacenter” option on their AI product. Then you might not care about energy efficiency at all; in which case you just care about profit.
To be clear, I'm only saying the unit is correct, not whether the number of watt-hours is accurate or (assuming it is correct) that it's a good amount to spend on each query.
[0] To my knowledge, no desktops are built to max out their power supply for an extended time; this is a simplification for illustration purposes.