Training is a physical production process
A finished model feels intangible: download weights, send a prompt, receive text. Training it is closer to industrial production. Thousands of accelerators move data, perform arithmetic, exchange updates, and discard heat for weeks or months. Electricity is not an incidental input. It is what makes each parameter update possible.
A language model starts with billions of adjustable values. During training it predicts the next token, compares that prediction with the target, calculates how much each parameter contributed to the error, and changes the parameters slightly. This cycle repeats across a large dataset. The useful behavior emerges from accumulated corrections, not from one dramatic computation.
Most of the arithmetic consists of matrix multiplication: large grids of numbers multiplied and added in regular patterns. GPUs can perform many of those operations in parallel, which is why they dominate modern training clusters.
Most training work is structured multiplication and addition repeated across enormous arrays of numbers.
More GPUs do not automatically mean less energy
The basic estimate is power multiplied by time. A ten-megawatt cluster running for one hour consumes ten megawatt-hours. Adding accelerators may shorten the run, but each chip needs the correct model state and training data at the correct moment. Networking and synchronization become harder as the cluster grows.
A faster run can therefore use more total energy if additional machines spend enough time communicating or waiting. The MLCommons power work is useful here because it measures energy as well as performance instead of assuming that the shortest run is the most efficient.
Meta’s Llama 3.1 model card reports 39.3 million H100 GPU-hours across the model family. That does not describe one uninterrupted arrangement of machines, but it makes the scale concrete. It also excludes the simple story of one clean final run. Research includes failed experiments, smaller trials, data work, post-training, safety evaluations, and repeated checkpoints.
The GPU is only part of the bill
Accelerators need CPUs, memory, storage, switches, and power-conversion equipment. Cooling moves the heat out of the building. Spare capacity waits for sudden demand. A chip-level calculation that ignores these systems understates the facility load.
A GPU’s rated draw captures only one part of the facility that keeps a training cluster operating.
The International Energy Agency notes that cooling’s share varies widely by facility. That matters when comparing estimates: a highly utilized hyperscale site in a cool climate is not equivalent to an older enterprise data center with inefficient cooling. Useful reporting must state the system boundary rather than present one universal number.
Inference can become the larger lifetime cost
Training creates the model. Inference happens every time someone uses it. One short, efficiently batched text request can be small compared with a training run. Billions of requests, long context windows, generated video, repeated reasoning, or agents that call models continuously change the comparison.
Lower cost per answer does not fix total demand. In a hypothetical service, making each answer use one tenth as much computation while serving twenty times as many answers would double total computation. The example holds the task and other conditions fixed; it illustrates why a unit-efficiency gain and a lower total electricity bill are separate claims.
The IEA estimates that global data-center electricity use could rise sharply through 2030. The global percentage can hide the local problem: data centers cluster around particular substations and transmission corridors. A facility may be built faster than the generation and grid upgrades needed to serve it.
Energy and emissions are not interchangeable
Two identical training runs can consume the same electricity while causing different operational emissions. A grid supplied mainly by low-carbon generation differs from one relying heavily on coal or gas. Timing matters too, because the marginal electricity mix changes during the day.
Renewable contracts can support new generation, but they do not remove the physical need for reliable power at every training hour. Credible accounting should disclose energy, location, timing, and emissions instead of using one figure as a substitute for the others.
Efficiency starts with choosing less computation
New accelerators can perform more operations per watt. Lower numerical precision reduces arithmetic and memory movement. Better software keeps chips occupied, and sparse models activate only part of a network for a given token. Distillation and quantization can move useful behavior into smaller systems.
Using the smallest capable model can improve latency and cost while avoiding unnecessary computation.
Match the model to a defined task, check that the smaller system completes it adequately and compare useful output with the energy used. Reserve additional computation for tasks whose results justify it. The inference-engine guide explains why prompt length, serving memory and generation behavior also affect the workload after training ends.
Visual guide
How the pieces fit together
