Inference Tokens-per-Watt Calculator: Measure AI Energy Efficiency
Tokens per watt measures AI productivity: how much useful model output — tokens — a chip, server, or data center produces for every unit of electrical power it draws. It has become a key metric for evaluating the true efficiency and ROI of AI infrastructure, sometimes described as running an "AI token factory," because it captures real-world output rather than just theoretical compute. A facility that generates more tokens per watt delivers more product per dollar of electricity, which is the number that ultimately decides whether an inference business scales profitably. This calculator computes tokens per watt at both the chip and the facility boundary, folds in data-center PUE overhead, and translates the result into cost per million tokens — the line item that shows up on the cloud bill. Enter your throughput, power draw, PUE, electricity rate, and monthly volume to see live results.
1 ÷ (power × PUE ÷ tps ÷ 3600)
watt-hours to generate 1k tokens
at selected electricity rate
4,000 kWh at 50,000M tokens
serving cost at volume
Results update live as you type. For planning and field-check estimates — always verify against applicable standards and equipment ratings.
How we calculate this →The core formula
Tokens per watt is calculated at two boundaries — the chip (or server) and the facility — because each tells you something different about where energy is being spent.
Chip / server level: Tokens per Watt = Throughput (tokens per second) ÷ Power draw (watts)
Facility level: Tokens per Watt-Hour = Tokens generated in one hour ÷ Total facility power draw (watts), where facility power includes PUE overhead (cooling, power delivery, power conversion, and building losses).
The chip-level formula isolates the silicon: how efficiently the accelerator turns electricity into model output. The facility-level formula steps back to the meter — the grid power actually pulled to keep that chip alive, including every watt of cooling, UPS, and distribution loss along the way. Measuring at both boundaries is what makes tokens-per-watt a complete efficiency picture rather than a single optimistic number.
A worked example
Consider a node running an open-source model on an NVIDIA H100-class accelerator:
- Throughput: 2,000 tokens/sec - GPU power draw (TDP): 700 watts - Chip-level result: 2,000 ÷ 700 = 2.86 tokens per watt
Now apply a data-center PUE of 1.35: total facility power becomes 700 × 1.35 = 945 watts.
- Facility-level result: 2,000 ÷ 945 = 2.11 tokens per watt at the facility boundary
That gap — 2.86 down to 2.11 — is why PUE matters, and why tokens-per-watt is measured at both boundaries. A chip can look superbly efficient in isolation while a poorly run facility throws away a third of the energy before it ever reaches the workload. Expressed per watt-hour (the unit this calculator uses for energy cost accounting), the facility figure of 2.11 tokens/watt becomes 2.11 × 3,600 ≈ 7,600 tokens per watt-hour.
Why tokens per watt matters
Beyond raw hardware. Metrics like TFLOPS-per-watt only measure theoretical math capacity — how many floating-point operations a chip can perform per unit of energy. Tokens-per-watt captures the whole inference stack: model architecture, quantization (e.g. FP8 vs FP16), batch size, and software efficiency. Those are the things that determine how much raw compute becomes useful output, and they can differ wildly between two systems with identical hardware.
The PUE blindspot. A data center can post a near-perfect PUE of 1.0 and still waste enormous amounts of energy if its servers sit idle or run inefficient software. PUE measures overhead, not productivity. Tokens-per-watt tracks actual useful work — the tokens delivered — so it exposes waste that PUE alone hides.
Economic scaling. Infrastructure operators use tokens-per-watt to link electricity cost directly to revenue per million tokens. Once you know how many watt-hours a million tokens consume, every input — GPU choice, model size, quantization, batching, electricity rate — rolls up into the inference total cost of ownership (TCO) that decides whether a serving business is profitable at a given price.
How this calculator works
The math starts with one question: how much energy does one token cost? Power draw (watts) times PUE gives total facility power attributable to this serving node. Dividing by tokens-per-second gives watt-seconds per token, and dividing by 3,600 converts to watt-hours per token — the natural unit for energy cost accounting. Flipping that gives tokens per watt-hour, the efficiency headline.
Cost per million tokens is the commercial translation: multiply kWh per token by one million, then by the electricity rate. At $0.08/kWh and a modest 5,000 tok/s serving rate at 1,200 W with a 1.2 PUE, energy cost alone is roughly $0.046 per million tokens — a fraction of the API prices cloud providers charge, but a real and growing line item as inference volumes scale.
Monthly cost scales directly with volume. At 50 billion tokens per month (50,000 M), that same setup draws about 320 MWh and costs roughly $25,600 in electricity alone — before amortized hardware, bandwidth, or labor. Understanding this per-token energy cost is the first step toward optimizing model serving: smaller models, quantization, speculative decoding, and batching all improve the tokens-per-watt figure and reduce cost at scale.