Whitepaper · October 2026
Large Language Model Inference Energy Efficiency Evaluation — Benchmark 1.5
Published by Tsinghua University — Information Energy Laboratory, Department of Electrical Engineering; and Sichuan Energy Internet Research Institute. Supported by Linux Foundation Research and the SODA Foundation through its Aitra project.
At a glance
13
models
3
scenarios
8× H100 80GB
TP8
48
formal results
Key findings
5.4×
Lifecycle vs steady state, text chat
Median full-lifecycle energy per output token was 4.002 J against 0.747 J in steady state. Short services cannot be costed from steady-state figures.
72%
Of lifecycle energy spent at startup
Median share across 13 models in text chat. The lowest was still 57%, so startup cost is systemic rather than model-specific.
0
Models or engines that lead everywhere
Results shift by scenario, by metric and between vLLM and SGLang. Selection has to start from the workload.
What was measured
- Steady-state J/token
- GPU energy ÷ output tokens inside a window where per-request variation stays below 3% across 100 requests. For resident services.
- Full-lifecycle J/token
- Startup + formal requests + resource release, ÷ output tokens of completed requests. For cold-start, short-task and elastic services.
- Throughput, TTFT, TPOT
- Reported per run at a unified test point (TP8, max-num-seqs 128).
Read results with these limits
- One formal run per configuration.
- GPU energy only (no CPU, memory, cooling or facility).
- Answer quality not evaluated.
- The unified test point is not each model's best configuration.
Data and reproduction
- Harness: ml-energy/benchmark v3.0 · Zeus 0.13.1
Cite
[Authors], Tsinghua University (2026). Large Language Model Inference Energy Efficiency Evaluation: Benchmark 1.5. [Publication URL]