What it measures
Energy
Steady-state and full-lifecycle J/token, with startup and release energy reported separately.
Utilization
GPU, memory and interconnect use under the same workloads and test points.
Cost and carbon
Whole-server overhead, PUE, electricity price and grid intensity — declared, never assumed.
Principles
Declared boundaries
Every number states what it includes: which stages, which components, which window.
Pinned artifacts
Model revisions, precision, engine version and container digest recorded for each run.
Like for like
Results shown within a scenario and model size group. No leaderboard, no overall winner.
Open records
Raw run data published with results, so anyone can recompute or challenge them.
Rounds and divisions
Aitra Bench Inference v0.5
Call for participation · Tokyo, December 7–9, 2026The first open round. Every model is measured two ways, so results stay comparable while each team can show its model at its best.
Closed division
A common test point: fixed hardware class, parallelism, workloads and thinking setting. Directly comparable across models.
Open division
Each model team recommends its engine, precision, parallelism and thinking setting. Every change is disclosed with the result.
Built on published research
-
13 models · 3 scenarios · full lifecycle
-
Benchmark 1.0 — Tsinghua University8 models · 3 scenarios · steady state
Built in the open
Built in the open.
Developed by a working group at SODA Foundation, building on published research from Tsinghua University.
Define the methodology
Review drafts and propose changes.
Build the software
Harness, Meter integration, result checks.
Submit results
Run the suite on your systems and publish.