AI infrastructure spending reached $89.7 billion in the first quarter of 2026 and is forecast to reach $497 billion for the year. Much of it is running below capacity: most enterprises deploying AI report exceeding their AI budgets while idle accelerator time remains the largest source of waste, and data center owners and operators rank cost, capacity forecasting, and power availability as their top management concerns.
The challenge is connecting information across layers. AI workloads are described in tokens, requests, training steps, latency, and other workload measures, while the infrastructure that runs them is procured, allocated, measured, and billed through different systems. These views are rarely connected, so it is hard to see how workloads use shared infrastructure or to plan on measured demand. Important work is already underway across the ecosystem, from the Open Compute Project and MLCommons to CNCF, the FinOps Foundation, and other Linux Foundation projects. We see an opportunity to build on these efforts with an open intelligence layer for AI infrastructure: one that connects workloads to the infrastructure they use, and measures and explains resource use, utilization, cost, and energy. It gives infrastructure teams, their control planes, and the automation agents increasingly built on them a measured basis for placement, capacity, configuration, and optimization decisions. The Open Compute Project's 2025 call for collaboration on AI data center infrastructure showed how an open letter can start shared work on the physical layer. This letter asks the same of the layer above it.
Each organization builds this today in its own way. We, the undersigned, propose to build it together, in the open.
What Aitra Is
Aitra is an open initiative to measure, understand, and optimize how AI uses infrastructure, from model to facility. It does not prescribe a hardware, software, or AI stack. It defines the methods and interfaces where agreement is needed for results to be comparable, and leaves the rest open to differentiation. Alongside those definitions, Aitra builds an open source software stack under Apache 2.0 as their reference implementation, so that any organization can deploy, extend, or replace components rather than start from a document. Aitra does not evaluate model quality, publishes methods rather than rankings, and works through the orchestrators, observability stacks, and facility systems infrastructure teams already run. Where it recommends or applies changes, it does so within human-defined policy, with every action explainable, auditable, and reversible.
Work began in 2026. Aitra Meter, the first component, was contributed to the SODA Foundation and demonstrated at KubeCon + CloudNativeCon in Yokohama and Shanghai. In parallel, Tsinghua University, with support from Linux Foundation Research and Aitra, has published two rounds of inference energy-efficiency evaluation under a unified, reproducible test point, showing that on the same hardware, energy per output token differs several-fold with model, precision, inference engine, and measurement boundary. This work is the basis for Aitra Benchmark, targeted for Open Source Summit Japan in December 2026. Methods and results: aitra.ai/research.
Four Areas of Work
We propose four areas of focus. They define the scope of Aitra; the working group will establish workstreams and projects within them and extend them as participants bring new work.
- Models & Workloads. What is running and what it demands. First work: define common workload-level measures of resource use, cost, and energy, including per-token measures where applicable, with the measurement boundary declared on every record, and the workload metadata model providers supply to make them meaningful per model. Toward: common definitions across inference, training, agentic, retrieval, and batch workloads.
- Infrastructure & Operations. What it runs on, what it consumes, and what sits idle. First work: define accepted counter sources across accelerator vendors and servers, and attribution methods for shared and partitioned accelerators. Toward: attribution from accelerator through server, rack, power, and cooling to facility, and capacity expressed against power and thermal limits. This is the boundary with OCP's physical infrastructure work.
- Telemetry & Interfaces. The common language. First work: define an open schema from model to facility and provider interfaces for accelerators and inference engines, mapped to Prometheus, OpenTelemetry, and Redfish. Toward: a graph of measured state and reference integrations for orchestrators, observability, cost tooling, and facility systems.
- Benchmarking & Research. Reproducibility of method, and the methods themselves. First work: turn the published evaluation procedure into the open Aitra Benchmark, at a workload-owner-defined quality threshold, with a manifest stating what was counted, and extend it to agentic, retrieval, and tool-calling workloads. Toward: open datasets of runs, peer-reviewed methods, and new categories as participants bring them, including training, storage, networking, and heterogeneous accelerators.
Alignment on these benefits both sides of the infrastructure. Hardware, power, cooling, and data center providers learn how real workloads use their systems and can plan on measured demand. Model providers get a common basis for the figures they publish. Enterprises and cloud providers get workload-level cost, utilization, and energy figures that hold across vendors, and a basis for controlling AI spend other than a cap.
How to Participate
Aitra is hosted by the SODA Foundation, a Linux Foundation project. The working group opens at Open Source Summit Japan, Tokyo, December 7–9, 2026. We invite model providers, inference engine and training framework teams, hardware, network, storage, power, and cooling providers, cloud and telecom providers, researchers and universities, and enterprises running AI in production to join, and to contribute code, test infrastructure, research, benchmarks, or production requirements.
We ask you to sign this letter and help build the intelligence layer for AI infrastructure, from model to facility.
Sincerely,
Footnotes. IDC, Worldwide Quarterly AI Infrastructure Tracker, Q1 2026, for the $89.7 billion first-quarter spend and $497 billion full-year forecast. McKinsey, The cost of intelligence: How CIOs can manage AI demand at scale, July 20, 2026, for most enterprises deploying AI exceeding their AI budgets. Uptime Institute, Global Data Center Survey 2026, July 29, 2026, for cost ranking as the top management concern for the fourth consecutive year. IDC, "Beyond the hype: AI infrastructure costs," Google Cloud blog, December 2025, for the idle accelerator time finding. Tsinghua University, Large Language Model Inference Energy Efficiency Evaluation Whitepaper, Benchmark 1.5, October 2026, aitra.ai/research.