At KubeCon + CloudNativeCon Japan 2026, we ran Aitra Meter live for the full event, attributing real inference energy on an 8×H100 cluster down to namespace, workload, model, and tenant. One theme came up again and again on the floor: as AI becomes production infrastructure, teams need a way to measure, understand, and optimize it.
KubeCon + CloudNativeCon Japan 2026 ran July 28–30 across Tokyo and Yokohama, the second edition of the event in Japan. AI was the throughline: CNCF, releasing its State of Cloud Native Development in Japan 2026 survey alongside the event, reported that roughly 100,000 of Japan's AI developers now build on cloud native technologies. (LF Events) (CNCF) That shift is what Aitra is built for. AI is no longer a workload beside the cluster; it runs on it.
What we presented
At the Cloud Native Telecom Meetup Japan, held at the NTT DOCOMO R&D Open Lab in Odaiba, Tokyo on July 28, Steven Tan of Xfusion gave a talk titled "Metering AI Inference: From PDU to Pod." His core point: as AI becomes production infrastructure, organizations need an operational layer to measure, understand, and optimize it. Aitra calls this AI Infrastructure Intelligence, and pursues it as an open initiative under the SODA Foundation.
The current implementation, Aitra Meter, handles the measure step. It joins two telemetry streams that today exist independently: infrastructure-native GPU energy counters (NVML, DCGM, AMD SMI) and AI-native token counts from the serving engine (vLLM, SGLang, TGI, Triton, Ollama). The result is a single unit, joules per token, attributed by namespace, workload, model, and tenant. It installs with one Helm chart, excludes idle windows, and tags every record with the method used to attribute it. Output feeds the tools teams already run: Prometheus, Grafana, OpenCost, and OpenTelemetry.
Model size alone does not predict energy per token. It has to be measured.
A production system, metered live
At the SODA Foundation project booth in Yokohama, we ran an end-to-end production demonstration for the duration of the event: live inference on the Xfusion Singapore Open Lab 8×H100 cluster, live workload energy attribution through Aitra Meter, and a real-time dashboard on the floor. Metered in Singapore, displayed live in Yokohama. The system ran on existing cloud-native tooling under live workloads, which moved the conversations from concept to deployment.
Working through the SODA Foundation
Aitra Meter is developed in the open as a community project under the SODA Foundation, and the SODA community ran the booth across the event. Neutral governance matters here: it lets a measurement and attribution layer be adopted across vendors and operators rather than living inside any one stack. Discussions at the booth ranged over benchmarking, network telemetry, distributed inference, and multi-tenant attribution.
What we heard
The technical feedback pointed consistently in one direction: broaden the scope from GPU measurement toward full-stack AI Infrastructure Intelligence.
- A major mobile network operator: MEC and edge infrastructure, pointing to distributed inference.
- A global IT services and systems integrator: broader infrastructure telemetry, toward full-stack telemetry.
- A telecom-backed AI cloud provider: customer workload optimization, toward cloud optimization.
- An enterprise IT and hardware vendor: networking efficiency, toward network telemetry.
- An application delivery and networking vendor: load-aware metering, toward traffic-aware scheduling.
- An AI model solution provider: efficiency benchmarking, toward cross-hardware benchmarking.
What we learned
Four observations stood out: existing telemetry remains fragmented; cost and utilization drive operational decisions; practical deployment matters more than new metrics; and AI is now operated as production infrastructure. Together they point one way. The scope expands from GPU inference measurement to AI Infrastructure Intelligence.
What comes next
Measurement starts at GPU inference today and widens across the stack (CPU workloads, network, storage, agentic workflows, and training), each step bringing another domain into scope across compute, accelerators, network, storage, power, and cooling. On the roadmap: extending the meter beyond GPUs, an AI infrastructure data model and benchmark suites in the understand phase, and attribution, capacity-planning, and cost-and-energy advisory tools in the optimize phase.
Get involved
Building AI Infrastructure Intelligence is an open, community effort. Join the conversation, shape the roadmap, and help define the metrics in the #soda-aitra channel on the SODA Foundation Slack.
Aitra is an open-source initiative for AI Infrastructure Intelligence (measure, understand, optimize), developed under the SODA Foundation. aitra-meter on GitHub.