LLM agents alternate GPU-bound inference with CPU-bound tool calls over many turns, so GPU-only energy numbers tell part of the story. We measure GPU, CPU, DRAM and whole-system power together on a dual RTX PRO 6000 Blackwell server, across agentic coding, mathematical reasoning and continuous serving.
LLM agents that iteratively reason, plan, and invoke tools create workload profiles fundamentally different from single-pass inference, yet how their energy consumption is distributed across hardware components and workload phases remains poorly understood. Characterizing these workloads therefore requires simultaneous visibility into both component-level power and phase-level execution. We conduct a full-stack energy profiling study combining NVML GPU counters, Intel RAPL CPU/DRAM counters, and Intelligent Platform Management Interface (IPMI) system-level sensors on 2× NVIDIA RTX PRO 6000 Blackwell GPUs, and profile three representative workloads with Qwen3.8-27B. For mathematical reasoning, we compare thinking-enabled and thinking-disabled modes. Our measurements reveal that GPU-only telemetry misses 41–45% of system energy across all three workloads, with non-GPU components accounting for the remainder. In our setup, the sequential agent workload consumes 63× more system energy per output token than saturated serving, reflecting the absence of batching, context growth across turns, and tool-induced idle periods. Extended thinking generates 21–75% more tokens per problem, while per-token energy differs by less than 1% between modes within each dataset, indicating that the resulting increase in energy is driven by output volume rather than a change in per-token efficiency. Continuous batching improves system-level energy efficiency by 3.2× from 1 to 16 requests per second, as GPU power plateaus while throughput continues to increase with batching depth. In agent and reasoning workloads, energy increases mainly because the model generates more tokens or runs longer as context grows, while system power changes little. For the memory-bandwidth-bound reasoning workload, throughput remains unchanged under per-GPU power caps of 400–600 W but drops sharply at 300 W. These findings suggest that context-management techniques such as summarization and selective retrieval may help reduce energy consumption in agent deployments.
Three interfaces are sampled in one timing loop on a dual-GPU server, so component-level power lines up with the phases of an agent run.
Cumulative per-GPU energy counters.
CPU package and DRAM energy counters.
CPU, memory, fan and total-system power from BMC sensors.
Pothers = Psystem − PGPU − PCPU − PDRAM RAPL CPU package power and the IPMI CPU rail differ by only 0.1–0.6%.
During agentic coding, system power averages 1,406 W. The two GPUs draw 785 W (55.9%), the CPUs 333 W (23.7%) and DRAM 14 W (1.0%). The remaining 274 W (19.5%) goes to fans, VRM and PSU conversion losses, PCIe links and the baseboard. Shell commands make up 46.7% of tool calls; they often launch subprocesses or run tests while the GPU is largely idle and the CPU stays active.
GPU power rises to about 800 W during inference and falls to about 150 W during tool execution, as utilization swings between near 100% and near 0%. System power generally remains elevated between inference bursts because the CPUs stay active during shell commands and test runs.
System energy and duration per turn both tend to rise over the course of a trajectory, while average system power changes little. Longer execution is the primary source of the increase, consistent with longer prefill as context accumulates across turns.
At the system level the agent workload consumes 58,383 mJ per output token, about 63× that of serving at 16 requests per second. Three characteristics explain the gap: no batching across requests, prefill work that grows with context, and tool execution between model calls. Under continuous batching, GPU power largely plateaus by 4 requests per second while throughput climbs from 401 to 1,494 tok/s, a 3.2× gain in system energy efficiency.
| System energy per output token | mJ/tok |
|---|---|
| Agent, SWE-bench | 58,383 |
| Reasoning, AIME, thinking on | 29,802 |
| Serving, 1 req/s | 2,955 |
| Serving, 4 req/s | 1,147 |
| Serving, 16 req/s | 920 |
Per task: about 2,989 kJ for one SWE-bench task, 313.2 kJ for a thinking-enabled AIME problem and 53.6 kJ for a MATH-500 problem.
Thinking-enabled runs generate 75% more tokens per problem on AIME and 21% more on MATH-500, with matching increases in energy per problem. Per-token energy differs by less than 0.4% between modes, so the added energy reflects output volume rather than a change in efficiency.
| Dataset | Thinking | Tokens / problem | System mJ/tok | Energy / problem |
|---|---|---|---|---|
| AIME | enabled | 10,506 | 29,802 | 313.2 kJ |
| AIME | disabled | 5,991 | 29,751 | 178.2 kJ |
| MATH-500 | enabled | 1,800 | 29,759 | 53.6 kJ |
| MATH-500 | disabled | 1,488 | 29,682 | 44.2 kJ |
Reasoning draws about 420 W per card, so 500 W and 600 W caps never bind, and a 400 W cap trims GPU power without affecting throughput (about 48 tok/s), consistent with memory-bandwidth-bound decoding. At 300 W, throughput falls by 3.1× and system energy per token rises to 2.5× its value at 400 W. Serving at 4 requests per second behaves differently: 300 W gives the lowest energy per token (1,167 vs. 1,215 mJ/tok at 400 W).
GPU telemetry captures 55–59% of system energy here. Carbon estimates should use full-system measurements or platform-specific corrections.
Per-turn energy follows turn duration as context grows. Context management, such as summarization and selective retrieval, may help reduce agent energy.
The energy cost of thinking is its extra output. Early stopping and budget-aware decoding may reduce it, subject to accuracy trade-offs.
A 400 W cap costs reasoning no throughput and 300 W is a cliff, while serving at 4 req/s is most efficient at 300 W.
Scope. One dual-GPU server and one 27B model, so generalization to other platforms and scales remains uncertain. There are no external PDU measurements for AC validation, and the “Others” category includes estimated contributions.
@inproceedings{luo2026energy,
title = {Where Does the Energy Go? Profiling {LLM} Agent Inference on {Blackwell} {GPUs}},
author = {Luo, Qi and Li, Kunlin and Wang, Ziwen and Chen, Yun},
booktitle = {NeurIPS 2026 Workshop on Resource-Aware Agentic AI},
year = {2026},
eprint = {2609.29707},
archivePrefix = {arXiv},
primaryClass = {cs.DC}
}