Power Caps the AI Factory. The Network Decides How Much It Produces

At GTC Berlin, NVIDIA networking chief Gilad Shainer traced AI-factory productivity through five infrastructure pillars. Agentic AI adds a second challenge: enforcing boundaries without losing the capacity those factories were built to deliver.

Most conversations about AI infrastructure begin with a GPU specification. Gilad Shainer began with the electricity supply.

 "Power is your limiting factor for everything you build,"

 NVIDIA's Senior Vice President of Networking told an on-the-record press roundtable at GTC Berlin on 22 October. From there, nearly every part of his presentation, including the choice between copper and optics, the design of storage and the security of autonomous agents, came back to the same question: how much useful work can an AI factory extract from the power it has secured?

Shainer made the stakes concrete with round numbers: an approximately 1-gigawatt AI facility costing $50 billion and generating as much as $40 billion a year in token revenue. NVIDIA's own estimate, published on 1 October, is around $60 million per megawatt, or roughly $60 billion per gigawatt, which confirms the order of magnitude. The precise figure matters less than the asymmetry it reveals. On either estimate, a 10% saving on construction cost is worth $5 to $6 billion once, while a 10% gain in tokens per watt is worth about $4 billion for every year the site runs at capacity, if Shainer's revenue illustration holds. NVIDIA's analysis calls power "the binding constraint on an AI factory," which is why throughput per megawatt, not component price, sets earning capacity.

The network is becoming part of the computer

Only a few years ago, Shainer recalled, an AI system could be described as GPUs and a network. The emerging factory is more heterogeneous. GPUs perform accelerated model computation; CPUs orchestrate agents and execute tools; and specialized LPUs accelerate latency-sensitive inference operations. Around those processors sit shared context memory, persistent storage and networks with different latency, bandwidth and security requirements. NVIDIA now describes five infrastructure pillars: scale-up, scale-out, scale-across, context memory and scale-in. Calling all five "networks" would miss an important distinction: context memory is a purpose-built storage tier, while scale-in is an infrastructure domain that combines connectivity, services and enforcement.

Scale-up shows why the vocabulary matters. "It's actually not a network. It's computing infrastructure," Shainer said of NVLink. In a synchronization-heavy workload, GPUs repeatedly need to exchange partial results before the next phase can proceed. NVIDIA NVLink switches do more than forward data: SHARP technology can execute parts of collective reduction operations inside the fabric, limiting data movement and synchronization overhead. The advantage is narrow but consequential: the fabric performs coordination work that would otherwise consume accelerator time.

The same economics extend to scale-out, where Spectrum-X Ethernet or Quantum InfiniBand connects server groups across an AI factory. A job may involve thousands of synchronized accelerators, and a late collective operation can leave all the other processors waiting. "This is where you lose money," Shainer said. NVIDIA designed Spectrum-X around adaptive routing, targeted congestion control and RDMA transport to address the jitter and congestion patterns of distributed AI. Rivals backing the open Ultra Ethernet specification argue that standards can close much of that gap. Buyers should ask for GPU utilization, job-completion time and tail-latency measurements on their own communication patterns, not simply line-rate bandwidth in a datasheet.

Scale-across uses Spectrum-XGS Ethernet to connect distinct data centers, potentially allowing a workload to draw on a larger resource pool. Distance, however, does not disappear because the equipment is faster. Its value depends on whether the application can tolerate cross-site latency and coordination costs. Across all three scaling domains, networking is now a component of delivered compute capacity rather than an overhead item purchased after the compute is chosen.

THE FIVE PILLARS: QUESTIONS TO ASK BEFORE BUYING

How watts disappear between the GPU and the workload

Shainer's preference is to use copper where distance permits. Copper is relatively robust and energy-efficient at short reach, but rising signaling rates shorten the practical distance over which it can carry data. He cited roughly a meter to a meter and a half at 200 Gb/s as an example. That makes rack density a networking decision as well as a computing one: place more of the required connectivity close to the processors, and fewer links need power-hungry optical conversion. Shainer tied NVIDIA's move toward 800-volt DC racks to the same logic: more power per rack means more compute within copper reach.

NVIDIA's scale-up roadmap illustrates why the problem grows: from current 72-GPU NVLink domains, it envisages a multirack NVL576 design for Vera Rubin Ultra and a future NVL1152 architecture associated with Kyber and Feynman. These are roadmap systems, but they share one consequence: once a scale-up domain leaves the rack, it needs optics. Beyond short electrical reach, co-packaged optics move optical engines closer to the switching silicon. Shainer offered an approximate comparison of five watts versus twenty watts for a 1.6-terabit optical connection, depending on the design. NVIDIA's published technical material claims up to a fivefold reduction in power per 1.6-Tb/s port compared with pluggable optics. Savings at site level will depend on port density, topology, optical reach, cooling and service practices.

The more revealing part of his explanation was about manufacturing. Earlier co-packaged designs struggled with the ability to qualify an assembled optical system at volume, rather than proving that an individual component could work. "You could validate every component by itself, but not when it's built together," Shainer said. NVIDIA's response combines small micro-ring modulators, package and test innovations developed with partners including TSMC, custom laser designs and improved fiber attachment. After more than three decades in semiconductors and industrial systems, I see this as the decisive step: performance does not become industrial capacity until it can be manufactured, tested, maintained and repaired reliably.

NVIDIA also reports a tenfold gain in network resiliency for its photonics architecture against specified comparisons. That is a vendor claim that operators should validate against their own fault and maintenance scenarios. Shainer connected the argument to fewer interventions around pluggable modules, where human handling can create outages. The same practical reasoning applies to cooling. Rubin systems can operate with coolant temperatures of up to 45 degrees Celsius, reducing dependence on energy-intensive chillers; whether mechanical cooling can be eliminated depends on local climate and facility design. Each saving matters because watts consumed outside productive computation cannot be used by the accelerators.

Efficiency gains in the fabric follow a rebound pattern. Shainer estimated that multiplane configurations cut the number of scale-out switches per deployment by roughly 30 to 40%, while bandwidth per GPU doubles from one generation to the next. Efficiency per deployment therefore need not translate into a smaller networking market. More AI factories, larger clusters and new workload patterns can absorb the savings. For capacity planners, the implication is to model future bandwidth and equipment generations rather than treating today's leaner topology as a permanent cost floor.

Storage follows a similar logic. NVIDIA's CMX context-memory tier holds reusable inference key-value (KV) cache close enough to serving infrastructure to avoid expensive recomputation and GPU stalls. Shainer asked why operators should pay for the same durability as a bank ledger when derived inference context can be rebuilt. The instinct is sound, but "recomputable" does not mean free to lose: regeneration consumes time, compute and power, and the underlying source records may still require strict preservation. The practical decision is which data must survive, which state may be reconstructed, and what the recovery delay costs the application. This is a workload classification question before it is a storage procurement question.

Agentic AI puts control on the same critical path

My first question to Shainer concerned what changes when the workload is no longer a single model response, but an autonomous agent. He called agentic AI "probably the most complex workload ever being built." An agent may make repeated model calls, execute code, access files, use external tools and collaborate with other agents. Its activity moves continuously between CPUs, GPUs, inference accelerators, memory and storage. Infrastructure optimized only for a model-serving request is incomplete once this continuous exchange becomes the dominant pattern.

Shainer described scale-in as NVIDIA's answer to this change. He rejected the idea that it merely renames a front-end access network. In the architecture he outlined, BlueField-4 DPUs, DOCA software and Spectrum-X Ethernet coordinate services across the agent servers, model servers, storage infrastructure and related systems. His shorthand was that "the front-end network is gone." The physical access path has not vanished; rather, the old separation between an access network and the AI back end is no longer a useful description of the new control and data flows.

The security reason is straightforward. An agent pursuing a legitimate task may still reach for information or privileges beyond what the business intended. OpenShell provides a software runtime boundary around the agent, with controlled access to resources and credentials. NVIDIA Sentry adds an isolated enforcement domain on BlueField-4. Its DOCA Tower gateway can inspect configured application traffic and enforce network-access policies, while DOCA Argus provides out-of-band visibility into the host execution environment, including processes, containers and selected memory regions. I examined the governance limits of this stack in my GTC Berlin analysis of OpenShell. What the roundtable added is where enforcement physically sits: in the network, outside the host the agent runs on, provided agent traffic is actually routed through it.

NVIDIA says Sentry running on BlueField-4 can increase agent capacity by up to 20%, partly by moving infrastructure work away from host CPUs. This is a company-reported capacity claim, not a demonstrated increase in realized token revenue, and the published technical documentation points to planned Sentry availability in early 2027. Independent benchmarking will need to measure both policy effectiveness and end-to-end performance overhead across representative agent workloads. If the gain is borne out, security offload may improve economics rather than simply add cost, but those two benefits should be tested separately.

Tokens per watt is the factory metric. It is not the business outcome.

My second question was whether the arrival of agents requires a new economic metric. Shainer's answer was no: tokens produced per unit of power remain central to an AI factory. He is right about the supply side. For an operator with a fixed megawatt allocation, the number of saleable tokens it can produce, at the required service quality and utilization, is fundamental to earning capacity. NVIDIA's own infrastructure analysis makes the same point while also acknowledging that demand, useful life and the ability to serve different workloads shape returns.

But an enterprise does not deploy an agent to maximize the number of tokens it generates. It deploys one to complete work. An agent that retries the same action, invents extra tool calls or crosses a policy boundary may consume more compute while producing less value. For that buyer, I would track a complementary measure: cost per authorized task completed. The numerator is the fully loaded cost of execution, including computation, controls and failed attempts. The denominator is not every task attempted; it is work completed to the required quality standard, inside policy and under a verifiable legitimate mandate.

That last condition matters. In my work on the chain of authority for AI agents, I have argued that identity and permission are necessary but do not by themselves establish the right to act. A compromised agent may possess valid credentials. A properly isolated agent may also receive an instruction that nobody with accountable business authority actually approved. OpenShell and Sentry create valuable points at which limits can be enforced and evidence preserved. They cannot determine an organization's risk appetite, create its delegations or replace human authorization where the consequence of an action demands it. Those are governance decisions that must become machine-readable if the infrastructure is to enforce them.

The executive test therefore has two parts. On the factory side, measure useful throughput per megawatt under realistic load, with network synchronization, context storage, cooling and failure recovery included. On the enterprise side, measure successful and policy-compliant task completion, the evidence behind each delegation, and the cost of exceptions, delays and reversals. Neither metric can substitute for the other. Together, they connect the economics of producing intelligence with the responsibility of allowing it to act.

Networking was once an implementation detail beneath the application. In Shainer's AI factory, it increasingly determines how effectively scarce power becomes computation and where technical controls can interrupt unsafe execution. The next competitive advantage will belong not only to those who generate more tokens from each watt, but to those who can turn that capacity into useful outcomes while proving that consequential actions remained within legitimate authority.

REPORTING NOTE  |  Based on an on-the-record GTC Berlin press roundtable with Gilad Shainer, 22 October 2026. Quotes lightly edited for clarity; NVIDIA performance figures are company claims.

SELECTED DOCUMENTATION

1. NVIDIA: BlueField-4 and the scale-in infrastructure (24 Aug 2026)

2. NVIDIA: OpenShell Reference Architecture, GTC Berlin announcement (21 Oct 2026)

3. NVIDIA: Inside Sentry: Secure AI Agents at Runtime (21 Oct 2026)

4. NVIDIA: NVLink in-network SHARP computation (5 Jan 2026)

5. NVIDIA: Spectrum-X Ethernet Photonics (6 Jan 2026)

6. NVIDIA: CMX Context Memory Storage (5 Jan 2026, updated)

7. NVIDIA: Rubin Ultra NVL576 and future Kyber NVL1152 roadmap (16 Mar 2026)

8. NVIDIA: AI factory economics and cost assumptions (1 Oct 2026)

9. NVIDIA: 45°C liquid cooling and geographic limits (21 Oct 2026)

10. S. Gervais: Why AI Agents Need a Chain of Authority (Forbes, 11 Sep 2026)