Main Page on The Infinite Unknown

Let the (AI) Bodies Hit the Floor

Jared Watkins — Thu, 07 May 2026 00:00:00 +0000

In 2001, an estimated 95% of all the fiber optic cable in the ground was dark. Telecom companies poured over $500 billion into it on the thesis that internet traffic was doubling every hundred days (it wasn’t, but everyone believed it was), and WorldCom, Global Crossing, and Williams Communications took on enormous debt to lay fiber across oceans and under highways. It took nearly 20 years for traffic to grow into that capacity. Several of those companies went bankrupt. The fiber itself was fine, sitting patiently in the ground, waiting for demand to catch up.

The GPU version of this is already happening. And unlike fiber, GPUs don’t wait.

The buildout that can’t be built

The numbers are staggering on paper. Microsoft, Amazon, Google, and Meta have committed roughly $725 billion in AI-related capital expenditure for 2026 alone, up 77% from the prior year. The announced pipeline of US datacenter capacity for 2026 is somewhere around 12 to 16 gigawatts. That’s a lot of zeros.

Here’s the problem: only about a third of it has broken ground. Close to half of planned US datacenter builds for 2026 have been delayed or canceled, according to Sightline Climate. Not because demand evaporated (the checks are being written), but because the physical world has hard constraints that press releases don’t.

The poster child is Stargate, the $500 billion joint venture between OpenAI, Oracle, and SoftBank that Trump personally announced in January 2025. More than a year later, the JV hasn’t hired staff and isn’t actively developing datacenters. The planned 600 MW expansion at the Abilene, Texas campus was canceled after negotiations broke down. Satellite imagery of the original 1,200-acre site shows six plots cleared, one with actual development. Oracle pushed delivery schedules for several large OpenAI facilities from 2027 to 2028, blaming labor and materials shortages.

But the construction delays aren’t even the most interesting problem with Stargate. The money isn’t real. SoftBank, the supposed primary financial backer, could only assemble 10% equity funding. The rest was going to come from debt. Their first $10 billion tranche was borrowed from Mizuho and other Japanese lenders. OpenAI tried to build its own datacenters but couldn't get financing because lenders weren't willing to back billion-dollar construction projects from a company losing $14 billion a year with no clear path to profitability. SoftBank eventually conditioned its investment on OpenAI restructuring into a public benefit corporation, or the commitment drops to $10 billion. The partners spent months arguing about control structure instead of breaking ground.

And then there’s the circular financing, which is the part that should make anyone who remembers the telecom bubble really nervous. NVIDIA invested in OpenAI. OpenAI uses that money to buy NVIDIA chips. Oracle committed to spending $40 billion on NVIDIA GPUs to power Stargate's Abilene facility. OpenAI signed a $300 billion deal to buy Oracle cloud capacity. So NVIDIA funds OpenAI, who pays Oracle, who pays NVIDIA. The money goes in a circle. Bloomberg published a whole investigation into these arrangements, calling them what they are: AI circular deals where Microsoft, OpenAI, and NVIDIA keep paying each other.

This is vendor financing with extra steps. If you drew it on a whiteboard, a first-year business student would circle it in red. Somewhere in Cupertino and Redmond, very smart people are nodding at this chart and calling it a partnership ecosystem. In the late 90s, telecom equipment makers like Nortel and Lucent lent money to their customers so those customers could buy their products. It inflated demand numbers beautifully right up until the loans went bad and the whole thing collapsed. The AI version is more sophisticated (it’s structured as equity investments, cloud commitments, and partnership agreements rather than simple loans), but the economic logic is identical. The demand looks enormous on paper because the same dollars are being counted multiple times as they circulate between a handful of companies. When actual external revenue has to support the structure instead of recycled internal capital, the math stops working.

Stargate isn’t an outlier. It’s the pattern.

To be clear, the fact that these companies are willing to spend this kind of money isn’t irrational. The underlying demand signal for AI compute is real and growing fast. The problem isn’t the bet. It’s the mismatch between the speed of the financial commitments and the speed of the physical world.

Everything bottlenecks at once

The constraint isn’t any single thing. It’s everything, simultaneously.

Large power transformers now have lead times of 128 to 144 weeks. That’s two and a half to nearly three years. Prices are up 77% since 2019, and Wood Mackenzie projects a 30% deficit in power transformer availability for 2026. These aren’t exotic components. They’re the things that connect a datacenter to the electrical grid. Without them, nothing turns on.

HBM (the specialized memory that goes into AI accelerators) has demand growing 80 to 100% annually against supply growing 50 to 60%. Only three companies on earth make it. NVIDIA’s Blackwell GPUs are sold out through mid-2026 with a massive backlog, and the company reportedly cut consumer RTX 50-series production significantly because the same memory capacity feeds HBM production lines. Datacenters will consume 70% of all memory chips produced worldwide in 2026.

Copper is at roughly $5.60 a pound and hit $6 earlier this year. A datacenter needs about 27 tonnes per megawatt of capacity. The same copper is being fought over by the renewable energy buildout that’s supposed to power these same facilities.

And then there’s the grid interconnection queue, which is the real binding constraint. You can have the land, the permits, the chips, and the money, but if you can’t get power to the site, you have a very expensive, very well-permitted patch of dirt.

None of these constraints are permanent. Transformer manufacturing is scaling (Hitachi Energy, Siemens, and others are expanding capacity). HBM production is ramping. But “scaling” and “ramping” operate on industrial timelines. Tech leadership hasn’t had to face that level of reality in recent decades.

Behind-the-meter won’t save you

The industry’s answer to the power bottleneck has been behind-the-meter generation: bring your own power plant, skip the grid entirely. It’s a smart instinct, and it’s the kind of creative problem-solving that eventually does work in infrastructure buildouts. But the near-term reality is messier than the pitch decks suggest.

Most BTM deals are centered on natural gas, with some nuclear restarts and fuel cell projects in the mix. AEP and Bloom Energy announced a 1 GW fuel cell deal (the largest utility-scale fuel cell procurement in US history). It hasn’t delivered yet. Some of the announced deals read more like science fiction (space-based solar, small modular reactor designs that don’t exist yet) with delivery timelines in the 2028 to 2030 range. That’s not giving you near-term relief. Anything more than 3 years out feels more like hope than a plan.

The fuel cell angle deserves its own reality check. Bloom Energy and others talk about fuel cells as clean, flexible BTM power, and the technology is real. But hydrogen isn’t an energy source. It’s an energy carrier. Unlike natural gas, which comes out of the ground ready to burn, hydrogen has to be manufactured first, usually by electrolysis (running electricity through water) or steam methane reforming (which uses natural gas anyway, so what’s the point). The round-trip energy efficiency of producing hydrogen, compressing it, and running it through a fuel cell is roughly 40%. You’re losing 60% of the energy you started with before a single GPU turns on. (There is a version of this where the clean energy answer is just a very complicated way to burn natural gas less efficiently, and some of these announcements are basically that.) And that’s before you deal with the storage and transport problems: hydrogen is the smallest molecule there is, it leaks through containment walls and pipe joints that are perfectly tight for other gases, it causes embrittlement in conventional steel (gradually weakening the metal until it cracks), it needs to be stored at extremely high pressures or cryogenic temperatures, and it’s explosive across a wide range of concentrations in air. There is no industrial-scale hydrogen supply chain today, and building one is a decades-long infrastructure project unto itself. Fuel cells running on natural gas are more practical, but at that point you’ve built a less efficient gas turbine with extra steps.

Even the relatively conventional BTM projects (gas turbines, which are the most deployable option) face the same transformer and switchgear bottlenecks as grid-connected builds. Gas turbines produce AC at medium voltage. The next-gen AI racks they’re supposed to power run on 400V to 800V DC. That means you still need the full power conversion chain between the generator and the rack: step-down transformers, rectifiers, and DC distribution infrastructure, all of which use the same components that are backordered for years. BTM doesn’t eliminate the supply chain. It just changes who you’re buying from. And the 800VDC ecosystem that NVIDIA’s latest architectures require won’t even be commercially available until the second half of 2026.

It also doesn’t eliminate the community opposition problem, and may actually make it worse. Nobody loves having a datacenter next door, but a datacenter with its own gas-fired power plant raises environmental and permitting issues that a grid-connected facility doesn’t. There are now 188 local opposition groups across 40 states. Over 300 state datacenter bills were filed in just the first six weeks of 2026. Maine enacted the first state-level moratorium on large datacenters. Virginia (home to 643 facilities) has a proposed moratorium halting all new applications until July 2028. Georgia’s Senate is considering a one-year ban. There’s even a federal moratorium bill now. While it’s not expected to pass the sentiment against these projects is growing.

Behind-the-meter power isn’t big enough and won’t come online fast enough to rescue the near-term pipeline. And the local opposition is only getting louder.

Dark GPUs are worse than dark fiber

Here’s where the analogy breaks down in a way that makes the current situation more dangerous to the economy than dark fiber of the past.

Dark fiber sits in the ground. It doesn’t rot. It doesn’t become obsolete. It costs almost nothing to maintain once installed. The fiber laid in 1998 was still perfectly usable in 2015 when traffic finally grew into it. Patience was rewarded, even if the investors who funded the original buildout went bankrupt waiting.

GPUs don’t work that way. A GPU that can’t be powered or deployed today isn’t going to sit on a shelf and be useful in three years. AI accelerator generations move on 18 to 24 month cycles. NVIDIA’s Blackwell is already being succeeded by Rubin. The H100s that were the hottest commodity in 2023 are already being displaced. A chip produced today that can’t be put to work has a shelf life measured in months before it’s obsolete, not decades.

And this isn’t hypothetical. It’s already happening. Microsoft’s Satya Nadella has said publicly that power, not compute, is their biggest datacenter constraint, and that Microsoft has AI GPUs “sitting in inventory” because it lacks the power to install them. In Santa Clara (literally minutes from NVIDIA’s headquarters), two freshly built datacenters, Digital Realty’s SJC37 and Stack Infrastructure’s SVY02 campus, are standing empty because the local utility can’t supply the electricity. They may sit empty for years. In Northern Virginia, the largest datacenter market in the country, connection delays are running multiple years as utilities struggle to reinforce high-voltage infrastructure. Regions in the Pacific Northwest and the Southeast are reporting wait times of two to five years for new power capacity.

So the dark GPUs aren’t a future risk. They exist right now. NVIDIA keeps shipping chips. The datacenters keep getting built (or half-built). And the power to run them isn’t there. Every GPU that sits in a warehouse or in a powered-down rack is depreciating toward obsolescence while the next generation rolls off the fab line.

The capital destruction isn’t deferred. It’s immediate. And it’s compounding. Unlike fiber that sat dark but held its value, a GPU that misses its deployment window doesn’t get a second chance. By the time the power arrives, the chip is last-generation and worth a fraction of what was paid for it.

This pricing reality probably isn’t fully baked into the market yet, because the worst of the delivery failures are still 12 to 18 months out. The commitments have been made, the purchase orders are in, but the physical constraints haven’t fully collided with the financial expectations. When they do, someone is going to be holding a lot of very expensive, very obsolete silicon.

The subsidy cliff

There’s a demand-side problem too, and it’s related.

OpenAI is projected to lose $14 billion in 2026 despite hitting $20 billion in annualized revenue and having 900 million weekly ChatGPT users. Ninety-five percent of those users don’t pay. The company’s cumulative losses between 2023 and 2029 are projected at an eye-watering $115 billion, (with a B) with profitability not expected until 2029 or 2030. That’s a lot of subsidized usage.

Anthropic is doing better on unit economics (they reportedly hit $30 billion ARR while spending a quarter of what OpenAI spends on training), but the broader pattern holds across the industry: AI services are being sold below cost to build market share, and the bills are coming due. The era of $20-a-month plans that cost the provider $100-plus to serve is ending as these companies approach IPOs and investor patience thins.

When prices rise to cover actual costs, how much of current usage survives? A 2025 MIT study found that 95% of enterprise AI pilot programs failed to deliver measurable financial returns. Only 6% of organizations qualify as “AI high performers” (generating 5% or more EBIT impact) per McKinsey. Now, to be fair, those numbers deserve some nuance. Enterprise adoption of anything is historically slow, and a lot of those early pilots were running older models with teams that were still figuring out how to use them. It’s not clear how much of that 95% failure rate reflects genuine limitations of the technology versus enterprises being bad at adoption (which they almost always are with new tools) versus a measurement problem where the ROI is real but shows up in places the study wasn’t looking. The tools have gotten dramatically better even in the last twelve months, and the organizations that started early are probably seeing compounding returns that newer adopters haven’t caught up to yet. But even granting all of that, the gap between the infrastructure investment and the demonstrable enterprise revenue is enormous, and it’s the revenue that has to justify the buildout.

Sequoia Capital’s David Cahn laid out the math starkly: take NVIDIA’s GPU revenue, double it for total datacenter costs, double again for the margins end users need to justify the spending, and you get a $600 billion annual revenue requirement from AI services. The actual revenue is a fraction of that. That gap tripled in twelve months.

The balance sheet problem

The hyperscalers funding this buildout are entering unfamiliar financial territory.

Historically, these companies spent about 40% of their operating cash flow on capital expenditure. In 2026, that number approaches 100%. Google’s free cash flow is projected to drop roughly 90%, from $73 billion to around $8 billion. Amazon is expected to go free-cash-flow negative (Morgan Stanley projects negative $17 billion, Bank of America projects negative $28 billion). Microsoft’s free cash flow drops an estimated 28%. Meta has $237 billion in non-cancelable contractual commitments.

These companies have always been valued partly on their enormous free cash flow generation. They didn’t need to borrow. They self-funded everything. That’s changing. Bank of America forecasts hyperscaler debt issuance will hit $175 billion in 2026, more than six times the annual average of the prior five years.

When tech companies that were valued like capital-light software businesses start borrowing like capital-intensive industrial companies, the market tends to re-rate them accordingly. Software companies trade at 25 to 35x earnings. Heavy industrials and utilities trade at 12 to 18x. If investors start pricing hyperscalers like the infrastructure-heavy companies they’re becoming, the multiple compression alone wipes out trillions in market cap before a single revenue target is missed.

That said, these companies aren’t utilities. They still have the advertising, cloud, and commerce businesses that generated the cash flow in the first place. The AI capex is an overlay on businesses that remain enormously profitable. The repricing risk is real, but it assumes the market ignores the base business entirely, which is the kind of overcorrection that creates buying opportunities as often as it creates crises.

There’s a workforce trap buried in that repricing. These companies have built their compensation structures around stock. At Google, Meta, and Microsoft, stock-based compensation is a huge chunk of total pay, especially for the engineering talent they can’t afford to lose during a buildout this complex. That worked beautifully when share prices climbed every year. But when multiples compress, stock comp stops being a retention tool and starts being a source of attrition. The employees who can leave, will. And the companies can’t just replace stock comp with cash, because they’ve spent all their cash (and then some) on datacenters and GPUs. You end up in a situation where the people you need most to execute the buildout are the ones most likely to walk, right when you have the least financial flexibility to keep them.

And here’s why that matters beyond tech investors: AI-related stocks now represent roughly 45% of S&P 500 market cap. Forty-one AI-linked stocks (about 8% of index constituents) account for 47% of the total index value and contributed 74% of the index’s gains since ChatGPT launched. The S&P 500 isn’t a broad market index anymore. It’s a leveraged bet on AI monetization.

AI-linked investment-grade debt has climbed to $1.4 trillion, representing 15% of the US credit market. If the revenue doesn’t materialize on schedule, the correction doesn’t stay in tech. It cascades through every retirement account, every index fund, every pension plan that’s passively allocated to the S&P 500.

Crowding out the reindustrialization

Here’s a dimension of this that I think is badly underappreciated: the AI buildout isn’t happening in a vacuum. It’s happening at the same time the US is trying to reshore semiconductor fabrication, battery manufacturing, and a whole range of industrial capacity that’s been offshore for decades.

The numbers on that reshoring push are enormous. Since 2020, over $630 billion has been invested across 140 semiconductor projects alone, creating roughly 500,000 jobs in 28 states. TSMC is building a $100 billion campus in Arizona. Micron announced $200 billion across Idaho, New York, and Virginia. Total manufacturing construction spending hit $234 billion annually by mid-2024, up 217% from 2019. The IRA, CHIPS Act, and IIJA together authorized over $2 trillion in federal funding. This is the most ambitious industrial policy the US has attempted in generations.

And it needs the same stuff the AI buildout needs. The same transformers, the same switchgear, the same copper, the same grid interconnection capacity, the same skilled electricians, the same construction labor. A single TSMC fab phase requires around 200 megawatts of power. Multiply that across dozens of fabs, battery plants, and related industrial facilities, and you’re talking about gigawatts of new industrial demand competing with the datacenter pipeline for grid capacity that doesn’t exist yet.

The grid interconnection queue now exceeds 2,100 gigawatts, which is more than the total installed capacity of the US grid. Everything is in line: datacenter projects, semiconductor fabs, battery plants, solar farms, wind farms. The queue itself has become the bottleneck, and the datacenter buildout is the 800-pound gorilla in that line.

The labor competition is just as bad. The datacenter construction industry faces a projected shortfall of up to 500k workers. TSMC’s Arizona fab was delayed six months largely because of skilled labor shortages, with Intel and other fab builders competing for the same pool of certified electricians and mechanical specialists. Construction unemployment hit a record low of 3.2% in August 2025. There’s no reserve workforce to absorb simultaneous megaproject buildouts across multiple industries.

What this means in practice is that every datacenter project that outbids an industrial project for transformers, power capacity, or construction crews is directly slowing down the reshoring effort. And the datacenter operators have deeper pockets. Hyperscalers can pay whatever it takes for a transformer allocation or a power interconnection because they’re spending hundreds of billions this year. A midsized semiconductor equipment supplier or battery plant builder can’t compete with that.

Data for Progress polling in early 2026 found that more than two-thirds of voters support new manufacturing, housing, and clean energy projects in their communities. Support for new datacenter development sits at 48%. People want the factories. They’re less sure about the datacenters. And the datacenters are eating the supply chain alive.

If the AI buildout stalls and the capital turns out to have been misallocated, the damage isn’t limited to tech company balance sheets. It will have crowded out and delayed the industrial projects that were supposed to reduce American dependence on foreign supply chains. That’s a strategic cost that goes well beyond stock prices.

The employment squeeze

And all of this is happening alongside a labor market disruption that’s already underway and accelerating.

Companies are cutting jobs in anticipation of AI’s impact, not because AI has actually proven it can replace those jobs. Harvard Business Review reported in January 2026 that firms are laying off workers based on AI’s potential, not its demonstrated performance. Fifty-five thousand job cuts were directly attributed to AI in 2025, with another 32,000 in the first two months of 2026. One in six employers expects AI to reduce headcount this year.

The irony is thick: the same AI that isn’t generating enough revenue to justify its infrastructure costs is already being used as justification for layoffs. Gartner predicts that by 2027, half of the companies that attributed headcount reductions to AI will rehire for similar functions under different titles, having overestimated what AI could actually do. But that’s cold comfort to the people being let go now.

Looking further out, the World Economic Forum projects 23% structural labor market churn through 2027, with a net loss of 14 million jobs globally. The introduction of capable, affordable robotics in the three to six year timeframe will sharpen this considerably, extending AI displacement from knowledge work into physical labor.

Where this all lands

What makes this moment different from a normal correction is that everything is converging at once. The physical buildout is stalling (transformers, power, chips, community opposition). The demand is softer than projected (subsidies ending, enterprise ROI still unproven at scale). The financial engineering is reaching its limits (free cash flow consumed, debt replacing equity, circular financing inflating demand numbers). The stock market is concentrated enough that a tech repricing ripples through every index fund and pension plan in the country. The labor market is absorbing AI-driven cuts based on hype rather than demonstrated capability. And the whole thing is competing for resources with the reindustrialization effort that’s supposed to reduce American dependence on foreign supply chains.

Let me try to put a timeline on how this plays out.

Late 2026 through mid-2027 is when the first wave of delivery failures becomes undeniable. The datacenter projects announced in 2024 and 2025 with 18 to 24 month timelines start missing their dates in volume. GPUs pile up in warehouses and unpowered facilities. The gap between announced capacity and operational capacity widens visibly. Hyperscaler earnings calls start featuring uncomfortable questions about returns on AI capex. The token factory companies (OpenAI, Anthropic, and the rest) face real pressure to raise prices as investor patience wears thin, and usage numbers start revealing how much of current demand was price-sensitive.

2027 through 2028 is when the financial consequences arrive. If hyperscaler free cash flow stays near zero or negative for multiple quarters, the market will reprice these companies. A shift from software-company multiples (25 to 35x) toward industrial multiples (12 to 18x) on companies that represent 45% of the S&P 500 would be a multi-trillion dollar repricing event. Stock-based compensation loses its pull, talent starts moving, and the companies can’t replace it with cash they don’t have. Credit markets tighten on AI-linked debt (currently $1.4 trillion). Meanwhile, the reindustrialization pipeline is two to three years behind schedule because the datacenters ate the transformers, the construction labor, and the grid interconnection capacity.

2028 through 2030 is where the employment picture gets sharp. By then, the AI tools will have matured enough (and the robotics will have arrived in early commercial form) that the job displacement moves from anticipatory layoffs to structural replacement. The companies doing the replacing will be under financial pressure themselves, creating a strange dynamic where firms cut headcount to save money on labor while simultaneously spending more on AI infrastructure that isn’t paying for itself yet.

The aggregate impact? If even half of this plays out on the timeline I’m sketching, you’re looking at a meaningful drag on US GDP. Not a recession caused by AI directly, but a combination of suppressed capital investment returns, stock market wealth destruction concentrated in the indices that most Americans are exposed to through retirement accounts, a delayed reindustrialization that leaves supply chain vulnerabilities unaddressed, and labor market disruption hitting both white-collar and (eventually) blue-collar workers simultaneously. The 2000 to 2002 tech correction erased about $5 trillion in market value and contributed to a mild recession. The current AI exposure is larger in both absolute and relative terms.

I’ve tried to be fair to the counterarguments throughout, because they’re real. The engineering talent being thrown at these constraints is world-class, the financial incentives to solve them are enormous, and the history of technology consistently embarrasses people who bet against human ingenuity on long enough timescales. I don’t think this is a bubble in the sense that the underlying technology is fake. I think it’s a buildout that’s outrunning its own supply chain and revenue base, and the correction when those things catch up will be painful.

The question isn’t whether AI will be transformative. I think it will. The question is whether the timeline of the buildout matches the timeline of the revenue, and what happens to the US economy during the gap between the two. The dark fiber era took twenty years to resolve. The companies that laid the fiber went bankrupt, but the fiber itself eventually became the backbone of the modern internet. With dark GPUs, the hardware won’t wait. The chips depreciate, the architectures move on, and the capital is gone. If there’s a resolution, it has to come faster than twenty years, because the assets don’t have twenty years in them. And I don’t think anyone knows the answer yet.

Megawatt Compute Racks!

Jared Watkins — Mon, 27 Apr 2026 00:00:00 +0000

I’ve designed a lot of racks. Most of them land somewhere between 10 and 15 kW. That’s the enterprise baseline, the design point that most of the world’s installed data center capacity is built around. Standard power distribution, off-the-shelf PDUs, hot aisle/cold aisle airflow. The physics are solved, the playbook is 30 years old, and nothing about it is surprising.

The racks going into AI facilities right now are a different species entirely. The ones being installed today are in the 80 to 100 kW range. The ones coming next are over a megawatt. Each step breaks assumptions from the one before it.

Glossary — acronyms and jargon used in this post

AC / DC — Alternating current / direct current. AC is what comes from the wall; DC is what processors actually run on. Every server has a power supply that converts AC to DC internally. The efficiency push in modern datacenters is about doing that conversion once, at high voltage, rather than repeatedly at lower voltages inside each server.

Aisle containment (hot aisle / cold aisle) — A layout convention where racks face alternating directions so that cold air intakes face a “cold aisle” and hot exhaust faces a “hot aisle.” Containment means physically enclosing one or both aisles with panels and doors to prevent cold and hot air from mixing, which makes cooling far more efficient.

AllReduce — A collective communication operation used in distributed GPU training where each GPU sends its gradient updates to all others and receives theirs back simultaneously. It’s the most bandwidth-intensive operation in large model training, and the reason interconnect bandwidth between GPUs is as important as raw compute.

Ampacity — The maximum current a conductor (wire, bus bar) can carry continuously without overheating. Higher ampacity requires either thicker conductors or higher voltage to carry the same power.

Blind-mate connector — A connector designed to make contact automatically as a component slides into position, without manual alignment or plugging. Used in high-density datacenter systems so a server tray makes both electrical and liquid cooling connections in a single insertion motion.

Bus bar — A solid copper or aluminum conductor that distributes power through a rack or row. Higher ampacity than wire bundles; used in datacenter power distribution because it handles high currents more efficiently than discrete cables.

Busway — An overhead or underfloor power distribution track (think: a giant extension cord rail) that runs the length of a server row and provides tap-off points for each rack. Replaces individual conduit runs at high rack densities where per-rack wiring becomes impractical.

CDU (Coolant Distribution Unit) — The rack-level or row-level appliance that circulates chilled water through a liquid-cooled system. It typically includes pumps, a heat exchanger that connects to the building’s facility water loop, and flow controls. Think of it as the “radiator unit” for a liquid-cooled rack.

CFM (Cubic Feet per Minute) — A measure of airflow volume. Used to quantify how much air needs to move through a rack for air cooling. At 100 kW densities, the CFM requirements become loud, physically challenging, and expensive.

Cold plate — A metal block (usually copper) bolted directly onto a GPU or CPU that has internal channels carrying coolant. Transfers heat from the chip directly into the liquid rather than into the surrounding air.

CRAC (Computer Room Air Conditioner) — The dedicated precision air conditioning units used in datacenters. Unlike home AC, they’re designed for high-sensible-heat loads (mostly heat, little humidity control) and run continuously. At 100 kW rack densities, you typically need one every few rows rather than around the perimeter.

DLC (Direct Liquid Cooling) — A cooling approach where liquid-carrying cold plates are attached directly to heat-generating components (GPUs, CPUs). The heat goes straight into the coolant rather than first into the air. Required at megawatt densities where air physically cannot carry enough heat.

GaN (Gallium Nitride) — A wide-bandgap semiconductor used in high-frequency power conversion. More efficient than silicon at high switching speeds; used in DC-DC conversion stages in datacenter power supplies and increasingly in consumer chargers.

GOES (Grain-Oriented Electrical Steel) — A specialty steel used in transformer cores. The grain alignment is optimized to reduce magnetic losses. There’s limited global production capacity, and both AI datacenter buildout and renewable energy interconnection are competing for it.

HVDC (High-Voltage DC) — A power distribution approach that distributes DC power at high voltage (48V, 400V, or 800V) through a datacenter rather than distributing AC and converting it at each server. Eliminates conversion stages and reduces energy losses.

IGBT (Insulated Gate Bipolar Transistor) — A power semiconductor switch used in UPS systems, solar inverters, and motor drives. Being progressively replaced by SiC in high-performance applications due to SiC’s better efficiency at high voltages and temperatures.

LPM (Liters per Minute) — The flow rate of coolant through a liquid cooling loop. At 1.2 LPM/kW (the industry rule of thumb for direct liquid cooling), an 85 kW rack requires around 102 LPM.

NVLink — NVIDIA’s proprietary high-bandwidth interconnect for GPU-to-GPU communication within a rack or system. Much faster than PCIe or Ethernet; allows multiple GPUs to act as a single unified compute resource.

OCP (Open Compute Project) — A Meta-founded industry consortium that publishes open hardware specifications for datacenter equipment: racks, power distribution, servers, networking. ORV3 (Open Rack Version 3) is their current rack standard; it defines bus bar voltage, connector specs, and physical dimensions.

ODM (Original Design Manufacturer) — Companies like Wiwynn, Quanta, and Supermicro that design and manufacture servers sold under another brand or sold directly to hyperscalers. The AI rack market is largely built on ODM hardware.

OpEx — Operating expenditure; ongoing costs like power, cooling, and staffing. Contrasted with CapEx (capital expenditure), which is the upfront cost of building or buying infrastructure.

PDU (Power Distribution Unit) — The rack-level power strip, essentially, but engineered for datacenter loads. Provides individual branch circuits to each server with metering and protection. At 100 kW densities they’re large, heavy, and custom-spec’d rather than off-the-shelf.

PUE (Power Usage Effectiveness) — The ratio of total facility power to IT equipment power. A PUE of 1.0 is theoretically perfect (all power goes to compute). 1.2 means 20% overhead for cooling and lighting. Lower is better; modern liquid-cooled facilities approach 1.1 to 1.2.

Raised floor — A floor system with removable tiles sitting above the structural slab, creating a plenum underneath for cabling and air distribution. Standard in enterprise datacenters; the underfloor space distributes cold air up through perforated tiles to server inlets.

RDHx / Rear-Door Heat Exchanger — A heat exchanger that replaces the rear door of a standard rack and captures heat from the rack’s exhaust airflow by running chilled water through a finned coil. A hybrid approach: rack fans still run, but the liquid loop captures a large portion of the heat before it reaches the room.

SiC (Silicon Carbide) — A wide-bandgap semiconductor with better high-voltage and high-temperature performance than silicon. Used in EV traction inverters, solar inverters, and increasingly in datacenter power conversion. The same 1,200V SiC MOSFET goes into both 800V EV drivetrains and 800VDC datacenter rectifiers.

Switchgear — High-voltage electrical equipment that controls, protects, and isolates power distribution circuits. The large metal cabinets you see at the entry point of a facility’s electrical system. Lead times for datacenter-grade switchgear have extended significantly as AI buildout demand accelerates.

UPS (Uninterruptible Power Supply) — A battery-backed power system that provides continuous power during utility outages or fluctuations. Datacenter UPS systems use a “double-conversion” topology where all power passes through the battery inverter continuously, giving true zero-transfer-time protection at the cost of some efficiency.

42U — A rack size designation where “U” is a rack unit (1.75 inches). A 42U rack is 73.5 inches tall, the most common standard height. Higher-density AI racks may run 48U or more.

A standard data center rack is roughly 42U to 48U tall (about 7 feet), 24 inches wide, and 36 to 48 inches deep. Call it 18 to 24 cubic feet of usable volume. That physical envelope is the constant across everything that follows. A 10 kW enterprise rack, a 100 kW AI rack, and a 1 MW hyperscale system all occupy roughly the same floor footprint. The density jump is what changes everything else.

The baseline: what 10–15 kW per rack looks like

The CERN data centre: rows of enclosed racks with hot/cold aisle containment, CRAC units along the back wall, raised floor, overhead cable trays. Classic enterprise design, refined over 30 years. Photo: Florian Hirzinger, CC BY-SA 3.0

A typical rack in this range has 20 to 40 servers, each pulling 300 to 500 watts at load. Three-phase AC, maybe 208V or 480V. PDUs are catalog items. Your biggest concern is cable management.

Cooling is straightforward: hot aisle/cold aisle containment, CRAC units pushing conditioned air, done. Air has enough thermal capacity to carry the heat load out of the rack before anything catches fire. Enterprise data centers are typically designed for 150 to 200 watts per square foot of raised floor, and these racks fit comfortably inside that.

This is the installed base (something like 90% of rack capacity in the world right now). The buildings that house it were designed for it. Thousands of these facilities exist and the design has been refined over 30 years.

The middle tier: 80–100 kW racks being deployed today

NVIDIA DGX SuperPOD — a cluster of DGX H100 racks. Each populated rack runs 40 to 50 kW; a full SuperPOD row approaches 100 kW per rack footprint. Image: NVIDIA

The first wave of purpose-built AI data centers isn’t running megawatt racks. It’s running GPU clusters in the 80 to 100 kW per rack range. Think DGX H100 clusters, or dense A100/H100 configurations from ODMs like Wiwynn, Quanta, or Supermicro. This is what’s actually getting installed at scale right now, and it already breaks the enterprise playbook in several important ways.

At 80 to 100 kW, air cooling is still technically possible but you’re working against physics rather than with it. The airflow volumes required are substantial: roughly 2,000 to 3,000 CFM through a single rack, which means high-velocity fans, significant acoustic load, and real structural air management. Hot aisle containment stops being optional and becomes mandatory. Cold aisle containment and blanking panels have to be perfect; any bypass airflow means hot spots. A lot of facilities running these densities are running at CRAC unit limits, with CRACs located every few rows rather than around the perimeter.

The power delivery changes significantly too. At 100 kW per rack you’re looking at 400 to 500 amps at 208V three-phase, which means you’re no longer running standard 30A or 60A branch circuits to a PDU. You need high-amperage three-phase feeds, often delivered via overhead busway (a dedicated power track running the length of the row) rather than individual conduit runs. The PDUs themselves are large, heavy, and rated for continuous 80%+ of their maximum load. Branch circuit protection, cord sizing, and PDU tap-off points all have to be engineered specifically for the load. This isn’t a catalog selection anymore.

Hyperscalers building for this tier are also pushing past 48V DC distribution and moving to 400VDC and 800VDC architectures, centralizing the AC-to-DC rectification closer to the utility feed and distributing high-voltage DC directly to the rack row. The efficiency gains at 100 kW density are real enough that Meta, Google, and Microsoft are all deploying medium-voltage distribution (some running as high as 13.8 kV) before stepping down to rack-level DC. Delta’s 800VDC “AI Power Cube” (co-developed with NVIDIA) is targeting 1.1 MW-scale racks, but the same architecture is relevant even at 100 kW because it eliminates conversion stages that compound into real money at this density.

The buildings designed for this tier look noticeably different from enterprise data centers. Power density per square foot goes from the 150 to 200W/sqft enterprise standard up to 500 to 800W/sqft for a dense GPU row. That changes transformer sizing, switchgear ratings, UPS topology, and generator capacity significantly. Floor loading is a separate hard constraint: racks with liquid cooling hardware at this density can weigh 2,000 to 3,000 pounds, and if you’re running a coolant distribution unit (CDU) per row, a fully flooded unit alone can weigh around 3 tons, so you need slab capacity around 800 kg/m² rather than the typical raised-floor spec. You also need extended rack depth (standard 42-inch racks won’t fit current NVIDIA HGX servers), and those deeper racks affect aisle spacing and the whole floor layout.

On the cooling side, 100 kW is where two distinct approaches are both in active use and worth understanding separately.

The first is rear-door heat exchangers. A rear-door HX (RDHx) replaces the rack’s back door with a chilled-water coil that the rack’s own fans blow exhaust air through. The liquid captures heat from the airstream before it reaches the room, but air is still the medium moving heat away from the chips. The fans keep running, and you still need hot/cold aisle management. Latest-generation units like OptiCool’s 120 kW RDHx can now absorb close to the full heat load of a 100 kW rack, up from the 40 to 70% capture typical of earlier units. A common 2025 deployment pattern runs about 70% liquid capture via RDHx with the remaining 30% handled by conventional room cooling. This approach works without redesigning the facility cooling loop from scratch, which is why it’s popular as a retrofit and for facilities not quite ready to commit to full direct liquid cooling.

The second approach is direct liquid cooling (DLC), where coolant runs through cold plates bolted directly onto the GPUs and CPUs. No air involved in moving heat away from the chips at all. Heat goes straight into the coolant. DLC is more efficient and handles higher densities, but it requires CDUs, supply and return manifold plumbing, and leak detection throughout. The industry sizing rule for a DLC loop is 1.2 liters per minute per kilowatt at 45°C inlet temperature: an 85 kW rack needs a CDU and manifold supporting roughly 102 LPM of flow. That’s not exotic hardware, but it has to be deliberately designed in rather than bolted on after the fact.

At 100 kW, both approaches are viable. The choice comes down to how the facility was built and what the next GPU generation will demand.

The critical point is that 100 kW racks are demanding but solvable within a purpose-built or heavily upgraded facility. Building new infrastructure to this spec costs somewhere between $200K and $300K per rack in facility-side capital (not counting the compute itself). That’s a real number. Retrofitting an existing facility up to 40 kW density is cheaper, around $50K to $100K per rack, but leaves headroom on the table when the next GPU generation arrives. The challenges are well understood, the vendor ecosystem is mature, and there’s enough operational experience to draw from. None of it requires fundamentally new infrastructure categories. It just requires actually building the right infrastructure rather than adapting what’s already there.

What it does require is supply chain access that’s getting harder to take for granted, because a lot of the components that make 100 kW infrastructure work are the same ones going into utility-scale solar farms by the thousands.

The most acute overlap is in transformers. A 100 kW GPU row drawing several megawatts across a facility hall requires large medium-voltage transformers to step utility power down to distribution voltage. Those same transformer types are going into solar interconnection projects in massive numbers: over 90% of new electric generating capacity installed globally in 2025 was solar and wind, and every one of those projects needs medium-voltage step-up transformers to get power onto the grid. Large power transformers now take two to three years to procure in some cases, versus weeks before 2020. The cores of those transformers require grain-oriented electrical steel (GOES), which in the US is produced by essentially one domestic mill (Cleveland-Cliffs). Hyperscalers have been documented outbidding utility grid suppliers for transformer allocations. That’s not a supply chain abstraction. That’s a literal bidding war between AI infrastructure buildout and the power grid that everyone depends on.

The UPS systems at 100 kW facilities have the same problem at the semiconductor level. Double-conversion UPS units (which virtually all purpose-built AI facilities use, since they can’t tolerate even a millisecond of power interruption during GPU training runs) rely on IGBTs and increasingly SiC MOSFETs for the conversion stages. Those devices are in the same demand pool as solar inverter switching components. A 650V GaN switch or a 1,200V SiC MOSFET doesn’t know if it’s going into a solar microinverter, a UPS module, or a datacenter PDU. The fabs don’t care either. Renesas, for example, is now explicitly marketing a single bidirectional 650V GaN device for both solar inverter and AI datacenter applications simultaneously. That’s convenient for the chip vendor and a scheduling problem for anyone trying to place a large order during a tight quarter.

The copper situation compounds everything at this tier too. Microsoft’s 80 MW Chicago facility used roughly 2,100 tonnes of copper across on-site and near-site power connections (about 26 tonnes per megawatt). Scale that to a 100-rack GPU hall at 10 MW of IT load and you’re sourcing 260 tonnes of copper just for the power infrastructure, before you run any cable to the racks themselves. That copper is competing with the solar farms and grid storage projects being built at unprecedented rates to supply the power those same facilities need. It is genuinely circular: the AI buildout is driving power demand that requires renewable buildout, and both the AI buildout and the renewable buildout are competing for the same copper, transformers, and power semiconductors to do it.

The new world: 1+ MW in the same box

The NVIDIA GB200 NVL72 — 72 Blackwell GPUs, 18 compute trays, 9 switch trays, direct liquid cooling throughout. Over a megawatt at peak load. Image: NVIDIA

The NVL72 (NVIDIA’s GB200 rack-scale system) fits in roughly the same floor footprint as all of the above. Same basic rack envelope. And it draws over a megawatt. That’s not a typo. One megawatt, in a box the size of a large refrigerator cabinet.

The physical layout of the NVL72 is worth understanding because it’s nothing like a conventional rack. Inside you have 18 liquid-cooled compute trays and 9 switch trays, all in 1U form factor, plus 4 NVLink cartridges mounted vertically at the rear. Those 4 cartridges alone contain over 5,000 active copper cables, the interconnect fabric that lets all 72 GPUs talk to each other as a single unified compute domain. Each GPU gets 1.8 TB/s of NVLink bandwidth, which is 36x faster than 400 Gbps Ethernet and about 2x faster than the previous H200 generation (which topped out at 900 GB/s per GPU). The aggregate AllReduce bandwidth across all 72 GPUs is 260 TB/s. That number exists because of those 5,000 copper cables crammed into the rear of the rack.

To put the compute density in physical terms: 1 MW sustained is enough to power somewhere around 800 to 900 average American homes. Coming out of a box that fits in a large office.

NVIDIA contributed the NVL72’s rack, compute tray, and switch tray designs to the Open Compute Project in late 2024, which means the full mechanical and electrical specs are now public. A few things in those specs are worth calling out because they show just how much had to be rethought from first principles.

The rack frame has over 100 lbs of steel reinforcements to handle 6,000 lbs of mating force as trays blind-mate into position. The bus bar carries 1,400 amps (double the existing ORV3 standard), same width as before but with a deeper profile for the increased ampacity. The cooling connections use a floating blind-mate liquid cooling manifold: each tray makes its coolant connection automatically as it slides in, the same mechanical motion that makes the electrical connection. No separate plumbing step, no hose connections. The tray goes in, everything connects.

That’s a 7 to 10x jump over the 100 kW racks being deployed today, and 70 to 100x over the enterprise baseline, all in the same floor footprint. And unlike the move from 15 kW to 100 kW (demanding but solvable), the jump to 1 MW broke categories. The existing OCP standards didn’t have answers. NVIDIA had to write new ones.

Power delivery: why you can’t just plug it in

Standard AC power delivery has a dirty secret: every conversion step wastes energy. Electricity comes in from the utility, goes through a transformer, hits a UPS, gets distributed through PDUs, and finally runs through server power supply units that convert it again to DC on the board. Each of those steps is 94 to 97% efficient. Cascade four or five of them and you’ve lost 15 to 25% of your input power to heat before a single computation runs.

At 10 kW per rack, this is annoying but manageable. At 100 kW, it’s a real cost that starts showing up in facility OpEx. At 1 MW, it’s a crisis.

Key highlights from NVIDIA’s GB200 NVL72 OCP submission: new bus bar spec, floating blind-mate cooling manifold, reinforced rack frame. Image: NVIDIA

High-voltage DC delivery eliminates one to two of those conversion stages. The OCP ORV3 standard uses a 48V DC bus bar delivered via blind-mate connector: the rack slides in and makes contact. No cord management, no intermediate conversion, direct DC to the server boards. Some hyperscale deployments push this further to 400V HVDC, eliminating another stage. The NVL72’s enhanced bus bar spec (1,400 amps, as mentioned above) is now part of NVIDIA’s OCP contribution, available to the industry rather than kept proprietary.

The difference between 85% end-to-end efficiency and 95% efficiency is 100 kW of waste heat per megawatt rack. A hundred kilowatts that you’re paying for, generating heat from, and then paying again to cool. That “paying again to cool” part is real and it compounds the loss. Modern liquid-cooled facilities run a PUE around 1.2 to 1.3, meaning roughly 0.2 to 0.3 kW of cooling energy is consumed for every kW of heat the facility has to reject. Apply that to the waste heat alone (the heat that never needed to exist in the first place) and the cooling overhead adds another 25% or so on top of the direct conversion loss cost.

At 1,000 racks (a medium-sized hyperscale hall), the annual cost difference between efficient and inefficient power delivery, counting both the losses and the cost to cool those losses, is somewhere between $130 and $175 million per year. That’s the business case for a 9-figure investment in HVDC infrastructure. The math isn’t subtle.

Here’s a rough comparison across delivery methods for a 1 MW rack, at $0.065/kWh with a 1.25 cooling overhead factor applied to waste heat:

Power delivery	Efficiency	Loss (kW)	Annual power cost of losses	Annual cooling cost of losses	Total annual waste cost
Single phase 120V AC	80 to 82%	180 to 200	$100K to $115K	$25K to $29K	$125K to $144K
Three phase 208V/480V AC	88 to 91%	90 to 120	$51K to $70K	$13K to $18K	$64K to $88K
48V HVDC (OCP ORV3)	94 to 96%	40 to 60	$23K to $35K	$6K to $9K	$29K to $44K

The gap between single-phase AC and HVDC, fully loaded, is roughly $96K to $115K per rack per year in pure waste: power you bought, converted to heat you didn’t want, and then spent more money to remove. These numbers are why you see hyperscalers spending billions on power infrastructure before they spend anything on compute.

Cooling: air physically cannot do this job

At 80 to 100 kW, air cooling is already working hard. You’re managing it with rear-door HXs, tight containment, and purpose-built facilities, but the physics are still on your side if you’re disciplined. At 1 MW, you’ve left the realm of “air cooling is expensive” and entered “air cooling is physically impossible in any meaningful sense.” I don’t mean difficult. I mean the airflow velocities required to move enough heat would damage components and make the room uninhabitable.

Here’s the physics. Air has a specific heat capacity of about 1 kJ/kg·°C. Water has about 4.18 kJ/kg·°C. But density matters too: water is about 830 times denser than air at standard conditions. So water carries roughly 3,400 times as much heat per unit volume as air. To remove 1 MW of heat with air at the temperature deltas you can realistically achieve in a data center (maybe 15 to 20°C rise across a rack), you’d need airflow rates that generate serious acoustic problems and create structural forces on lightweight components.

Direct liquid cooling (DLC) is the answer. Coolant flows through cold plates physically attached to GPUs, CPUs, and memory modules. The coolant absorbs heat at the source, carries it to a coolant distribution unit (CDU), and gets rejected to the facility cooling loop. With water as the coolant, you can pull 1 MW out of a rack with a flow rate measured in tens of liters per minute. Manageable with standard chilled water infrastructure, as long as that infrastructure was designed for it.

Cold plates themselves are also getting smarter. The standard approach runs parallel channels in a rectilinear grid across the chip surface, distributing coolant uniformly regardless of where the heat is actually being generated. Modern GPUs aren’t thermally uniform. There are intense hotspots at compute cores and memory interfaces sitting next to relatively cool regions, and a thermally blind cold plate treats all of it the same way. A Swiss EPFL spinout called Corintis is attacking this directly with microfluidic chip-scale cooling: channels roughly 100 microns in diameter (about the width of a human hair) etched into or just above the silicon die, with AI-optimized topologies that route more flow to hotspots and less to cool regions. Microsoft tested an early version on production server hardware and reported 3x better heat removal than advanced cold plates, a 65% reduction in GPU peak temperature, and a 55% drop in pressure compared to a parallel-channel baseline. The next generation goes further, embedding the channels directly in the chip die and co-designing the thermal structure alongside the electronics. I’ve got a full writeup on Corintis in the research section if you want the technical detail. The point here is that cold plates aren’t standing still. The ceiling on what DLC can do at the chip level keeps rising.

Rear-door heat exchangers (a useful stopgap for medium-density racks) get you into the 30 to 50 kW range without touching facility cooling loops. At 1 MW, they’re a footnote.

The facility requirements for DLC at megawatt density are not trivial. You need dedicated CDUs for each rack or row, supply and return manifold piping, leak detection at every connection point (a leak in a 1 MW liquid-cooled rack is a very bad day), and secondary containment for the coolant loop. NVIDIA’s reference architecture for a 7 MW GB200 NVL72 cluster (developed with Vertiv) shows what this looks like at scale: a purpose-built liquid-cooled floor plan where every element, from CDU placement to power distribution topology, is designed around the rack rather than adapted from a conventional air-cooled facility. That reference architecture reportedly cuts deployment time by up to 50% compared to custom-designed approaches, which says something about how standardized this problem is becoming even at megawatt densities.

Reference floor plan for a 7 MW GB200 NVL72 cluster, developed by NVIDIA and Vertiv. Every element is designed around the rack’s liquid cooling and HVDC requirements. Image: NVIDIA

This is a ground-up design requirement, and increasingly one with published reference architectures and OCP-standardized components rather than a bespoke engineering problem every time.

What it actually costs to run a 1 MW rack for a year

Let’s put numbers on the full picture. One megawatt of IT load, operating 24/7/365:

Raw power cost: 1 MW × 8,760 hours × $0.065/kWh = **$569,400 per year**
Apply a PUE (Power Usage Effectiveness) of 1.2, which is realistic for a modern liquid-cooled facility: total facility load is 1.2 MW
Total facility power cost: $683,000 per year per rack

That’s before amortizing the cost of the rack itself (the NVL72 is reportedly in the $3 to $4M range per system, before networking), the facility infrastructure, or the power delivery and cooling buildout.

Scale to 1,000 racks and you’re looking at roughly $683M per year in power costs alone. A mid-sized hyperscale AI hall. The infrastructure to support those racks (power substations, cooling towers, HVDC distribution, DLC manifolds) runs another $1 to $2 billion in capital. The compute itself is additional.

This is why the conversation in data center infrastructure has shifted so completely in the last two years. The decisions about PUE targets, power delivery topology, and cooling architecture are not engineering preferences. They’re P&L items. The difference between a 1.4 PUE facility and a 1.2 PUE facility, at this scale, is $136M per year in wasted power costs for that same 1,000-rack hall. Every tenth of a PUE point is worth fighting for.

The supply chain you didn’t expect: this stuff competes with electric cars

Here’s something that doesn’t come up enough in datacenter conversations: a meaningful chunk of the bill of materials for a 1 MW rack comes from the same supply chain as a high-end electric vehicle. Not metaphorically similar. Literally the same components, made by the same manufacturers, allocated from the same production capacity.

Start with the power semiconductors. The move to 800VDC datacenter distribution requires silicon carbide (SiC) MOSFETs rated at 1,200V for the front-end AC-DC conversion stages, and gallium nitride (GaN) transistors for the downstream DC-DC conversion. Those 1,200V SiC devices are the exact same class of component used in 800V EV traction inverters, the inverter that drives the motors in a Porsche Taycan, a Hyundai Ioniq 6, or a Lucid Air. Infineon, onsemi, STMicroelectronics, and Wolfspeed are the dominant suppliers to both markets, and they’re drawing from the same wafer fabrication capacity. NVIDIA’s 800V HVDC supplier alliance (announced May 2025 with Navitas and others) is specifically targeting this component class for the 1 MW rack generation. The SiC content per rack is projected to increase roughly 11x from GB200 to the Rubin Ultra generation. That’s not a rounding error in the supply chain.

The copper situation is similar. At 54V distribution, a single 1 MW rack requires around 200 kg of copper bus bar. That’s why the push to 800VDC matters beyond efficiency: running the same power at 15x higher voltage means roughly 45% less copper for equivalent current-carrying capacity. Even with that reduction, the aggregate copper demand from hyperscale AI buildout is enormous. Analysts project a 6 million-tonne global shortfall by 2035, driven jointly by AI infrastructure and clean energy electrification (including EVs and grid storage). These aren’t separate demand pools. They’re competing for the same mining output, the same refining capacity, and the same bus bar fabricators.

The coolant pumps are another one. The CDUs moving fluid through DLC loops at 1 MW densities use high-pressure centrifugal pumps with specifications (flow rate, pressure head, thermal tolerance) that overlap closely with the thermal management systems in EV battery packs. The same industrial suppliers (Grundfos, Ebara, and several automotive-derived vendors) serve both markets. This isn’t a theoretical concern; procurement teams at large datacenter operators have already run into allocation conflicts when trying to source at scale.

What makes this interesting is the timing mismatch. EV demand hit a rough patch in Western markets through 2024 and into 2025, which led SiC manufacturers to overcapitalize on production capacity that then looked underutilized when EV ramp rates slowed. Wolfspeed (historically one of the most important SiC suppliers) filed for bankruptcy restructuring in early 2026 after betting heavily on continued EV growth that didn’t materialize fast enough. Meanwhile datacenter demand for the same devices was accelerating sharply. The SiC market ended up with a strange combination of manufacturer financial stress and genuine tightening on specific high-specification parts. The long-term 800V EV trend is still intact (the physics of 800V drivetrains are compelling and won’t reverse), which means the demand competition is real and ongoing, just with a timing phase shift between the two application domains.

The practical implication for anyone building megawatt-class infrastructure: the supply chain for these racks isn’t just datacenter infrastructure suppliers. It’s also automotive tier-1 suppliers, SiC wafer fabs, and copper miners. Lead times on 1,200V SiC modules, high-ampacity bus bar stock, and specialty coolant pump assemblies are all being driven by a demand pool that extends well beyond the datacenter industry’s historical footprint.

Where this goes

I keep coming back to the physical constraint: same rack footprint, 100x the power density, and the world’s data center capacity was designed for the baseline, not the frontier. The greenfield buildout happening right now (the gigawatt-scale campus announcements, the utility partnerships, the dedicated substation builds) isn’t hype. It’s the physical infrastructure catching up to a compute density that existing facilities simply can’t support.

What comes after 1 MW per rack is a question I don’t have a clean answer to yet. There are 2 MW designs in discussion. Immersion cooling (fully submerging hardware in dielectric fluid) becomes more compelling as density increases further, though it introduces its own operational complexity. And at some point the silicon itself has thermal limits that packaging and cooling can’t engineer around.

Back to Writing

Jared Watkins — Sat, 18 Apr 2026 00:00:00 +0000

I know I’ve not posted much on this site the last few years… ok the last many years. For a long time the Hugo build process was broken and I didn’t spend enough time to get it fixed AND keep up with all the changes to Go which also broke some of my customizations. With AI tools being so good now though, I’ve been able to fix everything and simplify some writing workflows to the point that I actually have time to focus on content.

Originally I stopped posting when I joined Amazon and could no longer talk about the things I was working on. A crazy busy schedule and on-call didn’t help either. But these days life isn’t as crazy and it occurred to me that I’ve done a lot these last few years but most people have no idea. So I’m going to slowly start burning down some of that backlog. Some will be personal entries like about coffee roasting. Some will be more technical and driven by the new Research section I’ve added which is a lot of random cutting edge stuff that both interests me and is relevant for the work I do now.

Home Coffee Roasting - Hobby That Tastes Great

Jared Watkins — Sat, 18 Apr 2026 00:00:00 +0000

I never drank coffee before about three years ago, but I’ve been roasting and brewing it at home for much longer. The story starts about ten years back when I met my wife in Seattle. She’s a big coffee drinker, and one of the things I started doing was making her coffee in the mornings. I tried a bunch of different brewing methods and eventually landed on the Aeropress, which just gave me the most consistent results making one or two cups in a morning.

Actually, we didn’t get serious about coffee until we moved to the Dallas suburbs. There was this fantastic specialty roaster there, Addison Roasters, that carried single-origin coffees from all over the world. At any given time you could walk in and find a dozen or more different single origins available by the pound. I got fascinated by the regional differences, what makes something specialty grade, the whole business side of it. I wasn’t roasting coffee myself yet (or even drinking it, really)—just buying the good stuff and learning. I also picked up a few books on the subject.

Pandemic Opportunity

Fast forward a couple years, we’re in Northern Virginia (not long after Amazon picked the area for HQ2). Then the pandemic hit, and like the rest of the DC suburbs, lockdowns, restrictions, all of it. Bored and stuck at home, I decided to start roasting coffee. I bought a used Artisan 2.5 from a local shop that was upgrading their gear, set it up in a spare room with 220V power and a window for venting.

Over the next couple years I gradually hacked on it to increase performance. Airflow improvements, better chaff filtering, eventually it became an Artisan 3 (the 3-pound model) inside. It was cobbled together with hoses everywhere, not pretty, but it worked and I learned a lot. One of the problems I solved was with the exhaust blower. It was a cheap blower that vibrated like crazy. When I bought it came hard mounted to that little wire cart but it made a ton of noise and vibrated the bean cooler pretty badly. So.. simple fix.. I separated the two bits of plywood and added some little eyelets and some parachute cord zig-zagging between them to make a vibration isolated suspension mount. So the blower motor actually hangs under the lower shelf with the two boards only connected by the parachute cord. I didn’t expect it to work so well so didn’t bother to make it look good. It took me all of about 15 minutes but drastically improved the noise while roasting so I just left it.

Once I had the setup dialed in I started selling to the neighbors. People were stuck at home and didn’t want to go out, so I built a little pickup box outside and ran orders through Stripe and advertised through Nextdoor. The Stripe setup turned out to be overkill as everyone preferred to pay with cash in the dropbox. I wasn’t trying to build a business, just pay for my hobby. Three or four single origins in rotation, and it worked great for a while with enough repeat customers to pay for the coffee and make it worth my time. That lasted for a couple years until we moved again.

New Gear

I’ve been following Coffee Crafters for years as they make the roaster I’d been using. They’re an American company doing their own design and manufacturing, which I really respect. When they released the Valenta series in early 2025, I bought one. I crated up the the Artisan and went all in on the Valenta 3. It’s cleaner, way more compact, fits on a little rolling table in my office. I still roast periodically with a few single origins, though I’m not selling to neighbors anymore. (Could if I wanted to. This thing makes it pretty easy.) I had to make some improvements through. I added a dedicated Raspberry PI with screen to run the optional roasting software (also confusingly called Artisan) and then made a few other modifications.

The Airflow Problem and My Fix

Here’s the thing I’ve dealt with on both machines: airflow control during the roast is finicky. As beans lose density and expand, they float differently in the air, and the rate of heat transfer changes. The idea is simple: start at max airflow, dial it back as the beans dry and expand to maintain a sufficient loft without blowing beans out the top. It also helps you manage the transition from the drying phase to the development phase and you get faster roasts and better flavors.

Roasting temperature matters most (I aim for 415F for most beans), but the roast length and airflow management matter too. You want to hit temperature fast without over-extending the roast (baking the beans).

The factory loft air control is a little crude. A solid-state relay uses the voltage through a potentiometer to modulate power to the blower motor, but the dial doesn’t give you fine control. So I added a second pot in series, rated at about 10% of the factory range. Now I just set the factory knob at the max for this bean mass (usually around 4) and use my secondary adjustment which gives me a full rotation to play with rather than just 1/10 of the arc on the factory knob. Much finer control. That’s important because if you slip and cut loft air too much you risk stalling the bean fountain and scorching the beans or even starting a fire.

The Clock Is Already Running on Quantum Crypto Risk

Jared Watkins — Fri, 17 Apr 2026 00:00:00 +0000

On March 31, Google Quantum AI published a paper that got quiet but serious attention from the security world. The upshot: under optimistic error rate assumptions, you might need fewer than 500,000 physical qubits to break the elliptic curve cryptography that underpins almost everything on the internet. Previous estimates were in the millions. Their suggested migration target is 2029.

That’s not “quantum computers are coming someday.” That’s three years.

To understand why this matters, you need to know what “break” means here. RSA, ECDSA, and Diffie-Hellman, the signature and key exchange algorithms that secure HTTPS, VPNs, SSH, email, banking, and a substantial fraction of the world’s financial infrastructure, rely on mathematical problems (integer factorization and discrete logarithm) that are hard for classical computers. In 1994, Peter Shor showed that a quantum computer could solve these problems efficiently. We’ve been slowly accepting that this would eventually be a real problem ever since. “Eventually” keeps getting closer.

The threat isn’t only about what a quantum computer breaks in real-time. There’s a second threat model called “harvest now, decrypt later” (HNDL), and it’s already happening. Nation-state adversaries are almost certainly collecting encrypted traffic today (VPN sessions, TLS connections, classified communications) with the explicit intent to decrypt it retroactively once a CRQC (a cryptographically relevant quantum computer) becomes available. If your data needs to stay secret for more than 5 to 10 years, the security clock is already running, not starting when a CRQC is built.

Why now?

The NIST post-quantum cryptography standardization process has been grinding along since 2016, and in August 2024 it finally produced real standards: FIPS 203, 204, and 205. These are the CRYSTALS-Kyber and Dilithium-derived algorithms that are going to replace RSA and ECDH everywhere. The standards are done. The problem is that “done” at NIST and “deployed everywhere” are separated by a gap that historically takes a decade or more to close.

And Google’s paper just compressed the urgency. Prior estimates suggesting we had until the mid-2030s to get our act together were already making people uncomfortable; a plausible path to CRQC by 2029 is a different conversation entirely.

Network infrastructure

For enterprise networking, the migration path exists but it’s not plug-and-play. IKEv2/IPsec (the protocol behind most enterprise VPNs) has IETF extensions for PQC KEMs (RFC 9242, RFC 9370). TLS 1.3 has hybrid key exchange drafts; Chrome and Firefox have been running X25519Kyber768 in some configurations since 2023. The standards are real and the protocol work is largely done.

The vendor picture is messier. Fortinet, Palo Alto, Check Point, and Cisco all ship or are shipping ML-KEM IKEv2 support in recent software versions. But “ships support” and “deployed in production at scale across heterogeneous environments” are not the same thing. Most of these implementations are software-only right now, with meaningful CPU overhead. A lot of networking hardware doesn’t have the crypto acceleration to run post-quantum algorithms at line rate. Interoperability between vendor implementations is still an open problem. And then there’s the long tail of embedded devices, industrial systems, and legacy infrastructure that nobody’s going to upgrade on a 3-year timeline regardless of how urgent the threat becomes.

The cost to upgrade just the identifiable enterprise network perimeter globally (firewalls, VPN concentrators, core routing infrastructure) is probably in the low hundreds of billions of dollars when you account for hardware refresh cycles, migration labor, and the operational disruption of replacing cryptographic algorithms end-to-end. That’s before touching government classified systems (which have hard mandates under NSA’s CNSA 2.0 by 2030) or financial sector infrastructure. The Fed and CISA have both been clear that the financial sector needs to treat this as a critical infrastructure priority, but the actual upgrade spend is still largely ahead of us.

Cryptocurrency is a much harder problem

Enterprise networking is hard because of scale and operational inertia. Cryptocurrency is hard for a different and more fundamental reason: there’s no one in charge.

Bitcoin uses ECDSA secp256k1 and Schnorr signatures for transaction authorization. Both are fully quantum-vulnerable via Shor’s algorithm. Approximately 28 to 35% of the total Bitcoin supply sits in addresses where the public key is already visible on-chain, meaning a CRQC could derive the private key directly without the owner doing anything. That’s somewhere in the neighborhood of 6 to 7 million BTC, including most of Satoshi’s known holdings (around 1.1M BTC in early P2PK outputs). At current prices that’s over a trillion dollars in directly attackable assets across the crypto ecosystem.

Migrating Bitcoin requires the whole ecosystem to agree on a new signature scheme, ship it as a consensus change (probably a soft fork for the address infrastructure, definitely a hard fork for phasing out old signature types), and then get every wallet, exchange, node operator, and user to actually do the migration. The historical precedent here is not encouraging. SegWit took about two years from proposal to activation, and a significant portion of the network still doesn’t use it. Taproot activated in 2021 and most wallets still default to legacy address types. Core developers are estimating 5 to 10 years for full Bitcoin PQC migration from the point a change is activated. That’s a best case.

There’s an active governance debate happening right now. BIP-360 proposes quantum-resistant output infrastructure, and BIP-361 proposes a controversial phased sunset of legacy signature types that would effectively freeze coins in old address formats that don’t migrate. Jameson Lopp (Casa CTO) is driving the more aggressive approach. Adam Back is publicly against mandatory freezing. The fact that this is still being debated at the conceptual level, with no miner signaling, no algorithm formally selected, and a testnet that’s only a few weeks old, is not reassuring given the 2029 migration target that Google just put on the table.

Ethereum is in a similar position but with a more organized response. They have a dedicated post-quantum team, an active proposal (EIP-8141), and devnets running. Their self-imposed 2029 migration deadline implies roughly seven hard forks in the remaining time. It’s ambitious but at least there’s an organized program.

Some chains are further along. Algorand has Falcon-based signatures live on mainnet for state proofs as of November 2025. QRL was built from the ground up as a quantum-resistant blockchain in 2018. But the assets most at risk are on Bitcoin and Ethereum, which have the most value, the most decentralized governance, and therefore the hardest migration paths.

The broader picture

Beyond networks and crypto, there’s everything else. HTTPS certificates that secure web browsing. S/MIME email encryption. Code signing certificates that verify software updates. Document signing. The PKI underpinning most of the internet’s trust model. Healthcare record confidentiality, where HNDL is particularly concerning given how long medical data retains sensitivity. Financial records. Legal contracts. Smart grid infrastructure. Every layer of the stack that relies on public-key cryptography has to migrate, in some cases multiple times (signing and key exchange often use different algorithms).

Early estimates for migrating federal government systems put the number well north of $7 billion and the better part of this decade to complete. The private sector is an order of magnitude larger. The total cost of global crypto infrastructure migration, done properly, is probably in the trillions when you account for hardware, software, labor, testing, and the inevitable disruption to systems that can’t be taken offline cleanly.

The realistic outcome isn’t that we migrate everything in time. It’s that we prioritize the highest-sensitivity, highest-value assets and connections, get those migrated first, and accept that long-tail infrastructure will remain vulnerable for years longer than is comfortable. That means some HNDL-harvested traffic will eventually be decrypted. Some legacy embedded systems will get cracked once a CRQC is operational. The question is whether the critical stuff (financial clearing, classified communications, critical infrastructure control) is protected before the window closes.

The standards exist. The vendor implementations are starting to arrive. The governance debates in decentralized systems are happening, if contentiously. Whether 2029 proves out as the actual CRQC timeline or the real date turns out to be 2032 or 2035, the direction is clear and the work is late. The difference between treating this as a three-year problem versus a fifteen-year problem is whether we’re actually going to be ready.

I’ve been tracking this closely in the research section. There’s more depth there on specific vendor implementations, the cryptocurrency exposure numbers, and the NIST standards themselves, if you want to go further.

Picking hardware for local AI inference in 2026

Jared Watkins — Tue, 07 Apr 2026 00:00:00 +0000

Nobody buying AI hardware in 2026 is short on opinions. Everyone has a take. The forums are full of people who swear by their setup and can’t understand why anyone would choose differently. Most of those arguments are happening across completely different use cases which raises the noise floor for this subject.

“What’s the best hardware for running AI locally?” is roughly as useful as asking what’s the best vehicle without mentioning whether you’re hauling gravel or commuting to an office. The answer depends entirely on what you’re trying to do, and getting that wrong wastes real money.

Here’s my attempt to cut through it.

The three things that actually matter

Local AI hardware comes down to three variables: capacity, bandwidth, and software stack.

Capacity is whether the model fits in memory at all. If it doesn’t fit, nothing else matters. You’re either offloading to disk (more on that disaster in a bit) or you need a bigger box.

Bandwidth is how fast the hardware can feed data to the compute units. This is the single best first-pass predictor of how fast tokens actually come out. Memory bandwidth is not the same as tokens per second, but it’s the cleanest way to sort real performance tiers before you waste a weekend arguing with someone posting single-prompt screenshots.

Software stack is how much of the spec sheet you can actually cash out. A card with strong bandwidth numbers on paper does nothing useful if the inference framework doesn’t support it. This is still where CUDA’s dominance matters, and it’s where Tenstorrent’s fully open source stack is a genuine long-term bet worth watching.

The hardware landscape

Five distinct markets, same buzzword. Here’s what each one is actually good for.

Raw speed when the model fits: discrete GPUs

If the model fits in VRAM, discrete GPUs are still the fastest thing by a wide margin. Nothing else comes close on a per-token basis.

NVIDIA’s RTX PRO 6000 Blackwell (96GB, 1792 GB/s, around $8,000 to $9,200 retail right now) and the RTX 5090 (32GB, 1792 GB/s, street price has been running $3,000 to $5,000 and climbing due to supply issues) share identical bandwidth. The difference is capacity. The PRO 6000 can hold a 70B model at Q4 comfortably; the 5090 tops out around 30B quantized. The RTX 4090 (24GB, 1008 GB/s) is still worth knowing about if you find one at a good price on the secondary market.

AMD’s discrete cards deserve more credit than they typically get. The RX 7900 XTX (24GB, 960 GB/s) is genuinely competitive on bandwidth per dollar. The Radeon PRO W7900 (48GB, 864 GB/s) doubles the memory at workstation pricing. The newer AI PRO R9700 (32GB, 640 GB/s) sits in between. ROCm support has improved enough that AMD is a real option now, especially with llama.cpp and Ollama.

Intel showed up too. The Arc Pro B65 (32GB, ~608 GB/s) and B60 (24GB, ~456 GB/s) are interesting if you’re following where Intel’s headed with this. Not my first choice today, but they’re not irrelevant.

Discrete GPUs win because they can drink from a firehose. They lose the moment the model doesn’t fit.

Biggest one-box memory: Apple Silicon

Apple’s pitch is “not the fastest, but usable” combined with “more unified memory in a quiet box than anything else you can buy.”

The Mac Studio M3 Ultra is still the headliner here. Up to 512GB of unified memory at 819 GB/s. That’s enough to run a Llama 4 Maverick (400B MoE) at quantization, or DeepSeek-V3 (671B MoE) with aggressive quantization. Nothing else in a single consumer box gets anywhere near that capacity. The 512GB config is reportedly hard to find right now. Apple briefly pulled that upgrade option but the 96GB base config starts around $3,999 and the 192GB/256GB configs sit in the $6,000 to $10,000 range depending on CPU tier.

Below that you’ve got the Mac Studio M4 Max (up to 128GB, 546 GB/s, from around $2,000), MacBook Pro M5 Max (up to 128GB, 460 to 614 GB/s, from around $3,900), MacBook Pro M5 Pro (up to 64GB, 307 GB/s, from around $2,200), and Mac mini M4 Pro (up to 64GB, 273 GB/s, from around $1,400).

Apple wins when you want one box, you want silence, and you want to run models that simply won’t fit on a normal GPU. It loses when raw tokens per second and concurrency start to matter more than everything else.

Coherent NVIDIA appliance: DGX Spark

The DGX Spark (128GB unified, 273 GB/s) launched at $3,999 and has since been bumped to $4,699 due to memory supply constraints. It’s not a bandwidth monster. What it is, is a compact NVIDIA CUDA appliance with 128GB of coherent memory and NVFP4 support that hasn’t fully matured yet but is genuinely interesting for the future of quantization.

This is a developer appliance first. It’s for people who need the full NVIDIA stack, want 128GB in a small box, and aren’t optimizing for raw decode speed. The GB10-class machines like the ASUS Ascent GX10 live in the same category.

First real x86 unified-memory contender: Strix Halo

AMD’s Ryzen AI Max / Strix Halo is the most interesting new category in local AI hardware, in my opinion. Up to 128GB of LPDDR5X at ~256 GB/s, with up to ~96GB assignable as GPU memory on Windows. The Framework Desktop implements this starting at $1,099 for 32GB, $1,599 for 64GB, and $1,999 for the 128GB config.

This is not just another mini PC. It’s the first mainstream x86 box where local AI starts feeling like a serious hardware class rather than a laptop pretending very hard. The value proposition at 128GB for $1,999 is hard to beat, especially if you’re running MoE models where capacity matters more than raw bandwidth.

The fully open source bet: Tenstorrent

Tenstorrent’s Wormhole n300 (24GB, 576 GB/s, around $1,400) and Blackhole p150 (32GB, 512 GB/s, around $1,400 with 800G interconnect) run a fully open source stack from top to bottom. I’m genuinely rooting for this one to mature. The AI world needs more fully open stacks, and the bandwidth is competitive with mid-tier discrete GPUs. The Blackhole’s interconnect makes multi-card scaling worth watching as the software ecosystem develops.

The AI PC trap

Most machines wearing an “AI PC” sticker are still bandwidth-starved in any practical sense. Snapdragon X Elite (~135 GB/s), Intel Lunar Lake (~136 GB/s), MacBook Air M5 (~153 GB/s), Snapdragon X2 Elite (~152 to 228 GB/s depending on SKU). These are fine machines. They’re fine for small models, personal assistants, edge workloads. They are not serious local inference hardware for anything larger than a 7 to 8B dense model. Physics still applies, which is inconvenient but consistent.

The gimmicks section (or: technically possible doesn’t mean useful)

A pattern keeps coming up in local AI discussions that I want to name directly, because it costs people a lot of time and money chasing a thing that doesn’t actually work well in practice.

The pitch goes like this: “My hardware doesn’t have enough memory, but I can still run a big model by only loading part of it at a time.” This is usually presented as a clever hack. Sometimes it is. More often it’s a performance cliff dressed up as a feature.

Layer offloading. Tools like llama.cpp let you split model layers between GPU VRAM, system RAM, and even disk. The flag is -ngl (number of GPU layers). When you don’t have enough VRAM for the whole model, you offload some layers to CPU RAM. The problem is that every token generation step has to shuffle data across the PCIe bus between GPU and CPU. Real-world numbers here are brutal – people running 70B models with partial CPU offloading report around 1–3 tokens per second. That’s technically running the model. It’s also roughly the speed of reading text out loud to yourself. Not useful for interactive work.

Disk offloading. Some tools and frameworks support streaming model weights from NVMe directly. Modern NVMe drives can hit 7 GB/s reads in ideal conditions, which sounds fast until you realize your GPU memory bandwidth is 10 to 100x that. The energy penalty alone is significant – recent research puts SSD-offloaded decode at roughly 3 to 4x the energy cost versus in-memory inference on comparable hardware. Token generation with disk offload in practice tends to land below 1 token per second. I’ve seen people run 405B models this way. I’ve also seen them wait two minutes for a 60-token response.

Extreme quantization. Quantization is genuinely useful and I’m not criticizing it wholesale. Q4 is excellent, Q5 and Q8 are great when you can afford the memory for them. The cliff is at the bottom. Q2 quantization degrades quality enough that for many use cases you’d be better off running a smaller, better-quantized model. A Q2 70B model often loses to a Q4 7B on reasoning tasks while using four times the memory. The tradeoff is real.

The 30 tokens-per-second floor. For interactive use actual back-and-forth conversation or coding assistance where you’re watching the output stream 30 tok/s is roughly where it starts feeling like a tool rather than a waiting exercise. Below 15 tok/s it becomes noticeable. Below 5 tok/s it’s painful regardless of model quality. For batch processing or background tasks, slower is tolerable. But if you’re evaluating a hardware setup for daily driving, “it runs” and “it’s usable” are different things.

The test I’d apply: if your setup produces tokens slower than you read them, you’re probably past the gimmick threshold for interactive use.

What models are you actually trying to run?

Hardware decisions only make sense relative to the models you’re targeting. The good news is that open source models in 2026 have gotten genuinely good close enough to frontier API models on many tasks that the conversation has shifted from “is open source good enough?” to “which open source model is right for this?”

The big architectural shift is MoE (Mixture of Experts). These models have enormous total parameter counts but only activate a fraction of them per token. That changes the capacity-vs-speed tradeoff dramatically. A model that “needs” 192GB to load might only activate 17B parameters per forward pass.

24 to 32GB (RTX 5090, RX 7900 XTX, Arc Pro B65, MacBook Air M5 max): This is Llama 4 Scout territory at Q4 (109B total, 17B active – fits in roughly 55 to 60GB quantized, so you need a second GPU or larger box), or more realistically: Qwen3 30B-A3B (only 3B active per token), Gemma 4 26B MoE (~14GB at Q4, 85+ tok/s on consumer hardware, genuinely excellent for the size), Phi-4 14B for reasoning, and Qwen2.5-Coder 14B for coding work. Useful territory, not the frontier.

48 to 64GB (Mac Studio M4 Max, MacBook Pro M5 Pro, Framework Desktop 64GB): Dense 30 to 40B models at Q4 land comfortably here. Llama 4 Scout (109B MoE, 17B active) fits at reasonable quantization. Qwen3 235B-A22B MoE needs more room, but the smaller Qwen3 variants are excellent here. This is where local AI starts feeling like a real tool rather than an experiment.

96 to 128GB (RTX PRO 6000, Mac Studio M3 Ultra base, DGX Spark, Framework Desktop 128GB): Llama 4 Scout at Q8, DeepSeek-R1 70B for serious reasoning, Qwen3 235B-A22B MoE with 22B active parameters. The DGX Spark’s CUDA stack gives it an edge for frameworks that are optimized for NVIDIA. GPT-OSS-120B (OpenAI’s first open-weights release in years) also fits here. Single 80GB GPU in FP8, or 128GB unified at Q4.

192 to 512GB (Mac Studio M3 Ultra maxed, multi-GPU rigs): Llama 4 Maverick (400B MoE, 17B active), DeepSeek-V3 (671B MoE) at aggressive quantization, Qwen3 235B at higher precision. If you need frontier-class open source models running locally with zero cloud dependency, this is currently the only consumer-ish path to get there.

The quadrant chart

Here’s how these platforms map when you combine memory capacity, bandwidth, and cost into a single picture. The vertical axis is a synthesized “memory performance” score explained in detail in the collapsed section below. The horizontal axis is platform cost. Only min and max configs are shown for platforms with multiple options.

quadrantChart title Local AI Hardware - Memory Performance vs Cost 2026 x-axis Low Cost --> High Cost y-axis Low Performance --> High Performance quadrant-1 High Perf High Cost quadrant-2 High Perf Low Cost quadrant-3 Low Perf Low Cost quadrant-4 Low Perf High Cost RTX PRO 6000: [0.98, 0.75] Mac Ultra 192GB: [0.94, 0.7] RTX 5090: [0.45, 0.52] MBP M5 Max 128GB: [0.61, 0.43] Mac Ultra 96GB: [0.42, 0.42] DGX Spark: [0.51, 0.35] Radeon PRO W7900: [0.37, 0.29] Framework 128GB: [0.16, 0.34] RTX 4090: [0.23, 0.26] RX 7900 XTX: [0.04, 0.38] Framework 64GB: [0.13, 0.32] TT Wormhole: [0.09, 0.25] TT Blackhole: [0.15, 0.19] Arc Pro B65: [0.03, 0.13] Mac mini M4 Pro: [0.1, 0.08] MacBook Air M5: [0.06, 0.05]

How the memory performance score is calculated + data table

The score on the vertical axis synthesizes two things: memory capacity (GB) and memory bandwidth (GB/s). Neither alone tells the full story, A box with massive bandwidth but tiny capacity runs out of useful models quickly, and a box with massive capacity but slow bandwidth produces tokens at a crawl.

Scoring method:

I normalized both dimensions independently across the full set of platforms (0 = worst in set, 1 = best in set), then combined them with equal weighting (50/50). The formula is:

capacity_score = (GB - min_GB) / (max_GB - min_GB)
bandwidth_score = (GB_s - min_GB_s) / (max_GB_s - min_GB_s)
memory_performance = (capacity_score + bandwidth_score) / 2

The dataset spans 24GB (low end) to 192GB (practical max shown) for capacity, and 153 GB/s (MacBook Air M5) to 1792 GB/s (RTX 5090 / PRO 6000) for bandwidth.

Cost axis: Normalized from ~$750 (Arc Pro B65) to ~$8,500 (RTX PRO 6000). I used street price midpoints where ranges exist.

Platform	Memory (GB)	Bandwidth (GB/s)	Cap Score	BW Score	Mem Score	Cost ($)	Cost Score
RTX PRO 6000	96	1792	0.43	1.00	0.71	8,500	1.00
Mac Studio M3 Ultra 192GB	192	819	1.00	0.41	0.70	8,000	0.94
RTX 5090	32	1792	0.05	1.00	0.52	4,200	0.45
MacBook Pro M5 Max 128GB	128	546	0.62	0.24	0.43	5,500	0.61
Mac Studio M3 Ultra 96GB	96	819	0.43	0.41	0.42	3,999	0.42
DGX Spark	128	273	0.62	0.07	0.35	4,699	0.51
Framework 128GB	128	256	0.62	0.06	0.34	1,999	0.16
Radeon PRO W7900	48	864	0.14	0.43	0.29	3,600	0.37
RTX 4090	24	1008	0.00	0.52	0.26	2,500	0.23
RX 7900 XTX	24	960	0.00	0.49	0.25	950	0.03
Mac mini M4 Pro 64GB	64	273	0.24	0.07	0.16	1,400	0.08
Arc Pro B65	32	608	0.05	0.28	0.16	750	0.00
Framework 64GB	64	256	0.24	0.06	0.15	1,599	0.11
TT Blackhole p150	32	512	0.05	0.22	0.13	1,400	0.08
TT Wormhole n300	24	576	0.00	0.26	0.13	1,400	0.08
MacBook Air M5 32GB	32	153	0.05	0.00	0.02	1,100	0.05

Note: Cap Score of 0.00 for 24GB cards means that’s the minimum in the dataset – not that they have no capacity. Everything is relative to the range of platforms compared here.

A few things jump out. The RX 7900 XTX is the best value pure-bandwidth play if 24GB is enough for your models. The Framework 128GB is quietly the best price-to-capacity ratio in the whole field. The bandwidth score drags it down but nothing else gives you 128GB assignable to a GPU for $1,999. The DGX Spark is the most interesting chart anomaly: high capacity, middling bandwidth, high cost, and a software stack that might eventually justify all of it. The Mac Studio M3 Ultra at 192GB represents the upper right of what’s achievable in a single consumer box, and the price reflects it.

Which bottleneck are you buying?

Stop asking which hardware is best. Start asking which bottleneck you’re willing to pay to solve.

If you’re doing multi-agent workflows where you need fast concurrent inference, with multiple agents running in parallel each waiting on responses, bandwidth wins and you want discrete NVIDIA. If you’re running a single large reasoning model for deep analysis or long-context work, capacity wins and you want unified memory. If you’re experimenting and want the best flexibility per dollar, the Framework Desktop at 128GB or a Mac mini M4 Pro are hard to beat as starting points.

The local AI hardware market in 2026 is finally interesting enough that there’s no single right answer. That’s actually a good thing. It means the space has matured past the point where CUDA was the only viable path and a $10,000 GPU was the only serious option.

Sources and where to buy

Primary references

Where to buy

NVIDIA RTX 5090 – Best Buy, Newegg, B&H Photo (stock is spotty, prices above MSRP)
NVIDIA RTX PRO 6000 Blackwell – Newegg, Amazon RTX PRO 6000, Micro Center, B&H Photo RTX PRO 6000
RTX 4090 – secondary market, eBay RTX 4090, Newegg used
Apple Mac Studio – Mac Studio 128GB config, Mac Studio 192GB config
Apple MacBook Pro – MacBook Pro 16" M5 Max 128GB
Apple Mac mini – Mac mini M4 Pro config selector
Apple MacBook Air – MacBook Air M5 configs
NVIDIA DGX Spark – NVIDIA Marketplace direct (enterprise ordering)
ASUS Ascent GX10 – ASUS store (system integrator quotes)
Framework Desktop – Framework order page (direct from manufacturer, 128GB config available)
AMD RX 7900 XTX – Newegg RX 7900 XTX, Amazon RX 7900 XTX, B&H Photo RX 7900 XTX
AMD Radeon PRO W7900 – AMD.com Radeon PRO, CDW PRO W7900, B&H Photo PRO W7900
AMD Radeon AI PRO R9700 – AMD.com AI PRO, CDW R9700
Intel Arc Pro B65 – Intel Arc Pro B-series, CDW Arc Pro B65
Tenstorrent Wormhole / Blackhole – Tenstorrent store (direct ordering)

I built a research section with AI and it might be useful to you too

Jared Watkins — Sat, 04 Apr 2026 00:00:00 +0000

I’ve been spending a lot of time lately exploring what AI is actually good for beyond writing code… which is where most people seem to stop. One thing I’ve landed on that I think is genuinely useful is using it to maintain a structured research knowledge base on technical topics I care about. So I built one, and it now lives on this site.

It covers cutting-edge datacenters (cooling systems, power infrastructure, robotic server management, construction), energy (solar, SMR nuclear, batteries, grid resources), and robotics (actuators, sensors, edge compute, aerial and ground drones). These aren’t random topics. They’re pretty tightly interconnected when you start pulling on the threads, and that overlap is a big part of why I find them interesting.

I work at Amazon, which means I’m adjacent to infrastructure at a scale that makes you think differently about where things are heading. The question of how you build, cool, power, and automate a datacenter isn’t abstract to me. It’s the physical substrate underneath everything I work on. So I want to understand it at the component level. Who’s actually making the immersion cooling systems? Which SMR designs are closest to permitted? What startups are supplying the actuation systems that will eventually run the robotic logistics inside these facilities? That kind of thing.

What I found when I started digging is that most publicly available research in this space is either too high-level (analyst report fluff) or too narrow (a single company’s press releases). There isn’t a lot of good connective tissue between, say, perovskite solar cell efficiency breakthroughs and the energy sourcing decisions being made for the next generation of AI training clusters. But those things are connected, and if you’re trying to understand where the puck is going – for investing, for career positioning, or just because you find it fascinating – the connections are where the useful insight lives.

The research section is my attempt to build that connective tissue. Each entry is structured like a well-maintained Wikipedia stub: what it is, why it matters, recent developments, key people and organizations, sources. The focus is on smaller companies, university spinouts, and component suppliers, not the IBMs and Googles of the world, which already have plenty of coverage. The interesting stuff is one or two layers down from the obvious names.

It’s also AI-maintained on a schedule. I set up automated updates that keep the entries reasonably fresh as new developments happen. I do review things but I’m not manually rewriting every entry every time something changes. It’s an experiment in whether you can use AI not just to generate content but to curate and maintain a living knowledge base over time. So far the answer seems to be yes, with appropriate skepticism about any specific claim that would benefit from verification.

If you’re researching similar topics, hopefully this saves you some time. Why burn your tokens on what I’ve already gathered. And if you notice something wrong or missing, let me know.

My StartupWeekend Experience

Jared Watkins — Mon, 18 Dec 2017 12:15:56 -0600

One of the more fun things I did in Seattle back in 2014 was to participate in a StartupWeekend event with another guy from Amazon. I’d never heard of these events and he described it as being like a hackathon. So I was in and joined him on at the start of the event that Friday night at the Google campus in Kirkland. We quickly discovered that this wasn’t anything like a hackathon.

We were expected to write a business and marketing plan, do a customer analysis etc. so what we brought to work on wasn’t the right choice. So.. with minimal prep I pitched an idea to a room of about 200 people and managed to convince two others to join our team. One was a web developer and the other a business development person from another big tech company.

The Idea

The idea was to build a system to help eliminate traffic waves on busy roads which we called SmartCruize. If fully implemented it could also do things like have car Platoons and coordinated movement around traffic lights. It was a basically an implementation of a car-to-car communication network that could identify and track the movements of the cars around you and communicate your actions and intentions to the cars behind you. With such a network in place it would be possible to coordinate the speed and relative movements of cars in a very efficient way to increase the capacity of the existing roads and eliminate the most common causes of traffic jams.

It was ambitious to be sure.. but after a few rounds of simplifying the idea we decided to build an app that could roughly track the movements of a car and would gameify the driving experience to encourage good habits. For example using GPS we can identify what road you are on and the speed limit and you are going slower than that monitor how frequently you are using your brakes.. changing speeds etc. to encourage increased following distance and less braking.

At the time our thoughts on a full hardware implementation involved a mix of short range radio (mesh networking) combined with rear-facing infrared beacons that would broadcast a unique car ID to the vehicle directly behind you and a forward facing distance sensor to monitor forward car spacing and relative movements. If I were doing this project today I might replace some of sensors with a camera with some onboard video processing.

Failure is Always an Option

The idea with StartupWeekend is that by Sunday night, when the results are shared, you have a viable shell of a company which includes a business plan, market/customer analysis, and at least one paying customer. Our team was doing well until Saturday night. We had a website that did a walk through of how it was supposed to work with a mock up of the app. We had a customer survey written with some responses that helped shape the business plan. We also knew what we wanted the app to do.. but that’s where we hit a snag. Our two developers were not happy with the greatly reduced scope of what we were trying to deliver that weekend and they bailed on us.

Though we produced nothing of value I think it was a still a worthwhile experience. It was fun pulling together this group of strangers to try to build something and the experience of iterating to simplify a complex project was challenging and a useful exercise. Simple solutions are almost always better than complicated ones and fighting complexity should be one of the core missions of any engineer.

I’ve been asked if I would do one of these events again. I would.. but knowing now what I’m signing up for I’d be better prepared and not starting from scratch on a busy Friday night.

Hunting cold spots with the Flir One Thermal Camera

Jared Watkins — Thu, 05 Jan 2017 08:04:29 -0600

I like sharing useful gadgets with people and having an inexpensive thermal camera around the house can be very useful at times.

I’ve been in the new house for about six months now and with the cold snap making it in the mid 30s outside I thought this is a good time to break out the Flir One thermal camera and see what I can find around the house.

I’ve noticed certain rooms just feel cold and I think it’s from the older windows but I’m not sure how bad they are or if there is anything else going on behind the walls. From doing a little research I found that double glazed windows should have around an 8 degree C temperature difference to the interior walls under ideal conditions. This depends on the outside temperature but that’s a good ballpark. For single glazed windows this could be 20C and for triple glazed 4C. That’s the surface temperature of the glass vs the walls and does not take air leaks into account or the conductivity of the frame.

The windows in this house are mostly double pane with metal frames. The frames are the coldest part.. easily visible in the images. Several windows also seem to be leaking a lot of air on the bottom too and the front door also needs to be better sealed from air leaks.

In these images of the ceiling you can see how the insulation is not uniform above the vaulted ceilings leading to cold spots. The bright box in the corner is a heating vent.

The camera can colorize the image in different ways.. here I have it hi-lighting the coldest part of the image. In some of these I’m seeing gaps in the insulation behind the walls… not so easy to fix but also small enough that it’s probably not a factor. The ceiling behind the fan is about 20 feet away and you can tell the image isn’t as clear but you can still make out where the ceiling joists are behind the drywall.

Using the camera is pretty straight forward. There’s an app to install and the thermal camera plugs into the bottom of your phone. It’s powered by its own rechargeable battery. It feels a little delicate hanging by the lighting plug and using the rig pretty much requires two hands. It does seem to work pretty well though.. and seems to be great for this sort occasional use around the house. I did have to remove my phone from the Otterbox case I usually keep it in. I’m using this with an iPhone but it’s also available for Android phones or tablets.

Our cats are curious about what I’m up to. Notice how you can see the reflected heat of the cat on the door it’s standing next to. When using the camera around reflective objects be careful about not catching a reflection of heat from something nearby. You can even see your own body heat reflected in a window when standing in front of it.

Other fun uses around the house… Image your power panel to see if any circuits might be overloaded. Look at your gas grill to see exactly how and where the heat goes for better control while cooking. It can also be useful to find water leaks behind a wall.

Recommendations in Charlotte

Jared Watkins — Sun, 01 Sep 2013 00:00:00 +0000

I lived in Charlotte for a good long while… so here is a small list of my recommendations for businesses I’d come to know and trust. If you think I’ve forgotten to include you please let me know.

If you need a real estate agent for personal or investment property talk to Brenny Thompson
For help finding contractors I preferred Hocoa I always had good experiences from their suggestions.
For lawn care around downtown I liked Kevin at magnolialandscape.com

For solar energy or hot water or rainwater systems go to Energy Wise Solutions
For any sort of advertising, photography or graphic design talk to Kat at Paragon Studios
Tree issues/removal I had a good experience with Landman’s Tree Service 704 995-3473
For car detailing and glass tinting I like Brian at Exclusive Detail
For general car repair talk to Mikea at Almar Auto in southend
For guns and home safes I did a lot of business with Hyatt Guns
For a good range I preferred Shooters Express (ignore the totally 90s website)
If you need a barber talk to O’mar at No Grease Barber (at the Arena location downtown)