A staircase from the physical floor to off-chip memory
Follow a single bit from storage to the arithmetic unit. Each step it takes costs more energy, and the biggest jumps happen long before it reaches a network cable.
Every layer between compute and its data costs more than the Landauer floor
kT ln 2 at room temperature: the minimum thermodynamic dissipation for one irreversible bit erasure. No real device operates anywhere near this.
Four layers, each deciding how far a bit must travel
Only the bottom layer is set by physics. Everything above it is design: where memory sits, how chips talk, and whether software reuses the data it already has.
Physical state
Transistors, memory cells and wires hold and flip bits while fighting electrical noise.
- Measure
- J/switch · error rate
- Failure boundary
- Leakage, reliability margins and the need for speed all cost energy far beyond the theoretical minimum.
Where the frontier moves
Devices that hold a bit reliably with less energy, without slowing down.
Package and memory
Caches, high-bandwidth memory stacks and the silicon they sit on trade storage capacity against distance from the processor.
- Measure
- J/bit · GB/s · latency
- Failure boundary
- Heat and manufacturing yield limit how close memory can sit to the processor.
Where the frontier moves
Advanced packaging that keeps shrinking the distance data travels.
System interconnect
Cables, switches and network links move data between chips, servers and racks.
- Measure
- Bytes/result · network utilization
- Failure boundary
- Once data leaves the package, network layout and waiting for other chips can dominate.
Where the frontier moves
Optical links built into the chip package, closing the gap with on-package wiring.
Algorithms and software
Number precision, reuse, caching and scheduling decide how much data needs to move at all.
- Measure
- Verified outcomes/joule
- Failure boundary
- Poorly organized software fetches or recomputes the same data again and again.
Where the frontier moves
Software that treats data movement, not arithmetic, as the thing to optimize.
Erasing a bit has a floor. Moving one has a distance.
Until the mid-2000s, smaller transistors also used less power, so every new generation got efficiency for free. That automatic saving faded. Transistors kept shrinking, but wires did not get cheaper to charge, and real devices must hold their state firmly against noise. The practical gains now come from putting data closer and using it more, not from approaching the theoretical floor.
Efficiency improved, and the workload grew faster
Koomey and colleagues’ long-run series found computations per kWh roughly doubled every 1.5 years. That is a historical trend for a particular benchmark and set of machines, not a law of nature, and it slowed once voltage scaling stalled after the mid-2000s.
Meanwhile, total demand took off. The IEA estimates 485 TWh of global data-centre electricity in 2025, about 1.5% of world demand, and roughly 950 TWh in its 2030 case. Those totals include ordinary servers, AI accelerators, storage, networks and cooling, so they cannot be divided by a guessed number of AI prompts to get a real energy-per-answer figure. This is the paradox of efficient computing: each operation gets cheaper, so we buy far more of them.
How far above the floor are we?
Landauer's minimum at room temperature is about 2.9 × 10⁻²¹ joule per bit erased. It applies to an ideal operation, not to a transistor, a memory read, a network link, a power converter and a cooling system combined. Researchers exploring “reversible” logic have shown that some of the gap can in principle be closed, but no one has built a general-purpose computer that does so. For the foreseeable future, the big wins are engineering ones: shorter wires, fewer bytes, and better reuse.
The package becomes the computer
When memory sits right next to logic, the way chips are bonded, cooled, tested and supplied matters as much as the processor design. Four strategies are doing most of the work.
Reuse before moving
Reorganize algorithms so data fetched once is used many times while it is still nearby.
Use fewer bits
Store and move numbers at the lowest precision that still gives a correct answer.
Shorten the wire
Put memory right beside the processor with HBM, chiplets and optics where the workload justifies it.
Measure outcomes
Report energy per finished task for the whole system, not peak operations for one chip.
Who is building what
Co-packaged optics, silicon photonics, high-bandwidth memory and open chiplet standards all attack the same “memory wall.” Search the record, or filter by interconnect layer.
Celestial AIPhotonic FabricSilicon photonics platform delivering optical compute and memory interconnectivity, decoupling compute silicon from memory placement via optical links
- Reported evidence
- Secured major customer commitments and hyperscale investments; demonstrated multi-terabit low-latency die-to-die and die-to-memory interconnects.
- Announced next step
- Commercial deployment of disaggregated, optically pooled high-bandwidth memory (HBM) systems for multi-accelerator AI clusters.
- Unresolved risk
- Optical coupling losses at fiber-to-chip interfaces and manufacturing yield of dense silicon photonic waveguide arrays.
Ayar LabsTeraPHY & SuperNova LaserMonolithic electronic-photonic optical I/O chiplets utilizing micro-ring modulators driven by external multi-wavelength continuous-wave laser sources
- Reported evidence
- Demonstrated operational TeraPHY links integrated with Intel processors and NVIDIA GPUs; achieved <5 pJ/bit energy efficiency across multi-terabit links.
- Announced next step
- Integration of optical I/O directly into standard high-volume packaging substrates replacing electrical SerDes traces.
- Unresolved risk
- Thermal sensitivity of micro-ring modulators requiring active thermal tuning loops, and laser diode lifetime in data center environments.
LightmatterPassage Optical InterposerWafer-scale programmable silicon photonics interposer that routes optical signals between heterogeneous compute dies placed directly on top
- Reported evidence
- Demonstrated wafer-scale optical routing with orders-of-magnitude higher cross-sectional bandwidth density than organic packaging substrates.
- Announced next step
- Enabling rack-scale virtual compute chips consisting of hundreds of interconnected GPUs behaving as a single monolithic processor.
- Unresolved risk
- Mechanical stress and thermal expansion differences across full 300mm silicon interposers under high thermal dissipation loads.
BroadcomBailly 51.2 Tbps Co-Packaged OpticsCo-Packaged Optics (CPO) switch integrating 51.2 Tbps silicon switching ASICs with optical transceiver engines on a common substrate
- Reported evidence
- Manufactured and demonstrated working commercial 51.2 Tbps Bailly CPO systems, cutting optical interconnect power by over 30% versus pluggable transceivers.
- Announced next step
- Scaling CPO to 102.4 Tbps switch architectures, eliminating copper backplane trace losses in hyperscale AI network fabrics.
- Unresolved risk
- Serviceability and field repair: replacing a failed optical channel currently requires removing the entire multi-thousand-dollar switch package.
SK HynixHBM3E & HBM4 Memory Stacks12- and 16-high stacked DRAM dies interconnected via Through-Silicon Vias (TSVs) and Advanced Mass Reflow Molded Underfill (MR-MUF)
- Reported evidence
- Dominant supplier of HBM3E memory for NVIDIA H100 and B200 systems, delivering over 1.18 TB/s bandwidth per stack.
- Announced next step
- Transitioning to HBM4 featuring 2048-bit wide interfaces and logic base dies manufactured on advanced foundry nodes (TSMC).
- Unresolved risk
- Thermal dissipation from bottom DRAM layers trapped under top dies, and TSV micro-bump bonding yield across 16-layer stacks.
TSMC3DFabric & CoWoS (Co-Packaged Optics)Chip-on-Wafer-on-Substrate (CoWoS-R, CoWoS-L) and Compact Universal Photonic Engine (COUPE) integrating optics and compute in 3D packaging
- Reported evidence
- The manufacturing foundation for modern AI accelerators; expanding CoWoS capacity to multi-reticle dimensions exceeding 3.3× standard mask size.
- Announced next step
- Monolithic integration of electronic and photonic ICs (EPIC) with lowest parasitic capacitance and sub-picojoule transfer energy.
- Unresolved risk
- Substrate warpage during thermal cycling and severe global packaging substrate supply bottlenecks constraining AI accelerator shipments.
UCIe ConsortiumUniversal Chiplet Interconnect ExpressOpen industry standard defining physical, die-to-die adapter, and protocol layers (PCIe, CXL, streaming) for heterogeneously packaged chiplets
- Reported evidence
- Over 100 member companies including Intel, AMD, Arm, TSMC, and Qualcomm; validated interoperability across silicon vendor test chips.
- Announced next step
- Creating a thriving commercial merchant chiplet market where designers mix and match dies across different foundries and process nodes.
- Unresolved risk
- Testing and known-good-die (KGD) liability when chiplets from multiple competing vendors are integrated into a single expensive package.
IEEE Photonics / DARPA PIPESPhotonic Edge-to-Edge StandardsPublic research and standards initiatives benchmarking photonic link energy efficiency, latency, and optical interconnect architectures
- Reported evidence
- Funded foundational micro-ring, comb laser, and optical interposer developments that seeded leading commercial silicon photonics startups.
- Announced next step
- Establishing sub-1 pJ/bit interconnect metrics for exascale computing and disaggregated data center memory pools.
- Unresolved risk
- Academic prototypes demonstrating record efficiency in controlled settings struggling with manufacturing tolerances on commercial CMOS lines.
Optical I/O energy ratings (e.g. sub-5 pJ/bit) describe transceiver physical layer links; system-level energy must account for external continuous-wave laser thermal coolers and electrical serializer overhead.
The optimistic view, with conditions
The efficient computer moves less, not faster
The largest gains ahead come from designing algorithms, number formats, memory, packaging, networks and cooling together, so useful information travels less often and less far. There is enormous room left: we are still far closer to the cost of wires than to the limits of physics.
Keep shortening the wire
High-bandwidth memory already beats ordinary memory per bit because it sits closer; advanced packaging pushes the same logic further.
Bring light onto the chip
Optics built into the package can cut chip-to-chip energy well below today's plug-in modules.
Stop paying twice for the same bit
Algorithms that keep data nearby avoid fetching it again and again.
What actually lowers the energy of a useful answer
- Shorter distancesMemory placed physically close to where data is used.
- Fewer bits movedPrecision and compression matched to what the task needs.
- More reuseAlgorithms and caches that avoid fetching the same data twice.
- Whole-system measurementEnergy and latency reported per finished task, not peak operations.
- Layers designed togetherAlgorithms, packaging and networks optimized as one system.
Sources, method, and boundaries
The thermodynamic floor follows NIST's discussion of conventional and reversible computation. HBM figures come from NIST's project summary. The energy-per-bit ladder combines this floor with Horowitz's widely cited 45nm energy breakdown (ISSCC 2014) and public HBM and data-center optical-interconnect figures: representative orders of magnitude spanning different technologies and process nodes, not measurements of one system. The roofline relationship is a model, not a benchmark for a specific application.
- Landauer's principle
- The thermodynamic minimum energy required to irreversibly erase one bit of information at a given temperature.
- Roofline model
- A performance model showing that achievable compute throughput is capped by memory bandwidth for a given arithmetic intensity.
- Co-packaged optics
- Optical transceivers integrated directly onto a chip package rather than connected via a separate pluggable module.
Read More
Essays, books, talks, and research that shaped this field’s arguments. Influence is not endorsement; company communications and advocacy are labeled. Some publisher links require a subscription.
- Talk
Mark Horowitz — Computing’s Energy Problem (and what we can do about it)
The widely reused energy table makes data movement visible beside arithmetic.
- Foundational paper
Rolf Landauer — Irreversibility and Heat Generation in the Computing Process
The thermodynamic bound; compare it with the much larger costs of deployed architectures.
- Company technical writing
NVIDIA — Technical blog
Architecture and memory-bandwidth accounts from a hardware supplier, labeled as company writing.
- Company technical writing
Google — Research blog
Hyperscaler accounts of computing systems and efficiency, to read alongside accepted-work measurements.
- Paper / report
Wulf & McKee, “Hitting the Memory Wall: Implications of the Obvious” (1995)
The classic essay/paper that made the processor–memory performance gap a central computing-system problem.
Across the fields: learning curves, deployment, and rebound
- Theodore Wright — Factors Affecting the Cost of Airplanes (1936)
The original experience-curve formulation: costs change with accumulated production.
- Kenneth Arrow — The Economic Implications of Learning by Doing (1962)
The economics of productivity improvement through production experience.
- William Stanley Jevons — The Coal Question (1865)
The classic rebound argument: lower effective costs can expand total demand.
- Arnulf Grübler — The costs of the French nuclear scale-up: A case of negative learning by doing (2010)
A counterexample to assuming that greater deployment always lowers costs.



















