Arithmetic became abundant faster than proximity
HBM stacks memory beside compute on dense packages to create wide, short interfaces. Caches, tiling, operation fusion, quantization, sparsity, and near-memory designs attack the same problem from software and architecture: do not pay repeatedly to move the same information.
Three numbers that locate the frontier
Landauer's principle is a thermodynamic limit, not measured chip energy. Vendor bandwidth is a peak component figure, not application throughput or energy efficiency.
The roofline is an energy map as well as a speed limit
For a fixed arithmetic intensity, memory bandwidth caps achieved operations. Moving fewer bytes per result can raise performance and reduce energy simultaneously; adding compute units does neither once data delivery saturates.
Lower precision reduces storage, transfer, and arithmetic when task quality survives.
Specialization and larger batches improve utilization but can increase latency or become stranded when workloads change.
The headline metric sits on a system
Each layer can become the bottleneck even when the layer before it improves.
Physical state
Transistors, memory cells, wires, and emerging devices encode, retain, and switch information under noise.
- Measure
- J/switch · error rate
- Failure mode
- Leakage, reliability, and finite time
Package and memory
Registers, caches, HBM stacks, interposers, substrates, and host memory trade capacity against distance.
- Measure
- J/bit · GB/s · latency
- Failure mode
- Thermals and package yield
System interconnect
Links, switches, collective communication, and storage move state beyond one package.
- Measure
- Bytes/result · network utilization
- Failure mode
- Topology and synchronization
Algorithms and software
Precision, sparsity, locality, fusion, caching, routing, and scheduling decide which movement occurs.
- Measure
- Verified outcomes/joule
- Failure mode
- Generality and poor reuse
Erasure has a floor; communication has a distance
Reversible operations can approach lower dissipation only under restrictive assumptions and time trade-offs. Real systems must maintain energy barriers against noise and charge physical wires; placing data closer and reusing it are therefore near-term gains far larger than approaching Landauer's limit.
Packaging becomes architecture
When memory sits beside logic, bonding, interposers, thermals, test, repair, HBM supply, and software locality shape performance as directly as processor design. The bottleneck migrates from arithmetic units into the whole package and fabric.
Reuse before moving
Tile, fuse, cache, and restructure algorithms around locality.
Represent with fewer bits
Use the lowest precision and compression consistent with verified quality.
Shorten the link
Use HBM, advanced packaging, chiplets, optics, or near-memory compute where justified.
Measure outcomes
Report whole-system joules and latency per accepted task, not isolated peak operations.
An optimistic view, with conditions
The efficient computer is a balanced information machine
The largest practical gains will come from co-designing algorithms, numerics, memory hierarchy, packaging, networks, power, and cooling so that useful information travels less often and less far.
Sources, method, and boundaries
The thermodynamic floor follows NIST's discussion of conventional and reversible computation. HBM figures come from NIST's project summary. The roofline relationship is a model, not a benchmark for a specific application.
