Diagram

I didn’t set out to write about hardware. I was trying to stop burning my own bandwidth.

I dump raw thoughts into a prompt, let a local agent structure them, and save the result. That workflow only matters because the underlying physics is about to force the same split on every system we build.

The Bottleneck Is Physical

Modern GPU architectures spend most of their energy not computing. Up to 80 percent goes to moving data between separate memory and compute units.

This is the Von Neumann bottleneck. One bus. Two places. Constant shuttling.

Data centers are hitting hard caps — not just chip limits, but power grids and political permits. Scaling LLM inference to quadrillions of tokens per month on standard matrix multiplication is not expensive. It is unsustainable.

The Non-Von Neumann Answer

Analog compute-in-memory and neuromorphic spiking networks co-locate memory and compute on-die. No bus. No shuttle.

Processing becomes event-driven. A spike fires only when needed. No continuous clock-cycle matrix math.

The result is not incremental. Current research silicon reports 77+ TOPS/Watt — orders of magnitude above conventional accelerators for sparse, continuous workloads.

Trade-offs Are Real

Dimension Von Neumann (GPU/TPU) Non-Von Neumann (Neuromorphic)
Energy High, dominated by data movement Low, spikes only on events
Strength Dense matrix math, backpropagation Sparse inference, continuous listening
Weakness The bottleneck itself Conversion losses, limited training support
Software CUDA ecosystem, mature Immature compilers, no standard stack
Best use Training large models Edge agents, always-on sensing

The ecosystem inertia is the moat. CUDA took fifteen years to build. No neuromorphic compiler will replace it overnight.

The Hybrid Horizon

Von Neumann will not disappear. Software inertia guarantees it.

The likely split is functional. Standard GPUs handle heavy backpropagation training in the data center. Event-driven silicon handles low-latency edge work — continuous listening, sensor fusion, agentic loops.

Training stays centralized. Inference fragments to the edge. One system trains, another system reacts.

Human Hardware Is Not The Bottleneck

Digital consensus happens in milliseconds. A human prefrontal decision cycle takes roughly 10,000 ms.

The mistake is treating that as a bug to fix. The human system runs at 20W. It is homeostatic, low-power, and built for stability — not throughput.

The correct interface is asynchronous buffering. High-entropy digital streams should be summarized, filtered, and delivered in batches. The biology does not need to go faster. The machine needs to wait better.

Systems that force humans into machine clock cycles burn both.

Build the buffer, not the rush.