Afterword: What 'From the Gate Up' Teaches Us

We started with a single AND gate. Two inputs, one output. The simplest possible circuit. From that foundation, we built our way up through multiply-accumulate units, muxes, systolic arrays, pipeline registers, lookup tables, caches, and branch predictors — until we could see the entire shape of a modern AI chip and understand why it looks the way it does.

The journey reveals two things.

The One Problem

Every design decision in chip architecture is a response to the same problem: moving data is expensive.

This truth appears at every level of the stack.

At the gate level, the muxes that select inputs from the register file cost more than the computation unit itself. The ratio is worse for larger register files, which is why specialized accelerators keep their local storage minimal.

At the architecture level, the systolic array exists specifically to amortize data-movement costs. It loads the weight matrix once and reuses it across thousands of computations, transforming a data-movement-heavy process into a compute-heavy one.

At the system level, the choice between cache and scratchpad is a choice about how much non-determinism you are willing to accept in exchange for automatic data movement. The choice between FPGA and ASIC is a choice between paying the flexibility tax once or paying it on every operation.

At the organizational level — the level of the data center — the same trade-off appears as the batch-size problem: do you optimize for latency (small batches, more data movement per computation) or throughput (large batches, less data movement per computation)?

Chip design is not a collection of unrelated specialties. It is the same optimization, applied at different scales.

The One Question

The question that every chip designer faces, at every level, is the same: how much flexibility are you willing to trade for efficiency?

More flexibility means more muxes, more routing, more control logic, more overhead. Less flexibility means harder programming, narrower applicability, more risk if the workload changes.

The GPU chooses flexibility. It pays a heavy tax — the vast majority of its die is overhead — but it runs almost any workload.

The TPU chooses efficiency. It removes the overhead and gets more compute per square millimeter, but it only does matrix multiplication well.

The FPGA chooses extreme flexibility. It pays a 10× tax but can be rewired in the field for any digital circuit.

The brain chooses... something else entirely. It gets efficiency through co-location and 3D connectivity, speed through massive parallelism, and generality through a fundamentally different computational model.

Where We Are Headed

The AI chip industry is converging on a compromise: chips that can be both efficient and flexible, depending on the moment. The splittable systolic array is one example. The trend toward programmable scratchpads, configurable precision, and reconfigurable data paths points in the same direction.

But the fundamental physics will not change. Data movement will always be expensive. The area and power costs of moving a bit across a chip will always exceed the cost of computing on it. Every future innovation — optical interconnects, 3D stacking, analog compute, neuromorphic architectures — is a bet on a different way to manage this cost.

Understanding the trade-off does not make it go away. But it makes the industry's decisions legible. When NVIDIA announces FP4 is 3× FP8, you know the quadratic scaling law behind it. When a startup claims its chip is 10× more efficient for inference, you know to ask: "What did you give up to get that efficiency, and does my workload fit?"

The waiting game that defined the last few years of AI — the memory wall, the bandwidth bottleneck, the idle compute units — is not going away. But understanding the full stack, from the gate to the chip, makes it possible to see where the bottlenecks are and what kind of innovation would break them.

Everything in this book traces back to a single AND gate and a simple question about how two bits combine. The rest is just the same idea, scaled up.