3. Goodness
In 2022, Geoffrey Hinton — the same Geoffrey Hinton who co-authored the original backprop paper — published a paper proposing an alternative. It was called "The Forward-Forward Algorithm."
The timing was significant. Hinton had spent four decades as the most prominent defender of backprop. He had seen it through the AI winter of the 1990s, when funding dried up and researchers abandoned neural networks. He had watched it rise again in the 2010s, fueled by bigger datasets and faster GPUs. And now, at the height of its triumph, he was saying: maybe it's not how the brain works.
The forward-forward algorithm replaces backprop's forward-and-backward with two forward passes. In the first pass, real data flows through the network, and each layer computes its "goodness" — the sum of the squared activities of its neurons. In the second pass, negative data — data the network should not learn from — flows through, and each layer computes its goodness again. The learning rule is simple: increase the goodness for positive data, decrease it for negative data.
Each layer learns independently. There is no backward pass. There is no global error signal. Each layer sees only its own activity and adjusts its connections to make good patterns more likely and bad patterns less likely.
The elegance of the forward-forward algorithm is its simplicity. It replaces a complex, multi-phase process with something that looks almost obvious in retrospect. Want to learn? Compare how your neurons fire for real data versus fake data, and push the real responses higher. That is it.
The algorithm does have limitations. It is less accurate than backprop on many tasks. Hinton himself described it as a "preliminary investigation" that was "worth further investigation." But the significance of the forward-forward algorithm is not its performance. It is that one of the architects of backprop — the person most responsible for its dominance — was publicly exploring alternatives.
The forward-forward algorithm fits into a broader pattern. Over the past decade, a growing number of researchers have been circling the same problem from different angles. Each one has proposed a different way to escape backprop's constraints. And each one has arrived at a similar place: local learning rules, no global error signal, no dual-phase requirement.
The forward-forward algorithm is not the answer. But it is a signal that the search is serious, and that the destination is becoming clearer.