Technology

Teaching Machines to Learn How Neural Networks Went from Dead End to Modern AI

The researchers who kept faith through decades of ridicule, and the unsettling question their success has left us with

Listen to the opening

0:001:35

A 20-minute episode of CompoundThis episode is free to hear in the Compound app.

Compound is coming early 2027.

In this episode

  1. The First Spark: Rosenblatt's Perceptron
  2. The Long Winter: Minsky, Papert, and the Decade of Doubt
  3. The Godfathers and the Comeback: Backpropagation and Deep Belief
  4. The 2012 Earthquake: AlexNet and the GPU Revolution
  5. The Black Box Problem: What Machines Know That We Don't
  6. What It Means to Learn

The opening

In the spring of 1958, a psychologist at Cornell University named Frank Rosenblatt stood before a room of journalists and made a claim that would have seemed absurd to most serious scientists of the era. He told them that the machine he had built — a device he called the Perceptron — could learn. Not execute instructions. Not follow a program. Learn. The reporters wrote it up with breathless enthusiasm. The New York Times ran a headline suggesting the Navy had invented a machine that could walk, talk, see, write, reproduce itself, and be conscious of its own existence. Rosenblatt had said no such thing, but the public's imagination had already raced ahead of the science.

What followed was not a smooth march toward artificial intelligence. It was something far more turbulent, far more human — a decades-long story of brilliant ideas dismissed, careers derailed, and long winters of funding and faith, kept alive by a handful of stubborn researchers who refused to walk away. Their persistence eventually produced the most consequential technology of our time. And yet, even now, even after machines can diagnose cancer, translate between languages, compose music, and defeat every human at chess, we face a disquieting truth: we do not fully understand how they do it.

This is the story of how machines learned to learn — and why that story is far stranger, and far more unsettling, than the headlines suggest.

The episode continues in the Compound app.