Notes on Understanding Hebbian Learning

Hebbian learning describes how correlated neural activity can strengthen connections. It is named after psychologist Donald Hebb, who proposed it in 1949.

The entire concept boils down to one beautiful phrase:

“Neurons that fire together, wire together.”

That is a useful shorthand, not a complete account of learning. Timing and the synapses involved matter.

Breaking It Down

Repeated, correlated activity can strengthen the connection between two neurons. The effect depends on timing and the synapse, rather than simultaneous activity always producing the same change.

Think of it like this:

• First time: “Oh, these two things happened together”
• Fifth time: “Okay, these definitely seem related”
• Hundredth time: “These are basically the same signal now”

Real Examples That Make It Click

Learning to Type

Remember learning to type? At first, finding the ‘Q’ key required conscious thought. Your brain neurons for “want to type Q” and “move left pinky to top left” fired together… over and over. Now? It’s automatic. Those neurons are strongly wired.

Pavlov’s Dogs (The Classic)

Bell rings → food appears → dogs salivate

Do this enough times, and the neurons for “bell sound” and “food is coming” become so tightly linked that just the bell makes dogs salivate. No food needed.

Your Morning Coffee Ritual

Why does the smell of coffee make you feel more awake even before you drink it? Neurons fired together thousands of times:

• Smell of coffee + feeling of alertness
• Sound of coffee maker + anticipation of energy
• Taste of first sip + actual caffeine effect

Now they’re all wired together into one mega-circuit.

The Actual Mechanism (Simplified)

What’s physically happening:

1. Neuron A fires
2. Neuron B fires (around the same time)
3. The synapse (connection) between them releases more receptors
4. The connection physically strengthens
5. Next time A fires, it’s easier to activate B

It’s literally changing the structure of your brain. Learning is physical.

Why This Matters

For Understanding Yourself

• Habits: They form because neurons wire together through repetition
• Memory: Retrieving memories strengthens those neural pathways
• Skills: Practice literally builds stronger brain connections
• Trauma: Unfortunately, negative associations work the same way

For AI and Machine Learning

Hebbian learning inspired artificial neural networks. Modern AI doesn’t use pure Hebbian learning anymore (it’s evolved into more sophisticated algorithms), but it started here.

The basic idea - connections that prove useful should get stronger - is still foundational.

The Flip Side: “Use It or Lose It”

Here’s the thing nobody tells you: it works in reverse too.

Neurons that STOP firing together will gradually UNWIRE.

This is why:

• You forget languages you don’t practice
• Old skills get rusty
• Memories fade
• Bad habits can actually be broken

The Math (Optional)

For the nerds: The basic Hebbian rule can be expressed as:

Δw = η × x × y

Where:

• Δw = change in connection strength
• η = learning rate
• x = activity of first neuron
• y = activity of second neuron

If both are active (positive values), the connection strengthens. Simple, elegant, powerful.

Practical Takeaways

For Learning Anything:

• Repetition isn’t just helpful - it’s literally how learning works
• Practice needs to be consistent (neurons need to fire together repeatedly)
• Spacing matters (distributed practice > cramming)
• Context matters (learning in similar contexts strengthens related neural patterns)

For Breaking Bad Habits:

• You need to stop the neurons from firing together
• Replace the pattern with a new one (create competing neural pathways)
• It takes time - you’re literally rewiring your brain

For Building Skills:

• Deliberate practice works because it forces specific neurons to fire together
• Feedback loops help strengthen the right connections
• Muscle memory is real - it’s Hebbian learning in motor cortex

Cool Extensions of This Idea

Spike-timing-dependent plasticity (STDP): Order matters too. In this simplified account, neuron A firing before neuron B (within ~20ms) can strengthen the connection, while the reverse order can weaken it. The timing and effect depend on the synapse.

Competitive learning: When neurons wire together, they can also suppress other connections. This is how your brain specializes - certain neuron clusters become experts at specific patterns.

Associative memory: Hebbian associations offer a way to think about connections between faces, names, and songs. They are not a complete explanation of memory.

If you want to dive deeper:

• Donald Hebb’s original work: “The Organization of Behavior” (1949)
• Modern neuroscience textbooks on synaptic plasticity
• Research on long-term potentiation (LTP) - the molecular mechanism behind Hebbian learning
• AI/ML papers on Hebbian learning algorithms