Part 11/15:
- True progress demands fundamental architectural changes, like adding inhibitory mechanisms similar to those in biological brains.
Comparing the Human Brain and Large Language Models
While tempting, equating parameters to neurons or synapses is roughly approximate:
Human brains have ~90 billion neurons and approximately trillions of synapses.
Some estimate models like GPT-3 with 175 billion parameters equate to a fraction of neurons in the human brain—perhaps hundreds of millions of neurons, far below biological complexity.
Thus:
- Scaling parameters exponentially gets us closer to brain-level intelligence, but not directly comparable.