Part 8/9:
The discussion between Baker and Karpathy underscores a pivotal aspect of the AI evolution: while training model costs may decline, the demand for sophisticated inference capabilities—performing tasks based on learned data—will increase. By drawing comparisons between human intelligence and AI, the limitations of current AI model training hardware become apparent.
Historically, early computers occupied entire rooms, performing basic calculations. Today's devices, however, encapsulate increasingly powerful capabilities in more compact forms. The speculation is that AI could undergo a similar transformation, achieving higher levels of intelligence and usability within small, portable frameworks capable of performing complex tasks offline.