Redefining Scarcity and Abundance
The cost of having an AI take on a role will be reduced to the cost of compute, changing our understanding of which roles are scarce and abundant, and enabling AI firms to optimize for the most valuable abilities, regardless of their scarcity in human skill distributions.
Unlimited Access to Top Talent
With the ability to create multiple copies of top talent, such as Jeff Dean-level engineers or world-class researchers, at a marginal cost of pennies, AI firms will have unlimited access to the best skills and expertise, eliminating the constraints of finding and training rare human talent.
Compute as the Limiting Factor
The limiting factor for AI firms will no longer be the availability of skilled humans, but rather the availability of compute resources, which will determine the scale and scope of their operations, and will drive innovation in areas such as computing infrastructure, energy efficiency, and data storage.
New Era of Talent Acquisition and Management
This will usher in a new era of talent acquisition and management, where AI firms can focus on developing and deploying the most valuable skills and expertise, without being constrained by the limitations of human talent, and will likely lead to a significant shift in the way companies approach innovation, productivity, and competitiveness.
So what becomes expensive in this world? Roles which justify massive amounts of test- time compute. The CEO function is perhaps the clearest example. Would it be worth it for Google to spend $100 billion annually on inference compute for mega-Sundar? Sure! Just consider what this buys you: millions of subjective hours of strategic planning, Monte Carlo simulations of different five-year trajectories, deep analysis of every line of code and technical system, and exhaustive scenario planning.
Imagine mega-Sundar contemplating: "How would the FTC respond if we acquired eBay to challenge Amazon? Let me simulate the next three years of market dynamics... Ah, I see the likely outcome. I have five minutes of datacenter time left – let me evaluate 1,000 alternative strategies."
The more valuable the decisions, the more compute you'll want to throw at them. A single strategic insight from mega-Sundar could be worth billions. An overlooked risk could cost tens of billions. However many billions Google should optimally spend on inference for mega-Sundar, it's certainly more than one.
Distillation
What might distilled copies of AI Sundar (or AI Jeff) be like? Obviously, it makes sense for them to be highly specialized, especially when you can amortize the cost of that domain specific knowledge across all copies. You can give each distilled data center operator a deep technical understanding of every component in the cluster, for example.
I suspect you’ll see a lot of specialization in function, tacit knowledge, and complex skills, because they seem expensive to sustain in terms of parameter count. But I think the different models might share a lot more factual knowledge than you might expect. It’s true that plumber-GPT doesn’t need to know much about the standard model in physics, nor does physicist-GPT need to know why the drain is leaking. But the cost of storing raw information is so unbelievably cheap (and it’s only decreasing) that Llama-7B already knows more about the standard model and leaky drains than any non-expert.
If human-level intelligence is more than 1 trillion parameters, is it so much of an imposition to keep around what will, at the limit, be much less than 7 billion parameters to have most known facts right in your model? (Another helpful data point here is that “Good and Featured” Wikitext is less than 5 MB. I don’t see why all future models—except the esoteric ones, the digital equivalent of tardigrades—wouldn’t at least have Wikitext down.