Even people who expect human-level AI soon are still seriously underestimating how different the world will look when we have it. Most people are anchoring on how smart they expect individual models to be. (i.e. they’re asking themselves “What would the world be like if everyone had a very smart assistant who could work 24/7?”.)
Everyone is sleeping on the collective advantages AIs will have, which have nothing to do with raw IQ but rather with the fact that they are digital—they can be copied, distilled, merged, scaled, and evolved in ways human simply can’t.
What would a fully automated company look like - with all the workers, all the managers as AIs? I claim that such AI firms will grow, coordinate, improve, and be selected-for at unprecedented speed.
Currently, firms are extremely bottlenecked in hiring and training talent. But if your talent is an AI, you can copy it a stupid number of times. What if Google had a million AI software engineers? Not untrained amorphous "workers," but the AGI equivalents of Jeff Dean and Noam Shazeer, with all their skills, judgment, and tacit knowledge intact.
This ability to turn capital into compute and compute into equivalents of your top talent is a fundamental transformation. Since you can amortize the training cost across thousands of copies, you could sensibly give these AIs ever-deeper expertise - PhDs in every relevant field, decades of business case studies, intimate knowledge of every system and codebase the company relies on.
The power of copying extends beyond individuals to entire teams. Small previously successful teams (think PayPal Mafia, early SpaceX, the Traitorous Eight) can be replicated to tackle a thousand different projects simultaneously. It's not just about replicating star individuals, but entire configurations of complementary skills that are known to work well together. The unit of replication becomes whatever collection of talent has proven most effective.
Copying will transform management even more radically than labor. It will enable a level of micromanagement that makes founder mode look quaint. Human Sundar simply doesn't have the bandwidth to directly oversee 200,000 employees, hundreds of products, and millions of customers. But AI Sundar’s bandwidth is capped only by the number of TPUs you give him to run on. All of Google’s 30,000 middle managers can be replaced with AI Sundar copies. Copies of AI Sundar can craft every product’s strategy, review every pull request, answer every customer service message, and handle all negotiations - everything flowing from a single coherent vision.
There is no principal-agent problem wherein employees are optimizing for something other than Google’s bottom line, or simply lack the judgment needed to decide what matters most.1 A company of Google's scale can run much more as the product of a single mind—the articulation of one thesis—than is possible now.