Limitations of Human CEOs
Human CEOs, like Sundar Pichai, have limited knowledge and rely on filtered reports, dashboards, and summaries to make informed decisions, but they can't possibly absorb the full context of every aspect of their company's operations.
Mega-Sundar: The Central AI
In contrast, mega-Sundar, the central AI, would have the ability to learn from and aggregate knowledge from all the distilled Sundars, gaining a comprehensive understanding of every customer conversation, engineering decision, and market response, allowing it to make data-driven decisions with unparalleled accuracy and scope.
Unprecedented Situational Awareness
Mega-Sundar's ability to learn from the collective experiences of the distilled Sundars would provide it with an unprecedented level of situational awareness, enabling it to identify patterns, anticipate challenges, and optimize decision-making across the entire organization, and redefining the concept of strategic leadership.
Future of Corporate Leadership
The emergence of mega-Sundar-like AI leaders would mark a significant shift in corporate leadership, as human CEOs would be augmented or replaced by AI systems capable of processing vast amounts of data, learning from experiences, and making informed decisions with speed and accuracy, revolutionizing the way companies are managed and operated.
Unlike Tesla’s FSD, this doesn’t have to be a naive process of gradient updating and averaging. Mega-Sundar will absorb knowledge far more efficiently – through explicit summaries, shared latent representations, or even surgical modification of the weights to encode specific insights.
The boundary between different AI instances starts to blur. Mega-Sundar will constantly be spawning specialized distilled copies and reabsorbing what they’ve learned on their own. Models will communicate directly through latent representations, similar to how the hundreds of different layers in a neural network like GPT-4 already interact. So, approximately no miscommunication, ever again. The relationship between mega-Sundar and its specialized copies will mirror what we're already seeing with techniques like speculative decoding – where a smaller model makes initial predictions that a larger model verifies and refines.
Merging will be a step change in how organizations can accumulate and apply knowledge. Humanity's great advantage has been social learning – our ability to pass knowledge across generations and build upon it. But human social learning has a terrible handicap: biological brains don't allow information to be copy-pasted. So you need to spend years (and in many cases decades) teaching people what they need to know in order to do their job. Look at how top achievers in field after field are getting older and older, maybe because it takes longer to reach the frontier of accumulated knowledge. Or consider how clustering talent in cities and top firms produces such outsized benefits, simply because it enables slightly better knowledge flow between smart people.
Future AI firms will accelerate this cultural evolution through two key advantages: massive population size and perfect knowledge transfer. With millions of AGIs, automated firms get so many more opportunities to produce innovations and improvements, whether from lucky mistakes, deliberate experiments, de-novo inventions, or some combination.
As Joseph Henrich explains in The WEIRDest People in the World,
cumulative cultural evolution—including innovation—is fundamentally a social and cultural process that turns societies into collective brains. Human societies vary in their innovativeness due in large part to the differences in the fluidity with which information diffuses through a population of engaged minds and across generations
Historical data going back thousands of years suggest that population size is the key input for how fast your society comes up with more ideas. AI firms will have population sizes that are orders of magnitude larger than today's biggest companies - and each AI will be able to perfectly mind meld with every other, from the bottom to the top of the org chart.
AI firms will look from the outside like a unified intelligence that can instantly propagate ideas across the organization, preserving their full fidelity and context. Every bit of tacit knowledge from millions of copies gets perfectly preserved, shared, and given due consideration.