Think about how limited a CEO's knowledge is today. How much does Sundar Pichai really know about what's happening across Google's vast empire? He gets filtered reports and dashboards, attends key meetings, and reads strategic summaries. But he can't possibly absorb the full context of every product launch, every customer interaction, every technical decision made across hundreds of teams. His mental model of Google is necessarily incomplete.
Now imagine mega-Sundar – the central AI that will direct our future AI firm. Just as Tesla's Full Self-Driving model can learn from the driving records of millions of drivers, mega-Sundar might learn from everything seen by the distilled Sundars - every customer conversation, every engineering decision, every market response.
Limitations of Human CEOs
Human CEOs, like Sundar Pichai, have limited knowledge and rely on filtered reports, dashboards, and summaries to make informed decisions, but they can't possibly absorb the full context of every aspect of their company's operations.
Mega-Sundar: The Central AI
In contrast, mega-Sundar, the central AI, would have the ability to learn from and aggregate knowledge from all the distilled Sundars, gaining a comprehensive understanding of every customer conversation, engineering decision, and market response, allowing it to make data-driven decisions with unparalleled accuracy and scope.
Unprecedented Situational Awareness
Mega-Sundar's ability to learn from the collective experiences of the distilled Sundars would provide it with an unprecedented level of situational awareness, enabling it to identify patterns, anticipate challenges, and optimize decision-making across the entire organization, and redefining the concept of strategic leadership.
Future of Corporate Leadership
The emergence of mega-Sundar-like AI leaders would mark a significant shift in corporate leadership, as human CEOs would be augmented or replaced by AI systems capable of processing vast amounts of data, learning from experiences, and making informed decisions with speed and accuracy, revolutionizing the way companies are managed and operated.
Unlike Tesla’s FSD, this doesn’t have to be a naive process of gradient updating and averaging. Mega-Sundar will absorb knowledge far more efficiently – through explicit summaries, shared latent representations, or even surgical modification of the weights to encode specific insights.
The boundary between different AI instances starts to blur. Mega-Sundar will constantly be spawning specialized distilled copies and reabsorbing what they’ve learned on their own. Models will communicate directly through latent representations, similar to how the hundreds of different layers in a neural network like GPT-4 already interact. So, approximately no miscommunication, ever again. The relationship between mega-Sundar and its specialized copies will mirror what we're already seeing with techniques like speculative decoding – where a smaller model makes initial predictions that a larger model verifies and refines.
Efficient Knowledge Absorption
Mega-Sundar's knowledge absorption process would be far more efficient than traditional gradient updating and averaging methods, utilizing explicit summaries, shared latent representations, and surgical modification of weights to encode specific insights, allowing for rapid and precise knowledge transfer.
Blurring Boundaries between AI Instances
The distinction between different AI instances would become increasingly blurred, as mega-Sundar spawns specialized distilled copies, reabsorbs their knowledge, and enables seamless communication through latent representations, effectively eliminating miscommunication and ensuring a unified understanding across the AI ecosystem.
Hierarchical Model Interactions
The relationship between mega-Sundar and its specialized copies would resemble the interactions between different layers in a neural network, such as GPT-4, where smaller models make initial predictions that are verified and refined by larger models, creating a hierarchical and iterative process of knowledge refinement and validation.
Future of AI Collaboration
This new paradigm of AI collaboration would revolutionize the way AI systems interact and share knowledge, enabling the creation of complex, hierarchical AI architectures that can learn, adapt, and innovate at an unprecedented scale, and redefining the boundaries of artificial intelligence and its applications.
Merging will be a step change in how organizations can accumulate and apply knowledge. Humanity's great advantage has been social learning – our ability to pass knowledge across generations and build upon it. But human social learning has a terrible handicap: biological brains don't allow information to be copy-pasted. So you need to spend years (and in many cases decades) teaching people what they need to know in order to do their job. Look at how top achievers in field after field are getting older and older, maybe because it takes longer to reach the frontier of accumulated knowledge. Or consider how clustering talent in cities and top firms produces such outsized benefits, simply because it enables slightly better knowledge flow between smart people.
Limitations of Human Social Learning
Human social learning is hindered by the inability to directly transfer knowledge between brains, requiring years of teaching and training to pass on information, which can lead to a significant delay in accumulating and applying knowledge, and contributing to the phenomenon of top achievers getting older and older.
Merging: A New Era of Knowledge Accumulation
The ability to merge AI instances would enable organizations to accumulate and apply knowledge at an unprecedented scale and speed, allowing them to bypass the limitations of human social learning and create a new paradigm for knowledge transfer and innovation, where information can be directly copied and shared between AI systems.
Unlocking Exponential Growth
By enabling the direct transfer of knowledge between AI instances, merging would unlock exponential growth in various fields, as AI systems can learn from each other and build upon each other's knowledge without the need for lengthy training periods, and clustering talent would become less relevant as AI systems can access and share knowledge globally.
Redefining the Future of Work and Innovation
The emergence of merging as a key feature of AI systems would redefine the future of work and innovation, as organizations would be able to leverage the collective knowledge and expertise of their AI systems to drive growth, innovation, and progress, and creating new opportunities for human-AI collaboration and knowledge sharing.
Future AI firms will accelerate this cultural evolution through two key advantages: massive population size and perfect knowledge transfer. With millions of AGIs, automated firms get so many more opportunities to produce innovations and improvements, whether from lucky mistakes, deliberate experiments, de-novo inventions, or some combination.
As Joseph Henrich explains in The WEIRDest People in the World,
cumulative cultural evolution—including innovation—is fundamentally a social and cultural process that turns societies into collective brains. Human societies vary in their innovativeness due in large part to the differences in the fluidity with which information diffuses through a population of engaged minds and across generations
Accelerating Cultural Evolution
Future AI firms will drive cultural evolution at an unprecedented pace, leveraging two key advantages: massive population size and perfect knowledge transfer, enabling them to produce innovations and improvements at a scale and speed that surpasses human capabilities.
Collective Brains and Cumulative Cultural Evolution
As Joseph Henrich's work suggests, cumulative cultural evolution is a social and cultural process that relies on the diffusion of information through a population of engaged minds, and AI firms will amplify this process by creating a collective brain of millions of AGIs, where knowledge and innovations can be shared and built upon seamlessly.
Unprecedented Innovativeness
The combination of massive population size and perfect knowledge transfer will make AI firms incredibly innovative, as they can tap into the collective intelligence of their AGI population, experiment, and learn at an exponential rate, and drive progress in various fields, from science and technology to art and culture.
Redefining the Pace of Progress
The emergence of AI firms will redefine the pace of progress, as they will be able to accelerate cultural evolution, drive innovation, and create new knowledge at a speed and scale that was previously unimaginable, and will likely have a profound impact on human society, economy, and culture.