Part 2/9:
Gus initiated the dialogue by questioning the practical implications of recent advancements in AI, particularly regarding longer context windows and multimodal capabilities. Nathan responded by contrasting the belief held by some that AI progress has plateaued since the release of GPT-4, to a more nuanced view which acknowledges significant ongoing advancements, especially in user experience.
He noted the evolution of context windows, which have drastically increased. For example, the latest models now support context windows of over two million tokens, allowing for deeper interactions and richer conversations. This extended context significantly enhances the AI's ability to engage in discussions that involve complex information, resulting in more useful outputs and responses.