Mustafa Suleyman: The AGI Race Is Fake, Building Safe Superintelligence & the Agentic Economy | #216

Suleyman explains that AI models now personalize well, learning from your interactions to surface relevant papers and insights. This creates a feedback loop where the user becomes more effective, and the AI becomes more tailored. He recommends this as the single best accelerant for individuals to speedrun a Star Trek-like future. He also notes that this personal AI can act as an aid to your own line of inquiry, making you better over time. This is not just a tool, but a symbiotic relationship.

Suleyman believes that even as AI automates many tasks, the combination of deep technical knowledge and humanistic reflection is essential. He sees philosophy as critical for addressing alignment, ethics, and the meaning of intelligence, while CS provides the tools to build and understand the systems. He insists that formal education—the social and exploratory environment—is a huge privilege that should not be thrown away, even in a rush to build startups. He himself dropped out but still values the college experience.

Suleyman argues that over the past five decades, the status and respect for public service have been battered, leaving democratic institutions weak. He believes that AI will demand new regulatory frameworks and international cooperation, and that needs smart people inside governments. His own short stint in public service was formative, teaching him the importance of the humanist perspective. He sees this as a moral imperative to balance the accelerationist drive with stewardship.

Suleyman points out that historically, breakthrough technologies spread rapidly and globally, not constrained to a single winner. He argues that the race framing is misleading because it implies zero-sum outcomes, whereas in reality knowledge and science proliferate everywhere, all at once, at all scales. This viewpoint reflects his experience seeing AI advances like language models quickly disseminate once they are proven. He believes the focus should instead be on building safe, beneficial intelligence, as multiple actors will achieve powerful capabilities roughly simultaneously. He even says 'we're all going as fast as we possibly can,' so competition is real, but it's not a zero-sum race with a clear finish line. This stance has implications for policy, collaboration, and how companies like Microsoft position themselves—not as a single winner but as part of a broader ecosystem that will see many powerful agents.

Suleyman explains that as models moved from recognition to generation, the next step is agentic action. He predicts that in the near term, models will perform intelligent knowledge work—acting as project managers or startup founders. The natural way to measure performance is economic output: could an AI take a $100,000 initial capital and generate a $1 million return, a 10x ROI, within a set timeframe? This test captures real-world planning, resource allocation, and execution. He notes that we 'breezed past the cheuring test' without celebration, and AI capabilities are advancing so quickly we've become desensitized. He believes this economic test will be passed in a couple of years and will redefine how we value AI systems. It also aligns with his view that the ultimate product of AI companies might be certified agents that perform specific tasks with reliability and trust.

Suleyman outlines a future where the dominant interaction model is conversational agents and companions that have full context and can perform any task. He compares it to how software engineers already use coding assistants to debug and generate large amounts of code, replacing third-party libraries. This is a paradigm shift that will affect LinkedIn, M365, Windows, search, and gaming. He argues that the traditional GUI with apps and browsers will be replaced by a persistent assistant that becomes a 'real assistant in your pocket 24/7.' Microsoft is structuring itself to lead this transition, with Copilot as the unifying interface. The implication is that the value moves from the interface layer to the agent layer, and the company that provides trusted, reliable agents gains an edge.

Suleyman explains that the drop in inference cost was not something he anticipated, even after years in the field. He details how Inflection built one of the largest H100 clusters, only to see open-source models (like Meta's Llama) release around the same time as ChatGPT, drastically lowering the cost of performance. He says the inference cost per token has come down approximately 100x in two years (some estimates put it at 1000x for certain model sizes). This hyperdeflation means intelligence is becoming a commodity with near-zero marginal cost. He now sees this as both an enormous opportunity for widespread access and a disruptive force that will affect labor markets before the cost of services drops, creating a destabilizing transition mismatch. He acknowledges his error publicly, calling it 'the biggest surprise for me.'

Suleyman distinguishes AGI and superintelligence as points on a curve. He defines superintelligence as an AI that can perform all tasks better than all humans combined and can self-improve. He says it's very hard to judge timeline, but whether it's 1 year or 20, the urgency is the same: we need to declare what kind of superintelligence we will build and whether we will create an entity we provably cannot align or contain. He emphasizes that the bar for safety must be extremely high, requiring mathematical provability. This stance is a call to action for the industry to focus on defensive co-scaling of safety measures.