The Unfolding Utility of Intelligence: A Conversation with Sam Altman

The landscape of startups and technological progress is in constant flux, and few individuals have navigated these shifts as profoundly as Sam Altman. In a recent discussion, Altman reflected on his experiences, the evolution of AI, and the fundamental changes shaping the future of innovation. He emphasized the concept of "scale" as a critical, often underestimated, driver of progress and explored how the very nature of building and deploying technology is being redefined.

The Shifting Sands of Startups

Altman, who previously taught "How to Start a Startup" at Stanford in 2014, noted how dramatically the field has changed. He mused about updating the course, highlighting that the current environment for startups is vastly different from even a few years ago. The advent of powerful AI models has fundamentally altered the equation, enabling startups to achieve what once required massive engineering teams with relatively modest resources.

"What you can take on, the level of ambition you can have, the speed of which you can move, the amount of stuff you can do at once, is just totally different," Altman observed. He pointed out that the ability to leverage AI for tasks like automated coding has democratized capabilities, allowing a small team to accomplish what was previously the domain of hundreds.

Scale as an Emergent Property

A recurring theme in Altman's discourse is the power of scale, a concept he has observed repeatedly throughout his career. He posits that emergent properties often arise when pushing systems to scales previously unexplored, yielding returns far beyond initial expectations. This phenomenon is evident in AI model scaling, but also in collaborative research settings and the economic efficiencies of large organizations.

Altman recalled his time at Y Combinator, where the prevailing wisdom suggested the accelerator had become too large. However, he recognized that the network effects and emergent properties within larger batches of companies were a crucial part of YC's success, a benefit that simply didn't exist at smaller scales.

"Empirically speaking, when you find a time that you can push on, you can push something to a scale people have not tried before and it's already working in some interesting way at the smaller scale. More often than not, that seems to be a good idea," he stated. He also noted that many are hesitant to pursue such scaling, often due to the inherent challenges and the tendency for things to "break" in unpredictable ways.

Navigating Human Systems at Scale

Scaling technology also presents significant challenges in managing human elements. Altman stressed the importance of a clear goal, a well-defined plan, and transparent decision-making processes when organizing people at scale. He highlighted the power of a unified vision, using OpenAI's commitment to scaling deep learning as an example.

"We did not evolve to be good at thinking about exponentials," Altman remarked, explaining that humans often struggle to grasp the implications of exponential growth in scaling laws, revenue, or organizational complexity. He believes that first-principles reasoning is crucial for helping people understand and embrace these exponential trajectories.

From GPT-3 API to ChatGPT: Discovering the Killer App

The journey of OpenAI's models, from GPT-3 to ChatGPT, offers a compelling case study in identifying and scaling a product. Initially, the GPT-3 API, launched in 2020, struggled to gain traction. However, a serendipitous viral moment on Twitter revealed a surprising use case: users were primarily engaging with the API for conversational purposes, not for the intended business applications.

"We saw this a lot like more people were using they couldn't get the API to work for their business, but they were using their API key to just chat," Altman explained. This observation led to the development of ChatGPT, which, despite initial skepticism, became a global phenomenon. The rapid growth and user engagement signaled a "guaranteed hit," prompting OpenAI to rapidly scale the product and its underlying models.

The Promise and Evolution of Code Generation

The focus then shifted to Codex, OpenAI's code generation model. Altman revealed that the initial plan was to "go all in on code" before the breakthrough with ChatGPT. The belief was that code generation and robotics would be the key modalities for AI to interact with and control both digital and physical environments.

"Our kind of internal belief at the time was that coding was how these models would control things on computers and robots were how these models would control things in the physical world," he said. While the current pipeline for AI development, involving pre-training, mid-training, post-training, and reinforcement learning, has been effective, Altman anticipates a significant rewrite of this pipeline in the future.

Intelligence as the Next Utility

Altman drew a powerful analogy between the current development of AI and the advent of electricity as a utility. Just as early electricity companies didn't initially sell "electricity" but rather "light at night," Altman believes that the way intelligence is marketed and understood will need to evolve.

"I suspect that even if we're totally right and intelligence is going to become this new utility that every company, every customer, every government just needs access to... I kind of don't think at least right now the right way for us to analogize that is we're selling intelligence," he stated. The challenge lies in finding the right analogy to convey the value of accessible, on-demand intelligence to the broader public.

He further elaborated on the concept of utility, distinguishing between compute as a utility and tokens as a utility. For consumers and businesses, the focus will likely be on the latter – abstracting away the underlying hardware and focusing on accessibility, cost, and performance.

The One-Person Frontier Lab and the Future of Education

Reflecting on the "one-person frontier lab" project for students, Altman suggested focusing on the "inference part of the stack" – making intelligence cheap and abundant. He believes this area is currently underinvested in, despite the rapid advancements in model training.

Altman also expressed concern about the pace of adaptation in the education system. He predicted that the widespread availability of tools like ChatGPT would force a rapid redesign of curricula, emphasizing critical thinking and AI-augmented learning. However, he admitted to a "prediction error," noting the lack of significant systemic change in education since ChatGPT's launch.

The Exponential Trajectory of AI and Societal Repercussions

When asked about his "spiciest take," Altman reiterated his belief in the continued exponential progress of AI. He argued that if this trajectory holds, the world and society's capabilities will be "completely different" in the coming years.

He also addressed the debate around Large Language Models (LLMs) being a "dead end," refuting the notion by highlighting their ability to discover new knowledge and surpass human capabilities in certain domains. He criticized the field's past limitations, attributing them to a generation of scientists who were overly certain about what scaling would not produce.

Altman outlined two major "forks" for the future:

  1. Democratization vs. Concentration: The technology could become widely democratized, or it could remain concentrated in a few powerful companies. He strongly advocates for the former, viewing it as essential for a fair and representative future, despite potential safety and stability arguments for concentration.
  2. Compute Distribution: The equitable distribution of compute resources will be a critical economic challenge. As compute becomes an essential utility, its pricing and accessibility will shape future economic models, potentially necessitating new approaches beyond traditional capitalism or communism.

He emphasized that while solutions like universal basic income and universal ownership stakes are discussed, the specific challenge of distributing compute equitably is often overlooked.

Key Takeaways