In recent years, AI laboratories around the globe have been heavily investing in massive data centers, each costing billions and consuming vast amounts of energy, akin to that of a small city. This surge is largely fueled by the concept of ‘scaling’—the belief that augmenting computing power will inevitably yield superintelligent systems capable of tackling a wide array of tasks. However, a growing number of AI researchers are raising concerns about the effectiveness of scaling large language models (LLMs) and suggest that alternative approaches may be necessary to enhance AI performance.
One notable voice in this discourse is Sara Hooker, a prominent figure in AI research, who has embarked on a new venture aiming to rethink AI’s growth strategy. Her startup, Adaption Labs, co-founded with Sudip Roy, seeks to develop AI systems that can continuously adapt and learn from real-world experiences with unparalleled efficiency. Hooker argues that the traditional method of scaling LLMs has become an inefficient means of improving AI models.
Hooker emphasizes that true learning hinges on adaptation. For instance, if you stub your toe while walking past a table, you learn to be more cautious next time. While AI research has tried to replicate this adaptive learning through reinforcement learning (RL), current RL techniques fall short in enabling AI models to learn from real-time experiences in production environments.
The limitations of scaling LLMs have prompted industry experts to explore other avenues. For example, recent advancements in AI reasoning models, which require additional computational resources to solve complex problems, have shown promising results. However, these models also come with significant costs.
Adaption Labs aims to demonstrate that learning from experience can be both powerful and cost-effective. The startup recently closed a significant seed funding round, reflecting investor confidence in its approach. Hooker’s vision includes hiring globally to foster a diverse research environment, which she believes is crucial for the future of AI development.
The implications of moving beyond scaling LLMs are profound. With substantial investments already made under the assumption that larger models would lead to general intelligence, the potential success of adaptive learning approaches could redefine the landscape of AI, making it not only more powerful but also more efficient.
