Insiders in the tech industry are beginning to acknowledge that the growth of large language models (LLMs) is slowing, despite increased data and computing power.
In Silicon Valley, there is a growing belief that advancements in artificial intelligence (AI) are not keeping pace with earlier expectations, particularly after the enthusiastic launch of ChatGPT two years ago. Initially, proponents believed that exponential growth would follow as tech giants invested heavily in resources.
However, experts are now warning that relying solely on more data and computing power may not lead to the anticipated emergence of artificial general intelligence (AGI). Gary Marcus, a prominent AI critic, stated, "The astronomical valuations of companies like OpenAI and Microsoft are largely based on the idea that LLMs will become AGI if they continue to expand. This is just nonsense."
One significant challenge is the limited amount of linguistic data available for training models. Scott Stevenson, head of AI at Spellbook, noted that focusing exclusively on language data will inevitably lead to stagnation.
Sasha Luccioni from Hugging Face argued that the industry's obsession with size over purpose is a fundamental flaw. She remarked, "The pursuit of AGI has always been unrealistic, and the 'bigger is better' approach had to hit a limit, which I think we are seeing now."
Despite these concerns, some in the industry maintain that progress towards human-level AI remains unpredictable. OpenAI's CEO, Sam Altman, asserted, "There is no wall," without elaborating on his statement.
OpenAI has recently postponed the release of its anticipated GPT-4 successor due to unmet expectations regarding its capabilities. The company is now focusing on optimizing existing abilities rather than merely increasing data input.
Stevenson highlighted that OpenAI's shift towards enhancing reasoning rather than just data expansion has led to significant improvements. He compared the evolution of AI to the discovery of fire, suggesting that it's time to utilize advancements for specific tasks rather than simply adding fuel.