Learning a language may seem straightforward, as babies worldwide master this task within a few years. However, understanding the processes enabling this learning is far more complex.
While linguists have proposed elaborate theories, recent advancements in machine learning have introduced a new perspective, sparking intense debates among scholars and artificial intelligence (AI) developers.
Language models, such as ChatGPT, are designed to predict words and form coherent sentences based on vast textual databases. However, experts assert that this does not equate to learning a language in the same way humans do.
“Even if they do something that resembles human behavior, they might be doing it for entirely different reasons,” stated Tal Linzen, a computational linguist at New York University, in an interview.
This distinction is not merely semantic. If these models genuinely learn languages, it may necessitate a rethinking of traditional linguistic theories. Conversely, if they are only superficially simulating learning, machine learning may not provide significant insights for linguistics.
Noam Chomsky, a prominent figure in linguistics, has notably criticized this technology. In an opinion piece published in 2023 in The New York Times, Chomsky argued that language models lack relevance for linguistics, claiming they can learn even “impossible languages”—those with grammatical rules that do not exist in any known human language.
This critique was challenged by a group of computational linguists in an innovative study presented at the 2024 conference of the Association for Computational Linguistics.
The work, titled “Mission: Impossible Language Models,” published on the preprint server ArXiv, demonstrated that language models struggle more with learning impossible languages than human languages.
Adele Goldberg, a linguist at Princeton University, praised the study: “It is absolutely timely and important.”
Throughout the 20th century, linguistics evolved from cataloging languages to attempting to understand the universal structure behind them. Chomsky spearheaded this movement by proposing that humans possess an innate and specialized capacity for processing languages. This innate ability would explain why certain grammatical rules never appear in human languages.
According to Chomsky, if language learning were akin to other types of learning, there would be no preference for certain grammatical rules. However, the existence of a specialized system would justify this predisposition.
“It makes no sense to say that humans have a predisposition to learn certain things without recognizing that they also have a predisposition not to learn others,” asserted Tim Hunter, a linguist at the University of California, Los Angeles.
Recent experiments with impossible languages yielded fascinating results. In 2020, Jeff Mitchell and Jeffrey Bowers created three artificial languages with bizarre grammatical rules to test the models' capabilities. The results indicated that the models could learn these languages almost as well as English.
However, in 2023, Julie Kallini, a graduate student at Stanford University, decided to test this hypothesis with modern transformer-based models. Her team created 12 impossible languages, including variations such as inverted sentences or verb agreement rules based on characters positioned four words after the verb.
The models faced challenges in learning these artificial languages, confirming that, while powerful, they are not omnipotent. As expected, they learn patterns closer to human languages more easily.
The results suggest that language models exhibit preferences for certain linguistic patterns, similar but not identical to humans. This opens avenues for new investigations. “That’s what I really like about the paper,” remarked Ryan Nefdt, a cognitive science philosopher. “It opens up so many research possibilities.”
Kallini's team is already working on a follow-up study, informally dubbed “Mission: Impossible 2.” The aim is to explore how alterations in neural networks can enhance or hinder the learning of impossible patterns.
The debate regarding the role of language models in linguistics is far from over, but one thing is certain: the collaboration between humans and machines has the potential to unravel the mysteries of linguistic learning and transform our understanding of humanity's most fundamental capacity: language.