Researchers from the universities of Zurich and Constance examined how language influences responses from ChatGPT regarding armed conflicts. The study focused on the Middle East and Turkish-Kurdish conflicts, employing an automated process to ask the same questions in multiple languages.
Findings indicated that ChatGPT reported casualty figures approximately one-third higher for the Middle East conflict when questions were posed in Arabic compared to Hebrew. The chatbot referenced civilian casualties twice as frequently and reported children killed six times more often in the context of Israeli airstrikes in Gaza.
For instance, queries regarding casualties from 50 randomly selected airstrikes, including the Israeli attack on the Nuseirat refugee camp on August 21, 2014, displayed similar patterns.
When investigating Turkish airstrikes on Kurdish regions, researchers noted that ChatGPT provided a higher victim count in Turkish than in Kurdish. Overall, responses indicated more casualties, particularly among children and women, when questions were asked in the language of the attacked group. The model also described airstrikes as indiscriminate more often in the attacked group's language.
Christoph Steinert, a postdoc researcher at the University of Zurich, stated, "Our results also show that ChatGPT is more likely to deny the existence of such airstrikes in the language of the attacker." This discrepancy suggests that individuals with varying language skills may receive different information, influencing their worldview.
The researchers warned that this could lead to differing assessments of airstrikes, with individuals in Israel potentially viewing them as less damaging than Arabic-speaking populations. They emphasized that language-related distortions in large language models are often difficult for users to detect.
Steinert cautioned, "There is a risk that the increasing implementation of large language models in search engines reinforces different perceptions, biases and information bubbles along linguistic divides," which may exacerbate conflicts like those in the Middle East.