Brain Activity Aligns with Language Models During Natural Conversations: A Neural Study

A recent study explored the neural mechanisms underlying natural conversation by recording local field potentials (LFP) from 1910 channels across 39 brain areas in 14 participants undergoing epilepsy monitoring. The participants engaged in free-flowing conversations, and their neural activity was synchronized with transcribed words. The study found that changes in brain activity aligned with the NLP model, with a significant proportion of channels showing correlations. The research utilized a pre-trained GPT-2 model to vectorize word and sentence composition, enabling comparison with neural data. The left hemisphere exhibited more correlated activity than the right. Several brain areas, including the temporal and frontal cortex, thalamus, and limbic system, showed a high percentage of channels correlated to NLP embeddings. The highest ratio of correlated channels was observed in the left precentral cortex during speech production planning and in the left and right superior temporal cortex during comprehension. Comparison with a BERT model showed a significantly higher proportion of correlated channels than chance. The average correlation coefficients decreased when participants were passively involved in pseudo conversation. Real sentences elicited a significantly higher percentage of channels responding compared to jabberwocky. The highest percentage of correlated channels was observed in middle gamma frequencies (70-110 Hz) for both language comprehension and production. Neural activities preferentially aligned with higher network layers for both speech planning and comprehension. These findings reveal a dynamical organization of neural activities that subserve language production and comprehension during natural conversation and harness the use of deep learning models in understanding the neural mechanisms underlying human language.