Smaller AI Models Suffice For Brain-Language Studies, Says IIIT-H Study

Hyderabad: Larger artificial intelligence (AI) models may not necessarily be better for understanding how the human brain processes language, according to research from the International Institute of Information Technology Hyderabad (IIIT-H).

The researchers found that models with about 3 billion parameters performed almost as well as models with up to 14 billion parameters, potentially reducing the computing power needed for brain research.

Prof. In their study, ‘Linguistic Features and Model Scale in Brain Coding: From Small to Compressed Language Models’, presented at the International Conference on Machine Learning (ICML) in Seoul, Bapi Raju and PhD researcher Vijay Rowtula tested whether increasing the size of a language model necessarily improves its ability to predict human brain activity.

Researchers have studied small language models and made them more lightweight using quantization, which reduces the numerical precision used within a model, and pruning, which removes less important parts. Most of the techniques tested, including quantization and moderate pruning, reduced model size without significantly affecting brain activity estimates.

The findings challenge previous research that suggested moving from smaller to larger models increases brain prediction accuracy by about 15 percent.

However, even though brain alignment remained largely unchanged, compression affected performance on some traditional language tasks, including grammar, discourse, and morphology. Prof. “The new finding is the observed dissociation between brain alignment and linguistic proficiency,” said Raju.

He also suggested that the skills needed to score well on language measures may differ from the representations that are useful for modeling how the brain processes language.

The finding may have practical implications for computational neuroscience. Smaller models require less memory and computing resources, potentially making brain language studies cheaper and faster. Prof. This could be a “game changer” for brain decoding workflows used in the development of brain-computer interfaces, Raju said.

For Rowtula, who moved into computational neuroscience after working in computer vision and industry, the research is part of a broader effort to understand whether computational models can mimic human brain functions.

At ICML, researchers also exchanged ideas with computational neuroscience researchers, including members of the NeuroAI Laboratory at EPFL, opening up possibilities for future collaborations.