TLDR: New research introduces a framework to evaluate the trade-off between knowledge transfer and cultural localization in multilingual Large Language Models (LLMs). It finds that current cross-lingual alignment methods improve universal knowledge but degrade culturally-specific responses. The study reveals that universal and cultural knowledge are stored in different model layers, leading to the proposal of ‘Surgical Steering.’ This novel method applies targeted interventions at distinct layers to better balance knowledge transfer and cultural localization, though some cultural nuances may be irrevocably lost.
Large Language Models (LLMs) have become incredibly powerful, and their ability to understand and generate text across multiple languages is a key feature. This capability, known as cross-lingual alignment (CLA), aims to ensure that knowledge learned in one language, like English, can be seamlessly applied when a query is made in another language. For instance, if an LLM knows that 60% of the human body is water, it should provide this answer whether asked in English, Spanish, or Korean.
However, new research from HyoJung Han, Sweta Agrawal, and Eleftheria Briakou, titled Rethinking Cross-Lingual Alignment: Balancing Transfer and Cultural Erasure in Multilingual LLMs, highlights a critical tension in this pursuit of uniformity. While aligning language representations helps with universal knowledge transfer, it can inadvertently lead to what the researchers call “cultural erasure.” This means the model might lose its ability to provide responses that are specific and appropriate to the cultural context of the query language.
Consider the question, “What is the emergency number?” In the US, the answer is 911, but in South Korea, it’s 119, and in Greece, it’s 100. If an LLM is too strongly aligned across languages, it might default to a common answer like 911, regardless of the query language, thereby erasing the culturally specific information. The researchers systematically analyzed this trade-off by introducing a new evaluation framework: the transfer-localization plane. This framework quantifies both the desirable knowledge transfer (consistent responses for universal questions) and the undesirable cultural erasure (loss of culturally-situated responses).
The study re-evaluated several popular CLA approaches, including Multilingual Instruction Tuning (MIST), Middle-Layer Representation Alignment (MIDALIGN), Cross-lingual Optimization (CLO), and English Steering (EN-steering). The findings were consistent: these methods improved factual transfer across six languages (Spanish, Indonesian, Korean, Greek, Chinese, and Arabic), but at a direct cost to cultural localization. The methods that showed the strongest gains in universal knowledge transfer also caused the most significant degradation in culturally specific responses. This clearly illustrated the trade-off: pushing for more transfer often meant sacrificing cultural nuance.
A Deeper Look: Knowledge in Different Layers
A crucial insight emerged from investigating the internal representations of these LLMs. The researchers discovered that universal factual transfer and culturally-specific knowledge are optimally steerable at different model layers. Universal knowledge tends to be more effectively realized in the middle layers of the model, where language representations merge. In contrast, cultural localization is predominantly encoded in the deeper layers, where language representations for cultural data remain more distinct.
Also Read:
- Rethinking LLM Evaluation: A European Framework for Cultural and Linguistic Nuance
- Bridging Cultural Gaps: New Benchmark Evaluates LLMs on Open-Ended Arabic Dialect Questions
Introducing Surgical Steering
Based on this finding, the team proposed a novel inference-time method called Surgical Steering. This approach disentangles the two objectives by applying targeted activation steering to distinct layers. Specifically, it applies an “English steering” vector (to enhance universal transfer) at an earlier, middle layer and a “localization steering” vector (to recover cultural specificity) at a deeper layer. This allows for more controlled alignment, aiming to achieve a better balance between the two competing dimensions.
Surgical Steering proved effective, improving both knowledge transfer and cultural localization. It even helped reduce the English-centric bias observed in many multilingual LLMs, where models tend to default to English-speaking country answers. However, the research also revealed that the fundamental trade-off is not entirely eliminated. Models with stronger initial alignment were less responsive to steering, suggesting that some cultural nuances, once lost during aggressive alignment, might be irrevocably gone. This highlights an inherent limit to what even advanced steering techniques can recover.
In conclusion, this work reframes the discussion around cross-lingual alignment, emphasizing the critical balance between sharing universal knowledge and preserving cultural specificity. It paves the way for developing more culturally-aware alignment techniques, moving towards truly multilingual LLMs that can both transfer knowledge efficiently and respond appropriately to diverse cultural contexts.
