TLDR: EXAONE 4.0 is LG AI Research’s latest large language model, uniquely combining rapid non-reasoning and deep reasoning capabilities within a single model. It features enhanced agentic tool use, expanded multilingual support (English, Korean, Spanish), and comes in 32B and 1.2B sizes. Built with a hybrid attention mechanism and extensive pre-training, EXAONE 4.0 demonstrates strong performance in complex tasks like math, coding, and long-context understanding, outperforming many open-weight models and competing with frontier-class models.
LG AI Research has unveiled its latest breakthrough in artificial intelligence, EXAONE 4.0, a new series of large language models designed to bridge the gap between rapid, intuitive responses and deep, analytical reasoning. This innovative model integrates both a “NON-REASONING” mode for quick thinking and a “REASONING” mode for more accurate, in-depth problem-solving, offering users a versatile and powerful AI experience within a single framework.
A Unified Approach to AI Capabilities
Building upon the strengths of its predecessors, EXAONE 3.5 (known for its usability) and EXAONE Deep (recognized for its advanced reasoning), EXAONE 4.0 aims to usher in the era of agentic AI. A key feature is its enhanced ability for agentic tool use, allowing the model to seamlessly integrate and utilize various external tools to develop sophisticated AI agents and applications. Furthermore, its multilingual capabilities have been significantly expanded to include Spanish, in addition to its existing support for English and Korean.
The EXAONE 4.0 series comes in two sizes: a robust 32-billion parameter (32B) model optimized for high performance, and a compact 1.2-billion parameter (1.2B) model specifically designed for efficient on-device applications. Initial evaluations show that EXAONE 4.0 delivers superior performance compared to other open-weight models in its class and remains highly competitive even against much larger, frontier-class models.
Under the Hood: Architectural Innovations
Several key architectural changes contribute to EXAONE 4.0’s advanced capabilities. Unlike previous versions that relied solely on global attention, EXAONE 4.0 introduces a hybrid attention mechanism. This combines local attention (a sliding window approach) with global attention in a 3:1 ratio, allowing the model to efficiently process significantly longer contexts—up to 128,000 tokens for the 32B model and 64,000 tokens for the 1.2B model. This innovation helps manage the computational demands of long-context processing without sacrificing performance.
Another notable modification is the repositioning of layer normalization, adopting the QK-Reorder-LN method. This technique, which applies normalization after input queries and keys and again after attention output, has been shown to improve performance on various tasks. The model also benefits from a substantial increase in pre-training data, with the 32B model utilizing 14 trillion tokens (nearly double that of EXAONE 3.5) to enhance its world knowledge and reasoning abilities.
Training for Intelligence and Versatility
The development of EXAONE 4.0 involved a meticulous multi-stage post-training pipeline. This includes large-scale supervised fine-tuning (SFT) across diverse domains like world knowledge, math, coding, agentic tool use, and multilingual tasks. A unique aspect is the unified training of both non-reasoning and reasoning modes, carefully balancing their data ratios to ensure optimal behavior in each mode.
To further refine its reasoning prowess, EXAONE 4.0 employs an advanced reinforcement learning (RL) algorithm called AGAPO. This algorithm, an improvement over existing methods, uses verifiable rewards to enhance accuracy in complex domains like mathematics and coding. Following RL, a preference learning phase with a hybrid reward mechanism ensures the model aligns with human preferences, balancing correctness, conciseness, and language consistency.
Impressive Performance Across the Board
Evaluations across a wide range of benchmarks confirm EXAONE 4.0’s strong performance. It particularly excels in mathematical and coding challenges, often outperforming larger competitors. In agentic tool use scenarios, both model sizes show competitive results, highlighting their readiness for the agentic AI era. The models also demonstrate strong world knowledge, achieving high scores in benchmarks like GPQA-DIAMOND. Furthermore, EXAONE 4.0 showcases commendable performance in its supported languages: English, Korean, and Spanish.
Even when operating with a reduced “reasoning budget” (fewer tokens for the reasoning process), EXAONE 4.0 maintains competitive performance, demonstrating its efficiency and robustness.
Also Read:
- LG AI Research and FriendliAI Collaborate to Broaden Access to Advanced EXAONE 4.0 AI Model
- Athena: Boosting LLM Accuracy Through Seamless External Tool Integration
Looking Ahead
While EXAONE 4.0 represents a significant leap forward, LG AI Research acknowledges the inherent limitations of current language models, such as the potential for generating inappropriate or biased responses. The team emphasizes ongoing efforts to mitigate these risks and encourages ethical use of the model. For those interested in exploring the technical details further, the full research paper is available here.
LG AI Research continues its commitment to expanding the research ecosystem by making its models publicly available in an open-weight format, fostering continuous improvement based on user feedback.
