TLDR: JT-Safe is a new framework by China Mobile Jiutian Research that aims to make large language models (LLMs) intrinsically safer and more trustworthy. It addresses core issues like hallucination and bias by enhancing pre-training data with “Data with World Context” (DWC) – including metadata like source, time, and safety ratings – and incorporating extensive industrial scenario data. This data-centric approach, combined with multi-stage pre-training and advanced post-training alignment techniques, significantly improves the model’s performance across general, industry, safety, and trustworthiness benchmarks, demonstrating a fundamental shift towards building more reliable AI.
Large Language Models (LLMs) have rapidly transformed various industries, but their widespread adoption faces significant hurdles, primarily concerning their safety and trustworthiness. Issues like generating false information (hallucinations) or producing biased and harmful content are global challenges that researchers are actively trying to solve. While many advancements have focused on fixing these problems after the models are initially trained, a new research paper from China Mobile Jiutian Research suggests a more fundamental approach: addressing these issues right from the very beginning, during the pre-training phase.
The paper, titled JT-SAFE: Intrinsically Enhancing the Safety and Trustworthiness of LLMs, introduces a novel framework called JT-Safe. The core argument is that the roots of LLM unsafety and hallucinations often lie in the pre-training data itself and the way models learn from it. Current pre-training data, despite being vast, can contain factual errors, logical inconsistencies, and biases. More importantly, this data often lacks real-world context, treating information merely as sequences of tokens rather than representations of the world.
Data with World Context (DWC)
To tackle this, the JT-Safe team proposes enhancing pre-training data with what they call “Data with World Context” (DWC). This involves enriching the data with crucial metadata such as the time and location of its creation, the author, primary and secondary categories (e.g., “Physics/Dynamics”), source information (like website authority or journal credibility), dialogue attributes (if it’s a conversation), and even safety-related attributes. By embedding this contextual information, the model gains a more structured understanding of the world, reducing uncertainty during training and inherently improving its safety and trustworthiness.
For example, knowing the publication date helps the model distinguish current from outdated information, and understanding the source’s credibility can influence how the model treats a claim. Safety attributes explicitly guide the model on what is acceptable and what is not, fostering an intrinsic understanding of safe expressions rather than relying solely on post-hoc filtering.
Enhancing Industrial Relevance
Beyond general world context, JT-Safe also places a significant emphasis on incorporating a substantial amount of data reflecting industrial scenarios. This includes knowledge from sectors like energy, agriculture, manufacturing, finance, and healthcare. The goal is to strengthen the model’s understanding of specialized contexts, processes, and knowledge, thereby mitigating hallucinations that arise from insufficient training in these critical areas. This industrial data is meticulously curated, often transformed into structured knowledge graphs and scenario-based dialogues to enhance the model’s cognitive reasoning and application capabilities in vertical domains.
A Multi-Stage Training Approach
The JT-Safe framework employs a multi-stage pre-training process for its JT-Safe-35B model, which consists of 6.2 trillion tokens. The first stage focuses on general high-quality pre-training. The second stage is where the DWC safety-enhancement data is integrated, significantly improving the model’s safety and knowledge-intensive capabilities. The final stage involves long-context annealing, extending the model’s ability to process longer texts.
Activating Potential Through Post-Training
After pre-training, JT-Safe utilizes advanced post-training procedures to further activate the potential of the DWC data. This includes Supervised Fine-tuning (SFT) for general alignment and deep safety alignment, which trains the model not just to refuse unsafe content but also to ensure its internal reasoning processes adhere to safety principles. Reinforcement Learning (RL) is then used to achieve a higher level of trustworthiness alignment, with a sophisticated reward system that incentivizes honesty and awareness of the model’s own knowledge boundaries.
Also Read:
- Measuring the Depth of Belief in AI: How Language Models Internalize New Facts
- Building Reliable AI Agents: A Neuro-Symbolic Framework for Trustworthy Decisions
Impressive Results
Experimental results show that JT-Safe-35B achieves significant performance improvements. Compared to models of similar scale, it demonstrates an average improvement of 1.79% on Safety and Trustworthy evaluation benchmarks. It also outperforms baseline models in general capabilities, and notably, holds a commanding lead in industrial capabilities, showcasing its immense potential for real-world problem-solving. The research highlights a powerful synergy between the DWC data and the post-training techniques, validating that building safety and trustworthiness from the data source up is a highly effective strategy.
In conclusion, the JT-Safe framework represents a significant step towards creating inherently safer, more reliable, and industrially applicable large language models by fundamentally re-thinking the role of pre-training data and its contextual enrichment.
