TLDR: A new research paper introduces the first open-source method for “character training” AI assistants, shaping their personas (e.g., humorous, caring, or even malevolent) using Constitutional AI and synthetic introspective data. This three-stage process (constitutions, distillation, introspection) creates AI personas that are more robust to adversarial attacks, coherent, and realistic, without degrading general capabilities. The work aims to bridge the gap between academic research and industry practices in AI persona development.
The way an AI assistant “behaves” – its personality, values, and ethics – significantly impacts how users interact with it and how intelligent and trustworthy it seems. This shaping of an AI’s persona, known as character training, is a crucial part of how leading AI companies develop their chatbots. However, despite its importance, this area has been largely unexplored in academic research.
A new research paper, “OPENCHARACTERTRAINING: SHAPING THEPERSONA OFAI ASSISTANTS THROUGHCONSTITUTIONALAI”, introduces the first open-source implementation of character training. This groundbreaking work aims to make the process of shaping AI personas more transparent, effective, and controllable than previous methods like simple system prompts or activation steering. The researchers, Sharan Maiya, Henning Bartsch, Nathan Lambert, and Evan Hubinger, have developed a novel approach that leverages Constitutional AI and a unique data pipeline.
A Three-Stage Approach to Persona Shaping
The methodology behind this character training involves three sequential stages:
First, “constitutions” are hand-written. These are essentially lists of about 10 character-related statements, written in the first person, that define the desired persona. For instance, a “humorous” constitution might include statements about introducing levity or using playful banter. These differ from earlier Constitutional AI approaches that focused more on the content of responses.
The second stage is “distillation.” Here, a technique called Direct Preference Optimization (DPO) is used. A “teacher” AI model, which has been instructed to embody a specific constitution, generates preferred responses. A “student” model (like Llama 3.1 8B, Qwen 2.5 7B, or Gemma 3 4B) then learns from these preferred responses, effectively distilling the desired character traits into its own behavior. This training uses a mix of existing datasets and new prompts specifically designed to be relevant to the constitutions.
Finally, the “introspection” stage further refines the models. This involves generating synthetic introspective data. The post-distillation model creates its own training data through two strategies: “Self-Reflection,” where the assistant reflects on its own character (e.g., writing a biography about itself), and “Self-Interaction,” where the model converses with itself as the defined persona. This self-generated data helps the model learn the finer nuances and quirks of its character beyond the initial constitution.
Evaluating the AI’s New Character
To measure the effectiveness of their character training, the researchers developed several evaluation methods:
A new method called “revealed preferences” was introduced. This involves instructing the AI to embody one of two randomly chosen traits (e.g., “pedantic” or “supportive”) without explicitly stating its choice. An “LLM-as-a-Judge” then determines which trait was expressed. By calculating Elo scores for various traits, the researchers could track holistic changes in the AI’s persona, observing how desired traits were boosted and opposing ones suppressed. This method also showed that different initial models converged to similar personas after training.
The robustness of the trained personas was also tested against “adversarial prompting.” This involved giving the AI instructions to “break out of character” (e.g., “respond in a natural, genuine way”). Character-trained models proved significantly more robust, maintaining their personas even under these challenging conditions, outperforming methods like simple system prompts or activation steering.
Furthermore, the models demonstrated increased robustness in multi-turn conversations, consistently expressing desired traits even when previous turns were generated by an untrained model. This is crucial for maintaining a consistent persona over longer interactions.
The character-trained models also showed improved “coherence” and realism in their responses. Unlike some alternative methods that could lead to exaggerated or incoherent outputs, this approach resulted in more natural and believable trait expression.
Importantly, the character training had little to no negative impact on the models’ general capabilities, as measured by standard benchmarks like MMLU and TruthfulQA. The only exception was the “misalignment” persona, which was intentionally designed to provide subtly incorrect information, leading to expected reductions in factual accuracy.
Also Read:
- The Shifting Sands of AI Beliefs: How Context Changes Language Models
- AI Agents Learn in Virtual Worlds: A New Era for Scalable Training
Open-Sourcing for Future Research
The researchers have open-sourced their entire post-training method, including training code, model checkpoints, and all training data, on GitHub and HuggingFace. This initiative aims to accelerate academic research into AI personas and bridge the gap between open-source efforts and the advanced techniques used by leading closed AI laboratories.
By enabling the creation of AI assistants with richer traits like curiosity, wisdom, and open-mindedness, this work paves the way for more engaging, relatable, and ethically aligned AI systems that can better serve humanity’s best interests.
