TLDR: The research introduces Targeted Persuasion Score (TPS), a new metric based on Wasserstein distance, to measure how effectively a context shifts a language model’s answer distribution towards a specific target. Unlike previous methods, TPS accounts for the direction and magnitude of change, using a flexible cost function to define answer relationships. Experiments reveal subtle model behaviors, such as negative contexts being more persuasive and a “lost-in-the-middle” effect where contradictory information at the start or end of a context has a greater impact. TPS is a valuable tool for understanding and controlling how contexts influence language models.
Language models (LMs) are incredibly versatile, capable of recalling vast amounts of pre-trained knowledge and adapting to new information provided in a prompt. But how do we truly measure how much a piece of information, or “context,” actually sways a language model’s answer? A new research paper titled “How Persuasive is Your Context?” by Tu Nguyen, Kevin Du, Alexander Miserlis Hoyle, and Ryan Cotterell introduces a novel metric called the Targeted Persuasion Score (TPS) to provide a more nuanced understanding of this phenomenon.
Traditionally, evaluating how a context influences an LM often involved simply checking if the model’s top answer changed. While straightforward, this method misses subtle shifts in the model’s underlying probability distribution over all possible answers. Another approach, using KL divergence, could detect these distribution changes but didn’t indicate the direction of the change – whether the model was moving closer to or further away from a desired target answer.
Introducing the Targeted Persuasion Score (TPS)
The Targeted Persuasion Score (TPS) addresses these limitations by leveraging the Wasserstein distance, a mathematical concept that measures the “cost” of transforming one probability distribution into another. In the context of LMs, TPS quantifies how much a given context shifts the model’s original answer distribution towards a specific, user-defined target distribution. A positive TPS indicates that the context successfully moved the model closer to the target, while a negative score means it moved further away.
What makes TPS particularly powerful is its flexibility through a “cost function.” This function allows researchers to define the relationships between different possible answers. For instance, if answers are numerical ratings, the cost function can reflect how far apart two numbers are. If answers are natural language phrases, it can use semantic similarity (e.g., “lovely” and “great” are closer than “lovely” and “meh”).
TPS in Action: Uncovering Hidden Model Behaviors
The researchers demonstrated TPS through several compelling case studies:
First, using a simplified version called BasicTPS, they showed that while it aligns with instances where the model’s top answer changes, it also reveals cases where a context significantly influences the model’s probability for a correct answer, even if that answer isn’t the top choice.
Second, in a word-sense disambiguation task, Distance-based TPS (which incorporates semantic similarity) proved superior to BasicTPS. It accurately reflected how contexts persuaded the model towards semantically similar word senses, a detail missed by metrics that treat all incorrect answers as equally wrong.
Third, in a more realistic scenario of rating movies based on reviews, Distance-based TPS revealed intriguing patterns. For example, a small number of negative reviews were found to be more persuasive than positive ones for the Qwen-2.5 7B Instruct model. This effect, however, diminished as the number of reviews increased.
Fourth, and perhaps one of the most significant findings, was the discovery of a “lost-in-the-middle” effect. When a contradictory review was placed within a longer context of positive reviews, the Distance-based TPS showed that the model was influenced significantly more if the contradictory review was at the beginning or end of the context, rather than in the middle. This crucial insight was completely invisible when only looking at the greedily decoded answers.
Also Read:
- Improving LLM Problem Solving with Guided Pivotal Optimization
- DivLogicEval: A New Benchmark Uncovers Gaps in LLM Logical Reasoning
Real-World Applications: Understanding LLM Annotation
The paper also explored TPS in an applied setting: automated text annotation for social sciences. Researchers often adapt human-designed “codebooks” (detailed instructions and definitions) into prompts for language models. Using TPS, the study measured how well these technical definitions or few-shot examples persuaded an LM to align with expert annotations on a political left-to-right scale. The results showed that while technical definitions had a minor impact, random few-shot examples could sometimes even push the model away from expert consensus, highlighting the inconsistent influence of different prompting strategies.
While TPS offers a powerful new lens for understanding language model behavior, the authors acknowledge some limitations. It currently requires a finite answer space, making it challenging for truly open-ended text generation tasks. However, the potential for TPS to shed light on how LMs integrate context and prior knowledge, especially in areas like retrieval-augmented generation and in-context learning, is immense.
To delve deeper into the technical details and experimental results, you can read the full research paper: How Persuasive is Your Context?
