TLDR: New research introduces Aligned Scoring Rules (ASR) for textual evaluation, ensuring provable truthfulness while also aligning with human preferences. By optimizing proper scoring rules against reference scores (like instructor or LLM-Judge scores), ASR significantly improves alignment compared to previous methods, offering a reliable and interpretable way to score text, especially useful for peer grading.
In the realm of artificial intelligence and data-driven systems, ensuring the quality and truthfulness of information provided by strategic agents is paramount. This is where the concept of “scoring rules” comes into play. Traditionally, scoring rules have been well-established for eliciting numerical information, such as probabilities or means, by comparing a prediction against a ground truth state. A key property of these rules is “properness,” meaning that an agent is incentivized to report their true beliefs to maximize their expected score.
With the rapid advancements in large language models (LLMs), there’s a growing interest in eliciting textual information, which can be far richer and more nuanced than simple numerical predictions. Imagine a peer grading scenario where students provide open-ended reviews of their peers’ homework. While LLMs can evaluate text quality, a significant challenge arises: these language model-generated evaluations often lack provable guarantees like truthfulness, making them susceptible to strategic manipulation. For instance, a student might fabricate comments to get a higher score, even if they don’t reflect their true assessment.
Addressing this, prior work by Wu & Hartline (2024) proposed a method to reduce the complex problem of textual information elicitation to the more understood numerical elicitation problem. This approach leverages LLMs as “oracles” for summarization and question-answering, thereby achieving provable properness for textual elicitation. However, a new challenge emerged: even if a scoring rule is provably proper, it might not align well with human preferences or established scoring rubrics. This misalignment can lead to scores that are technically truthful but don’t feel “right” to human evaluators, such as instructors.
This new research introduces the “Aligned Scoring Rule” (ASR) for text, designed to bridge this gap. The core idea behind ASR is to optimize and minimize the difference (specifically, the mean squared error) between a proper scoring rule and a “reference score,” which could be a human instructor’s score or a score generated by an LLM-as-Judge. By doing so, ASR aims to create a scoring mechanism that is not only provably truthful but also closely reflects human judgment.
The methodology involves optimizing over a specific type of proper scoring rules called “separate scoring rules.” These rules apply a single-dimensional scoring rule to each summary point and then average these individual scores. This framework leads to a convex optimization problem, which can be efficiently solved using algorithms like gradient descent. The paper highlights that this approach also offers interpretability, allowing researchers to identify which rubric points are considered more important for scoring based on the “convexity” of their single-dimensional scoring rules.
Empirical evaluations were conducted using peer grading datasets from undergraduate algorithm classes. The results demonstrate that ASR significantly outperforms previous methods, including non-aligned ElicitationGPT approaches, in terms of aligning with reference scores. This alignment is measured by metrics such as Mean Squared Error (MSE), Pearson correlation, and Spearman rank correlation. The ASR scores showed a nearly-identity linear relationship with the reference scores, indicating a strong fit. Furthermore, the case studies revealed that ASR effectively identifies more important rubric points (e.g., correctness of algorithm logic) by assigning them more “convex” V-shape scoring rules, while less important aspects (e.g., clarity) receive more linear scoring lines.
Also Read:
- Rethinking LLM Evaluation: Why ‘Answer Matching’ Outperforms Multiple Choice
- Balancing AI Values: A New Approach to Multi-Objective Language Model Alignment
In essence, Aligned Textual Scoring Rules offer a robust solution for incentivizing truthful and human-aligned evaluations in textual contexts, particularly valuable for applications like peer grading. This work provides a significant step towards creating more reliable and fair automated assessment systems. You can read the full research paper for more details here.
