TLDR: A new framework, EK-ICL, improves early Alzheimer’s Disease detection using large language models (LLMs) in data-scarce and unfamiliar scenarios. It integrates explicit knowledge through SLM-derived confidence scores, parsing feature scores for better example selection, and “Good/Bad” label word replacement to align with LLM understanding. This multi-component approach, combined with parsing-based retrieval and ensemble prediction, significantly outperforms existing methods by ensuring better task and label alignment.
Detecting Alzheimer’s Disease (AD) early is crucial for effective intervention, but traditional diagnostic methods are often invasive and costly. Non-invasive, text-based approaches, which analyze linguistic patterns in narrative transcripts, offer a promising alternative. However, using large language models (LLMs) for this task, especially in situations with limited data or unfamiliar patterns (out-of-distribution scenarios), has been challenging.
Existing methods for in-context learning (ICL), a technique that allows LLMs to adapt to new tasks with minimal data without updating their parameters, often struggle with AD detection. These struggles stem from issues like failing to recognize the task correctly, selecting suboptimal examples (demonstrations), and a mismatch between the label words used (e.g., “Alzheimer” or “Control”) and the actual task objectives. This is particularly problematic in clinical fields like AD detection, where the nuances of language are critical.
Researchers have introduced a new framework called Explicit Knowledge In-Context Learners (EK-ICL) to address these challenges. EK-ICL aims to improve the stability of reasoning and task alignment in ICL by integrating structured explicit knowledge. This framework incorporates three key knowledge components:
Confidence Scores from Small Language Models (SLMs)
EK-ICL uses confidence scores derived from smaller language models (SLMs) to help ground predictions in patterns that are highly relevant to the AD detection task. These scores provide reliable insights into predictions and help stabilize the LLM’s task recognition, especially when dealing with complex clinical narratives.
Parsing Feature Scores for Better Demo Selection
Another component is parsing feature scores, which capture the structural differences in language patterns between different transcripts. Unlike traditional ICL methods that rely on semantic similarity, EK-ICL uses these parsing features to improve the selection of “demonstrations” (examples) for the LLM. This ensures that the selected examples highlight meaningful structural distinctions between AD and control participants, rather than just superficial semantic similarities, which are common in AD transcripts where both groups might describe the same picture.
Also Read:
- Enhancing Language Models for Early Alzheimer’s Detection
- Boosting LLM Performance with Implicit Federated In-Context Learning
Label Word Replacement for Semantic Alignment
The framework also tackles the issue of semantic misalignment between label words and the LLM’s pre-trained knowledge. Instead of using conventional labels like “Alzheimer” or “Control,” EK-ICL replaces them with “in-distribution” labels such as “Good” or “Bad.” This alignment helps LLMs better leverage their existing understanding of universal concepts, preventing performance drops caused by mismatched label semantics in new or unfamiliar scenarios.
Beyond these core knowledge components, EK-ICL employs a parsing-based retrieval strategy for selecting demonstrations. This strategy prioritizes structural similarity over semantic similarity, which is crucial given the semantic homogeneity often found in AD transcripts. Additionally, an ensemble prediction method is used, where predictions from multiple learners are aggregated through majority voting. This helps to mitigate label conflicts and further stabilize the overall predictions.
Extensive experiments were conducted across three AD datasets: ADReSS challenge, Lu, and Pitt corpora. The results showed that EK-ICL significantly outperformed state-of-the-art fine-tuning and existing ICL baselines. For instance, EK-ICL achieved 93.75% accuracy on the Test dataset, demonstrating its effectiveness. Further analysis revealed that the performance of ICL in AD detection is highly sensitive to how well label semantics align with task-specific context, emphasizing the critical role of explicit knowledge in clinical reasoning, especially when data resources are limited.
The ablation study, which involved removing individual components of EK-ICL, confirmed the importance of each part. Removing confidence scores led to a dramatic drop in accuracy, highlighting their foundational role. Parsing-based feature scores and search also provided critical benefits, improving the model’s ability to capture fine-grained structural cues and enhancing demo retrieval quality. The study also underscored that even with optimal explicit knowledge integration, model performance is highly sensitive to the choice and alignment of label words, with “Good/Bad” proving to be the most effective pair.
In conclusion, EK-ICL represents a significant advancement in using LLMs for early AD detection. By explicitly injecting structured knowledge—through confidence scores, parsing feature scores, and strategic label word replacement—it effectively addresses the challenges of task recognition failure and suboptimal demonstration selection in data-scarce and out-of-distribution clinical settings. This work highlights that robust AD detection requires a joint alignment across label, context, and structure, which cannot be achieved by any single knowledge source alone. For more details, you can refer to the full research paper: Explicit Knowledge-Guided In-Context Learning for Early Detection of Alzheimer’s Disease.
