TLDR: A new AI method called VS-LLM automates depression assessment using drawing projection tests, specifically “a person picking an apple from a tree” (PPAT) sketches. It analyzes both visual drawing details and generates psychological captions using a Large Language Model (LLM). This approach significantly improves accuracy by 17.6% compared to traditional psychologist assessments, offering a scalable and less biased method for mental health screening.
The Drawing Projection Test (DPT) is a valuable tool in art therapy, allowing psychologists to gain insights into a person’s mental state through their drawings. One specific task, known as “a person picking an apple from a tree (PPAT),” can reveal indicators of mental states like depression. While DPT offers a richer understanding compared to standard questionnaires, its interpretation is often time-consuming and heavily relies on the individual experience of psychologists, making large-scale application challenging.
To overcome these limitations, researchers have developed an innovative method called Visual-Semantic Depression Assessment based on LLM (VS-LLM). This AI-powered system aims to support psychologists by enabling large-scale, automatic assessment of DPT sketches. Unlike traditional sketch recognition, VS-LLM focuses on a holistic evaluation, considering aspects such as color usage and space utilization within the drawing.
The VS-LLM system is designed with three main components. First, the Visual Perception Module meticulously analyzes the drawing process by breaking down the sketch into sequences of strokes. This allows for a more detailed understanding of how the drawing was created. Second, the Mental Semantic Caption Generation Module uses a Large Language Model (LLM), specifically Qwen-VL, to generate psychological captions. These captions are created by providing the LLM with tailored prompts that emphasize key psychological elements like the use of colors and how space is utilized in the drawing. Finally, the Mental Classification Module integrates both the visual and semantic information to classify the mental state, such as identifying depression.
To validate the effectiveness of VS-LLM, a dedicated PPAT dataset was constructed, comprising 690 sketches. The participants also completed the Patient Health Questionnaire (PHQ-9) scale, which served as the ground truth for depression assessment. The sketches were further evaluated by psychologists across 14 dimensions, providing a comprehensive human assessment baseline.
The experimental results demonstrate a significant improvement in accuracy with the VS-LLM method. It achieved an accuracy of 87.8% in depression assessment, which is a remarkable 17.6% improvement compared to the best psychologist assessment method (Random Forest, which achieved 70.2%). This superior performance is attributed to the AI’s ability to extract richer visual features from the sketches and integrate detailed psychological semantic descriptions generated by the LLM, overcoming potential biases and simplifications inherent in human-only evaluations.
Also Read:
- Advancing Depression Assessment: A New Dataset and AI Reasoning Approach
- TalkDep: Creating Realistic AI Patients for Depression Screening Training
This research not only provides an experimental environment for automated analysis of PPAT sketches but also confirms the crucial role of incorporating mental descriptions assisted by LLMs in mental state assessment. The success of VS-LLM suggests a promising future for automated depression assessment, offering a scalable and objective approach to support mental health professionals. Future work will explore assessing other mental states like anxiety and developing a comprehensive platform for both researchers and patients. For more details, you can read the full research paper here.
