TLDR: The TAI Scan Tool is a new RAG-based system designed for easy self-assessment of AI systems, focusing on legal compliance with the EU AI Act. It uses minimal user input, features a two-step pre-screening and assessment process, and provides risk-level classification along with relevant legal references, making AI trustworthiness assessment more accessible for organizations, especially SMEs.
The rapid growth of Artificial Intelligence (AI) has brought many benefits, but also new challenges related to ethical use and legal rules. This has highlighted the importance of Trustworthy AI (TAI), which aims to build confidence in AI systems. The European Commission’s High Level Expert Group (HLEG) has defined TAI with three key components: ethical, legal, and robust, along with a set of principles to achieve it. These components ensure AI systems respect human values, follow laws, and are technically sound throughout their entire lifecycle.
However, aligning with these TAI principles can be resource-intensive, especially for smaller organizations and startups. They often focus their resources on developing new AI solutions and may lack the specialized legal or ethical knowledge needed for compliance. To address this, a new tool called the TAI Scan Tool has been developed.
The TAI Scan Tool is designed to help organizations, particularly Small-Medium Enterprises (SMEs), self-assess their AI systems for trustworthiness with minimal effort. This tool is based on the Retrieval Augmented Generation (RAG) framework, which helps it provide accurate and relevant information by drawing from original documents. The current version of the tool specifically focuses on the legal aspects of TAI, with a strong emphasis on helping users comply with the EU Artificial Intelligence Act (AI Act).
Unlike many existing compliance tools that rely on rigid, manually updated rule-based systems, the TAI Scan Tool uses a RAG system. This makes it more adaptable to changes in regulatory frameworks and allows for more nuanced reasoning and guidance. The assessment process is divided into two main steps: a pre-screening phase and a legal TAI assessment phase.
The pre-screening step acts as an initial check, informing users about prohibited or high-risk AI practices and helping them understand if their system falls under the AI Act’s definition of an AI system or a General Purpose AI (GPAI) system. It also serves as a safety measure to prevent the involvement of AI systems with unacceptable properties. Users must pass this pre-screening by confirming their system does not involve prohibited or high-risk practices to proceed to the full legal TAI assessment.
The legal TAI assessment provides a more detailed evaluation. Users provide minimalistic input about their AI system, such as its role (provider or deployer), domain of application, type of AI system, input data type, and intended use. Based on this information, the RAG system determines the AI system’s risk level—categorizing it as Low-Risk, Medium-Risk, High-Risk, or Prohibited—according to the AI Act. Crucially, it also retrieves and presents relevant articles, recitals, and annexes from the AI Act to guide users on their compliance obligations.
The development of the TAI Scan Tool is part of the EU-funded DIGITAL-CSA Europe project, DeployAI, which aims to build the AI-On-Demand Platform (AIoDP) to promote trustworthy European AI solutions. The tool’s design is user-centric, incorporating feedback from stakeholders to ensure it meets their needs for ease of use, minimal input, trusted results, and transparent guidance. Its modular architecture allows for scalability and easy integration into larger AI governance frameworks.
Qualitative evaluations of the tool have shown promising results, accurately predicting risk levels for various scenarios, including prohibited, high-risk, and low-risk systems. The tool consistently retrieves relevant articles from the AI Act, which can be grouped into horizontal obligations, classification rules, and scenario-specific obligations. This indicates that the tool effectively grounds its reasoning in the legal text, helping users understand the specific requirements applicable to their AI systems.
Also Read:
- A RAG Chatbot Enhances Regulatory Compliance for Risk and Quality Assurance
- Navigating the Path to Trustworthy Federated Learning: A Comprehensive Overview
The TAI Scan Tool represents a significant step forward in making TAI self-assessment more accessible and efficient, especially for organizations with limited resources. Future work aims to expand the tool’s knowledge base to include more legal documents and to incorporate ethical and robust TAI components, moving towards a comprehensive assessment solution. For more details, you can refer to the full research paper here.
