TLDR: This research paper details the successful real-world deployment of AI-assisted evaluation systems, powered by Large Language Models (LLMs) like GPT-4o, in two Portuguese government initiatives. These systems significantly improved efficiency and reduced workload in reviewing public funding applications for corporate international expansion and citizen energy-efficient home improvement claims, demonstrating a 20.1% increase in reviewer productivity for the latter and an overall reduction of over two months in evaluation time, while maintaining human oversight. The study highlights the practical potential of AI in public administration, alongside challenges related to regulation, integration, and user adoption.
Public funding programs across the European Union are vital for economic growth, sustainability, and innovation, but they often face a significant challenge: an overwhelming number of applications. This high volume can lead to bottlenecks, delays, and dissatisfaction among applicants. A recent research paper explores how Large Language Models (LLMs) are being leveraged to tackle this issue, streamlining the review process and enhancing efficiency in real-world government initiatives.
The paper, titled “Leveraging LLMs to Streamline the Review of Public Funding Applications,” details the deployment of AI-assisted evaluation systems in two distinct programs in Portugal. These initiatives include corporate applications for international business expansion and citizen reimbursement claims for investments in energy-efficient home improvements. Despite their different evaluation procedures, the findings consistently show that AI effectively boosts processing efficiency and reduces the workload for human reviewers.
AI in Action: Two Case Studies
In the first initiative, focusing on corporate applications for international business expansion (IExp), the main goal was to produce high-quality summaries of lengthy applications, which often span over 50 pages. Manual summarization was identified as a major bottleneck. The AI system, utilizing GPT-4o, was designed to automate six time-consuming tasks, including summarization, detecting internal inconsistencies, and assigning preliminary scores. By providing the LLM with only the most relevant sections of the application, identified by human experts, the system not only improved accuracy but also cost-efficiency.
The second initiative, known as ReClaim, addressed citizen reimbursement claims for energy-efficient home improvements. This program received approximately 80,000 applications, each with an average of eleven supporting documents. The challenge here was not subjective interpretation but managing the vast variability in document formats and ensuring consistency between claimed expenses and supporting evidence. The ReClaim solution employs a hybrid pipeline combining classical document parsing (like Optical Character Recognition) with Visual Language Models (VLMs) such as GPT-4o. It extracts key details from unstructured documents and performs automated consistency checks, flagging only discrepancies for manual review. This significantly reduces the need for human intervention in routine verifications.
Tangible Benefits and Human Oversight
The deployment of these systems has yielded impressive quantitative and qualitative improvements. In the citizen reimbursement claims initiative, reviewer productivity increased by 20.1%, leading to an overall reduction of more than two months in the total evaluation time. The system also enabled reviewers to skip manual validation in about 76% of field verifications. Furthermore, both initiatives saw a decrease in clarification requests sent to applicants and a lower applicant appeal rate, suggesting fewer human errors and greater consistency in assessments.
Crucially, the approach emphasizes a “human-in-the-loop” design. LLMs serve as supportive tools, providing recommendations, while human reviewers retain ultimate oversight and accountability. This mitigates concerns about AI reliability and potential biases, ensuring that critical decisions remain under human control.
Also Read:
- AI’s Role in Software Teams: Boosting Collaboration and Productivity
- Software Project Management in the GenAI Era: A Look at Practitioner Perspectives
Challenges and Lessons Learned
The real-world deployment also highlighted several challenges. Organizational and regulatory barriers, such as strict GDPR requirements and complex authorization workflows, often caused delays and limited technical options. Integrating the AI solutions into existing workflows, especially when platforms were managed by external providers, also proved difficult. Moreover, reviewer adoption varied, with some embracing the tool readily while others became cautious after encountering minor errors. The paper stresses the importance of effective change management and setting realistic expectations for AI capabilities to foster broader acceptance.
Despite these hurdles, the project underscores the high practical application potential of AI-assisted evaluation systems. The ReClaim system, for instance, processed approximately 80,000 applications in less than three weeks, demonstrating the immense speedup possible in parts of the evaluation process. As LLMs continue to advance, the potential for even more dramatic efficiencies in public sector workflows is significant.
For more detailed information, you can read the full research paper here.
