TLDR: CP-Model-Zoo is a new system that helps non-experts find the right Constraint Programming (CP) models for their problems. Instead of generating code from scratch, which often leads to errors, it uses natural language queries to retrieve the most similar expert-validated CP models from a database. By combining source code with AI-generated descriptions of varying expertise, it achieves high accuracy in matching user problems to existing solutions, acting as an intelligent tutoring system for CP.
Constraint Programming (CP) is a powerful approach for solving complex problems, often referred to as the “holy grail of problem-solving” where a user describes a problem and the computer solves it. However, despite its potential, CP has traditionally been challenging for non-experts due to the complexity of its modeling languages, the vast number of global constraints available, and the nuanced “art” involved in creating effective models.
While the dream of generating expert-level CP models directly from natural language descriptions remains a future goal, a new tutoring system called CP-Model-Zoo offers a pragmatic solution. Developed by Augustin Crespin, Ioannis Kostis, Hélène Verhaeghe, and Pierre Schaus from UCLouvain, this system aims to bridge the gap between a user’s natural language problem description and high-quality, expert-validated CP models.
The core idea behind CP-Model-Zoo is to leverage a vast database of expert-written CP models accumulated over the years through solver competitions, libraries like CSPLib, and system tutorials. When a user provides a natural language description of a combinatorial problem, CP-Model-Zoo retrieves the closest matching source code model from its database. This approach ensures that users are presented with reliable, expert-validated models, eliminating the common issues of “hallucinations” or incorrect code often seen when large language models (LLMs) attempt to generate CP models from scratch.
For instance, the researchers demonstrated how state-of-the-art LLMs like ChatGPT-4o, Gemini 2.5 Pro, and Mistral Lechat Pro struggled with a complex cube puzzle. The generated code contained syntactical errors, misunderstood decision variables, used inconsistent data, and failed to employ efficient global constraints or enforce all necessary rules. Fixing these LLM-generated models proved more time-consuming than starting anew, highlighting the need for a system that provides robust, pre-vetted solutions.
CP-Model-Zoo operates by converting both the user’s query and the models in its database into numerical representations called embeddings. These embeddings are created using a pre-trained text embedding model, such as a modern BERT-based encoder. The system then measures the semantic similarity between the user’s query embedding and the database model embeddings using cosine similarity, returning the top-k most similar models.
A key innovation of CP-Model-Zoo is its use of LLM-augmented descriptions. Instead of relying solely on the raw source code, the system generates synthetic descriptions of each problem at different levels of expertise: Novice (simple explanations), Intermediate (domain-specific terminology), and Expert (formal mathematical formulations). These descriptions are then combined with the source code to create richer embeddings, significantly improving retrieval accuracy.
Experiments conducted on datasets from MiniZinc examples and CSPLib problems demonstrated the system’s effectiveness. The Mean Reciprocal Rank (MRR) metric, which measures the ranking position of the correct model, showed that embeddings incorporating LLM-generated textual descriptions consistently outperformed those based solely on source code. Notably, when queries were at a Novice level, embeddings combining source code with Intermediate-level descriptions achieved a perfect MRR of 1.0, meaning the correct model was ranked first every time. For queries based on human-written CSPLib descriptions, the combination of source code and Intermediate-level descriptions yielded the highest MRR of 0.9051, suggesting that these descriptions strike the best balance between detail and conciseness for real-world problem statements.
The system is designed to be fast and incremental. Adding a new model simply requires generating its natural language description using an LLM and computing its embedding. To make this research accessible, a web-based application prototype has been developed using the Gradio Python framework, allowing users to input problem descriptions and retrieve relevant CP models. This tool, available online, uses the SC+D2 embeddings (source code plus Intermediate-level descriptions) as its knowledge base.
Also Read:
- Unlocking Complex Proofs: BFS-Prover-V2 Advances AI in Formal Mathematics
- CogGuide: A Human-Inspired Approach to Smarter Multimodal AI Reasoning
CP-Model-Zoo represents a significant step towards making Constraint Programming more accessible to a broader audience. By acting as an intelligent tutoring system, it helps users learn and apply CP effectively by providing access to high-quality, expert-validated models. The researchers envision this tool being hosted on platforms like CSPLib, fostering a community effort to continuously expand its database of clean, well-described models. You can find more details about this research in the full paper: CP-Model-Zoo: A Natural Language Query System for Constraint Programming Models.
