TLDR: LinkedIn has developed an internal Text-to-SQL chatbot designed to make enterprise data analytics accessible to all employees, regardless of their SQL expertise. The solution features a dynamic knowledge graph for semantic understanding, an intelligent Text-to-SQL agent for accurate query generation and error correction, and an interactive multi-agent user interface that supports diverse user intents. This comprehensive approach has significantly improved data accessibility and user satisfaction, demonstrating a practical framework for deploying Text-to-SQL in large enterprise environments.
In the world of enterprise data, getting insights often means navigating complex databases and writing intricate SQL queries. This can be a significant barrier for many employees, especially those without a strong technical background. A new research paper, “Text-to-SQL for Enterprise Data Analytics,” sheds light on how LinkedIn has tackled this challenge by building an advanced internal chatbot.
The paper, authored by Albert Chen, Manas Bundele, Gaurav Ahlawat, Patrick Stetz, Zhitao Wang, Qiang Fei, Donghoon Jung, Audrey Chu, Bharadwaj Jayaraman, Ayushi Panth, Yatin Arora, Sourav Jain, Renjith Varma, Alexey Ilin, Iuliia Melnychuk, Chelsea Chueh, Joyan Sil, and Xiaofeng Wang, all from LinkedIn, details their comprehensive approach to making data accessible to everyone in the company.
Bridging the Gap Between Natural Language and SQL
While large language models (LLMs) have shown impressive progress in converting text to SQL on benchmarks, deploying a robust solution in a real-world enterprise setting presents unique hurdles. LinkedIn’s data lake, for instance, contains millions of tables, some with over a hundred columns, and is constantly evolving. Users also expect personalized responses and assistance with various data-related tasks beyond just query writing.
To address these complexities, LinkedIn’s solution is built on three core components:
1. A Dynamic Knowledge Graph
Understanding the specific meaning of data within a company is crucial. LinkedIn constructed a knowledge graph that acts as a central repository of up-to-date semantic information. This graph indexes everything from database metadata and historical query logs to internal wikis and code. It even applies clustering techniques to identify relevant tables for different teams or product areas, ensuring that the chatbot provides contextually appropriate information. This graph is regularly refreshed to keep pace with changes in the data landscape.
2. An Intelligent Text-to-SQL Agent
At the heart of the system is the Query Writer Agent, responsible for generating accurate SQL queries. This agent employs a multi-stage process: first, it retrieves and ranks relevant context from the knowledge graph, identifying the most useful tables, columns, and example queries. Then, it writes the SQL query. Crucially, it includes an automatic correction mechanism to fix common issues like ‘hallucinations’ (making up non-existent tables or columns) and syntax errors. A ‘Researcher LLM Agent’ within this component can even search for additional information to resolve these issues, making the query generation process highly robust.
3. An Interactive Multi-Agent Chatbot
The user experience is paramount. LinkedIn designed an interactive chatbot that goes beyond just translating text to SQL. It supports a variety of user intents, from discovering data to writing queries and even debugging them. The chatbot uses a multi-agent architecture, routing questions to specialized agents (e.g., for query writing, data finding, or query fixing) to ensure high-quality and timely responses. The user interface displays responses in rich, interactive elements, such as generated queries with explanations, suggested tables, and quick reply buttons, encouraging further interaction and making data insights easier to consume. Users can even provide feedback and certify example queries, contributing back to the knowledge graph.
Also Read:
- Unpacking LLM Performance in SPARQL Query Generation: The Role of Knowledge and Memorization
- Unlocking Data Connections: The SCHEMORA Framework for Schema Matching
Real-World Impact and Evaluation
LinkedIn’s internal benchmark, created with input from product area experts, showed impressive results. The full system achieved 48% of answers rated as ‘correct or close to correct’ and a 96% successful compilation rate. Ablation studies highlighted the critical role of the knowledge graph components, especially example queries, table clustering, and table/column attributes, in improving semantic understanding and overall query quality. The modeling components, like the query fixer and context rankers, significantly reduced errors and improved recall.
Since its launch in July 2024, the chatbot has garnered over 300 weekly active users, with 33% of chat sessions leading to code being pasted directly into the SQL editor. User surveys indicate high satisfaction, with 39% rating queries as “Very good” or “Excellent.” This demonstrates the practical utility and success of their approach in empowering non-experts to self-serve data insights.
This paper offers valuable insights for any organization looking to build an enterprise-grade Text-to-SQL solution, emphasizing the importance of a robust knowledge graph, intelligent query generation, and a user-friendly interactive experience. You can read the full research paper for more technical details and findings here: Text-to-SQL for Enterprise Data Analytics.
