TLDR: OASBuilder is a novel framework that uses large language models and rule-based algorithms to automatically convert unstructured online API documentation (HTML) into standardized, machine-readable OpenAPI Specifications (OAS). It employs a multi-stage pipeline involving scraping, generating partial OAS from both demonstrative examples and descriptive text, and then enhancing the combined specification with missing metadata. This system has been successfully deployed in enterprise settings, significantly reducing manual effort and improving the accessibility of APIs for AI agents and automation tools.
In today’s digital landscape, AI agents and business automation tools are increasingly reliant on external web services. For these systems to interact effectively, they need clear, machine-readable instructions about how to use these services, often provided in a format called OpenAPI Specifications (OAS). However, a significant challenge arises because much of the API information available online is in unstructured, human-readable HTML documentation, making it difficult for automated systems to understand and use.
This is where OASBuilder comes in. Developed by researchers at IBM Research, OASBuilder is a new framework designed to automatically convert these lengthy and varied online API documentation pages into consistent, machine-readable OpenAPI specifications. This innovation is crucial because manually converting this documentation is a time-consuming and error-prone process, especially for the large and complex APIs common in real-world applications.
The Challenge of Unstructured API Documentation
Many API providers, despite the benefits of standardized specifications like OAS, still present their API details as HTML webpages. These pages often lack structural consistency and don’t follow standard conventions, making automated parsing very difficult. Existing attempts to automate this conversion, whether through traditional parsing or direct use of large language models (LLMs), have struggled due to the sheer variability in documentation formats, inconsistent layouts, and the presence of dynamically generated content.
How OASBuilder Works: A Multi-Stage Approach
OASBuilder tackles these challenges with a clever, multi-stage pipeline that combines the power of large language models with rule-based algorithms. The process is broken down into smaller, more manageable sub-tasks:
- Scraping and Segmentation: First, OASBuilder “scrapes” the API documentation webpage. It uses tools like Selenium to navigate and interact with the page, identifying key elements like API operation signatures (e.g., “GET /status”) and usage examples (like cURL commands). It then segments the long webpage into smaller, continuous sections, each corresponding to a specific API operation.
- Demonstrative OAS Generation: For each identified operation, OASBuilder extracts “demonstrative examples” such as sample requests and responses. It then uses parallel LLM calls to convert these examples into a partial OpenAPI specification. This is preferred over rigid rule-based parsing because LLMs can better handle the variations and potential errors in human-created examples.
- Descriptive OAS Generation: In parallel, OASBuilder also processes the “descriptive documentation” – the textual overview of an operation’s purpose, security details, and tabular breakdowns of request and response fields. LLMs are again leveraged here to generalize over the diverse HTML structures. The information from both demonstrative and descriptive generations is then merged, prioritizing the most reliable attributes from each source.
- OAS Enhancement: After the initial OAS is generated, OASBuilder further enriches it by filling in missing metadata. This includes extracting details like default values or allowed options (enums) from parameter descriptions and generating missing descriptions or examples based on the surrounding context within the OAS. This step ensures a more complete and useful specification.
Also Read:
- AI-Powered Framework for Extracting Relationships from Data
- AI Models Streamline Clinical Data Standardization with HL7 FHIR
Real-World Impact and Performance
OASBuilder has been successfully implemented in an enterprise environment, where it has generated hundreds of API specifications, saving thousands of hours of manual effort. This makes complex enterprise APIs more accessible as tools for other AI systems.
Experiments evaluating OASBuilder’s performance used various open-source LLMs. The results showed high rates of valid JSON and OpenAPI document generation, with some models achieving over 89% valid OAS documents. Notably, direct attempts to generate OAS documents from original HTML using powerful LLMs like GPT-4 without OASBuilder’s multi-stage approach were far less successful, often failing due to the large size of the documentation pages exceeding the models’ context limits. This highlights the effectiveness of OASBuilder’s modular design.
The system also demonstrated strong semantic performance, accurately capturing most of the relevant information from the documentation, including parameters, descriptions, and attributes like ‘required’, ‘default’, and ‘enum’. While response generation recall was sometimes lower due to the complexity of nested structures, the overall outcome significantly reduces the manual effort required for creating high-quality API specifications.
OASBuilder represents a significant step forward in automating the creation of machine-readable API specifications from human-readable documentation. By intelligently combining rule-based processing with the generalization capabilities of large language models, it provides a robust and scalable solution to a long-standing challenge in software development and AI integration. You can find the full research paper here: OASBuilder Research Paper.
