TLDR: The paper introduces AutoLit, an AI-driven pipeline that automates literature reviews by using transformer-based embeddings and cosine similarity. It generates keywords from a paper’s title and abstract, fetches relevant papers from open-access repositories like arXiv, and then filters, summarizes, and tags them. The system evaluates different embedding models, with all-MiniLM-L6-v2 offering a balanced performance. AutoLit aims to reduce manual effort and provide a scalable tool for preliminary research, acknowledging limitations such as the absence of ground-truth relevance labels and the need for future enhancements like adaptive thresholding and human-in-the-loop evaluation.
Conducting a thorough literature review is a cornerstone of any research project, providing essential context, identifying gaps, and guiding future studies. However, traditional methods are often time-consuming and demand extensive manual effort for paper selection, filtering, and organization. This challenge has led to the development of automated approaches, yet many still require significant configuration or user expertise.
Addressing this, researchers have proposed AutoLit, an innovative artificial intelligence-driven pipeline designed to streamline the literature review process using semantic similarity. Unlike systems that require complex setups, AutoLit focuses on minimal input and high relevance, making it a practical tool for preliminary research and exploratory analysis.
The core of AutoLit’s functionality lies in its use of transformer-based embeddings and cosine similarity. When provided with just a paper’s title and abstract, the system automatically generates relevant keywords. It then fetches potentially relevant papers from open-access repositories, such as arXiv, and ranks them based on their semantic closeness to the initial input. This semantic search capability allows the system to identify conceptually related papers, even if they don’t share exact keywords.
The pipeline consists of several key stages. First, a large language model (LLM), specifically Gemini-2.0-flash, is used to generate 5 to 10 keywords from the input paper’s title and abstract. These keywords then guide the retrieval of up to 20 papers per keyword from arXiv via its API, with duplicates filtered out to ensure a unique corpus.
Next, to refine the selection, a semantic similarity assessment is performed. The all-MiniLM-L6-v2 sentence transformer model generates dense vector embeddings for both the input query and the fetched papers. Cosine similarity is then calculated between these embeddings. A statistical thresholding method, based on the interquartile range (IQR), is applied to filter out less relevant papers, ensuring high precision in the selection of semantically similar documents.
The researchers evaluated three embedding models: TF-IDF, all-MiniLM-L6-v2, and Specter2. TF-IDF, a classical lexical model, struggled to capture deeper semantic meaning. Specter2, fine-tuned for scientific texts, achieved high similarity scores but suffered from score saturation, requiring extremely precise thresholding. All-MiniLM-L6-v2 emerged as a balanced choice, effectively capturing semantic relations and providing a good separation between relevant and less relevant documents.
After filtering, the system extracts structured textual content from the PDFs of the selected papers using the PyMuPDF library. Regular expression patterns are used to identify and delineate key sections like the Abstract, Introduction, Methodology, Results, and Conclusion. The extracted text from each section is then summarized using the Gemini-2.0-flash LLM, structuring the summaries into categories such as problem statement, methodology, key findings, and conclusion/recommendations.
Further analysis involves citation intent tagging (e.g., Background, Comparison, Extension) and contribution classification (e.g., Dataset, Algorithm, Framework) for each paper, also performed by the Gemini LLM. Finally, the metadata, structured summaries, and tags are synthesized to generate BibTeX entries and a comprehensive literature review paragraph, providing a high-level overview of the existing literature.
While AutoLit demonstrates significant potential in reducing manual overhead, the researchers acknowledge several limitations. These include the absence of human-annotated ground-truth relevance labels, which makes direct assessment of true relevance challenging. General-purpose embedding models like all-MiniLM-L6-v2 may also underperform in highly specialized domains, and the heuristic thresholding method doesn’t guarantee semantic relevance. Future work aims to address these by implementing adaptive thresholding, incorporating human-in-the-loop evaluation, exploring re-ranking techniques, and integrating with citation network data sources like Semantic Scholar.
Also Read:
- CLIMB: A New Framework for Data-Driven Occupation Taxonomies
- AI Learns to Fix C++ Code: A New Approach to Compilation Repair
This work represents a significant step towards automating literature reviews, offering a scalable and accessible alternative to traditional systematic review systems. For more details, you can refer to the original research paper: An Artificial Intelligence Driven Semantic Similarity-Based Pipeline for Rapid Literature Review.
