TLDR: MMORE is an open-source pipeline designed to process and retrieve knowledge from over 15 heterogeneous document formats, including text, images, audio, and video, at scale. It transforms this data into a unified format for Large Language Models (LLMs), offering significant speed improvements (3.8x faster than single-node baselines) and higher accuracy (40% over Docling on scanned PDFs). MMORE enables robust Retrieval-Augmented Generation (RAG) systems, improving LLM performance on tasks like biomedical question answering by providing accurate, contextually grounded information from diverse real-world data.
In the rapidly evolving landscape of artificial intelligence, Large Language Models (LLMs) are becoming increasingly powerful. However, their ability to access and utilize the vast amount of information locked away in diverse, real-world document formats remains a significant challenge. A new open-source pipeline, MMORE (Massive Multimodal Open Retrieval-Augmented Generation and Extraction), aims to bridge this gap by providing a comprehensive solution for ingesting, transforming, and retrieving knowledge from a wide array of heterogeneous data at scale.
MMORE is designed to handle more than fifteen different file types, including traditional text documents, tables, images, emails, audio, and video. It processes these varied formats into a single, unified structure, making them readily accessible for downstream LLM applications. This unified approach is crucial because, as of 2025, trillions of PDF documents and petabytes of other mixed-modality content on the public web are largely underrepresented in machine learning corpora due to parsing difficulties.
One of MMORE’s standout features is its architecture, which supports modular and distributed processing. This design allows for scalable parallelization across multiple CPUs and GPUs, leading to impressive performance gains. On processing benchmarks, MMORE demonstrated a 3.8-fold speedup compared to single-node baselines. Furthermore, it achieved 40% higher accuracy than Docling, another document processing tool, particularly on challenging scanned PDFs.
The pipeline integrates a sophisticated hybrid dense-sparse retrieval mechanism, which combines lexical matching with semantic search capabilities. This allows users to choose or combine retrieval strategies based on their specific task requirements. MMORE also offers both interactive APIs and batch RAG (Retrieval-Augmented Generation) endpoints, providing flexibility for various deployment scenarios.
Addressing LLM Challenges with RAG
The rise of LLMs has also brought challenges like “hallucinations” and “factual drift.” Robust RAG pipelines are increasingly seen as a way to mitigate these issues by grounding LLM outputs in trustworthy source material. MMORE significantly contributes to this by enabling LLMs to access accurate, contextually grounded information from diverse data through the RAG paradigm.
Evaluations on the PubMedQA benchmark, a biomedical question-answering task, showed that MMORE-augmented medical LLMs improved accuracy with increasing retrieval depth. This highlights the pipeline’s effectiveness in injecting domain-specific context at inference time, leading to more reliable and accurate answers.
Also Read:
- HANRAG: Enhancing AI’s Question Answering Through Adaptive Retrieval and Noise Filtering
- GeoGPT: Advancing Geoscience Research with Retrieval-Augmented AI
Under the Hood: Architecture and Performance
MMORE’s architecture is built for efficiency and extensibility. It reuses existing open-source extraction tools like Surya for PDF parsing and Whisper for audio transcription, focusing its innovation on scalable orchestration and integration. The system leverages Dask for automatic workload balancing, fault tolerance, and seamless scaling across different deployment settings, from single machines to large Kubernetes clusters.
The processing module offers two modes: a ‘fast mode’ optimized for speed (by omitting OCR) and a ‘default mode’ prioritizing accuracy. While the fast mode delivers significant speedups, the default mode maintains high extraction fidelity, especially crucial for image-based documents requiring OCR, where it clearly outperformed Docling.
The RAG pipeline is composed of independent components for post-processing (filtering extracted text for quality), indexing and retrieval (using hybrid sparse and dense embeddings), and an integrated RAG service. This modularity ensures flexibility and allows components to be run independently.
In conclusion, MMORE provides a robust and extensible foundation for deploying task-agnostic RAG systems on diverse, real-world multimodal data. Its open-source nature and comprehensive capabilities make it a valuable tool for researchers and organizations looking to unlock the full potential of their unstructured data for advanced AI applications. You can find more details about this research paper here: MMORE: Massive Multimodal Open RAG & Extraction.
