TLDR: SPIRA is an intelligent system that uses machine learning to detect respiratory insufficiency from voice recordings. This paper details the development journey, highlighting challenges in data collection, model training, and system inference across two iterations, offering valuable lessons for building similar AI-powered health systems. It covers the evolution from initial attempts with existing platforms to a robust microservices architecture with a Progressive Web App for data collection and an asynchronous inference system.
Respiratory insufficiency is a medical condition where a person’s blood receives a reduced amount of oxygen, leading to symptoms like coughing, tiredness, and shortness of breath, and in severe cases, can be fatal. This condition can stem from various diseases, including the flu, severe asthma, or heart conditions. During the 2020 COVID-19 pandemic, it also emerged as a key indicator of infection before clinical tests were widely available.
Introducing SPIRA: An Intelligent System for Voice-Based Detection
Born out of the COVID-19 pandemic, SPIRA is a multidisciplinary project that leverages Machine Learning (ML) to identify respiratory insufficiency through voice analysis. This initiative brought together medical doctors, linguists, speech therapists, and computer scientists. While previous research focused on the modeling challenges using Deep Learning (DL) techniques, this paper details the practical experience of building SPIRA as an intelligent system, focusing on the challenges and lessons learned in its development.
The SPIRA Architecture: Training and Inference Workflows
Intelligent systems, which utilize artificial intelligence and machine learning to achieve specific goals, often separate their operations into two main workflows: training and inference. SPIRA adopted a microservices architectural style to accommodate this, allowing different system components to use specialized software and enabling developers to work on multiple parts simultaneously. The system includes components for data collection, an MLFlow model repository, and separate APIs for training and inference.
The data collection process involves gathering IDs, basic information, and voice samples from volunteers using a Progressive Web App (PWA). This data is sent to a Data Collection API, which stores metadata in a document database and audio files in a file system. These audios are preprocessed, transformed into features, and then used to train a classifier model, which is stored in the MLFlow repository. This training workflow is experimental, aimed at building better models.
Once a classifier model is ready, it’s used for inference on new data. The Inference PWA collects data from potential patients, which generates events in a requests queue. A Model Server consumes these events, applies the classifier model, and places the results in a responses queue, which the Inference API then receives and displays via the Inference PWA. Using a queue enhances system resilience, ensuring all collections are eventually processed.
Lessons from the First Version
The initial version of SPIRA aimed to support data collection for training the Deep Learning model. An early idea was to use WhatsApp for data collection due to its popularity and built-in audio features in Brazil. However, preliminary tests revealed that WhatsApp’s audio compression and filtering (using the Opus codec) significantly degraded audio quality, removing crucial signals needed for training. This led developers to build a custom website for data collection, emphasizing the need for strict control over audio capture.
Training the initial model also presented challenges. Developed experimentally, the training pipeline lacked specific design patterns or conventions, making it difficult to integrate the model into an API. Refactoring was necessary, involving checking dependency compatibility, creating automated tests (especially for data transformation functions), and analyzing results. The absence of experiment tracking tools made exact reproducibility impossible, leading to efforts to minimize metric disparities rather than eliminate them. These experiences highlighted the importance of well-defined model interfaces and automated testing for maintainability and debugging.
Improvements in the Second Version
Building on the lessons learned, the second version of SPIRA focused on creating an improved model trained with hospital patient data, integrated into an app for pre-diagnosis. The data collection app was redesigned to gather raw audio signals without compression or filtering, crucial for capturing biomarkers in speech. This new PWA, built with MongoDB, Quarkus (Java framework), and Vue.js (JavaScript framework), allowed for more control over audio recording and provided an offline experience, storing recordings locally before syncing when internet access was available. This significantly improved resilience and user experience in hospital environments.
The inference system in the second version addressed response resiliency and service architecture. It adopted an asynchronous event-driven API using NATS as a message broker, ensuring predictions are processed regardless of server conditions. The hexagonal architecture pattern was chosen for the API service, centralizing business logic and abstracting external dependencies. This design also facilitated a clear interface between the server and the prediction model, allowing for the use of various models. To improve reproducibility, MLFlow was adopted as a model registry platform to track experiments, store parameters, and version the training pipeline source code.
Also Read:
- Optimizing Large AI Model Training: A System-Level Approach to Federated Mixture-of-Experts
- Advancing Speech Quality Assessment with a Mixture of Experts Model
Conclusion and Future Outlook
SPIRA represents a significant effort in building an intelligent system for respiratory insufficiency detection from voice. The project’s journey, detailed in this paper, highlights critical challenges in UX design, audio recording, system deployment, response resiliency, service architecture, coupling, and reproducibility. Overcoming these challenges provided valuable insights into using PWAs, managing audio, leveraging message brokers, and applying hexagonal architecture. The data collection app is currently in production, gathering data from Brazilian hospitals to further refine the detection model. The inference system is also under development, with plans to test multiple models in hospitals to identify the most effective one for pre-diagnosis. For more details, you can refer to the full research paper: SPIRA: Building an Intelligent System for Respiratory Insufficiency Detection.
