TLDR: The AIRE-KIDS research developed machine learning models to predict repeat severe asthma exacerbations in children. Using electronic medical records, the LGBM model proved most effective, significantly outperforming current clinical decision rules and large language models in predicting future emergency department visits or hospitalizations. Key predictors included prior asthma ED visits, medical complexity, and triage acuity. These models aim to improve targeted preventative care for children with asthma.
Asthma is a common chronic disease among children, often leading to repeated emergency department (ED) visits and hospitalizations due to severe flare-ups. Despite existing treatments, a significant number of children experience recurrent exacerbations, highlighting a need for better ways to identify those most at risk for these severe events.
Researchers at the CHEO Research Institute and the University of Ottawa have developed a new set of machine learning models, called AIRE-KIDS, to predict future severe asthma exacerbations in children. These models aim to identify children who are likely to have repeat asthma-related ED visits or hospital admissions within one year of an initial asthma ED visit. The goal is to facilitate early referral to preventative comprehensive care, thereby reducing morbidity and the burden on healthcare systems.
The study utilized retrospective electronic medical record (EMR) data from the Children’s Hospital of Eastern Ontario (CHEO), covering periods before and after the COVID-19 pandemic (February 2017 – February 2019 for training, and July 2022 – April 2023 for validation). This data was enriched with information on environmental pollutant exposure and neighborhood marginalization.
The researchers evaluated several machine learning approaches, including boosted trees (LGBM, XGBoost) and three open-source large language models (LLMs): DistilGPT2, Llama 3.2 1B, and Llama-8b-UltraMedical. The models were trained and then validated on a separate dataset to ensure their effectiveness in a real-world setting.
Among the tested models, the Light Gradient-Boosting Machine (LGBM) model consistently showed the best performance. For predicting repeat asthma ED visits, the AIRE-KIDSED model achieved an F1 score of 0.51 and an AUC of 0.712. For predicting future asthma hospitalizations, the AIRE-KIDSHOSP model had an F1 score of 0.375 and an AUC of 0.65. These results represent a notable improvement over the current decision rule used at CHEO, which has an F1 score of 0.334 for ED visits and 0.313 for hospitalizations.
Interestingly, despite the growing popularity of LLMs in various fields, the traditional LGBM approach proved superior for this specific clinical prediction task. The study suggests that while LLMs offer flexibility, they currently lack the precision required for targeted clinical decision-making in this context. The LGBM model, being task-specific and interpretable, offers greater transparency and trustworthiness, which are crucial for clinical adoption.
Key features that were most predictive for repeat ED visits included a prior asthma ED visit, the Canadian Triage Acuity Scale (CTAS), medical complexity, food allergy, prior ED visits for non-asthma respiratory diagnoses, and age. For predicting future hospitalizations, medical complexity, prior asthma ED visit, average wait time in the ED, the Pediatric Respiratory Assessment Measure (PRAM) score at triage, and food allergy were the most important features. Environmental and social variables, such as air quality and marginalization index, did not significantly improve model performance and were excluded from the final, more streamlined models.
The AIRE-KIDS models offer a significant opportunity to enhance how children at high risk for severe asthma exacerbations are identified and referred for preventative care directly from the ED. This could lead to more equitable access to comprehensive asthma management and ultimately reduce the frequency of acute healthcare visits. Further details on this research can be found in the full paper: AI for pRedicting Exacerbations in KIDs with aSthma (AIRE-KIDS).
Also Read:
- Intelligent AI System Enhances Medical Pre-Consultation with Proactive Inquiry
- Mapping Process Success and Failure: A New Discovery Method
Future steps for AIRE-KIDS involve local deployment and prospective evaluation as a clinical decision support system at CHEO, followed by validation at other pediatric centers. Successful implementation could transform pediatric asthma management in acute care settings, benefiting vulnerable children and alleviating healthcare system strain.
