TLDR: ProFees is an LLM-based framework designed to automate CPT E/M coding, a complex and error-prone process crucial for healthcare billing. It tackles real-world challenges like label noise and explainability through a modular architecture, dynamic few-shot prompting, recursive criticism, and self-consistency. Tested on a real-world dataset, ProFees significantly improves coding accuracy by over 36% compared to commercial systems, demonstrating its effectiveness in streamlining medical documentation and enhancing billing efficiency.
Accurate and efficient medical coding is a cornerstone of healthcare operations, directly influencing billing, regulatory compliance, and revenue cycles. Specifically, Evaluation and Management (E/M) coding, under the Current Procedural Terminology (CPT) taxonomy, is used to document the medical services provided to patients by physicians. Traditionally, this complex task is performed manually by physicians and trained human coders who review extensive clinical notes and electronic health records (EHR) to assign appropriate codes. This manual process is not only resource-intensive but also prone to inconsistencies and errors, leading to significant financial ramifications, such as billions of dollars in inappropriate Medicare payments due to incorrect coding or insufficient documentation.
The challenges in automating CPT E/M coding are numerous and complex. Production data often lacks intermediate labels, such as the underlying medical decision-making (MDM) levels, which are crucial for accurate coding. Even expert human coders frequently disagree on code assignments, highlighting the inherent ambiguity. Furthermore, any automated system must provide human-readable reasoning for its predictions to satisfy clinicians and auditors, as black-box predictions are unacceptable in healthcare. Ensuring robustness and repeatable results from inherently stochastic Large Language Models (LLMs) is another hurdle, as is the need for deep and wide-ranging familiarity with coding guidelines and edge cases.
Introducing ProFees: An LLM-Based Framework
To address these real-world complexities, researchers have developed ProFees, an innovative LLM-based framework designed to automate CPT E/M coding. ProFees aims to alleviate the documentation burden on physicians, improve billing efficiency, and ultimately enhance patient care by providing accurate and consistent coding. The framework has been developed and deployed within a production EHR system, offering practical solutions to the aforementioned challenges.
ProFees employs a modular architecture that integrates LLM-based classifiers and self-critics, leveraging dynamic few-shot in-context learning boosted by retrieval with chain-of-thought exemplars, and traditional rule-based decision trees. The system is broken down into two primary LLM-driven modules: an Encounter Type Classifier, which identifies the nature of the clinical visit, and an MDM Complexity Classifier, which predicts the level of medical decision-making involved. The MDM prediction is further refined through a Recursive Criticism and Improvement (RCI) process to ensure strict adherence to guidelines.
How ProFees Works
The model’s design is directly motivated by CPT E/M coding guidelines, which determine the final code based on encounter type, MDM level, and patient type. ProFees focuses on predicting the encounter type and MDM level. The MDM Complexity Classifier assesses three key elements: the number and complexity of problems addressed, the amount and/or complexity of data reviewed and analyzed, and the risk of complications and/or morbidity or mortality. The final MDM level is determined using a “2-out-of-3” rule, meaning at least two of these elements must meet the criteria for a given complexity level (e.g., straightforward, low, moderate, or high).
To enhance its capabilities, ProFees utilizes a dynamic few-shot prompting strategy. This involves retrieving contextually relevant examples from a Vector Database (VDB) during the prediction process. These examples include expert-curated justifications, model-generated justifications, and chain-of-thought reasoning steps, all verified by human coding experts. The RCI process then systematically audits and refines the initial predictions by following checklists designed to catch common errors. To mitigate the inherent stochasticity of LLMs, ProFees also adopts a self-consistency strategy, executing the MDM classifier multiple times concurrently and using majority voting to achieve reliable and consistent coding predictions.
Significant Improvements in Accuracy
On an expert-curated real-world dataset, ProFees demonstrated remarkable effectiveness. It achieved an increase in CPT coding accuracy of more than 36% over a commercial CPT E/M coding system and almost 5% over the strongest single-prompt baseline. This significant improvement validates its efficacy in addressing the complexities of CPT E/M coding automation. The addition of the RCI module alone boosted MDM accuracy by 4% in the few-shot setting, confirming the value of structured self-critique.
Also Read:
- Self-Evolving LLMs: How Ontology Rules Enhance Domain Knowledge Without Extensive Data
- Enhancing FrameNet Annotation with AI: A Look at LLM-Assisted Workflows
Ethical Considerations and Future Directions
While ProFees represents a significant leap forward, its integration into healthcare systems requires careful ethical consideration. The researchers emphasize privacy and data integrity, ensuring development and deployment in HIPAA-compliant environments. Accountability and liability are also addressed, with ProFees acting as an assistive tool, not an autonomous decision-making one, meaning physicians and professional coders retain final responsibility. Algorithmic biases and fairness are being mitigated through the curation of synthetic datasets for targeted evaluation. Furthermore, the framework is designed for interpretability and transparency, providing detailed rationales for coding outputs to build trust and enable human oversight.
Future work for ProFees includes extending the model to support multiple codes per encounter, CPT modifiers, and generating synthetic datasets for edge-case testing and enriching the VDB. This ongoing development aims to further refine and expand the capabilities of this promising AI solution for medical coding. You can read the full research paper here.
