Demos
Risk adjustment models are only as complete as the codes they can see. A physician note describing “poorly controlled diabetes with early nephropathy and peripheral neuropathy, complicated by depression and social isolation” documents four risk-relevant conditions, but if that documentation never produces matching ICD-10 entries, the model is blind to all of them. Natural language processing that reads notes as they are written can close that visibility gap. Building extraction accurate enough to trust and fast enough to run continuously, in a form that survives an audit, is a harder infrastructure problem than the underlying NLP task suggests.
From unstructured clinical text to an auditor-ready evidence record – extraction, assertion resolution, normalization, risk-adjustment mapping, human review, and provenance logging, using synthetic examples throughout.
Why does clinical complexity get lost between the note and the code?
Structured coding captures what a clinician documented in codeable form: a diagnosis entered as a discrete field, a problem added to a list, an order tied to an ICD-10 value. It does not reliably capture what a clinician wrote in prose, and prose is where most clinical reasoning happens. A comorbidity mentioned in an assessment paragraph, a complication described in a progress note, a social determinant noted in passing during an office visit: all of it is real, all of it affects care complexity and cost, and none of it reaches a risk adjustment model that only reads structured fields.
This is not a documentation failure. Clinicians write notes to communicate with other clinicians, not to populate a payment model. The gap between what gets written and what gets coded is structural, and it means any risk adjustment approach that relies solely on structured data will systematically understate the complexity of the patients it is supposed to describe.
What does continuous extraction require from data infrastructure?
Ohio State University Wexner Medical Center’s NLP pipeline processes more than 200 million Epic clinical notes across seven hospitals, combining cohort selection, de-identification, information extraction, and human-in-the-loop validation in a single production workflow that runs on Azure Databricks and the university’s on-premises high-performance computing environment (Huerta, 2025). That scale illustrates the gap between batch and continuous processing. A batch job that runs nightly on the prior day’s notes is standard for retrospective research and introduces a lag that research tolerates but risk adjustment cannot, since the goal is to reflect a patient’s risk profile as care events occur, not as they were documented a day earlier.
Continuous extraction changes the integration point. It requires the NLP pipeline to connect to EHR data streams at the API level rather than to periodic exports, infrastructure that absorbs burst volume when documentation spikes at shift changes or during high-acuity periods, and storage that updates incrementally rather than rebuilding a full extraction run from scratch. Across a multi-facility network, it also requires consistent normalization: a comorbidity documented with one term at one facility and a different term at another has to resolve to the same code, or the resulting risk distribution reflects normalization inconsistency rather than the actual complexity of the patient population.
What does an audit-ready evidence trail have to document?
Risk adjustment in value-based care is subject to audit, and that audit exposure just increased materially. In May 2025, CMS announced it would expand Risk Adjustment Data Validation audits to cover all eligible Medicare Advantage contracts in every newly initiated audit, scale its medical coder workforce from roughly 40 to about 2,000, and increase the number of plans audited annually from around 60 to approximately 550, with per-plan record reviews ranging up to 200 depending on plan size (CMS, 2025). CMS has set a target of completing all outstanding audits for payment years 2018 through 2024 by early 2026.
An automated extraction system that influences a risk code has to be able to produce, for any single code assignment, the note that supports it, the model version that performed the extraction, the confidence score assigned, and whether a human reviewed the extraction before it was accepted. That is a longer chain than a human chart reviewer’s audit trail, because the model itself is an additional link between the documentation and the code. Provenance has to be captured at extraction time, not reconstructed after the fact, which means the pipeline architecture has to log model version, confidence, and reviewer decision as part of the extraction event rather than as a separate downstream process.
Why doesn’t negation handling alone solve overcoding?
The failure mode that matters most for risk adjustment is not missing a condition. It is coding one that should not have been coded. A model that reliably surfaces undocumented comorbidities will, without careful design, also surface conditions that are mentioned in the text but not active: a diagnosis that has resolved, one that was considered and ruled out, one documented as a family history rather than the patient’s own. “History of hypertension, now resolved following weight loss” is not a negation. “We considered CHF, but the echocardiogram was normal” is not a negation either. “Mother had type 2 diabetes” is risk-relevant family context that should never generate a diagnosis code for the patient in front of the clinician.
Negation detection catches the explicit cases: “no evidence of,” “denies,” “ruled out.” It does not catch these. A 2025 assertion-classification study built to distinguish present, absent, hypothetical, conditional, and third-party clinical statements found that a fine-tuned healthcare-specific model reached 0.962 combined accuracy, ahead of GPT-4o’s 0.901 on the same test set, but that even the fine-tuned model dropped to 0.599 accuracy on the hardest category, conditional statements, versus 0.976 on straightforward present-or-absent cases (Kocaman et al., 2025). Conditional and historical framing is exactly where overcoding risk concentrates, and it is the category every model, purpose-built or general, still struggles with most. That is the argument for routing high-weight HCC codes to human review before acceptance: not because aggregate accuracy is inadequate, but because a single systematic error pattern, a condition class extracted from family-history text as if it were an active diagnosis, produces a pattern that looks deliberate under audit even when it originated as a model error.
How does purpose-built extraction compare with general-purpose models?
The evidence on general-purpose LLMs performing risk adjustment and coding tasks without healthcare-specific engineering is consistent and unfavorable. In a benchmarking study published in NEJM AI, GPT-4, the best-performing model tested, produced an exact-match rate of just 33.9% on ICD-10-CM code assignment from clinical text (Soroush et al., 2024). Independent of any JSL benchmark, a 2025 evaluation of LLMs mapping clinical terms to ontology concepts found GPT-4o reaching 93.75% precision on the task while an open-source alternative, Llama 3.3 70B, managed only 19.19% on identical inputs, with both models weakest on complex, multi-parameter measurements (Mavridis et al., 2025). The gap to purpose-built models is wide, and the variance between general-purpose models is nearly as wide, which means no single one can be assumed reliable without task-specific validation.
Normalization, mapping an extracted mention to the standard code that a risk adjustment engine actually consumes, shows the same pattern. On RxNorm terminology mapping, John Snow Labs’ vector-based resolver reached 85% top-five accuracy against GPT-4o’s 9% on the same records, at roughly a fifth of the per-record cost (Bonis, 2025). None of these gaps make general-purpose LLMs unusable for every healthcare task. They do mean that a risk adjustment pipeline built to be defensible under a CMS RADV audit, where a single systematic overcoding pattern can trigger extrapolated financial recovery, needs extraction and normalization components validated on the specific task and specific documentation it will run against, not a general model assumed to transfer.
How does John Snow Labs support audit-ready risk adjustment extraction?
Real-time NLP extraction turns unstructured clinical documentation into structured, audit-ready risk adjustment data – complete with provenance and confidence scoring for every code.
John Snow Labs’ Healthcare NLP and Medical LLM models are built for the assertion, temporality, and normalization problems this kind of pipeline depends on, evaluated against commercial cloud APIs and frontier LLMs under matched test conditions rather than adapted from general text models after the fact. Martlet AI, John Snow Labs’ risk adjustment platform, applies that extraction layer to HCC coding specifically: it maps extracted clinical evidence to MEAT criteria and ICD-10-to-HCC logic, and its RADV audit readiness capability is built to produce the note-level provenance, confidence scoring, and reviewer trail that CMS’s expanded audit program now expects as a matter of course (Martlet.ai, 2025). Generative AI Lab supplies the human-in-the-loop layer: a no-code interface that routes low-confidence or high-weight extractions to a reviewer, and, as of its 7.8 release, built-in HIPAA audit logging that keeps every extraction, review, and code decision attached to its source (John Snow Labs, 2026).
For teams evaluating a real-time extraction pipeline for risk adjustment, the diligence questions worth asking a vendor follow directly from the evidence above: what does assertion accuracy look like on conditional and historical statements specifically, not just on the easy present-or-absent cases; what is normalization accuracy on the terminologies your HCC codes actually depend on; and can the system produce, for any single code, the note, model version, confidence score, and reviewer decision behind it, on demand. Request a technical demonstration scoped to your own note volume and documentation patterns to get answers specific to your environment rather than benchmark averages.
Frequently asked questions
Why can’t risk adjustment models just rely on structured diagnosis codes? Structured codes capture what clinicians document in codeable form, but most clinical reasoning happens in prose. A note describing poorly controlled diabetes with early nephropathy, neuropathy, and complicating social factors documents real, risk-relevant conditions, but if that language never produces a matching ICD-10 entry, the risk adjustment model has no way to see it. NLP extraction reads what was actually written rather than only what was coded.
What makes real-time extraction harder to build than batch processing? Batch pipelines run on periodic exports and tolerate a lag of hours to a day, which retrospective research applications accept. Real-time extraction requires API-level EHR integration, infrastructure that absorbs burst volume during high-documentation periods, and incremental storage updates rather than full pipeline reruns. Across multiple facilities, it also requires consistent normalization so the same comorbidity, documented with different terminology at different sites, resolves to the same code.
What does an audit-ready NLP risk adjustment pipeline need to document? For any single code, the pipeline has to produce the supporting note, the model version that performed the extraction, the confidence score assigned, and whether a human reviewed the extraction before it was accepted. That is more documentation than a human chart-review process typically produces, because the model itself is an additional link in the chain between the note and the code.
How much did CMS expand RADV audit scope, and when? In May 2025, CMS announced it would audit all eligible Medicare Advantage contracts in every newly initiated RADV audit, grow its coder workforce from about 40 to roughly 2,000, and raise the number of plans audited annually from about 60 to approximately 550, with a goal of completing all audits for payment years 2018 through 2024 by early 2026 (CMS, 2025).
Is negation detection enough to prevent overcoding? No. Negation detection catches explicit constructions like “no evidence of” or “denies,” but it misses resolved historical conditions, diagnoses that were considered and ruled out, and family-history mentions, all of which can be extracted as if they were active patient diagnoses. A 2025 study found that even a fine-tuned assertion model dropped from 0.976 accuracy on straightforward present-or-absent statements to 0.599 on conditional statements, the category where most overcoding risk concentrates (Kocaman et al., 2025).
Can a general-purpose LLM like GPT-4 replace purpose-built clinical NLP for risk adjustment? The published evidence says no without task-specific validation. GPT-4 reached only 33.9% exact-match accuracy on ICD-10-CM coding in an independent benchmark (Soroush et al., 2024), and on ontology mapping, general models ranged from 93.75% to 19.19% precision depending on which model was tested (Mavridis et al., 2025). That variability means a general model’s performance cannot be assumed; it has to be measured on the specific task before it supports a financial or regulatory outcome.
Why does human review remain necessary even when extraction accuracy is high? Aggregate accuracy does not eliminate the risk of a consequential error in a specific high-stakes case. Codes that carry significant HCC weight, and therefore significant financial and regulatory consequence, warrant human review of the supporting evidence before the code is accepted. That review is a governance layer, not a substitute for model accuracy, and it is also what generates the reviewer-level audit trail that demonstrates the organization’s risk adjustment process has real controls in place.
