Demos
HEDIS and Medicare Advantage Star Ratings both depend on evidence that often lives only in unstructured clinical text: discharge summaries, referral letters, physician progress notes. Structured claims data alone misses much of it. NCQA is now retiring hybrid, chart-review-dependent measures on a fixed schedule running through measurement year 2030 (NCQA, 2024). Health plans that wait for that deadline to build extraction infrastructure will be building it under pressure. Here is a six-step sequence for doing it now.
A handful of measures decide most Star Ratings, and much of the evidence behind them lives in clinical notes, not claims data. NCQA is phasing out manual chart review by 2030. What does replacing it actually take?
Why can’t manual chart review keep pace with quality reporting?
Star Ratings for 2026 evaluate Medicare Advantage prescription drug contracts against up to 43 measures, MA-only plans against up to 33, and standalone Part D plans against up to 12, with CMS reweighting patient-experience and access measures downward this cycle (CMS, 2025). About 40% of MA-PD contracts reached four stars or higher for 2026, representing roughly 64% of enrollment once weighted by plan size, which means the majority of the market is competing inside a narrow band where a handful of measures decide the rating (CMS, 2025). HEDIS works the same way: dozens of measures, each with its own numerator logic, each capable of moving a plan’s overall score.
NCQA’s own rationale for phasing out hybrid measures, which currently require manual medical-record review to supplement claims data, is that this review no longer scales against the volume and complexity of what plans need to report (NCQA, 2024). Lead screening in children moves to administrative-only reporting for measurement year 2026, weight assessment for children and adolescents follows in 2027, prenatal and postpartum care in 2028, and glycemic status, transitions of care, and care for older adults shift to electronic clinical data systems (ECDS) by 2029. The direction is unambiguous: NCQA expects plans to source this evidence from electronic clinical data, not chart pulls, and the compliance clock is already running.
The problem is that “electronic clinical data” does not mean structured data. A colonoscopy documented only in a gastroenterology consult note, a smoking-cessation counseling session buried in a progress note, a follow-up visit referenced in a discharge summary but never coded as a follow-up: none of this shows up in a claims feed or a structured EHR field. TriNetX’s clinical informatics team found that only 22.3% of patients had smoking status captured in structured fields, and that an NLP extraction pipeline recovered status for 66.9% of patients once applied to the unstructured note text (Temple, 2026). Smoking status is not a HEDIS or Star measure on its own, but the pattern generalizes directly: roughly three-quarters of a data element that clinicians document routinely was invisible to a system that only reads structured fields.
What does automated evidence extraction actually require?
Getting from free text to a defensible measure-compliance record takes more than an accurate named-entity model. It requires the same three-layer architecture as any regulatory-grade clinical NLP deployment: entity and relation extraction to find the clinical facts, assertion and temporal logic to determine whether a fact is confirmed, negated, planned, or historical and when it happened, and normalization to a standard code system so an extracted concept is comparable across notes, encounters, and members.
For HEDIS measure logic specifically, that middle layer, assertion and temporality, is where most of the compliance risk sits. “Follow-up after ED visit” is only satisfied if the follow-up encounter falls inside the measure’s required window, not merely if a follow-up is mentioned somewhere in the chart. “Patient declined screening” and “screening scheduled for next visit” both mention the screening, but only one should count against the numerator. A pipeline that extracts entities without correctly resolving negation, timing, and hedged language does not reduce audit risk; it just moves the error from a human abstractor’s desk to a model’s output, with less visibility into why the error happened.
That is also why frontier general-purpose language models are not automatically the safer choice. In a blinded evaluation of clinical information extraction, three physician reviewers preferred outputs from a healthcare-specific language model over GPT-4o on factuality 35% of the time versus 24% for GPT-4o, a 10-point-plus preference gap, with corresponding advantages in clinical relevance and conciseness (Kocaman et al., 2025). For a workflow that feeds regulatory reporting, that gap is the difference between evidence a compliance team can stand behind and evidence that needs a second manual check anyway.
What is a 6-step blueprint for extracting HEDIS and Stars evidence?
A six-step blueprint for HEDIS and Star Ratings evidence extraction: scope, build one pipeline, encode measure logic separately, route uncertain cases to review, keep provenance, validate at scale
Step 1: Scope the measures and the data you actually have
Start narrow. Inventory which document types you can access (EHR exports, HIE feeds, scanned referral letters, discharge summaries) and map them to the specific measures where unstructured evidence is likely to close the biggest gaps: follow-up-after-ED-visit, controlling blood pressure, statin therapy documentation, and the chronic-disease measures NCQA is not yet fully digitizing. Trying to automate all measures on day one is how these programs stall.
Step 2: Build extraction and normalization as one pipeline, not two projects
Entity extraction and terminology mapping (to SNOMED CT, ICD-10, LOINC, RxNorm) need to run in the same pipeline, versioned together, so a re-run six months from now on refreshed models produces traceable, comparable output. Healthcare NLP is built specifically for this: named entity recognition, assertion detection, and terminology normalization running as a single, on-premises pipeline rather than separate tools stitched together after the fact.
Step 3: Encode measure logic separately from extraction logic
Measure specifications change every year. Keep the clinical-logic layer, the rules that decide whether an extracted, dated, and negation-resolved fact satisfies a specific HEDIS or Star numerator, configurable and separate from the underlying NLP models. This is what let WVU Medicine’s HCC-coding deployment extract diagnosis codes from clinical notes and route them to physicians through best-practice alerts inside the EHR: the extraction layer stayed stable while the coding logic on top of it stayed current (Godse, 2024).
Step 4: Route uncertain cases to human reviewers, not into the compliance report
No extraction pipeline should output a compliance assertion at 100% automation. A framework built by Care-Connect and Spryfox uses clinician-developed logic to filter and rank which extracted entities need manual review, explicitly aiming to reduce reviewer workload “while maintaining a high level of safety” rather than eliminating review entirely (Debes & Kiernan, 2024). Generative AI Lab supports that same pattern for quality teams: a no-code interface that surfaces low-confidence extractions to a reviewer and captures the correction as training signal, rather than requiring every note to go through the same manual path.
Step 5: Keep full evidence provenance, not just a final answer
Every compliance assertion needs to trace back to the source sentence, the document ID, the encounter, and the date, because a Star Ratings audit or a RADV review does not accept a summary number without a citation. Martlet.ai, John Snow Labs’ HCC-coding and risk-adjustment platform, builds evidence packets that link each coded assertion to the chart sentence, encounter ID, date of service, provider, and signature it came from, and its RADV Audit Readiness Platform runs the same encounter, credential, and signature checks CMS auditors run, ahead of the audit rather than in response to it (Martlet AI, 2026). That platform launched specifically because, as Martlet.ai co-founder Ritwik Jain put it, “RADV audits are no longer retrospective events, but a year-round operational priority” (Martlet.ai, 2025).
Step 6: Validate at production scale before you rely on it
A pipeline validated on a few hundred curated notes is not validated for a member population of a million or more. Ohio State University Wexner Medical Center runs cohort selection, de-identification, extraction and coding, and human-in-the-loop validation across more than 200 million Epic notes from seven hospitals, on both Azure Databricks and on-premise infrastructure (Huerta, 2025). At Providence St. Joseph Health, an independent audit of 790 randomly selected patients across roughly 2 billion de-identified notes found 98.6% PHI recall and 99% obfuscation, with no successful re-identifications (Kocaman et al., 2025). That is the validation standard, run on the full diversity of production notes, that a quality-measure extraction pipeline should be held to before its output feeds a Star Ratings submission.
For a broader look at where healthcare-specific language models are changing HEDIS and Star Ratings work more generally, see how healthcare-specific language models are transforming HEDIS and Stars performance; this blueprint focuses on the implementation sequence underneath that shift.
What do production deployments show about the payoff?
The efficiency case for this infrastructure does not depend on quality-measure use cases alone; it compounds across every application built on the same extraction layer. Intermountain Health runs medical-text summarization and de-identification on a Databricks Lakehouse spanning hundreds of millions of clinical documents, cutting document review time from 10 minutes to 3 minutes per document, a 70% reduction, on the same infrastructure that also supports natural-language querying of the underlying data (Bonis, 2026). A health plan that builds its HEDIS and Star extraction pipeline on reusable Healthcare NLP infrastructure, rather than a point solution scoped to one measure set, gets the same compounding return: the same models and normalization layer support risk adjustment, RADV preparation, and population-health stratification without a separate build for each.
That is the throughline across WVU Medicine’s HCC-coding deployment and Martlet.ai’s RADV platform: both treat evidence extraction from clinical notes as shared infrastructure, not a one-off project tied to a single reporting cycle. A quality team building toward measurement year 2029, when NCQA expects glycemic status and care-for-older-adults measures to move fully to electronic clinical data, is better served starting from that same infrastructure now than building a narrow tool it will have to replace.
Where should a health plan start?
Manual chart review was never going to scale to NCQA’s digital-quality timeline or to the narrow margins that separate Star Ratings tiers. The plans already ahead of that curve- WVU Medicine on HCC coding, Ohio State and Providence on enterprise-scale extraction and de-identification, Martlet.ai’s Medicare Advantage clients on RADV readiness- are treating extraction, normalization, human review, and provenance as one connected pipeline rather than four separate purchases. Start with the one or two measures where unstructured evidence is costing you the most points, build the pipeline with full provenance from day one, and expand it as NCQA’s hybrid-measure retirement schedule narrows your other options. For what the underlying NLP infrastructure needs to look like once you’re reporting continuously against digital quality measures and ECDS, rather than scoping the first one or two measures, see our companion post, “dQMs and ECDS: why real-time NLP pipelines must replace manual chart abstraction,” on the John Snow Labs blog. For a technical walkthrough of how Healthcare NLP and Martlet AI apply to your specific measure set and document mix, request an architecture consultation scoped to your plan’s data.
Frequently asked questions
What is the difference between HEDIS and Medicare Advantage Star Ratings? HEDIS is a standardized set of quality measures maintained by NCQA and used across commercial, Medicaid, and Medicare plans to assess care quality. Star Ratings are CMS’s overall scoring system for Medicare Advantage and Part D plans, which incorporates many HEDIS measures alongside patient-experience surveys, safety measures, and Part D-specific metrics, evaluated against up to 43 measures for MA-PD contracts in 2026 (CMS, 2025).
Why is NCQA retiring hybrid HEDIS measures? NCQA is phasing out measures that rely on manual chart-review supplementation to claims data, moving them to electronic clinical data systems (ECDS) or administrative-only reporting on a schedule running from measurement year 2026 through 2030, with the stated goal of reducing reliance on manual medical-record review as digital clinical data quality improves (NCQA, 2024).
Can general-purpose LLMs like GPT-4o handle HEDIS evidence extraction accurately enough? Not consistently, without validation against a healthcare-specific alternative. In a blinded physician evaluation of clinical information extraction, reviewers preferred a healthcare-specific language model’s output over GPT-4o’s on factuality 35% of the time versus 24% (Kocaman et al., 2025). For evidence that feeds a regulatory submission, that gap needs to be measured on your own document set, not assumed.
What is the biggest technical risk in automating quality-measure evidence extraction? Assertion and temporal logic, not entity recognition. Correctly distinguishing a confirmed follow-up visit from a scheduled-but-not-completed one, or a documented screening from a declined one, and anchoring each to the correct date, determines whether extracted evidence actually satisfies a measure’s numerator logic rather than just mentioning the right clinical concept.
Does automating evidence extraction eliminate the need for human reviewers? No. Production frameworks route only uncertain or high-impact extractions to human review rather than replacing review entirely. Care-Connect and Spryfox built a framework that uses clinician-developed logic to filter and rank entities for manual review specifically to reduce reviewer workload while maintaining safety, not to remove the human step (Debes & Kiernan, 2024).
How does this evidence-extraction infrastructure relate to RADV audit preparation? The underlying requirement, linking every coded assertion to verifiable, date-bounded, provider-appropriate documentation, is the same for HEDIS/Stars evidence and for RADV audit defense. Martlet.ai’s RADV Audit Readiness Platform runs encounter detection, provider credential checks, and signature verification on every chart before submission, the same checks CMS auditors run after the fact (Martlet.ai, 2025).
What scale has this kind of clinical NLP pipeline been validated at? Ohio State University Wexner Medical Center runs its extraction and de-identification pipeline across more than 200 million Epic notes from seven hospitals (Huerta, 2025), and an independent audit of Providence St. Joseph Health’s roughly 2-billion-note de-identification pipeline found 98.6% PHI recall across 790 randomly sampled patients (Kocaman et al., 2025). Both validations ran on production data rather than curated test sets.
Where should a health plan start if it has no automated evidence-extraction pipeline today? Scope the one or two measures where unstructured evidence is most likely to be closing gaps you cannot currently see, such as follow-up-after-ED-visit or a chronic-disease measure with heavy narrative documentation, and build the extraction, normalization, and human-review pipeline around those measures first, with full source-document provenance from the start. Expand to additional measures as NCQA’s hybrid-measure retirement schedule reduces how much longer manual review remains an option at all.
