PII coverage is not HIPAA or GDPR coverage: what clinical de-identification requires

Clinical de-identification requires removing the 18 HIPAA Safe Harbor identifier categories, the GDPR Article 9 special categories, and the contextual identifiers that make a patient re-identifiable in a longitudinal record. General-purpose PII tools cover between 8 and 150 entity types, and none of them carries a label for a medical record number, a health plan beneficiary number, or the distinction between a patient’s name and their physician’s. Coverage gaps do not appear in an F1 score.

HIPAA-Grade PII Coverage: Where Presidio, AWS, Databricks and Google DLP Fall Short

Every major platform now ships a way to find and mask personal data in text. All of them work. The question for a healthcare team is what taxonomy they were built against.

Google’s list is the broadest, and Google’s own documentation is candid about what that breadth does not buy you. The infoType reference states that built-in detectors cannot guarantee compliance with regulatory requirements, and the concepts page explains why: categories like medical record numbers have no common industry definition, so the recommended approach is a generic identifier detector or a custom regex. Many infoTypes also need a contextual clue nearby before they fire, so a number in a discharge summary without MRN next to it goes unflagged.

The same limit runs through all six tools. They were designed for data that follows a format: a credit card has a checksum, an SSN has nine digits in a known pattern, an email has an @. Clinical identifiers are defined by their role in a document rather than by their form. OpenAI says as much in its own model card, which states that Privacy Filter is not an anonymization or compliance tool and that accuracy degrades on domain-specific text including medical.

What HIPAA Safe Harbor requires that PII taxonomies do not name

HIPAA offers two routes to de-identified status: Safe Harbor, requiring removal of all 18 enumerated identifier categories, and Expert Determination, requiring a qualified statistician to certify that re-identification risk is very small. Both are binary. The categories with no counterpart in a general PII taxonomy are the ones worth checking:

Two fields sit outside the enumerated list and matter in practice. Provider names are not patient PHI alone, but in a small specialty at a named facility, the treating physician plus a date range narrows the patient population sharply, and a single undifferentiated PERSON label cannot mask one while keeping the other for care-pathway analysis. Hospital and facility names carry the same risk.

None of the six general-purpose tools make the patient/provider split. Presidio’s default entity set has no facility-name recognizer and a single PERSON type. AWS Comprehend’s NAME type is explicitly defined to exclude organization names, which means a hospital name either goes undetected or falls to a separate, non-PII entity category, and there’s no split between a patient’s NAME and a doctor’s. Google Cloud DLP’s PERSON_NAME infoType has the same shape: one label for everyone in the document, and no infoType for a facility. OpenAI’s privacy-filter reduces every person to a single private_person span, and Databricks ai_mask() inherits whatever granularity the caller types into the label list, so the distinction exists only if someone thought to request “patient” and “doctor” as separate labels and the model reliably tells them apart, which the benchmark above shows it does not do reliably even for the labels it is given.

Naming more identifier types can make detection worse – the Databricks ai_mask results

A tool with no fixed taxonomy looks like the flexible option: name whatever the regulation requires and the tool masks it. Databricks ai_mask() was run against a public expert-annotated clinical set of 1,479 PHI chunks across 48 documents, using the same evaluation code as the published John Snow Labs comparison. Because ai_mask() returns masked text rather than spans, a token-level diff against the original recovered the offsets; everything downstream is the original scoring code.

Asking for the more specific identifier set a HIPAA-aligned taxonomy requires, including medical record number, social security number, account number, hospital name, and the patient-versus-doctor distinction, drops PHI F1 from 0.71 to 0.61. The configuration closest to the regulation performs worst.

Precision is competitive in every class while recall is not, so the shortfall is coverage rather than false positives.

At chunk level, 28.0% of annotated identifiers are never touched and another 20.8% are only partially covered. What the function does find, it labels correctly, with the entity type right on 422 of 423 matched dates, so the problem is not classification. Configurable labels are worth having, but they are not coverage: the completeness of your de-identification equals the completeness of a list someone typed into a SQL array literal, nothing checks that list against the regulation, and lengthening it toward the regulation costs accuracy.

What GDPR adds

GDPR takes a different approach: no enumerated list, a risk-based standard, and a distinction that catches teams out.

Pseudonymized data is still personal data. Article 4(5) defines pseudonymization as processing that prevents attribution without additional information, and Recital 26 is explicit that pseudonymized data stays within scope. Masking a name to [MASKED] produces pseudonymized data, and the full obligations still apply. Anonymization is the state in which GDPR no longer applies, and reaching it requires that re-identification be reasonably impossible given all means likely to be used, a standard that depends on the release context and the whole downstream pipeline, not on a model’s F1 score.

Most general-purpose PII tools don’t draw this line at all, let alone name it. OpenAI’s own model card for privacy-filter, for instance, states plainly that it is not an anonymization or compliance tool.

Article 9 special categories are a separate axis. Racial or ethnic origin, political opinions, religious or philosophical beliefs, trade union membership, genetic data, biometric data processed for identification, health data, and sex life or sexual orientation all receive heightened protection. Clinical notes are dense with these. A social history section routinely records ethnicity, religious practice affecting treatment, and sexual orientation, none of which is PII in the ordinary sense and none of which appears in a general PII taxonomy. Presidio’s NRP recognizer covers nationality, religious, and political groups in part; the other tools have nothing.

Beyond Masking: What Production-Grade De-Identification Requires

Suppose detection is solved and every identifier is found. Replacing each one with [MASKED] still produces a corpus that is compliant and close to useless. A note reading patient was seen by [MASKED] at [MASKED] on [MASKED] supports no cohort building, no downstream extraction, and no manual review. Masking also advertises its own gaps: a reader of a corpus of placeholders can see where the tool found something and, by absence, where it did not. Obfuscation removes that signal, because a missed identifier looks the same as a replaced one.

Production use of longitudinal clinical data needs four things beyond detection.

Consistent obfuscation. PHI is replaced with realistic surrogates rather than placeholders, deterministically, so the same patient maps to the same surrogate everywhere. Several constraints hold at once: if “Jane Doe” becomes “Nancy Smith,” the later bare “Jane” becomes “Nancy”; the surrogate stays feminine because the note records breast cancer; an age of 78 is replaced within the same band; and the mapping is keyed to patient ID, so the same name for a different patient gets a different surrogate.

Deterministic tokenization. An MRN, or a composite of name and date of birth, becomes a cryptographic hash, so records about the same person written years apart tokenize identically and link across systems without exposing the identifier.

Date shifting that preserves intervals. Each patient gets a consistent offset, so time-to-progression and readmission analyses survive. Dates have to be normalized before they can be shifted and returned in format: “April 2020” shifted by a random number of days comes back as “March 2020,” not “3/3/2020.”

The same treatment across modalities. A patient journey lives in notes, claims, scanned PDFs, DICOM images with burned-in text, FHIR bundles, and structured tables. Tokenization and obfuscation have to agree across all of them, or linkage breaks at the first modality boundary. A re-identification vault belongs alongside them: emergency unblinding, safety follow-up, and regulatory audit all need the mapping recoverable under access control, which is a different design than one-way masking.

What regulatory-grade coverage looks like in production

John Snow Labs Healthcare De-identification is purpose-built for this task rather than adapted from a general PII model, which is why the entity list starts where HIPAA and GDPR start: 23+ named PHI entity types covering all 18 Safe Harbor categories, with PATIENT and DOCTOR as separate labels, dedicated MEDICALRECORD, HEALTHPLAN, IDNUM, and PROFESSION entities, the ages-over-89 rule enforced in a contextual rule layer, and the Article 9 special categories included. The same pipeline handles masking, consistent obfuscation, deterministic tokenization, date shifting, and re-identification across text, PDF, DOCX, DICOM, and image inputs, in 7+ languages, inside your own environment.

The accuracy evidence is peer-reviewed and reproducible. On the official 2014 i2b2 test set the NER models reach 0.955 micro F1 with 7 labels and 0.978 with 13, published at ML4H 2023, and binary PHI recognition exceeds 95% in every supported language. On the expert-annotated clinical set used above, PHI F1 is 0.96, against 0.71 for ai_mask() and 0.86 to 0.91 for the current frontier models. At Providence, the pipeline de-identified 2 billion patient notes at over 99% PHI detection, validated by three months of external red-teaming, manual review of 35,000+ notes, equity analysis across gender, age, ethnicity, and geography, and adversarial re-identification testing on 790 patients that produced zero re-identifications. That methodology is published.

Before committing to a de-identification approach, run the coverage check rather than the accuracy check. List the identifier types in your own notes, map them against the 18 Safe Harbor categories and Article 9, and ask which the tool has a label for. A tool scoring 0.95 on the six entity types it recognizes has told you nothing about the twelve it does not, and a tool with configurable labels may score worse the more of them you ask for.

Learn more

Try it on a note of your own on the live demos and notebooks page, or run the clinical de-identification notebook on your own data in one click.[/vc_column_text][/vc_column][/vc_row]