Ingest external HL7 CDA documents without patient linkage

Upload HL7 and CDA files, ingest them as documents in their own right with no patient or OMOP person record attached, and group them into a dataset you can curate, extract from, or de-identify. The page stops at the point where the dataset is handed to a downstream job. If your external documents arrive as FHIR rather than CDA, use Ingest external FHIR documents without patient linkage instead.

Ingest External CDA Documents Without Patient Linkage

This video has no audio. It is a visual walkthrough of the screen.

Who this is for

Anyone who has to do something useful with clinical documents that arrived from outside, about people who are not in the local database: interoperability and integration engineers who receive the feed, data engineers who run ingestion jobs, privacy and de-identification officers preparing external documents for release, real-world evidence teams sourcing facts that never reached a structured system, and the platform administrator who grants the permissions.

Why it matters

Clinical Document Architecture (CDA) and its Consolidated CDA (C-CDA) templates are what health information exchanges, referral networks, and prior-vendor archives actually send. They arrive in volume, often as a bulk drop rather than a per-patient request, and a large share of them describe people your OMOP database has never seen.

The documents still hold the facts a study or a de-identification job needs. A C-CDA continuity of care document carries the discharge summary, the pathology report narrative, the medication list, and the progress notes behind a referral, sectioned and coded well enough to work with. Forcing each one into a patient record before you can touch it means creating person records for people you may never link, on identifiers you have not verified, which is expensive to unwind once cohorts are built on top of them.

Document-first ingestion makes the files addressable without writing anything to the patient layer. The record stays clean, and the option to link later stays open.

What you gain

A set of external CDA and HL7 documents that are ingested, validated, and queryable through their ingestion lineage, with no patient or OMOP person record created. From those you get a reusable dataset, defined by the ingestion job it came from, that feeds document-level curation, information extraction, and de-identification.

The limits are worth stating plainly: these documents do not appear in patient-level views such as Patient Timeline or Cohort Builder, because nothing was linked. That is the trade you are making on purpose, and the ingestion lineage is what lets you audit it, or link the same documents later once identity is confirmed.

Before you start

• Valid HL7 and CDA files, prepared and readable. Malformed files surface as validation errors during upload, not silently later
• Confirmation that the file format is supported by the ingestion configuration you plan to select. An unsupported format fails at ingestion or, worse, at the downstream job after you have built a dataset on it
• An appropriate pipeline or template available in the deployment
• For De-Identification, a profile compatible with the document format. Profiles are format-specific, and a mismatch here is the most common reason a de-identification job produces nothing usable
• Permission to create ingestion jobs and datasets

Step 1: Upload the external documents

1. In the left navigation, go to Data Integration > Ingestion Jobs.
2. Select Add Ingestion.
3. Choose Local Upload.
4. Upload the HL7 and CDA files.
5. Wait for file validation to complete.
6. Review validation warnings or errors before continuing.
7. Continue without assigning the documents to an existing or new patient.

Step 2: Configure document-first ingestion

1. Enter a clear ingestion job name and description.
2. Select an ingestion purpose or configuration that supports external documents.
3. Select or create the appropriate pipeline configuration.
4. Enable the processing steps required for the intended workflow.
5. Leave patient- and OMOP-dependent steps disabled unless the documents will later be linked.
6. Review the detected document metadata and format information.
7. Continue to Review & Start.
8. Verify that patient assignment is not required.
9. Select Start Ingestion.

Step 3: Validate the ingestion result

1. Monitor the job from the Ingestion Jobs page.
2. Open the job details page after processing completes.
3. Confirm that the expected number of documents was ingested.
4. Review the document-ingestion step, its logs, and its validation results.
5. Confirm that the documents are available even though no patient or OMOP record was assigned.
6. Resolve failed or unsupported documents before creating the dataset.

Step 4: Create a dataset from the ingested documents

1. In the left navigation, go to Data Curation Studio > Dataset Builder.
2. Select Source Document Filter as the dataset mode.
3. Select Unstructured, or the applicable document data type.
4. Select the completed external-document ingestion job.
5. Apply document filters if needed, such as file name, document type, source, ingestion job, or document metadata.
6. Review the document preview and the total document count.
7. Confirm that the dataset does not require a cohort or patient selection.
8. Enter a dataset name and description.
9. Review the dataset configuration.
10. Select Create Dataset.

Step 5: Use the dataset in a downstream workflow

Once the dataset exists, it behaves like any other document dataset in the supported document-level workflows.

Data Curation

1. Open Data Curation Studio > Curation Jobs.
2. Start a new curation workflow.
3. Select the external-document dataset.
4. Use document-level extraction when patient-level consolidation is not required.
5. Review the extracted results and their linked source evidence.

Information Extraction

1. Open Information Extraction.
2. Create a new extraction job.
3. Select the external-document dataset.
4. Configure the required extraction pipeline.
5. Review the extracted entities, attributes, and evidence.

De-Identification

1. Open De-Identification.
2. Start a new de-identification job.
3. Select the external-document dataset.
4. Select a profile compatible with the document format.
5. Run the job and review the detected PHI.
6. Review and export the de-identified results according to your governance requirements.

Expected result

Your HL7 and CDA documents are ingested without patient assignment, and no new patient or OMOP person record is required to hold them. The documents remain available through their ingestion lineage, a reusable dataset can be created directly from the ingestion output, and that dataset can be used for document-level curation, information extraction, or de-identification.

Confirm the ingestion is correct

• The uploaded file count matches the ingested document count.
• Unsupported or failed files are clearly identified.
• No unintended patient or OMOP records were created.
• The dataset references the correct ingestion job.
• Dataset preview counts match the intended document scope.
• The selected downstream workflow supports the document format.
• The original ingestion and dataset lineage remain available for auditing.

Limitations

• Supported formats depend on the ingestion configuration you select. Confirm format support before uploading rather than after.
• The pipeline or template you need must already be available in the deployment.
• De-identification profiles are format-specific. A profile that does not match the document format will not process it.
• Failed or unsupported documents must be resolved before the dataset is created, otherwise the dataset scope will not match what you uploaded.
• Documents ingested this way stay outside patient-level analysis. They are not reachable from cohort or patient-level workflows until they are linked.
• Creating ingestion jobs and datasets requires permissions that depend on how your deployment assigns roles.