Demos
The HIPAA Security Rule requires covered entities to implement mechanisms that record and examine activity in systems containing electronic PHI (45 CFR §164.312(b)). Generative AI Lab 7.8 adds an Audit Logs Dashboard that records user and project activity across annotation, de-identification and LLM evaluation projects.
Designed to support HIPAA-compliant AI workflows, the Audit Logs Dashboard combines detailed audit trails, interactive visualizations, and filtering capabilities to enabling administrators investigate platform activity, monitor data movement, and maintain accountability across annotation, de-identification, and LLM evaluation projects.
Administrators can also block local imports and exports and limit project creation to admin users, these capabilities provide healthcare organizations with a framework for monitoring and managing sensitive AI workflows.
Why Do Healthcare AI Workflows Require Comprehensive Audit Logging?
Healthcare AI projects often involve multiple teams working with sensitive clinical information. Annotators correct model predictions and clinical experts review the results, and de-identification teams prepare datasets for research and model development.
Each interaction generates activity that may be relevant to compliance and security investigations.
When an organization needs to determine who accessed a project, when annotations changed, whether a dataset was exported, and how many API requests each user sent.
Traditional logging systems may capture these events, but investigating activity across multiple projects and users can require considerable manual effort.
Generative AI Lab addresses this challenge by bringing audit records and monitoring capabilities into a centralized interface, allowing administrators to examine platform activity in the context of the projects and workflows involved.
What Does the Generative AI Lab Audit Logs Dashboard Track?
The Audit Logs Dashboard records platform activity across users and projects, including annotation operations, data access and modifications, project lifecycle events, exports, and API activity.
Recorded events include information such as user identifiers, timestamps, API methods and operational context, creating an audit trail that administrators can examine during internal reviews or security investigations.
The Project Lifecycle Dashboard tracks into project creation, deletion, and export activity.
It consolidates project-level events into dedicated views, allowing administrators to examine when projects were created or removed, who performed those operations, and when project or task exports occurred.
For healthcare teams managing sensitive datasets, this provides a centralized record of important project operations and data movement.
The User Behavior and Access Dashboard focuses on how users interact with the platform.
Visualizations show API request volumes, activity patterns across different times and days, and activity trends across projects.
These views help administrators understand how the platform is being used and identify patterns that may require further investigation.
How Can Audit Logs Help Investigate Unusual Activity?
One of the advantages of centralized audit logging is the ability to investigate specific events without manually examining unrelated server logs.
Generative AI Lab allows administrators to filter recorded activity by project, user, event type, and date range. Interactive visualizations facilitate them recognize activity patterns and examine the events behind them.
Consider a synthetic de-identification project involving 400 fabricated discharge summaries assigned to three reviewers.
During a routine review of export activity, an administrator notices that one reviewer ran 46 exports in a week; the other two ran 4 and 2.
Using the Audit Logs Dashboard, the administrator can examine the export records, filter activity by user and date, and investigate whether the operations correspond to an authorized workflow or require further attention.
The dashboard shows potentially unusual activity, allowing administrators to investigate it. It does not independently detect security incidents or replace an organization’s security-monitoring procedures.
How Does Generative AI Lab Protect PHI While Maintaining Audit Trails?
Audit logging must provide sufficient information for investigations without exposing the sensitive data being processed.
Generative AI Lab records event metadata, including user identifiers, timestamps, API methods, and operational context, without storing sensitive request payloads.
This allows administrators to examine recorded activity without exposing the underlying clinical documents through the audit logs.
Log retention is configurable, enabling organizations to align audit-data retention with their internal policies and applicable regulatory requirements.
As with any auditing capability, its effectiveness depends on appropriate configuration and the broader security and governance measures implemented by the organization.
How Does Generative AI Lab Help Control Data Movement?
Visibility into platform activity is important, but healthcare organizations also need preventive controls over how sensitive information enters and leaves their systems.
Generative AI Lab complements audit logging with administrative settings that allow organizations to restrict local file imports and exports.
Administrators can disable local imports and require datasets to be imported from configured cloud storage services such as Amazon S3 or Azure Blob Storage. Similarly, local exports can be restricted so that datasets are directed to approved storage destinations rather than individual workstations.
Project-level exceptions provide flexibility for workflows with different data-handling requirements, such as development projects using synthetic data.
Administrators can also restrict project creation to users with administrative privileges.
Combined with audit logging, these controls help organizations establish more consistent data-governance practices across their healthcare AI workflows.
Supporting Accountability Across Healthcare AI Workflows
The value of centralized audit logging extends beyond individual security investigations.
For human-in-the-loop annotation projects, audit trails provide insight into reviewer activity and project operations. Administrators can examine recorded actions associated with annotation and validation workflows while maintaining accountability across teams.
In de-identification projects, audit logs allow organizations monitor activities associated with preparing, reviewing, and exporting sensitive datasets.
For compliance teams, centralized records provide documentation that can support internal reviews and regulatory audits. For security teams, visualizations and filtering capabilities make it easier to investigate unusual activity and reconstruct relevant events.
By combining audit logging with data-movement restrictions and administrative controls, Generative AI Lab provides a framework for monitoring AI workflows that involve sensitive clinical information.
Teams interested in the broader governance process can also explore the HIPAA-Compliant Human-in-the-Loop Workflows in Generative AI Lab webinar.
Frequently Asked Questions
What does the Generative AI Lab Audit Logs Dashboard track?
The dashboard records platform activity across projects and users, including annotation operations, project lifecycle events, data access and modifications, exports, and API activity. Events include timestamps and user information to support investigations and accountability.
Is the Audit Logs Dashboard HIPAA-compliant by default?
The dashboard is designed to support HIPAA compliance, but audit logging must be enabled during installation or upgrade. Organizations must also configure appropriate access controls, retention policies, and other safeguards to meet their compliance requirements.
Does audit logging expose PHI in the logs themselves?
No. The logging infrastructure captures event metadata without recording sensitive request payloads, allowing administrators to investigate platform activity without exposing the underlying clinical documents through the logs.
Can administrators restrict local file imports and exports?
Yes. Administrators can disable local imports and exports globally, directing data movement through configured cloud storage services. Project-level exceptions accommodate workflows with different requirements.
Does the dashboard automatically detect suspicious activity?
The dashboard provides visualizations and filtering capabilities for identifying and investigating potentially unusual activity. It does not independently classify events as security incidents or replace dedicated security-monitoring procedures.
Does Generative AI Lab support human-in-the-loop validation alongside audit logging?
Yes. Generative AI Lab supports human-in-the-loop workflows in which clinical experts validate and correct AI-generated annotations. Audit trails provide accountability for relevant annotation activity, while the Audit Logs Dashboard offers broader oversight of platform operations.
Deploy Generative AI Lab
Generative AI Lab is available on AWS Marketplace and Azure Marketplace, with audit logging and governance controls available out of the box.
