AI Compliance Checklist for Healthcare Organizations: State Laws and HIPAA

Healthcare organizations deploying artificial intelligence face compliance obligations from multiple directions simultaneously: state AI laws, HIPAA requirements, FDA guidance for clinical decision support software, and emerging state-specific rules for particular care delivery contexts. The intersection of these frameworks is not always clean.

This checklist focuses on state AI regulations that have taken effect or are scheduled to take effect through 2026, with particular attention to California and Colorado's comprehensive AI Act, which covers healthcare as a category of consequential decision-making.

California SB 1120: The Physicians Make Decisions Act (Effective January 1, 2025)

SB 1120 amends the California Health and Safety Code and Insurance Code to restrict how health care service plans (HCSPs) and disability insurers use AI in utilization review.

Core Prohibition

A health plan or disability insurer may not deny, delay, or modify health care services based, in whole or in part, on a medical necessity determination made solely by an AI or algorithmic tool. A licensed healthcare professional must make every such determination individually, based on the specific member's clinical data.

Who Is Covered

Health care service plans licensed under the California Knox-Keene Act; disability insurers regulated under the California Insurance Code; and delegated entities performing utilization review on behalf of covered plans.

What Is Permitted

SB 1120 does not prohibit AI tools in utilization review. Plans may use AI to assist data review, flag cases, or streamline prior authorization workflows—provided a qualified human professional makes every final determination to deny, delay, or modify care.

Compliance Checkpoints

• ☐ Map every AI or algorithmic tool used in utilization review or prior authorization workflows
• ☐ Confirm no AI tool is configured to issue final denials, delays, or modifications autonomously
• ☐ Document the human review step in each workflow as a compliance record
• ☐ Train utilization management staff on SB 1120 requirements and escalation procedures
• ☐ Review vendor contracts to confirm AI vendors are not processing final adverse determinations on your behalf

California AB 3030: Generative AI Disclosure in Patient Communications (Effective January 1, 2025)

AB 3030 requires licensed health care providers to disclose when generative AI has been used to create written or verbal communications with patients.

Scope and Exemptions

The law covers licensed health care providers and applies when generative AI produces content communicated to patients, including clinical notes, care instructions, discharge summaries, or patient portal messages. Exemptions apply to communications substantially reviewed and modified by a licensed professional before delivery, and to purely administrative communications (scheduling, billing, appointment reminders).

Compliance Checkpoints

• ☐ Identify all patient-facing communications generated or drafted using generative AI
• ☐ Implement disclosure language where AI was used (e.g., "This message was drafted with AI assistance and reviewed by your care team.")
• ☐ Document review and modification processes where the provider substantially edits AI-drafted content
• ☐ Train clinical and administrative staff on which communications trigger disclosure obligations

AI Companion Chatbots in Mental Health: AB 2905

California's AB 2905 imposes specific disclosure and safeguard requirements for AI companion chatbots—systems designed to provide social or emotional support. Key requirements include disclosing AI identity at the start of each session and at defined intervals; implementing safeguards for users exhibiting signs of distress or who may be minors; and ensuring users cannot be deceived into believing they are communicating with a licensed therapist or other healthcare professional.

Colorado SB 205: Healthcare as a Consequential Decision Domain (Effective June 30, 2026)

Colorado's AI Act (SB 24-205) covers AI systems that make or materially influence decisions about healthcare services as high-risk AI. Healthcare organizations in Colorado using AI for clinical decision support, patient risk stratification, care pathway recommendations, or resource allocation should assess their obligations. Key obligations include pre-deployment and annual impact assessments; consumer notices before AI substantially influences a healthcare decision; human review rights for adverse AI-influenced decisions; and a risk management program aligned with NIST AI RMF or ISO/IEC 42001.

HIPAA and AI: Where the Frameworks Intersect

HIPAA does not specifically regulate AI, but its core requirements apply whenever AI systems process protected health information (PHI):

• Business Associate Agreements (BAAs): Any AI vendor that receives, creates, or transmits PHI on your behalf is a Business Associate under HIPAA. BAAs must be in place before deploying such systems.
• Minimum necessary standard: AI training datasets derived from patient data must be limited to the PHI necessary for the intended purpose.
• De-identification: PHI used to train or fine-tune AI models must meet HIPAA's de-identification standards (Safe Harbor or Expert Determination) unless a valid authorization or HIPAA exception applies.
• Risk analysis: HIPAA's Security Rule requires periodic risk analyses. AI systems that process ePHI should be included in scope.
• Right of access: If an AI system generates clinical notes or records that constitute PHI, patients retain their HIPAA right of access.

FDA Considerations for Clinical Decision Support

Clinical decision support (CDS) software meeting the FDA's definition of a device under the 21st Century Cures Act is subject to FDA oversight. The FDA's final CDS guidance indicates that AI-based tools presenting findings clinicians cannot independently review (black-box AI) are likely regulated as medical devices. Non-device CDS tools are still subject to state AI laws and HIPAA.

Healthcare AI Compliance Priorities by Organization Type

For a full index of state-by-state healthcare AI laws, visit our AI in healthcare regulations page.

This article is for informational purposes only and does not constitute legal advice. Consult a qualified attorney for guidance specific to your situation. Healthcare regulatory analysis requires qualified health law counsel.