The Utah Medical Licensing Board recently ordered the immediate suspension of a pilot program involving Doctronic, a company deploying AI chatbots to manage prescription renewals for chronic conditions. The board intervened because the state launched the system without established clinical oversight, raising concerns about potential risks to patient safety. This incident underscores the challenges of regulating adaptive clinical software within existing frameworks.

Traditional regulatory mechanisms, such as the FDA’s device-approval process, are designed for static products rather than adaptive systems that evolve through updated data and capabilities. Consequently, states are enacting fragmented regulations, with at least 47 states currently considering over 250 bills governing clinical AI. Evidence supporting these systems is growing: a 2025 prospective study of nearly 40,000 primary care visits in Kenya found that AI-supported clinicians made fewer diagnostic errors. Additionally, the NOHARM trial, published in December, indicated that large language models perform comparably to physicians on routine clinical tasks.

Alon Bergman, an assistant professor of medical ethics and health policy at the University of Pennsylvania, proposed a national licensure framework to address these regulatory gaps. This model suggests shifting authority from the FDA to a new federal Office of Clinical AI Oversight within the Department of Health and Human Services (HHS). The proposed framework includes four core requirements: models must meet or exceed median human scores on USMLE and specialty boards, maintain defined scopes of practice, undergo biennial performance monitoring, and operate under federal preemption to prevent duplicative state-level assessments.

The Utah Medical Licensing Board recently ordered the immediate suspension of a pilot program involving Doctronic, a company deploying AI chatbots to manage prescription renewals for chronic conditions. The board intervened because the state launched the system without established clinical oversight, raising concerns about potential risks to patient safety. This incident underscores the challenges of regulating adaptive clinical software within existing frameworks.

Traditional regulatory mechanisms, such as the FDA’s device-approval process, are designed for static products rather than adaptive systems that evolve through updated data and capabilities. Consequently, states are enacting fragmented regulations, with at least 47 states currently considering over 250 bills governing clinical AI. Evidence supporting these systems is growing: a 2025 prospective study of nearly 40,000 primary care visits in Kenya found that AI-supported clinicians made fewer diagnostic errors. Additionally, the NOHARM trial, published in December, indicated that large language models perform comparably to physicians on routine clinical tasks.

Alon Bergman, an assistant professor of medical ethics and health policy at the University of Pennsylvania, proposed a national licensure framework to address these regulatory gaps. This model suggests shifting authority from the FDA to a new federal Office of Clinical AI Oversight within the Department of Health and Human Services (HHS). The proposed framework includes four core requirements: models must meet or exceed median human scores on USMLE and specialty boards, maintain defined scopes of practice, undergo biennial performance monitoring, and operate under federal preemption to prevent duplicative state-level assessments.