Skip to main content

Skill Guide

Risk management per ISO 14971 with AI-specific hazard analysis (data drift, distributional shift, automation bias)

The systematic application of the ISO 14971 risk management process to medical devices, extended with specific risk analysis techniques for hazards arising from the unique failure modes of artificial intelligence components, such as data drift, distributional shift, and automation bias.

This skill is critical for ensuring the safety and regulatory compliance of AI-enabled medical devices, directly mitigating patient harm and enabling market access. It translates directly to reduced liability, accelerated regulatory approval, and the commercial viability of AI-driven healthcare products.
1 Careers
1 Categories
9.1 Avg Demand
15% Avg AI Risk

How to Learn Risk management per ISO 14971 with AI-specific hazard analysis (data drift, distributional shift, automation bias)

1. Master the core ISO 14971 standard lifecycle (risk analysis, evaluation, control, monitoring). 2. Learn the fundamental AI/ML concepts (training data, model performance, inference) and their potential failure modes. 3. Understand how traditional hazards (e.g., electrical, mechanical) are distinct from data-driven hazards.
1. Apply the risk management process to specific AI use cases (e.g., an algorithm for detecting sepsis from EHR data). 2. Develop and document hazard analyses that explicitly include AI-specific scenarios (e.g., 'Model performs well on training data but fails on data from a different hospital due to distributional shift'). 3. Avoid common mistakes like treating the AI model as a 'black box' or failing to trace hazards back to data sources.
1. Architect risk management systems for complex, adaptive AI platforms (e.g., continuously learning systems). 2. Align risk management plans with broader QMS (e.g., ISO 13485) and software lifecycle (e.g., IEC 62304) requirements. 3. Mentor teams on interpreting regulatory expectations (e.g., FDA's AI/ML SaMD Action Plan) and defending risk control measures to auditors.

Practice Projects

Beginner
Project

Risk Management File for a Static AI Diagnostic Model

Scenario

You are tasked with creating the initial risk management file for a chest X-ray analysis algorithm that is trained on a fixed dataset and not updated post-deployment.

How to Execute
1. Define the intended use and misuses. 2. Use a top-down (FTA) and bottom-up (FMEA) approach to identify hazards, focusing on model false negatives/positives and input data quality failures. 3. Create a hazard traceability matrix linking each hazard to a source, risk control (e.g., human-in-the-loop), and residual risk evaluation. 4. Write a risk management report summarizing the file's contents and conclusion.
Intermediate
Case Study/Exercise

Post-Market Surveillance for a Drifting Algorithm

Scenario

A deployed AI algorithm for diabetic retinopathy screening is showing a gradual decline in performance at a new clinic partner, suspected to be due to differences in camera hardware (distributional shift).

How to Execute
1. Define the monitoring metrics (sensitivity, specificity) and statistical process control thresholds. 2. Trace the performance drop to a specific input feature (image resolution/color balance). 3. Update the risk management file to add a new hazard: 'Inadequate image quality from unvalidated hardware.' 4. Propose and document risk controls: a) input data validation layer, b) mandatory re-validation for new hardware, c) labeling requirement for source data quality.
Advanced
Case Study/Exercise

Defining the Predetermined Change Control Plan (PCCP) for a Continuously Learning AI

Scenario

A company wants to deploy an AI model for predicting patient deterioration that is designed to retrain monthly on new clinical data from the deployment site. The FDA requires a PCCP for the anticipated modifications.

How to Execute
1. Define the algorithm's desired modifications (retraining with site-specific data) within a locked performance envelope. 2. Specify the exact metrics and acceptance criteria for the retrained model (e.g., AUC must not decrease by more than 5%). 3. Document the change protocol, data sources, and automated validation tests. 4. Integrate this PCCP into the overall risk management plan, showing how it controls the hazard of model degradation due to data drift without requiring a new 510(k) for each update.

Tools & Frameworks

Standards & Regulatory Guidance

ISO 14971:2019IEC 62304:2006+AMD1:2015FDA Guidance: AI/ML-Based Software as a Medical Device (SaMD) Action PlanEU MDR & IVDR Annex I (General Safety and Performance Requirements)

The foundational framework. ISO 14971 provides the process. IEC 62304 governs the software lifecycle. The FDA guidance defines modern expectations for AI/ML, including PCCPs. EU MDR provides the legal safety requirements.

Risk Analysis Techniques

Fault Tree Analysis (FTA)Failure Modes and Effects Analysis (FMEA)Hazard Traceability Matrix (HTM)Use-Related Risk Analysis (URRA)

FTA is top-down for system-level hazards. FMEA is bottom-up for component failures (e.g., specific AI model outputs). HTM is the critical document linking hazards to controls and verification. URRA is essential for assessing automation bias and clinician over-reliance.

AI/ML Development & Monitoring Tools

MLflow / Weights & Biases (for experiment tracking)Great Expectations / TensorFlow Data Validation (for data drift detection)Fairlearn / AIF360 (for bias detection)Custom dashboards for model performance (AUC, recall, precision over time)

These tools are used to implement technical risk controls. Track model versions and data lineage. Continuously monitor input data distributions and model performance metrics in production to detect drift and trigger predefined responses.

Interview Questions

Answer Strategy

Structure the answer using the ISO 14971 process phases. For the hazards, move beyond generic software risks to concrete AI failure modes. Sample Answer: 'I'd start with intended use definition, then apply both FTA and FMEA. My top three AI-specific hazards would be: 1) **False Negative Leading to Delayed Diagnosis**, traced to a failure in detecting subtle findings due to a lack of representative training data. 2) **Automation Bias Leading to a Missed Finding**, a use-related hazard where the clinician defers to the AI and fails to review the study themselves. 3) **Performance Degradation from Data Drift**, where the model's accuracy declines over time as the patient population or imaging equipment changes without detection.'

Answer Strategy

This tests operational risk management and post-market surveillance. The answer must demonstrate a closed-loop process. Sample Answer: 'Immediately, I would trigger the post-market surveillance protocol: 1) Verify the performance metric and assess if it breaches a predefined acceptance threshold, constituting a safety signal. 2) If so, initiate a correction under the risk management plan, potentially adding a human review step for this site while investigating. Long-term, I'd root-cause the drift-likely a distributional shift in the data. I'd then update the risk management file, adding or modifying this hazard with new controls like site-specific recalibration or data pre-processing, and document the decision-making per ISO 14971.'

Careers That Require Risk management per ISO 14971 with AI-specific hazard analysis (data drift, distributional shift, automation bias)

1 career found