how automated healthcare fails, how you'd know, and what to do at each tier — every claim sourced, reviewed continuously
The moment automation hands work back to a person who has stopped watching, stopped practising, or stopped hearing the alarms.
The human handoff layer is everything that has to be true of people for an automated hospital to fail safely: that clinicians notice when a machine is wrong, that someone owns each alarm and alert, that staff can take over when automation stops, and that they can still run the ward on paper. It covers automation bias (following or waiting for the machine), out-of-the-loop performance (taking over without situation awareness), skill atrophy and deskilling, alarm and alert fatigue, and downtime competence.
The problem is old. Bainbridge's 1983 'Ironies of Automation' observed that automating a process leaves the operator with the tasks the designer could not automate, including taking over in abnormal conditions, while the manual skills needed for that takeover deteriorate when they are not used. Parasuraman and Riley (1997) described misuse (over-reliance) and disuse (ignoring automation, commonly after false alarms). Endsley and Kiris (1995) showed that operators of an automated system were slower to decide after it failed, and linked this to lower situation awareness from passive monitoring.
Healthcare now has direct evidence of each mechanism. Experienced radiologists' accuracy collapsed when a purported AI suggested the wrong BI-RADS category (Dratsch 2023). Endoscopists' adenoma detection rate in non-AI colonoscopies fell from 28.4% to 22.4% after AI was introduced (Budzyń 2025). The Joint Commission logged 98 alarm-related sentinel events, 80 of them deaths, between January 2009 and June 2012. During EHR downtime, safety reports show downtime procedures missing or not followed in 46% of cases (Larsen 2018), and clinicians at Ascension in 2024 described medication errors and delayed labs after the switch to paper.
FailSystems viewIn an automated hospital the human is no longer the primary operator but the fallback, and a fallback that is never exercised decays. Every other layer on this site eventually fails into this one: when power, network, devices or models go, the plan is 'staff take over', and that plan silently assumes skills, attention and alarm ownership that automation has been eroding. The dangerous moment is not the outage itself but the handback, when a person with stale skills and little context is asked to act fast. We judge that rehearsal (unaided practice at tier 0, drills at tiers 2 and 3) is the only defense that addresses the cause rather than the symptom, and that it is the one hospitals most often cut.
Clinicians over-rely on decision support, making errors of commission (following incorrect advice) or omission (not acting because the system did not prompt). Reliance increases with workload, time pressure and task complexity, and when verifying the output is hard. Experience reduces but does not remove it: in a mammography experiment, very experienced readers' accuracy dropped from 82.3% to 45.5% when the AI suggestion was wrong.[1,2,3,4,5,6]
Warning signs
Seen inWrong AI suggestions pull radiologists' mammogram ratings off
When automation disconnects or fails, the person taking over has been monitoring passively and lacks current situation awareness. Endsley and Kiris found slower decisions after an expert system failed, attributed mainly to the shift from active to passive processing. Bainbridge noted that manual operators need 15 to 30 minutes to build a feel for a process before taking over, time an automated handback rarely allows.[7,8,9,10]
Warning signs
Seen inAir France 447: autopilot hands a stalled aircraft back to a crew (non-clinical analogue), Uber automated test vehicle kills pedestrian while safety operator is distracted (non-clinical analogue)
Skills that automation performs routinely are practised less and deteriorate, so a formerly experienced operator becomes an inexperienced one at the moment of takeover. In four Polish endoscopy centres, the adenoma detection rate of standard (non-AI) colonoscopy fell 6.0 percentage points in the three months after AI was introduced. Trainees who learn with automation from the start may never build the unaided skill at all (judgement; not yet measured).[11,8]
Warning signs
Seen inEndoscopists detect fewer adenomas without AI after AI is introduced
Most device alarm signals do not require clinical intervention (the Joint Commission cites estimates of 85 to 99 percent), so clinicians become desensitised and turn alarms down, off, or outside safe limits. Parasuraman and Riley described this as disuse: false alarms teach operators to ignore automation. Alarm fatigue was the most common contributing factor in alarm-related sentinel events reported to the Joint Commission.[12,13,14,5]
Warning signs
Seen inICU patient dies after alarms go unanswered for an hour, Patient dies while a heart monitor's crisis alarm is switched off
An alarm only protects a patient if a specific person hears it and is responsible for acting. Joint Commission data list alarms not audible in all areas (25 events), alarms inappropriately turned off (36) and inadequate staffing to respond among contributors. The Joint Commission's alarm-safety goal (NPG.01.05.01 from January 2026, previously NPSG.06.01.01) requires hospitals to define who may set, change and turn off alarm parameters, but it generally excludes CPOE and other IT alerts, so AI and EHR alerts can fall outside any alarm-ownership policy.[12,15,16,17,18]
Warning signs
Seen inICU patient dies after alarms go unanswered for an hour, Epic Sepsis Model missed two-thirds of sepsis cases in external validation, Patient dies while a heart monitor's crisis alarm is switched off
Decision-support alerts with low specificity are overridden routinely: a review found drug safety alerts overridden in 49% to 96% of cases. A widely deployed sepsis model generated alerts for 18% of all hospitalised patients while missing 67% of sepsis cases at one academic centre, a burden the authors described as alert fatigue. Clinicians learn to click through, including on the rare correct alert.[19,20]
Warning signs
Seen inEpic Sepsis Model missed two-thirds of sepsis cases in external validation
When the EHR or other systems go down, staff must run care on paper, but ONC notes that many organisations have employees who do not know how to work in a paper-based environment. Safety reports tied to downtime cluster in lab orders and results and medication, and patient identification and communication fail. Paper workflows are slower (lab results averaged 62% longer in one study) and the supporting infrastructure, such as forms and fax machines, may no longer exist.[21,22,23,24,25]
Warning signs
Ontario's Auditor General found every AI scribe on the province's approved vendor list had fabricated, wrong or missing content in procurement tests, including 12 of 20 recording the wrong drug.[26,27]
A pharmacy chain's AI voice and text assistant for refills mispronounced medications, ordered wrong dosages and duplicate refills, and gave callers no keypad alternative; after hundreds of complaints the chain pulled it back.[28,29]
After AI polyp detection was introduced, the adenoma detection rate of standard non-AI colonoscopy fell from 28.4% to 22.4%.[11]
An automated sepsis alert triggered a protocol for large-volume IV fluids in a dialysis patient; the nurse objected, was told to follow the protocol, and a physician intervened.[30,31]
A Class I software correction found that backlogged EHR-to-pump automated programming requests could load stale rate, dose or volume parameters.[32,33]
Researchers and an AP investigation found OpenAI's Whisper speech-to-text model inserting fabricated sentences, and a Whisper-based clinical scribe used by over 30,000 clinicians erased the source audio, removing the way to check.[34,35]
A faulty Rapid Response Content update to CrowdStrike's Falcon sensor crashed about 8.5 million Windows devices worldwide. Outside-in measurement found disrupted services at 759 of 2,232 US hospitals studied.[36,37,38,39,40,41,42]
PathDevices → Connectivity & data → Human handoff
Ransomware hit Synnovis, the pathology provider for several south-east London NHS trusts and GP practices. Blood testing and matching collapsed, more than 11,000 appointments and procedures were postponed, O-type blood ran short nationally, and one death was later partly attributed to a delayed result.[43,44,45,46,47,48,49,50]
PathConnectivity & data → Human handoff
A ransomware attack took Ascension's electronic records offline for about five weeks. Clinicians told KFF Health News of medication errors and delayed lab results, and one said he had no training for the attack; Ascension said its care teams were trained for such disruptions.[24,51]
Attackers used stolen credentials on a Change Healthcare Citrix remote-access portal that had no multi-factor authentication, then deployed ransomware nine days later. Disconnecting the clearinghouse stalled pharmacy claims, medical claims and payments across the US.[52,53,54,55,56]
PathConnectivity & data → Human handoff
A class action alleges UnitedHealth used naviHealth's nH Predict model to cut off post-acute care; a Senate investigation found UnitedHealthcare's post-acute denial rate nearly tripled while it automated prior authorization.[57,58,59,60]
In a controlled experiment, 27 radiologists' accuracy on mammograms fell sharply when a purported AI suggested an incorrect BI-RADS category.[3]
An independent validation of Epic's proprietary sepsis score found it far less accurate than the vendor reported: it missed 67% of sepsis cases while alerting on 18% of all hospitalizations.[20,61]
A month-long ransomware attack on a health system with about 25% of regional inpatient discharges drove patients and ambulances to two unaffected academic EDs, raising their census, waits and stroke activations.[62,63]
PathConnectivity & data → Human handoff
Freezing weather knocked out generation and forced the largest controlled load shed in US history. Power loss spread to water systems and hospitals, and to patients at home on powered medical equipment.[64,65,66,67,68,69]
PathPower → Devices → Human handoff
After go-live, the new EHR routed more than 11,000 clinical orders to a hidden queue instead of the intended service, without telling the ordering clinician; VHA identified 149 adverse events.[70]
Weeks after its first COVID-19 admissions, the University of Michigan paused Epic sepsis alerts because dataset shift produced spurious alerting; its clinical AI committee decommissioned the model.[71,72]
Non-clinical analogue. The human operator meant to back up an automated driving system was looking at her phone; the NTSB cited automation complacency.[10]
A self-spreading ransomware worm infected 34 English trusts and 603 primary-care and other NHS organisations, and at least 46 more trusts were disrupted. Thousands of appointments were cancelled and five hospitals diverted ambulances.[73,74,75]
PathConnectivity & data → Devices → Human handoff
Ransomware made the EHR inaccessible; the hospital moved to paper within an hour and restored computers after 36 hours.[63,76,77]
Storm surge flooded basements holding fuel tanks and pumps at two Manhattan hospitals whose generators sat on upper floors. Both hospitals evacuated.[78,79,80]
A patient on a cardiac monitor died after the monitor's crisis alarm had been left off; lower-level alarms sounded at the nurses' station but went unheeded.[81,82]
A 60-year-old ICU patient's monitor alarmed for rising heart rate and falling oxygen saturation; staff responded only after about an hour, when he had stopped breathing.[12]
Non-clinical analogue. Iced pitot probes caused inconsistent airspeeds, the autopilot disconnected, and the crew did not recognise or recover from a stall.[9,83]
After city power failed, Memorial ran on generators that failed as floodwater rose. 45 bodies were later recovered from the hospital.[84,85,86]
A network loop took down clinical applications at an academic medical centre for about four days, forcing a return to paper it had abandoned years earlier.[87,88,89,21]
What should already be in place at each degradation tier for this layer. Tier 0 is normal automated running; tier 3 is paper, batteries and judgement.
These are practices reported or recommended in the cited sources, gathered for reference. They are not a prescription for your organisation; judge what fits your setting, and check the current official text of any standard.
| Instrument | What it requires |
|---|---|
| Joint Commission NPG.01.05.01, clinical alarm safety (hospitals and critical access hospitals, from January 2026; previously NPSG.06.01.01, effective 2014) | Leaders make alarm safety a priority, identify the most important alarm signals, and set policies on alarm settings, when alarms may be disabled or changed, who has authority to set, change or turn them off, and monitoring and response; staff must be educated on the alarm systems they are responsible for. It generally does not cover CPOE or other IT alerts.[16] |
| 42 CFR 482.15(d), CMS Conditions of Participation: emergency preparedness training and testing | Hospitals must train all staff in emergency procedures initially and at least every 2 years, demonstrate staff knowledge, and run at least two exercises a year (one full-scale or functional, one additional such as a tabletop), analysing and documenting each.[93] |
| ONC SAFER Guide: Contingency Planning (2025 revision) | Voluntary self-assessment. Recommends paper forms for key EHR functions, training and testing staff on downtime and recovery (including unannounced drills at least yearly and read-only backup EHR use), EHR-independent communication, and review of downtimes over 24 hours.[21] |
| FDA, Clinical Decision Support Software guidance (January 29, 2026) | Non-binding. Defines automation bias and treats the level of automation and the time-critical nature of the decision as factors in whether a clinician can independently review the basis of a recommendation (criterion 4 for non-device CDS).[4] |
| IEC 60601-1-8:2006+A1:2012 (Ed. 2.1), alarm systems collateral standard | Requirements and tests for alarm systems in medical electrical equipment: alarm categories by urgency, consistent alarm signals and control states, and their marking.[91] |
The EU AI Act (Regulation (EU) 2024/1689) is the first law to name automation bias. Article 14(4)(b) requires high-risk AI systems to be provided so that overseers can remain aware of the tendency to over-rely on outputs, and 14(4)(d)-(e) require that they can disregard or override an output and stop the system safely. Article 26(2) requires deployers, such as hospitals, to assign human oversight to people with the necessary competence, training, authority and support. In England, DCB0160, mandated under section 250 of the Health and Social Care Act 2012, requires care organisations to apply clinical risk management to the deployment and use of health IT, which is where local handoff and downtime hazards are expected to be logged.[94,95,96]
FailSystems judgementJudgement. Likelihood is high because alarm fatigue, alert override and weak downtime readiness are documented as common, not exceptional. Blast radius is usually one patient or one unit per event, but rises to system-wide when a large outage forces a whole network onto paper. Detectability is poor because skill loss and automation bias stay invisible until the handback, and routine metrics measure assisted rather than unaided performance.
Each factor is scored 1–5 and multiplied, as in a classic FMEA risk priority number. This is our first-draft judgement, not a measurement; see how scoring works and how it will be revised.
These gaps drive what the nightly research pass looks for. If you have evidence, send it.
Cite this pageFailSystems. “The human handoff.” https://failsystems.health201.com/layers/handoff/ (reviewed 2026-09-27). Health 201 / AstroNexus LLC. CC BY 4.0.
Information only, not advice. FailSystems is an aggregation and synthesis of published sources. It is not consulting, engineering, legal, regulatory or medical advice, and using it creates no professional relationship. Health systems are complex and no approach fits every organisation: anything you adopt is your own decision, at your own risk, and should be checked against the current official sources and by qualified people who know your setting. Full disclaimer.