Across industries, much of the conversation around safe and effective AI use has focused on bias and inaccurate outputs, like hallucinations. While these risks are real and important, as AI technologies evolve and become more accessible, the healthcare industry cannot overlook another critical factor: how humans are interacting with these systems. As AI gets better, there is a risk that humans get worse as we naturally drive for efficiency. As our systems from 80% to 99% accuracy, the human brain naturally stops looking for the 1% error, making vigilance an uphill battle. Healthcare leaders and clinicians must understand automation complacency and take intentional efforts to safeguard against it.

Automation complacency occurs when individuals become overly reliant on automated systems, leading to reduced vigilance and a decreased ability to critically evaluate outputs. Over time, this can result in a low index of skepticism. While this pattern appears across industries, it poses particularly high stakes in healthcare, where decisions directly affect patient outcomes. It is helpful to frame AI as an intern that performs research and provides a “first draft”.

Automation complacency occurs when individuals become overly reliant on automated systems, leading to reduced vigilance and a decreased ability to critically evaluate outputs.

Judgement can’t be automated

With ambient listening emerging as a widespread AI use case, even seemingly minor instances of automation complacency can impact care decisions in big ways. When automated documentation appears complete and accurate, it can be easy to assume it no longer requires a closer review. This is where automation complacency becomes a risk. Even when notes are read through, subtle inaccuracies can be missed. While documentation may look correct, clinician judgment is still required. Expertise still matters. Small errors can accumulate over time, gradually creating an inaccurate clinical story for both the patient and the care team.

Keeping clinicians engaged

Although clinical AI systems are carefully trained and governed, it remains imperative that clinicians stay engaged and skeptical consumers of AI. Curiosity in practice and a commitment to hands‑on, real‑world experience cannot be replaced by technology. AI can support clinical work, but it cannot substitute for deep clinical expertise or contextual understanding built over years of practice. The most effective way to combat automation complacency is to ensure correct clinical usage, with clinician judgment firmly in the driver’s seat.  AI‑generated results should prompt review and questioning, not automatic acceptance. Organizations can also support this mindset by implementing processes that require provider engagement and approval, rather than default acceptance. Even simple reminders and tools designed to keep clinicians in the loop can make a meaningful difference. From a software design standpoint, it is important that AI systems are transparent about confidence scores, reasoning, and references so that a user can take full ownership of reviewing and understanding the output before signing off.

Care at the center, tools in the background

With increased awareness and education around AI adoption and automation complacency, healthcare organizations can help providers remain in control while still benefiting from technologies that ease administrative burden and burnout. AI adoption should never be the goal on its own, nor should it simply be a matter of saving time. In the long term, AI adoption isn’t about adding modules, it’s about getting to an invisible EHR, reducing the friction of screens such that the technology recedes in the background, allowing human connection to take center stage AI is a means to enable clinicians to apply their best judgment and deliver exceptional care. These tools are meant to support not replace the expertise at the heart of medicine.