Recent research shows that 25% of adults in the U.S. use consumer AI tools at least once a month to find health information and advice. However, these chatbots were not designed for healthcare and patient safety. These AI chatbots have shown limitations as a clinical tool, which is why it is critical for organizations, such as health plans, to intervene and offer plan members access to AI-enabled care solutions with built-in guardrails and a sensitivity and specificity framework.
Striking a balance between sensitivity and specificity is crucial for any organization offering an AI health care solution. As medical AI chatbots converse with and triage users, they must be able to accurately identify diseases while ruling out false positives.
In the medical field, the “traditional” definitions of sensitivity and specificity revolve around the goal and accuracy of a test. When it comes to AI triage, a technology that automatically guides patients to the best treatment options, these words have slightly different meanings.
Sensitivity is all about correctly detecting a condition. The more sensitive a test is, the better it is at identifying patients who have a particular condition. This is also known as a true positive.
For an AI triage solution to be sensitive, it must ask the right questions and suggest appropriate tests, with supervision from a licensed physician, of course. The goal of incorporating sensitivity into the system is to prevent underdiagnosing diseases or reporting false negatives, particularly for conditions that require prompt treatment and intervention.
Specificity is all about verifying the absence of a condition. The more specific a test is, the less likely it is to incorrectly identify a particular condition. This is also known as a false positive.
For an AI triage solution to be specific, it must ask strategic questions that help rule out various diseases. The goal is to confirm, as accurately as possible, that a patient does not have a particular condition. In this way, specificity avoids scaring people with uncertain diagnoses or advising them to continue testing.
The specificity and sensitivity difference is simple enough: The former minimizes false positives, while the latter reduces false negatives.
In AI-enabled virtual care, both matter. A false positive can lead to stress and anxiety about a non-existent condition as well as potential escalation to an ER visit, which costs, on average, $750. For payers with members in metropolitan areas, this translates to about $260 per insured member per year in potential ER costs. Meanwhile, a false negative can stop a patient from seeking the professional care they need, which could result in avoidable harm to the patient and potentially higher-cost care when they do eventually seek care.
One of the biggest challenges in developing a medical AI triage solution is striking a balance between specificity and sensitivity. For an AI to be useful, it needs to be simultaneously cautious about offering a diagnosis too soon and careful about missing a positive case. And it needs to do so without simply hiding behind a “This information is not medical advice” disclaimer.
At Counsel, we have implemented monitoring of both specificity and sensitivity into our agentic infrastructure, effectively and safely triaging care for our members. By combining the best of artificial and human intelligence, we can ask the right questions and more accurately identify condition status, leading to impactful value for members:
A smarter approach to sensitivity and specificity in triage can drive meaningful value for a health plan’s AI strategy:
Counsel is the safe and responsible medical AI solution designed for the next era. With built-in enterprise-grade security, we deliver care that is secure, continuous, and always available. By combining the speed of AI with physician oversight, our platform optimizes triage accuracy while continuously improving member guidance.
Counsel can become your responsible front door to care. Members receive instant AI answers and can chat with an in-house physician in minutes, enhancing care quality and operational efficiency.
Bring AI-enabled care to every member today. Learn more about partnering with Counsel.
The Counsel Health editorial team is a multidisciplinary group of writers and editors dedicated to delivering clinically grounded, evidence-based health information. Their work is informed by real-world care delivery and guided by physician expertise, ensuring content is accurate, accessible, and trustworthy. By translating complex medical topics into clear, practical guidance, the team helps readers understand their health, explore care options, and make informed decisions in a rapidly evolving healthcare landscape.
Dr. Cían Hughes is a physician-scientist with over a decade of experience in health AI research. He began his career as an academic surgeon and, in 2015, joined Google DeepMind as its first Clinical Research Scientist, helping to found the DeepMind Health team. Prior to DeepMind, he was an NIHR Academic Clinical Fellow in Otolaryngology at University College London, working across the UCL Ear Institute and the Farr Institute while maintaining clinical practice.
Our content is created for informational purposes and should not replace professional medical care. For personalized guidance, talk to a licensed physician. Learn more about our editorial standards and review process.
