Seattle's AI 911 Routing: Critical Failure in Transparency

Quick Verdict

Seattle's implementation of an AI system to triage and route 911 medical calls, developed by Denmark-based Corti, is a deeply problematic deployment of technology in a critical public safety service. Operating without the knowledge of callers or any formal public review, this system fundamentally undermines trust and accountability. While the stated intention might be efficiency, the execution demonstrates a severe lapse in ethical governance, transparency, and public oversight, leading to potentially dangerous outcomes and raising serious questions about the city's approach to AI.

Key Details: What This AI System Does

At its core, the Seattle Fire Department (SFD) has been using artificial intelligence to assist dispatchers with medical 911 calls since December 2023. This AI, provided by Corti, actively listens to all incoming medical 911 calls. Its primary function is to analyze these conversations and provide prompts to human dispatchers, suggesting that certain patients be diverted from an ambulance response to a nurse-staffed call center located in Texas. This system has been operating live for over two years, influencing critical decisions about emergency response without public awareness. The SFD's medical director has indicated that this technology has contributed to an increase in calls routed to the nurse line, though specific figures are reportedly disputed. Meanwhile, other areas in Washington, including Snohomish, Kitsap, and the Tri-Cities, are reportedly adopting similar AI agents for their emergency and non-emergency lines.

Design & User Experience: A Veil of Secrecy

From a design and user experience perspective, this system is a profound failure of public-facing technology. The most glaring flaw is the absolute lack of transparency. Callers to 911, often in moments of extreme distress or emergency, have no knowledge that an AI is actively listening to their medical calls and influencing the routing decision. This constitutes an implicit user experience where a critical element of service delivery is hidden. Seattle's own surveillance ordinance requires review for technologies that observe individuals and raise social justice concerns, yet this AI system was never assessed under it. This oversight directly impacts the user's right to know how their emergency is being handled and undermines trust in public services. University of Washington law professor Ryan Calo aptly pointed out that "A person who is erroneously routed outside of the 911 environment has a right to know how it happened." The 'design' here prioritizes backend efficiency over transparent, informed public interaction.

Pros and Cons: Balancing Efficiency with Ethical Blunders

Pros:

Potential for Resource Optimization: The SFD's medical director suggests the AI has driven an increase in calls routed to a nurse line, implying a potential for diverting non-critical cases from ambulance dispatches, thereby saving resources and potentially speeding up responses for more severe emergencies. Dispatchers reportedly retain final authority, theoretically acting as a human failsafe.

Cons:

Lack of Transparency and Public Consent: This is arguably the most significant con. The system operated for over two years without public disclosure, formal review, or caller knowledge. For a service as vital as 911, this opacity is unacceptable and erodes public trust.
Ethical Concerns and Accountability: The absence of review under Seattle's surveillance ordinance is a major ethical lapse. When AI influences life-or-death decisions, rigorous public debate and oversight are paramount. Questions arise regarding accountability if the AI's prompts lead to negative outcomes.
Safety Risks: The article highlights a troubling 2022 case where a retiree, routed to the nurse line, waited over 10 hours for an ambulance and was later found deceased. While the AI wasn't directly responsible for that specific initial routing (it started in Dec 2023), its role in increasing such diversions raises serious safety concerns given past incidents associated with the nurse line.
Limited Public Review Framework: While the Mayor's office states they are developing a public-facing framework for AI governance, this comes after a critical system has been operating covertly for years. This reactive approach is concerning.

Comparison to Alternatives (Conceptual)

The source material doesn't present alternative AI systems in a comparable manner, but we can draw a conceptual comparison between the current opaque AI deployment and a hypothetical, transparent, and publicly vetted system.

Feature	Current Seattle AI (Corti) Approach	Transparent, Vetted AI / Human-Only Triage (Conceptual Alternative)
Transparency	None. Operated without caller knowledge or public review for 2+ years.	Full public disclosure, community engagement, clear explanation to callers.
Oversight	None. Bypassed Seattle's surveillance ordinance.	Rigorous independent audits, ongoing performance monitoring, clear grievance mechanisms.
Public Trust	Severely compromised due to secrecy and potential safety issues.	Fosters trust through openness, accountability, and demonstrable safety measures.
Decision Making	AI prompts dispatchers who retain final authority, but influence is present.	Dispatchers make decisions with or without AI assistance, with clear guidelines and oversight on AI's role and impact.
Accountability	Unclear, especially in cases of adverse outcomes.	Clear lines of responsibility and liability for system failures.
Safety Focus	Efficiency-driven, with documented past safety concerns related to the nurse line.	Safety-first design, comprehensive risk assessments, continuous improvement based on public feedback.

Buying Recommendation: Demand Transparency and Oversight

As a tech reviewer, my "buying recommendation" for a civic technology like this is directed at the public and policymakers. The current deployment of Corti's AI in Seattle's 911 system is simply not ready for prime time and should be immediately paused for comprehensive public review and re-evaluation. While AI holds promise for improving public services, its implementation, particularly in life-critical scenarios, must be predicated on unwavering transparency, rigorous public oversight, and clear ethical guidelines.

I strongly recommend that:

All AI involvement in 911 and emergency services be immediately disclosed to callers.
A comprehensive, independent public review of Corti's AI system be conducted, assessing its accuracy, biases, safety record, and adherence to Seattle's surveillance ordinance.
Clear accountability mechanisms be established for any adverse outcomes linked to AI-assisted routing.
Public dialogue and citizen input be actively sought before any re-implementation or expansion of such AI systems.

Operating such a system in the shadows is a dangerous precedent and a disservice to the community it is meant to protect. Innovation should not come at the cost of public trust and safety.

FAQ

Q: How long has Seattle been using this AI to route 911 calls? A: The Seattle Fire Department (SFD) started using live AI prompts from Denmark-based Corti in December 2023, meaning it has been in operation for over two years without public disclosure or formal review.

Q: What are the main concerns with Seattle's AI 911 routing system? A: The primary concerns include the lack of caller knowledge or public review, bypassing Seattle's surveillance ordinance, potential safety risks (highlighted by a previous case of a delayed ambulance after nurse line diversion), and the ethical implications of AI influencing life-or-death emergency decisions without transparency.

Q: Is the AI directly deciding whether to send an ambulance? A: No, the SFD states that dispatchers retain final authority over every call, even when they receive AI prompts. However, the AI's role is to listen to calls and prompt dispatchers to route certain patients to a nurse-staffed call center instead of sending an ambulance, thus influencing the decision-making process.