Council Post: Protecting The Frontline From AI’s Silent Failure
Mahesh Saptharishi, Executive Vice President and Chief Technology Officer at Motorola Solutions.getty“Is it good enough for my loved ones?” As a technology leader in the public safety space, I ask myself this ...
Mahesh Saptharishi, Executive Vice President and Chief Technology Officer at Motorola Solutions.
getty
“Is it good enough for my loved ones?”
As a technology leader in the public safety space, I ask myself this question frequently. In these roles, we intentionally imagine the unthinkable: A loved one in an emergency, dialing 911. At moments like those, the public safety technology we build as an industry is all that’s connecting them to the help they need.
If there’s any hesitation with the designs, the product isn’t ready. It requires more testing, more rigorous thought, more frontline feedback.
While the technology sector celebrates the rapid pivot and the iterative prototype—operating on a belief that speed is a competitive advantage—in mission-critical environments, speed without reliability is a liability.
Public safety software cannot afford a margin of error, because a technical flaw will directly dictate human outcomes. When these systems misfire, the consequences aren’t contained within the application—immediate risk is introduced to the dispatchers, the first responders and the public relying on them.
The Threat Of AI’s Silent Failure
As AI becomes central to emergency workflows, we are facing a fundamental shift in risk.
Legacy hardware failures are generally explicit and self-evident—a computer-aided dispatch (CAD) system crashes or a radio goes silent.
AI, however, introduces the threat of "silent failure." A large language model or a speech-to-text algorithm hallucinates instead of crashing. It provides a plausible lie with the same confidence as a life-saving truth.
When 911 call takers are already operating with 25% vacancy rates while fielding a deluge of non-emergency calls, the temptation to automate operations is high. But if AI operates without rigorous safeguards, it risks bypassing human validation, compromising the integrity of the entire emergency response.
To mitigate this, we must move beyond the allure of passive automation to a roadmap of active human augmentation, built on four design pillars.
1. Architectural Accountability
The staffing crisis in emergency services is critical, but the solution isn't to replace experts with algorithms.
That said, there are ways to use AI to reclaim vital human bandwidth, but this requires architectural accountability: designing systems that autonomously manage non-emergency calls, while operating within strict guardrails and protocols.
Algorithmic automation is, of course, not a new frontier in mission-critical engineering. The ethical stakes of public safety have always dictated that technology integrate carefully within operational workflows, explicitly accounting for the cost of systemic breakdown.
Consider a noise complaint call. An AI agent can handle the initial intake, but it must be designed with "escalation intelligence." If it recognizes linguistic cues of violence or immediate danger, it must detect the shift in urgency and seamlessly hand the interaction back to the dispatcher to manage.
This demands an intentional architectural constraint: prioritizing safety over efficiency. The system optimizes for a reliable human handoff, ensuring caller well-being takes precedence over machine call duration.
2. Contextual Intelligence
As Gartner researchers have pointed out, while general-purpose LLMs provide robust language capabilities, their response accuracy declines for tasks requiring specific business domain context.
Contextual design ensures that technology adapts to the user’s unique cognitive state and the evolving nature of an incident. For instance, while a call taker initially prioritizes establishing the caller's location, the system must dynamically pivot as the incident unfolds, detecting keywords, executing background queries and surfacing hidden hazards to shape an active response.
The technology’s primary role is to save time and enhance operational performance, without burying the user in pedantic guidance or excessive data. Contextual intelligence values data curation over data volume, elevating the delivery of critical information while intentionally withholding the rest.
3. Cognitive Engagement
In everyday digital interactions, frictionless design is the goal. In mission-critical environments, however, unchecked autonomy is dangerous.
As behavioral scientist Dr. Harriet Rowthorn argued in a recent conversation with me on the Motorola Solutions podcast, AI should be "mind-extending" rather than "mind-replacing." This means it should function as a collaborative partner that pushes a person's insights beyond their natural limits, rather than eroding their critical thinking.
Building for active engagement requires deliberate, architectural friction points. For instance, if an AI detects the faint sound of a gunshot in the background of a call, it should not automatically categorize the incident. Instead, it must present the data for manual verification from the dispatcher.
This forced interaction acts as a cognitive speed bump, ensuring the professional remains present, attentive and in control.
4. Radical Transparency
Trust in AI should not be a feeling. In legal and ethical contexts, it is a matter of evidence.
Decisions assisted by AI must be reconstructible and backed by an auditable chain of evidence, a necessity driven by the integration of the “Daubert standard” to evaluate the validity of machine outputs. Whether during a post-incident review or courtroom testimony, the "why" behind an AI's suggestion, synthesis or analysis should be clear.
If an AI flags a call as a high-priority cardiac event, radical transparency requires the system to provide specific indicators—such as keywords or acoustic signatures—that triggered the classification.
Transparency transforms shrouded decision-making into visible evidence, ensuring that every action is defensible and grounded in clear reasoning. This allows the professional—and indeed the court—to challenge the system rather than blindly accept its outputs.
Beyond The Prototype
When applying the loved ones test to public safety AI, it becomes clear that "good enough" is a dangerous standard.
While consumer technology utilizes the minimum viable product, mission-critical environments demand a much higher threshold. True operational integrity means establishing a new engineering benchmark, the minimum trustable product, where automation is deployed only after proving its reliability under extreme stress.
Ultimately, success will not be measured by the speed of a launch, but by the confidence of the dispatcher, the safety of the officer and the protection of those we love on the other end of the line. In these moments, there is only one standard: the one we would demand for our own.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?