On the evening of March 8th, PSAPs in multiple states were notified that wireless AT&T customers were unable to access 9-1-1 with a voice call, or were having difficulty reaching a 9-1-1 calltaker.
It appears that the outage was extensive, perhaps nationwide, and lasted for several hours before being resolved. PSAPs in Tennessee, Texas, California, Florida, Colorado, and Pennsylvania are just a few states that publicly confirmed 9-1-1 service degradation. The full impact is still unknown at this point.
As the vendors involved work to investigate the root cause of the system failure, many within the public safety sector are beginning to reflect on the situation and how the industry can be better prepared for when—not if—a similar problem occurs in the future.
Meanwhile, at the federal level, the FCC has announced an investigation that will likely to take some time to complete.
A seemingly uncoordinated effort
Many of our 9-1-1 clients reported that notifications to state 9-1-1 leaders and PSAPs were sporadic and narrowly targeted. Agencies shared bits of information via email with their colleagues as the situation unfolded. Network operation centers from service providers such as West Safety Services and Comtech forwarded notifications to some of their PSAP customers; however, many were never notified by a service provider during the incident or after the situation was resolved. The notification process and subsequent communication appeared to be an uncoordinated effort that leaves significant room for improvement.
Although requirements codified in Federal regulations (§47 CFR 4.9 - Outage reporting requirements - threshold) mandate notification to the FCC within 120 minutes for system outages, contacting affected 9-1-1 authorities is required “as soon as possible” which unfortunately is not well defined.
It's critical that the 9-1-1 community mitigate the effects and impacts associated with unplanned outages. Agencies need to prepare themselves to respond effectively to emergency situations like this outage.
time for a policy and procedure refresh
In light of the outage, Mission Critical Partners urges each 9-1-1 authority to review their local/regional policies and procedures to determine if revisions (or testing) are necessary. The National Emergency Number Association (NENA) describes contingency planning in its “PSAP Disaster & Contingency Plans Recommendations.”
A few steps we suggest include: