Even in Seemingly Diverse 911 Centers, Failure Points Still Exist

In a recent blog, my colleague Phil Rizzo made a strong case for 911 center diversity and suggested several ways to achieve it. His thoughts are based on the truism that network and system outages,

both internal and external, will continue to occur, potentially with devastating consequences. This blog discusses some of the failure points that exist even in highly diversified 911 centers and how to avoid them.                                                                                                                               

A common failure point concerns the fact that 911 centers often do not perform enough due diligence and thus tend to take the word of vendors regarding the facilities they provide. When such facilities are first implemented, the vendor might claim that they are completely diverse and that claim might very well be true — in fact, most of the time it is true. But as weeks and months and even years go by, things happen — and the center’s officials usually are completely unaware.

For example, a fire might have occurred somewhere in a 911 service provider’s footprint that required cable to be rerouted. This is not something that the vendor typically would inform the 911 center about because service was unaffected, but it’s possible that some portion of the diversity that previously existed now no longer does.

Another example involves circuit grooming, a tactic that telecommunications carriers commonly use to achieve greater network efficiency while saving money. Let’s say that two network devices exist that each have capacity for 100 circuits, but each only is using half that amount. So, the carrier decides to combine all circuits onto one of the devices. Now let’s say that the 911 center previously had a circuit on each device. Consequently, the carrier’s decision to groom circuits cost the 911 center its diversity.

This latter example makes a strong case for 911 centers to leverage fiber-optic cable, which does not touch and 911 service provider equipment. When a center deploys fiber-optic cable, what they’re really doing is leasing two strands of glass in a cable provided by a vendor that connects two points, and those strands are lit by equipment provided by the 911 center — which gives the center complete control over its diversity.

The key takeaway from this blog is that 911 centers need to be constantly vigilant. This means monitoring and assessing every network, system, device, and connection that it uses to provide service to emergency responders and citizens, at least annually. This is not always easy to do and often becomes a battle for a variety of reasons.

A big one is that many 911 centers do not have the personnel and financial resources needed to pull this off. Another is that service providers and technology vendors try to avoid providing any visibility for fear of exposing vulnerabilities that could be exploited by their competitors, or worse, bad actors. Or they might want to execute a strategy that will serve their interests without harming the client’s interests but does not want the client to know for one reason or another.

A common example concerns a carrier that does not want to implement a fiber run that a 911 center has been seeking due to cost considerations. Instead, the carrier might choose to lease capacity on a run that already has been deployed by a competitor. In the first carrier’s thinking, this might fall under “no harm, no foul,” but this is information that the 911 center needs to have — and should have. If the second carrier suffers a catastrophic event that affects service, the 911 center unwittingly would be in the line of fire.

So how should a 911 center go about ensuring its vigilance? We humbly suggest that they enlist the aid of a third-party consultant like MCP that can help them fight these battles. We also can help develop procurement documents that not only define the level of diversity to which the service provider or vendor is committing, but also the service-level agreements that will enforce those commitments and the penalties for failing to meet them. We have a lot of experience in such matters — so, please reach out.

Milton Schober is an MCP technology specialist. Email him at MiltonSchober@MissionCriticalPartners.com.