Imagine this: It’s 1PM on a summer Saturday, and you get a phone call from Chuck E. Cheese. They’ve had a small electrical fire. No one is hurt, but they have to close for the rest of the day. This is a problem for you, because at 2PM, you are hosting 15 rambunctious 7-year-olds for your kid’s birthday party. You feel a tension headache begin immediately.
Now imagine that the phone call went like this: “Hello, we have to cancel your reservation.” But you don’t worry, because you had a backup reservation at Dave and Buster’s. As soon as you called to confirm at Dave and Buster’s an automated email and text message was sent to the parents of each of the 15 kids, updating them on the change of plan and providing them with directions.
No headache, just a smoothly unfolding backup plan.
We don’t do this with a birthday party because a) what are the odds? and b) it’s not a big deal in the grand scheme of things (although every parent who just read that account felt their blood pressure rise). But in the business world, where an interruption in data services would be somewhere between ‘catastrophic’ and ‘business-killing,’ it is imperative that you plan for the worst.
Where To Start Thinking About Disaster Preparedness
The first problem when considering “what could go wrong” is that the answer is “everything.” Before very long, you’re left with a stack of eventualities that no one is going to read, much less implement.
An easy place to start is industry compliance standards. SOX, PCI-DSS, HIPAA, et al, all these are not only legally required, they actually form a good starting point for a plan customized to your organization. Why customized? Because the plan, whatever it entails, must have specific protocols that will be unique to the business. After all, if things are going pear-shaped, you want the plan to read, “Call [Name], [Phone Number], not “Call CTO,”
The second thing to do is prioritize critical services. What needs to come back online first? Do they have dependencies that trump that? Does the activation rely on a human operator, and if so, who? (And can you automate that?) Yes, you need everything back online. But you need to have a strict order of operations so that resources can be triaged effectively.
The next step is to focus on the effect, not the situation itself. It doesn’t matter what happened to cause a communication outage at the data center, just that there is a backup plan in place. This will systematize your responses and ensure that they are usable.
Too Much Redundancy?
At this point, people start to consider redundancy. You don’t want too much redundancy, as it can cause almost as many problems as too little, and it’s expensive, too.
There are several types of redundancy to take into account when planning your disaster preparedness procedures.
- Network Redundancy — You mainly get this through multi-source facilities.
- Server Redundancy — Automatic failovers here have saved many a headache.
- Data Storage — Many people think data backup is all they need to ensure business continuity, but the reality is more complex. Multiple types of data storage are often required.
- Location Diversity — About 40% of firms only have one data center, making them vulnerable to things like hurricanes and floods. Maintaining services in even one other data center would greatly reduce this risk.
Automate, Automate, Automate
The great benefit to a technical problem is that there is often a technical solution. A well-thought-out disaster preparedness plan can put into place numerous procedures that do not require human intervention. This is highly desirable, as it reduces potential downtime substantially.
This might be the most rewarding aspect of setting up backup and recovery solutions. Where, before, you might have relied on one or two people who will hopefully be at work when something goes down, by setting up processes upfront, you can devise solutions such that the appropriate action is taken automatically, and the administrator is informed — not the other way around. Just think what a difference that would make!
If you’ve considered designing a disaster recovery solution but considered the project too expensive or in-depth, we’re happy to help! We can help you explore your options and evaluate potential solutions for your specific needs.