Helping with Disaster Recovery

When everything goes wrong know what to do right.

June 20, 2023

What do you do when there is a disaster? What recovery plans do you have? Are those plans being acted upon? Have those plans been tested? We have been helping a customer get through a building fire. Here is what we have learned.

Backups are critical

The building fire did not occur in the server room, and those machines were thankfully spared. However, due to smoke damage, we did replace the servers. While doing this, we tested our backup strategies to ensure that had we lost all the servers our data would have been intact.

We discovered that our backups worked perfectly. All data needed to completely rebuild the systems was available and usuable. Our recovery strategy centered around having good backups and this proved to be well executed.

Understand the business

Having good backups was key, but we also relied on understanding how all of the various systems worked together. We knew what systems existed and what task each system performed. We were able to identify each business process and what underlying systems those processes depended on. This allowed us to immediately know the impact the building fire had on computer operations and work with the customer to prioritize recovery steps. The customer was able to have peace of mind during a stressful period because of our deep understanding of their business processes and the computer resources that support them.

More than just technology

Another important lesson that this experience highlighted is that recovery from a disaster is about a lot more than just recovering computer systems - even for a technology company. Clearly recovering computer system was the first priority, but it is not what we have spent most of our time doing since the fire. The computer systems were recovered quickly and successfully. But the customer has further relied on us to use our accounting knowledge (and specifically how that knowledge is applied to accounting systems) to help them with recording expenses, insurance payments, lost sales and equipment, special payroll runs, and more. Recovering the systems and data is only the beginning of the total disaster recovery process. The much more involved process of accounting for all of the disruptions caused by the disaster can stretch for months and even years. This is where we have been of most help to our customer.