DENVER – The city needs a comprehensive technology strategy to ensure timely and effective recovery of data and systems in the case of a disaster, according to a recent audit from Denver Auditor Timothy M. O’Brien, CPA.
“We look at the city’s systems often and I am confident in the steps Technology Services takes to keep the city’s data secure in response to our work,” Auditor O’Brien said. “But it pays to plan ahead for the worst-case scenario, because the city must be right every single time while hackers only have to be right once.”
The audit team found the city’s Technology Services agency has not effectively completed a comprehensive strategy for prioritizing disaster recovery procedures for mission-essential city functions. Disaster recovery programs should address backup restoration, alternative processing and data storage sites, telecommunications, communications and training plans, and annual program tests.
Comprehensive and well-designed disaster planning can help the city respond more effectively to disasters such as floods, pandemics, technological accidents, or cyberattacks. Cyberattacks are an increasingly common risk factor for any entity that can cost time, money, and data loss and can potentially cause more serious damage to property and lives.
The most recent example of what can happen is the hack of the Colonial Pipeline in the Southeastern U.S. that led to days of empty gas stations and panic buying. People across Texas were without power and running water in February 2021, after a natural disaster overwhelmed the electricity infrastructure and at least 57 people died. Another example in 2019 was a human-caused blackout in Venezuela where inadequate planning led to blackouts at hospitals and 26 deaths. In 2018, the City of Atlanta’s employees were locked out of files for five days while cybercriminals held them for ransom.
“If there’s a citywide disaster, we need to be sure we know how long it will take to get key systems back online,” Auditor O’Brien said. “Technology Services is taking steps to keep our files secure and backed up in case of emergency, but there’s no such thing as too much planning in this case.”
Although Technology Services does have a disaster recovery policy, it has not been prioritized in strategic planning and operations. This has resulted in inadequate governance, a less-than-comprehensive disaster recovery program that lacks documentation and maintenance, and insufficient communication and training strategies.
Whether a disaster is natural or human-caused, by not having a comprehensive plan, an emergency could cripple city operations — causing excessive downtime, lost data, and irreparable damage to the systems. Contingency planning helps ensure systems and data are up and running as soon as possible for the continuity of an organization’s operations.
A comprehensive plan should include the development of metrics and measurable goals. Common metrics include recovery point objectives — meaning, should a disaster happen how far back in time before the disruption do files need to be recovered. Metrics also often include goals for how long it takes for key systems to get back up and running after a disruption and the maximum tolerable downtime.
Although there is no comprehensive disaster recovery program, Technology Services has made some progress, including moving into a more modern data center. The new data center will provide real-time backup of the entire network.
Our audit team made several recommendations, including that Technology Services needs to include representatives of other agencies on the disaster recovery committee, develop a committee charter, update documentation, develop training, and improve its disaster recovery strategic plan. The plan should include goals that are timebound, specific, measurable, and actionable.
Technology Services agreed to all of our recommendations.