Within traditional business-technology environments, keeping software systems up to date should (theoretically) be straightforward—a flaw is identified in the software, the software vendor publishes a patch, and the customer deploys the patch to their systems and endpoints. All systems are now updated.
While it's that simple, it's rarely that easy, especially in operational technology environments that are intolerant to downtime. In these environments, industrial control systems abound, and operate and often communicate through proprietary protocols. Patches must be tested before they can be safely delivered. And, sometimes, OT/ICS systems can't be quickly updated because of regulatory, contractual, or performance concerns.
Control systems and field devices, as a result, are often left exposed despite the availability of patches for known vulnerabilities—sometimes for months.
What does this mean when patching OT/ICS systems successfully?
"Patching in operational technology environments operates very differently to standard IT, and to a large extent IoT, environments," says Jonathan Sword, director at security services provider Agility Cyber. Sword explains that standard IT systems typically operate under regular patch release and internal maintenance cycles. These patches generally can be applied as a batch shortly after their release.
However, OT environments are different. In OT/ICS environments, patches are tailored to specific hardware deployments. Think backward compatibility, Sword explains, because the patch may be designed for the software or firmware of currently sold devices, not the devices the organization has presently installed.
There are several significant challenges associated with patching OT/ICS systems. These challenges include:
Patching Constraints. Within OT/ICS environments, patching is not often allowed because of downtime costs. In too many industrial and other OT organizations, the timed windows for permitted patching are spread far apart.
Legacy Systems, Proprietary Protocol Complexities. The OT/ICS devices deployed in these environments have been used for decades. Decades ago, secure environment design wasn't a strong consideration always, and the air gaps that were put into place vanished long ago as these systems grew more connected.
Vulnerability Assessment woes. Traditional vulnerability scanning tools are often incompatible within OT/ICS environments and can cause significant disruption. This makes it challenging to identify systems that need to be updated or verify that systems have been patched.
Diversity of Systems. OT environments have a lot of variety in terms of the systems used, and the job becomes more complicated when OT/ICS devices are installed from multiple vendors, making effective patch management crucial to identifying vulnerabilities and reducing risk to an acceptable level.
Still, despite the challenges, patching is possible and essential.
"Keeping OT systems patched is crucial to reduce vulnerabilities. Network owners should establish a comprehensive patch management process that includes regular updates and patches for all OT devices and systems," says Robin Berthier, one of the world's foremost experts on OT cybersecurity and the CEO at Network Perception.
Sword adds that patching is best achieved through centralized management that tracks both device component versions and their risk exposure." A lot of organizations have moved toward cloud-based management solutions for this as they can offer a security dashboard which can be utilized by an organization's security operations center," Sword says.
The practices required to patch OT/ICS environments successfully must consider the unique challenges and requirements of these environments. The key is to follow a number of beneficial methods:
Comprehensive Patch Management Program. Due to the diversity of systems in OT environments, it's essential to patch intelligently. That includes prioritizing patching based on risk, such as patching those devices that provide high business value or those systems exposed to the Internet, such as those operating in a DMZ. This requires a specialized patch management plan explicitly developed for OT systems.
Validate patches. As part of the patch management process developed for OT/ICS systems, ICS patching requires diligent patch testing and validation. This will significantly reduce the risk of downtime and costly operational outages.
Lean on device vendors. The device manufacturers are the best source of technical support and possibly help in the OT/ICS patching process. Use the resources they provide and, if these resources are subpar, demand that they be improved.
Maintain a comprehensive understanding of assets. An accurate and up-to-date inventory of networked devices and systems in the OT/ICS environment is crucial for success.
That last item is essential as operations and security teams can't patch what they don't know is installed.
"The simpler it is for an organization to see what's in use, the risk it poses, including attack events, and what actions are outstanding on it, the greater the organization's ability to remain secure," Sword says. "It is worth organizations ensuring they are kept informed about emerging threats too, through sourcing relevant threat intelligence specific to OT infrastructure and their industry sector as well as opening up communication channels with national security initiatives and other organizations to be proactive in mitigating threats," he adds.
That's solid advice, as patch management programs don't run in an organizational vacuum — or at least shouldn't. Organizations must ensure these programs are in place and mitigate the most pressing risks first.
George V. Hulme is an award-winning journalist and internationally recognized information security and business technology writer. He has covered business, technology, and IT security topics for more than 20 years. His work has appeared in CSOOnline, ComputerWorld, InformationWeek, Security Boulevard, and dozens of other technology publications. He is also a founding editor at DevOps.com.
