Securing The Move To Ip-Based Scada/Plc Networks: November 2011
Securing The Move To Ip-Based Scada/Plc Networks: November 2011
Securing The Move To Ip-Based Scada/Plc Networks: November 2011
November 2011
Purpose The purpose of this guide is to provide examples/types of SCADA and control systems and their typical use in industry. This document then describes the functional areas of those SCADA systems which typically use analogue serial telecommunications technologies, those components which are impacted with the move to ethernet TCP/IP communications; the security risks to consider when migrating to IP-based communications, and practical tips for ensuring the reliability and security of SCADA systems when leveraging ethernet TCP/IP protocols and communications links. This guidance is designed to impart good practice advice for securing industrial control systems such as process control, industrial automation, distributed control systems (DCS) and supervisory control and data acquisition (SCADA) systems. Such systems are used extensively across the nations critical national infrastructure. The document provides valuable advice on protecting these systems from electronic attack. It has been produced by the Red Tiger Security firm on behalf of CPNI and peer reviewed by members of the SCADA and Control Systems Information Exchange. Disclaimer Reference to any specific commercial product, process or service by trade name, trademark manufacturer, or otherwise, does not constitute or imply its endorsement, recommendation or favouring by CPNI or Red Tiger Security. The views and opinions of authors expressed within this document shall not be used for advertising or product endorsement purposes. CPNI and Red Tiger Security accept no responsibility for any errors or omissions contained within this document. In particular, CPNI and Red Tiger Security shall not be liable for any loss or damage whatsoever arising from the usage of information contained in this document.
Contents
Executive summary
Overview Best practices: the design and management of IP-enabled SCADA networks
4
4 5
Introduction
Terms and definitions Target audience
8
8 8
9
9
9
9
10
Distributed Control System (DCS) Manufacturing Execution Systems (MES) Energy Management System (EMS) Automatic Meter Reading (AMR) Automatic Metering Infrastructure (AMI) Building automation systems (building management systems) Industrial Control Systems (ICS)
Figure 3: Example of a Building Automation or Building Management System
10 11 11 11 12 12 12
13
14
14 14 15 15
17
17 17 18 18
19 19
20
20
22
22 22 23 24 24 25 25
26
Common control system components likely to move to IP Most typical migration path from serial to IP Changes required at the front end processor (FEP) Implementing the new ethernet communications path from HMI to RTU Zones, conduits, boundaries and security levels
Figure 11: Original concept of zones and conduits (from the first version of ISA99 standard in 2003) Figure 12: Security levels as depicted in the ISA 99 standard
27
28
28 28 28 29 29
Serial to IP converters (convert leased and dial line modems to cellular broadband service) 31
Figure 15: Cellular communications for SCADA (option 1) Figure 16: Cellular communications for SCADA (option 2)
31 32 33 34
Multi-network design
Figure 17: Modernisation in process: legacy and IP devices on the same IP network
35
36
36 36 37 38
Executive summary
Overview
Supervisory control and data acquisition (SCADA) and control systems were initially designed to monitor and control industrial processes using proprietary protocols, and were typically kept isolated from other computer systems. Since these SCADA systems were never designed with security in mind, and are now being connected with business networks and the internet, they are now more than ever at risk. With the move to 21st Century networks taking place, many traditional telecommunications providers are moving from a circuit switched to a packet based infrastructure. This allows the convergence of many different network types (e.g. voice, data, video, physical security and control signals) onto a single network and can represent significant cost savings to a business. Many businesses have increased connectivity between the SCADA and corporate network in order to gain improved business and allow more informed decisions to be made, with no consideration for security of the SCADA and corporate systems. With an increased level of risk in the Internet Protocol (IP) world and the increasing interest in the security of control systems, it is important that asset owners understand the various IPbased solutions available in order to make an effective security based risk management decision. With routable IP-based communications, threats can come from anywhere, and these networks are much more dynamic in nature. New management activities are now required with ethernet TCP/IP SCADA systems that did not exist in legacy serial communications. These activities include keeping networked devices up to the latest firmware, providing network access controls to block unauthorised access, and monitoring the use of these devices. This best practice guidance document first defines the various components of a SCADA or control system, and then explains how various parts of these SCADA systems have evolved to operate over switched IP networks. This document also covers several aspects to consider when deciding to convert from dedicated serialised telecommunications circuits to IP-based packet switched networks. Managing SCADA systems that operate over IP networks requires much more security oversight, as well as additional security solutions and considerations. After discussing these topics, this document provides a section on future technologies and Next Generation Networks (NGN).
Keep the SCADA IP network physically separate and isolated from the Corporate IT Network and other un-trusted networks. Where physical separation is not a practical option, logical separation must be applied due to the heightened risk of IP-enabled SCADA systems. It should be noted that logical separation is more complicated to implement effectively, and runs the risk of ineffective configuration over the life of the assets. Use quality assurance techniques to ensure that all security requirements identified during the design phase are developed, tested and implemented within the final product. Also consider using the ISA S99 security levels as a model when designing SCADA and control systems based on IPbased protocols. The ISA S99 security standard defines security levels 0 (Instrumentation) all the way up to level 5 (internet-facing systems). Each security level is designed to support specific SCADA components based on functionality requirements and security. For example, FEPs (Front End Processors) are typically placed in level 2, right alongside the HMI or operator console. They log data to a data historian, which resides in the DMZ or level 3. IP communications which originate from untrusted networks outside of the SCADA and control system networks should terminate in a DMZ or buffer network; they should not be allowed to make direct connections with components in the SCADA and Control Systems networks. Devices on the inside of the SCADA and control systems network should not be able to communicate directly with the internet, nor should they be used with external email. Avoid the use of Virtual LAN (VLAN) technology as a method of keeping SCADA IP communications logically separated from Corporate IT communications: VLAN technology was not designed as a security method. If existing corporate IT network infrastructure such as switches, routers and WAN links must be used as a transport method for portions of the SCADA communications, then the SCADA communications should be encrypted and routed through a VPN tunnel through corporate IT or other non-critical networks. When building a complete end-to-end IP network, avoid using devices which use layer 3 separation between SCADA and other non-critical networks. Devices claiming layer 3 network isolation use VLAN (Virtual Local Area Network) technology to virtually isolate network traffic. VLAN technology was initially designed as a bandwidth-shaping tool, and was never designed with security in mind. Several free hacking tools allow VLAN-walking capability that allows an attacker or malware to jump across any layer 3 VLAN network. For true network isolation use equipment that can provide isolation at layer 2.
Do not permit SCADA or control system devices to be dual-homed to two or more networks that are at different security zones or trust levels. For example Plant Data Historians should only be connected to one security zone, such as the DMZ, then a firewall rule or access control rule should be created to specifically allow traffic from the Plant Data Historian to be exchanged with the SCADA or control system over specified ports at specified rates. If remote access is allowed into the SCADA or control system over an IP-based network, then a different authentication check should be made to remote users in addition to the authentication process used to log into the corporate IT network. In addition, remote access should not be permanently enabled; it should only be granted upon request from a known and trusted source, with an end time agreed, following which the access should be discontinued. A good cyber defence must compliment active blocking devices like firewalls, IPS, and network inline antivirus appliances with passive detective technologies which can detect, log and alert to what is happening at the network, server and application layers. NIDS (network intrusion detection sensors), HIDS (host intrusion detection sensors), Syslog events, and SNMP traps can all be configured to send their logs and alerts to a centralised SIEM (Security Incident and Event Management) appliance where the logs and alerts can be reviewed and escalated as needed.
When managing IP-enabled telecommunication devices, network equipment, PLCs, RTUs, protocol gateway converters, and any other embedded devices, disable any unnecessary services and limit the attack footprint by disabling or blocking unnecessary TCP and UDP ports. Limit the use of clear text protocols such as telnet, ftp, and http. Instead, force the use of encrypted protocols, where technically feasible. Most IP-enabled telecommunication devices such as switches, routers, firewalls, protocol converters, media converters and serial terminal servers support SNMP (Simple Network Management Protocol) for monitoring the health and performance of the devices. Ensure that the latest version of SNMP is enabled, and configure the security and performance based alerts to be collected by a SEM (Security Event Management) system. The default SNMP community strings should also be changed. Most IP-enabled telecommunication devices such as switches, routers, firewalls, protocol converters, media converters and serial terminal servers support the logging of security events such as denied attempts. Logging should be enabled in such a way that a log is kept resident on the device and a copy sent out to a Defence syslog server. If an SEM is already present on the network collecting SNMP events, then send the syslog information to the same SEM. Organisations will have to determine the appropriate frequency and scale of operation in reviewing logs generated by IP-enabled devices. Several industry best practices such as ISA 99 suggest that logs be reviewed on at least a monthly basis. Government regulations such as NERC CIP require that logs be reviewed within 30 days, and that the reviewer must sign off that the action was performed. As indicated in the previous point, if SNMP is used, the latest version of the protocol should be used that supports encryption and the default community strings should be changed.
For SCADA and control system networks that contain critical components, system administrators should also consider deploying in-line network appliances at the choke points that perform Network Intrusion Prevention and Anti-Virus to filter and drop packets that are a known signature match for malicious traffic. Firmware for IP-enabled telecommunications equipment and control devices should be kept up to date within the current patch cycle. Due to its real-time nature, patching SCADA system components is problematic, and in many instances not a practical option due to the consequent overhead of retesting the system. Purchasing industrial control equipment that have gone through an industry security certification like the ISA Certification process, helps ensure that the components have gone through rigorous vendor and industry inspection. Some control system vendors will test the latest security patches on their systems first, and release a certified patch for their end users. Leveraging industry and vendor certification processes can reduce the periodicity of system patching. Control devices such as PLCs, RTUs, Smart Meters, ethernet I/O and IP-enabled instrumentation should be deployed with a password or PIN code that must be required to make changes to the configuration over an ethernet connection. Based on the level of risk, some devices may be configured to only allow configuration changes over a serial console cable. Control devices such as PLCs, RTUs, Smart Meters, ethernet I/O and IP-enabled instrumentation have known TCP/IP stack vulnerabilities. Switches, routers, firewalls, or any network devices in front of the control devices should be configured with rate limiting commands to restrict data from flooding the device. The alerts and logs from network equipment can be either configured for sending to a SEM (Security Event Management) system and viewed by IT security staff, or be translated to OPC tags and viewed by SCADA and control system operators. Translation software is available to convert SNMP and Syslog data to OPC. Whether the network is monitored by IT staff or SCADA operators, responsibility for the care and management of the networks should be clearly defined, and system logs should be reviewed on at least a monthly basis.
Introduction
Terms and definitions
Throughout this document, the terms SCADA and control system are used as generic terms to refer to all industrial automation and control systems, distributed control systems (DCS), energy management systems (EMS), and related real-time monitoring and control systems. The follow list defines some of the acronyms and industry-specific terms used within this report. APT AD CIP DCS DMZ ESP ISA PLC RTU SCADA TCP/IP Advanced Persistent Threat Active Directory Critical Infrastructure Protection Distributive Control System Demilitarised Zone Electronic Security Perimeter International Society of Automation Programmable Logic Controller Remote Terminal Unit Supervisory Control and Data Acquisition Transmission Control Protocol (TCP) / Internet Protocol (IP)
Target audience
This guide is aimed at anyone involved in the security of process control, SCADA and industrial automation systems including: process control, automation, SCADA and telemetry engineers; information security specialists; physical security specialists; government and policy makers; business leaders; risk managers; health and safety officers; operations engineers; security professionals (either in-house staff or outsourced).
Note: the role of cyber security can be outsourced to a 3rd party through the use of MSSP (Managed Security Service Program), however the responsibility and accountability for security can never be outsourced; it must be owned by a senior designated manager.
Automated systems
Automation is embedding logic in a controller to allow a process to run efficiently without constant human intervention. An automated system contains at least one controller, but may also include a local touch screen or terminal for the system operators to interact with the controller. An automation system is typically the lowest level building block for higher level SCADA, DCS, and process control systems and can either exist as an isolated disconnected system, or be networked with other systems.
PLC or RTU
10
o o o o o o
managing work orders, receipt of goods, shipping, quality control, maintenance, scheduling or other related tasks; link enterprise accounting systems like SAP to plant floor industrial control systems; automate the management of batches or recipes; track and control raw material and finished goods inventory; track Key Performance Indicators (KPI); make intelligent decisions to influence production and cost management; manage resources, including inventory and personnel.
centralised servers at the primary and backup control centres; distributed monitoring and control elements at remote substations; disconnect and reconnect switchgear at substations and on top of electrical poles near power transformers; EMS systems also typically include demand scheduling/load balancing systems, OMS (Outage Management Systems), GIS Mapping Systems, and NMS (Network Management Systems).
Also used in Electricity Transmission & Distribution systems to monitor and control distribution of power to large industrial users and neighbourhoods. During the last five years there has been a movement in North America to integrate EMS systems with AMR (Automated Meter Reading) systems.
11
NOTE: The SCADA and control systems industry has now standardised the term ICS. The United States Department of Homeland Security formulated the ICSJWG (ICS Joint Working Group) specifically to address cyber security issues related to SCADA and control systems. The ICS-CERT organisation has been set up within US-CERT specifically to address SCADA and control system security vulnerabilities and provide incident response guidance.
12
13
14
15
In legacy systems, the operations systems were not typically connected to the business LAN, and the field controllers and plant equipment located at level 1 and level 0 used serial circuit communications as shown in the diagram above. Since levels 3 and above in SCADA and control systems are typically already leveraging ethernet TCP/IP networking to connect the SCADA computer workstations and servers to other networks, we can focus on the lower levels of the system which involve acquiring data from controllers and field devices. This next section will review various architectures of legacy SCADA systems and examine how modems, wired copper and fibre circuits, and wireless RF systems were used to provide analogue serial communications between the field equipment such as RTUs and PLCs and the central control room computer systems.
16
Serial communications require each device on the same channel to use the same settings so that the master device can communicate with each slave device on the channel. The following communications settings are used to define how a serial channel functions: Communications channel and IRQ Baud rate (typically: 300, 1200, 2400, 4800, 9600, 19,200) Number of bits (8 or 7) Parity (none, even or odd) Number of stop bits (typically 1) Example: 9600, 8, N, 1 Flow control and error-checking (CRC, none, Xon/Xoff) NOTE: Flow control is rarely used in SCADA communications
All devices on a RS-232, RS-422, or RS-485 must be configured with the same serial communications settings. The following sections provide examples of serial analogue systems over various wired, fibre, or wireless media.
17
Figure 7: Legacy SCADA system using modems (mapped to the ISA 99 model)
Slow limited bandwidth and latency issues are the leading drivers for migrating away from serial communications over modems, cellular infrastructure, or ISDN networks. For remote locations like offshore oil and gas platforms, the only option is satellite communications, and the timing and latency issues are greater for satellite. The following sections provide examples of how serial communications can be sent over wired, fibre and RF media.
18
Use of wired (copper or fibre) circuits for carrying several types of serial communications
For SCADA, Energy Management Systems and Distributed Control Systems contained within a regional area that can be covered with copper or fibre circuits, the controllers, RTUs and other field equipment are typically connected over TDM (Time Division Multiplexed) circuits. These circuits can break out either large bandwidth circuits like T1 and DS3, specialised proprietary trunks for CCTV, or smaller individual RS-232 circuits for polling PLCs and RTUs. The underlying technology is still based on serial analogue channels, whereby attackers have to gain access to the physical channel to subvert the system. Modernisation brings additional risk, since switched IP networks can be routed into from foreign external networks if security is not applied correctly.
Figure 8: Legacy SCADA system using fibre (mapped to the ISA 99 model)
19
20
These wireless systems were developed with either licensed radios that used known fixed transmit and receive frequencies, or with spread spectrum radios that used frequency hopping techniques to avoid having to use fixed RF frequencies. Both the licensed and non-licensed wireless RF systems were initially only designed to transport serial communications. In legacy wireless RF systems the master radio can connect directly to slave radios to poll the field devices for data, or in some cases, might have to connect through a repeater site to get the signal down to the slave devices if line of sight is not permitted. Repeater sites are typically used due to geographical constraints such as large hills, mountains, or buildings that might be in the way blocking the signal from the master radio. Serial protocols only allow for one master to communicate to one slave at a time, and while the slave is responding to the master, it is consuming a serial channel that cannot be used for any other purpose. Technicians could not remotely troubleshoot or program the field devices using serial communications, so all modifications and device management had to be performed out in the field.
21
Data link
Data move from the application layer on the client side down through the layers to the physical layer where it is transported to the host. From here the data move up through the layers again to be presented on the host side.
Figure 10: 7-layer and 4-layer OSI models and data flow diagram
22
Routable TCP/IP protocols built on IP communications allow the data to be separated from the transport method, thus opening up many ways to route data from the source to final destination. Instead of requiring a dedicated end-to-end link (as with serial channels), packetised communications based on IP communications can be routed over multiple methods such as wireless RF, microwave, satellite, fibre, DSL or even cellular broadband. This provides more options for those deploying or integrating SCADA systems. Also, since routable communications based on IP technology are not one-for-one master-slave communications as in serial channels. Multiple communication masters and slave devices can communicate at the same time over the same or diverse channels, providing greater redundancy should any one link go down. For instance, if IP communications are used, an RTU can be polled by a satellite uplink and by a cellular broadband connection at the same time. Another benefit of IP communications is that while the master servers and other applications are polling field data, technicians can log into the field equipment over the same IP network without interrupting the system. This makes system administration much easier.
23
Devices which are directly connected to physical equipment and are responsible for controlling and monitoring the safe operations of real-world processes are the devices which are most vulnerable. Field devices such as PLCs, RTUs, Smart Instrumentation, Smart Meters and other embedded devices are the most critical element of the SCADA system, since they control the operation of physical equipment such as valves, pumps, drives, boilers, heaters, generators, switching re-closures, compressors, conveyor belts, etc. Instead of making the system more closed and protected, the move to IP communications opened up devices in the field to other networks and systems by connecting them with TCP/IP communications and open clear-text protocols. These embedded devices do not have the ability to support the security features that most computing systems require, such as Antivirus, Authentication, Encryption, and End Point Security, and many do not support network monitoring and logging either. So while the industry waits for security features to be added to embedded devices, the best way to secure IP-based SCADA systems is to restrict access to them by implementing secure architectures which prevent access to the SCADA systems from Corporate IT and other third party IP networks.
Most typical migration path from serial to IP (from the field device perspective)
When considering migration from a serial system that uses analogue communications to an ethernet networked environment, it is not as simple as configuring the PLC, RTU, or field equipment to accept communications from IP networks. Changes will most likely need to be made at the Front End Processor (FEP) server to switch the SCADA driver from a serial protocol to an ethernet protocol. The telecommunications media and hardware may need to be upgraded or replaced, and finally changes will need to be made out at the field controller hardware as well.
24
25
The ISA standard document S99-00-01 defines a security zone as: A logical grouping of physical, informational and application assets sharing common security requirements. This concept applies to the electronic environment where some systems are included in the security zone and all others are outside the zone. There can also be zones within zones, or sub-zones, which provide layered security, giving defence-in-depth and addressing multiple levels of security requirements. Defence-in-depth can also be accomplished by assigning different properties to security zones.
The diagram below, provided by the ISA S99 standard, depicts typical Enterprise, Plant and Control zones, with firewalls providing boundaries between zones creating conduits for data flow.
Figure 11: Original concept of zones and conduits (from the first version of ISA 99 standard in 2003)
Each zone has a border which is the boundary between systems included on the inside of the zone, and those that would be outside of the zone. Assets which are outside the security zone are by definition at a lesser or different security level. Security levels can be defined for each functional area of the system, and the ISA S99 model uses level 5 for the most untrusted public security level for internet-facing systems, and reserves level 0 for the most trusted part of the control systems where the instrumentation is directly connected to the physical equipment. The next diagram places the various functional components of typical SCADA and DCS systems into a logical diagram in the appropriate security levels.
26
The architecture proposed by the ISA S99 standard provides the SCADA and control systems industry with a model for segmenting networks at levels 2 and above to ensure that SCADA and control systems are isolated from the Enterprise IT networks through a DMZ (demilitarised zone). As the lower levels 0 and 1 typically contain instrumentation, meters, PLCs, RTUs and other field embedded devices move to IP communications, additional considerations for security must be taken into consideration. The next section discusses typical scenarios in which legacy serial communications methods are being replaced by TCP/IP communications.
27
Modems (modulator-demodulator) are devices which encode digital data by modulating an analogue carrier signal. In most cases this is to enable transmission over POTS (Plain Old Telephone Service). SCADA applications and other serial data networks use serial protocols such as DNP3, IEC 60870-5101/103/104, Modbus RTU, Modbus ASCI, CDC, S/NET, CONITEL, and ABB that are typically transported via modems such as Bell 202 or V.XX type modems over leased or dial tone telephone lines for communications between the central site and remote facilities. In SCADA systems that used serial communications, a separate leased or dial tone telephone line had to be supplied for each modem link to connect the centralised control room computing systems to remote devices in the field. Escalating costs from leased and dial line service providers along with limited leased line availability create access problems for SCADA utility and services users. Recurring monthly costs, together with the limitations of legacy modem equipment may limit the expansion and capability of the services which can be supported over these slow analogue lines. Such cost and operational constraints have motivated utility SCADA operators to move to newer telecommunication technologies such as Internet Protocol (IP) based data and wireless networks. These are now widely available and offer expanded services and lower recurring costs, provide alternative remote site access and comply with new regulatory standards such as the NERC CIP requirements in North America.
Tunnelling RS-232 over ethernet IP networks With diverse hardware platforms out in the field, rarely can a SCADA system completely migrate from legacy serial communications to ethernet TCP/IP protocols all at the same time. Typically the legacy and modern hardware are run in parallel mode as the system slowly migrates away from the serial protocols. In this scenario, technicians would typically install an RS-232 to IP converter at the Master SCADA servers and also out at the remote sites where older legacy hardware does not support a native ethernet port. The serial protocol is essentially tunnelled through the IP network, and the converter out at the field site removes the IP header, footer, and encapsulation information, then passes the original RS-232 serial protocol to the DB9 serial port so that any connected serial hardware would receive the serial protocol as expected.
28
Figure 13: Leveraging TCP/IP networks to transmit RS-232 serial data over IP
IP-enabled communications to the new upgraded hardware that supports IP communications can be implemented in parallel so that both the legacy serial and new ethernet IP devices can co-exist in the same system. Definitions for various types of SCADA firewall or gateways The sample network diagram above shows the introduction of a device entitled the SCADA Firewall. This device will also be shown in the remaining examples. The purpose of the device is to provide a security gateway or access control point for network traffic desiring to enter, leave or move between networks carrying critical real-time SCADA traffic. Depending on the level of risk, this gateway device could take on the form of several classes or types of network devices. The table on the next page illustrates several types of typical network gateways which can be used to secure SCADA networks, the ease of deployment, the level of security the device provides, and finally some overall comments on the scenarios when best to use each type of gateway.
29
Layer 3 device using VLANs and ACLs (Access Control Lists) to filter traffic based on source/ destination IP address and/or source/destination port. Cannot maintain state or perform deep packet inspection of the traffic payload.
Low
Only use in very low-risk environments as ACLs and VLANs can be compromised.
Firewalls
Purpose-built embedded network appliances which can maintain state as well as performing the same ACLs and VLAN separation as layer 3 switches and routers. Some switching components can be upgraded to have firewall feature set added to the switch.
MEDIUM: new firewall devices most likely needed; new IP networks may need to be created. Specific rules will have to be created for each type of network traffic that must traverse various network interfaces.
Medium
Firewalls are the first level of security appliance that can maintain state for each session. They provide true network isolation and specific rules must be made for each network path.
UTM devices can perform all of the security features of the switch/router and firewall, plus they can also perform deep packet inspection. They can block traffic when the payload of the network pockets are a match on a known virus or exploit signature.
HARD: UTM devices require a greater level of detail during configuration. All the same switching, routing and firewall features must be programmed, and in addition, specific actions can be taken when malicious payloads are detected.
High
If upgrading from a layer 3 device, the UTM is the best choice since it provides the highest level of protection without jumping into the next class of application-aware firewalls. The packet inspection capability can be set to detect and not to block to start.
SCADA Application-aware firewalls, also known as Layer 7 firewalls, are programmed to be able to parse the network traffic down to the protocol layer, and can either DENY or ACCEPT traffic based on the SCADA command, such as READ or WRITE data blocks from specific PLC registers.
HARD: to properly deploy Application-aware firewalls the programmer must have intimate knowledge of the process or HMI application. Rules must be made to either block or allow traffic based on the SCADA operator command and the specific area of PLC memory.
High
This class of firewall can lock the SCADA system down to the memory address to only allow network traffic that is required to operate the plant or facility. This class of firewall can prevent a Stuxnet-like attack by blocking data writes to specific PLC memory addresses.
Data diode
Uses optics to physically ensure that data can only be transferred in one direction.
HARDEST: since data can only transfer in one direction, some protocols that require a connection handshake will need to be tested. The way data is written may need to be altered from a built in API to an SFTP or PUSH type of data transfer.
Highest
Data diodes only allow data to be transferred in one direction. In very high risk scenarios such as nuclear plants and hazardous chemical plants, data diodes can ensure that network traffic cannot pass down to the controllers.
30
Serial to IP converters
broadband service)
With rising costs of lease and dial line service, utility operators are looking for options to replace modems in the field without having to replace any of the hardware, software, or protocol drivers. Cellular broadband services have dropped in price, and several industrial vendors offer a cellular broadband modem that communicates on all major carrier technologies, while still providing a resident RS-232 port for legacy systems. Earlier in this document, the issue of latency was raised with cellular and satellite communications. When considering migration of leased line or modem communications to cellular or satellite options, testing must be done to ensure that the communication performance can still support the real-time requirements of the SCADA system.
Usually, two installation options are found when using cellular broadband modems for SCADA communications. In the first option, the cellular carrier provides static public addresses for each device and uses the internet as the backhaul for connectivity to the field devices. For some applications where the system is in monitor-mode and can only read from the devices in the field, using this option whereby the carrier assigns public static IP addresses to the end points provides greater benefits than the risk. With the rise in large installation projects involving data acquisition and smart metering, and with the advent of MPLS (Multiprotocol Label Switching), several major international cellular carriers began offering private IP networks that leverage cellular broadband as the last mile to the end devices.
32
The entry and exit points of an MPLS network are called label edge routers (LER), which, respectively, push an MPLS label onto an incoming packet and pop it off the outgoing packet. The major carriers can drop a wired router anywhere onto the network that acts as the wired MPLS node and configure the remote cellular nodes to operate in the same private network. While the second option is more secure and uses private IP address spaces, VPN (virtual private networks) can be established over MPLS, and there exists two methods for building MPLS VPNs. A layer 3 MPLS VPN, or L3VPN, combines enhanced BGP signalling, MPLS traffic isolation and router support for VRFs (Virtual Routing/Forwarding) to create an IP-based VPN. Compared to other types of VPN such as IPSec VPN or ATM, MPLS L3VPN is more cost efficient and can provide more service options. However, the carriers do not guarantee the privacy of traffic sent over the MPLS network, and a lot of trust must be placed in the operator of the MPLS equipment to keep the data secure and private along the way. If privacy is a concern, other encryption methods such as end-to-end IPSec tunnels should be considered to encrypt the data at injection points before entering the MPLS backbone. Another consideration is service level agreements (SLAs). When using telecommunications infrastructure from cellular or 3rd party service companies, SCADA asset owners should be aware that the performance of the system is no longer under their control. Backup communication paths should be implemented to avoid a single point of failure; i.e. the system should be capable of suffering interruptions from the communications provider.
Multi-network design
IP conversion has spurred the growth of multi-network multiplexors that support multiple network transport interfaces on the same hardware platform. These class of devices typically also support 2 to 6 RS-232 communication ports in addition to ethernet IP interfaces. This allows for the best combination of backward compatibility with legacy serial devices with the ability to connect to the site over multiple network configurations. For instance, at one site the cellular broadband may have a strong signal, so a cellular broadband interface is used on the carrier side of the multiplexor. At the next site, there may already have an existing PSTN or Frame Relay network present, so those interface options would be used. There can be a diverse set of telecommunications options that exist throughout the system, and since the multiplexor devices support both IP and serial protocols, the system can be upgraded over time. Depending on the level of risk, various encryption standards can be implemented over the telecommunication links.
33
Figure 17: Modernisation in process: legacy and IP devices on the same IP network
34
Future technologies
Next Generation Networks (NGN) is a broad term used to describe key architectural evolutions in telecommunication networks. The overall concept behind NGN is that one network transports all information and services (i.e. voice, data, video, etc.) by encapsulating these into packets. NGNs are commonly built using the Internet Protocol, and therefore many people describe these NGN systems as all-IP networks. The key difference with Next Generation Networks is a more defined separation between the transport (connectivity) portion of the network and the services which run on top of that transport layer. This means that whenever a carrier wants to add on a new service such as voice-over-IP, Video-over-IP, broadband internet or even cable TV, the new service can be commissioned at the central locations, and new hardware deployed to the consumer, but the network transport or core functioning components of the network do not have to be changed. When considering the use of NGN technology with SCADA and control systems, network isolation and redundancy must be considered. In terms of isolation, some MPLS and NGN technologies support layer 2 separation. Layer 2 is to be preferred over layer 3 separation because the carrier typically manages layer 3, and the system users must put their faith in the carrier. This concern is valid for existing technology today but may change in the future. Where significant risk is involved, consider using the telecommunications carrier as only the transport layer, and inject encryption before traffic enters the MPLS or NGN system, then decrypt it again on the other side after it comes out of the cloud. The other consideration is that these Next Generation Networks are carrying non-critical data over the same physical infrastructure as the critical SCADA and control systems network protocols, which means that any failure of the infrastructure due to high network traffic, congestion, or security concerns on the other non-critical functions can impact the reliability and availability of the SCADA and control systems protocols. While this concern is valid for the existing technology today, keep in mind that this particular risk may change in the future. For higher risk applications, multiple redundant network transport methods should be considered, and in some cases building a private network just for SCADA and control systems communications may be required.
35
36
In the network diagrams and examples shown in this section, the security levels shown align with the ISA S99 SCADA security architecture model.
Figure 18: Harvesting security alerts and logs off of SCADA system devices
37
For SCADA and control system networks containing critical components, system administrators should also consider deploying in-line network appliances at choke points performing Network Intrusion Prevention and Anti-Virus, to filter and drop packets which are a known signature match for malicious traffic.
Figure 19: Use of intrusion prevention system (IPS) technology for networks directly upstream of SCADA environments
38
Firmware for IP-enabled telecommunications equipment and control devices should be kept up to date within the current patch cycle. Control devices such as PLCs, RTUs, Smart Meters, ethernet I/O, and IP-enabled instrumentation should be deployed with a password or PIN code that must be required to make changes to the configuration over an ethernet connection. Based on the level of risk, some devices may be configured to only allow configuration changes over a serial console cable. Control devices such as PLCs, RTUs, Smart Meters, ethernet I/O, and IP-enabled instrumentation have known TCP/IP stack vulnerabilities. Switches, routers, firewalls, or any network devices in front of the control devices should be configured with rate limiting commands to restrict data from flooding the device. The alerts and logs from network equipment can be either configured to be sent to a SEM (Security Event Management) system and viewed by IT Security staff, or can be translated to OPC tags and viewed by SCADA and Control System operators. Translation software is available to convert SNMP and Syslog data to OPC. Whether the network is monitored by IT staff or SCADA operators, responsibility for the care and management of the networks should be clearly defined, and system logs should be reviewed on at least a monthly basis.
39