Professional Documents
Culture Documents
SANS 2022 ICS Field Manual Vol2 v5
SANS 2022 ICS Field Manual Vol2 v5
D
ICS C y b
EL
FI UAL
M AN Vol.
2
Author:
Dean Parsons
B.SC., GICSP, GRID, CISSP,
GSLC, GCIA
Certified SANS Instructor |
CEO ICS Defense Force Inc. &
ICS Cybersecurity Leader
Contents
What to expect from these ICS security field manuals 3 What to expect from these
Introduction to Volume 2 4 ICS security field manuals
Sliding scale of cybersecurity 5
Establishing an ICS asset inventory 7 If you are new to industrial control system (ICS) security, the
Industrial control network protocols 12 SANS ICS Cybersecurity Field Manuals will get you up to speed
quickly with long-lasting reference materials, free resources,
Defining network security monitoring for ICS 14
and a training path in control system security for you and your
Set-up of ICS network security monitoring 20 teams. The manuals consist of several sections and volumes,
each focusing on a different aspect of ICS cyber defense.
ICS network security monitoring in practice 23
Epilogue to Volume 2 30
The consequences of modern ICS cyber- Volume 2 of the ICS Cybersecurity Field The Sliding Scale of Cybersecurity can be used to categorize the security maturity,
attacks can include but are not limited Manual provides insight into the active actions, and investments that build a cybersecurity program.¹ The scale has five
to widespread power grid blackouts, cyber defense cycle, presents effective progressive categories: Architecture, Passive Defense, Active Defense, Intelligence, and
failure or physical destruction of ways to establish an ICS asset inventory Offense. Each category builds on the previous one to make the upcoming categories
critical engineering equipment, massive and obtain network visibility to apply stronger. Architecture is a foundational and affordable starting point to which there
business financial losses, paralysis of network security monitoring (NSM) is high return on investment, and from which all following categories of the scale will
smart city emergency infrastructure through data collection, network traffic benefit. Each category in the scale is described below.
in large municipalities, human injury analysis, and network threat detection.
or death, and possibly devastating It also serves as a resource for budget- Passive Active
Architecture Intelligence Of fense
Defense Defense
environmental impacts. ICS intrusions constrained ICS security programs to
Figure 1: Sliding Scale of Cybersecurity
will continue to occur and likely leverage no-cost or low-cost tools as
increase in their severity and range of they start their journey to mature efforts 1. Architecture. The planning, establishment, and maintenance of systems with security
consequences across multiple critical to protect control systems and critical and reliability as the priority, including the supply chain, patching, and network
managing control system cyber risk and 2. Passive Defense. Systems added to Architecture that do not require consistent
effectively applying tactical ICS defenses human interaction and provide reliable defense or insight into a subset of less-
is achievable! advanced threats.
3. Active Defense. The process of human analysis consistently involved in proactive
defense. It involves using ICS-specific tools, monitoring for, actively responding to,
ICS418 ICS Security Leadership and learning from adversaries internal to the control networks.
Simulation Game - Industrial
4. Intelligence. Collecting data, exploiting collected data and processing it into
Cyber42
information, obtaining or adding
SANS has extended the Cyber42 Leadership
Simulation Game to the ICS418 course as context, and producing actionable Career Development Opportunity
Industrial Cyber42 (https://www.sans.org/ threat intelligence to inform proactive - ICS515
blog/cyber42). Students participate in defense. The SANS ICS515: ICS Visibility, Detection, and
various ICS risk-based and management Response course teaches students all steps of
5. Offense. Partaking in legal
decision scenarios to protect a control the Active Cyber Defense Cycle through hands-
countermeasures and self-defense
system using their risk management skills. on technical labs and real-world industrial
actions against the adversary.
The object of the game is to finish with the attack scenarios and related lessons learned.
highest safety culture score.
1 For more information, see the SANS White Paper, The Sliding Scale of Cybersecurity, available
here: www.sans.org/white-papers/36240/
ICS security managers must support ICS security managers must map It is difficult to protect a control environment and keep engineering operations resilient
their teams to lead them to success. This existing technical ICS security controls without knowing which engineering assets are in production and which assets are
means positioning team members and to the sliding scale. They can start deemed critical. An established ICS asset inventory of operational technology and
technologies, at a minimum, in an Active maturing their Industrial Control engineering assets will improve the ICS security program’s vulnerability management,
Defense position within the Sliding Scale System and Operational Technology network security monitoring, and incident response scenarios. The common
of Cybersecurity. Active cyber defense cybersecurity program with the methodologies to establish and maintain an inventory are physical inspection,
configuration analysis, active scanning, and passive network traffic analysis. These
for control systems involves trained Architecture and Passive Defense
methods can refine an existing asset inventory or be used to build and maintain an
ICS analysts leveraging technology and categories, then move to the Active
inventory.
ICS-specific knowledge and protocols Defense category by documenting a
to monitor, respond to, and learn from control system asset inventory as a best
Physical Inspection: This involves Configuration Analysis: A review of
threats targeting control networks. practice.
physically walking through industrial configuration settings may require
In parallel, ICS defenders and risk
Tactical ICS security team members facilities, documenting the hardware access to many controls system and
managers must work with engineering seen in racks and network cabinets, network devices. Switch and firewall
must ensure that the tools deployed in
teams to define incident response inspecting the software and protocols configurations can reveal IP address
control system environments are “ICS-
processes, outcomes, and recovery used, and taking other proactive and MAC address pairings through
aware” – that is, they are specifically
steps, as safety is prioritized above all steps. Physical inspection is time- Address Resolution Protocol (ARP)
designed or adapted to suit ICS for both
else. consuming and expensive if it tables to indicate devices allowed or
endpoint and network defense. For
involves traveling to remote sites. denied access to the network. Traffic
example, network intrusion detection
Some potential physical risk exists, so and port information at a quintuple
systems (IDS) must be capable of deep
Career Development personal protective equipment will be level could reveal general protocols
Opportunity - ICS418 packet inspection and perform complete
required at sites. in use. Collection and interpretation
ICS protocol packet dissection.
The SANS ICS418: ICS Security Essentials of configuration settings from
for Managers course includes an ICS attack ICS systems (programmable logic
history walkthrough for new and existing controllers (PLCs), remote terminal
ICS/OT security managers, with a focus units (RTUs), intelligent electronic
on lessons learned to improve ICS risk devices) can also be used to generate
management and reporting to the board. a holistic inventory of hardware,
software, and firmware installed on
these devices.
Active Scanning: This is intrusive and augmenting it with passive control A PRACTICAL EXAMPLE TO AN ICS ASSET INVENTORY
to ICS operations and an unnatural system network traffic capture and The steps below are an example of a practical approach to an ICS
representation of network analysis can be highly effective in a asset inventory that combines both physical and passive network
communications. However, this maturing program. traffic capture.
method of asset identification is 1. Start by reviewing any already-created network diagrams and
We can see several benefits when
very fast and can provide detailed engineering documentation such as “as-built documents.”
combining asset inventory methods. In
information about devices, 2. Use an encrypted laptop with at least a basic spreadsheet application to start
the above example, physical inspection
services, etc. It should be tested cataloging and storing ICS asset information during a physical site walk-through, as
takes advantage of face-to-face security
in a development environment seen below in Table 1: Sample Asset Inventory Attributes.
awareness discussions on-site with
prior to scanning any production 3. Augment physical inspection with passive network packet captures on critical
engineering, safety, and operational
environment. network segments that host critical ICS assets by using either a SPAN or mirrored
teams. This goes a long way when the
Passive Network Traffic Analysis: port configuration off a fully managed switch or hardware TAP.
teams need to perform ICS incident
Nonintrusive to industrial operations, response in the field but rely on 4. Ensure field device configurations are backed up during an incident and securely
this analysis can provide an accurate engineering staff to help with forensic stored for later comparison to detect whether an unauthorized change occurred and
representation of natural control reload trusted configurations and project files (controller logic), if needed.
data acquisition, log collection and/
system network communications. It or engineering network changes during 5. At a minimum, record attributes from the commonly targeted critical assets such
can provide a visual network diagram containment, or threat eradication, as data historians, human machine interfaces (HMIs), PLCs, RTUs, engineering
that can be printed and used for workstations, core network devices, and active safety instrumented systems.
for example. Passive control system
engineering troubleshooting and ICS network traffic captures are safer and Table 1: Sample Asset Inventory Attributes
incident response. Where feasible quicker and can create or verify an
Sample Asset Inventory Attributes
and for best results, it is beneficial existing inventory. They provide network
Site location
(though not always possible) to data to analyze when performing threat
Facility type
capture and analyze network traffic hunting or threat detection exercises.
Asset type and ID tag
during different modes of operation
Asset location room, cabinet, rack
(startup, normal operations, and
Description of asset function for operations
emergency modes).
Physical Configuration Impact to operations if assets are unavailable
Time to complete
Inspection Analysis
Each asset inventory method poses IP and MAC address
different risks to operations and takes Network protocols used
different times to complete. Tactical ICS Model, manufacturer, serial number
defenders and engineering staff must Firmware version for controllers and related modules, chassis information
Passive Traf fic Active
work together to weigh the risk versus Analysis Scanning Applications installed on critical assets with versions
time and related returns on investment Assets deemed critical – data historians, HMIs, primary controllers, control system
for each method. Methods can be network switches
Risk to operations
combined based on the ICS security Project files and configuration (last change date, secure storage location, etc.)
program maturity and budget. For Figure 2: Asset Inventory Method Analysis Dependencies – systems, networks, other assets, etc.
example, performing physical inspection Primary and secondary contact for asset
determine the requirements applicable to native protocols in control networks to help safely identify assets, perform
specific implementations. threat detection, and understand threats that may be “living off the land.”
To start network detection in ICS on a limited budget, facilities can leverage sector- ICS NETWORK SECURITY to perform DNS name resolution, which
specific ICS threat intelligence using freely available tools such as tcpreplay, Snort, MONITORING – ANALYSIS can help with asset identification. Each
Zeek, and Suricata with built-in or added ICS rulesets/dissectors. Known IP addresses A triggered detection rule, IP address and associated ports should
associated with attack campaigns can be used in a search across network 5-tuple or such as a match on a malicious IP be recorded and analyzed to identify all
full-packet captures. The pseudo rules and logic detailed below can be expanded or address from ongoing attack campaign, active assets for legitimate operation.
changed to suit an organization’s control network, tools deployed, and general setup. will lead to the NSM Analysis phase.
Wireshark:
It is important to know which assets
ICS ASPECT – DETECTION Wireshark > Statistics >
on the network are critical for safety Conversations
An IDS is preferred for threat detection in ICS environments over an intrusion
and operations. This makes it easier to
prevention system (IPS). IDS is also preferred to prioritize safety – that is, to Provides statistics about conversations
identify anomalous network connections
reduce false positive detections that could cause legitimate control commands to be in the traffic between devices, displayed
around critical assets to determine
blocked if detected by an IPS and that may cause operational and safety disruptions. as IP addresses. Information such as
when ICS incident response should be
The NSM Detection phase is primarily about understanding what is “normal” for the the start, stop, and duration of the
performed. The tshark or Wireshark
industrial operations to be better at spotting “abnormal” activity. For example, with conversations is notable. The devices
filters discussed below can be expanded
engineering knowledge and through analysis of normal operations, expected function communicating, protocols in use, and
or changed to suit the hunt for malicious
codes, other operations and elements, and anonymous activity can be discovered. Using their communication pattern should
network activity.
these tools and filters are a great start when developing an ICS NSM program. be noted. A single device having
Pseudo rules and Logic:
ICS ASPECT – ANALYSIS conversations with multiple devices
ICS environments have far could indicate an HMI.
Replay packet captures against a listening network IDS such as Snort to alert to known less connectivity to the
threats: Wireshark:
Internet and use far fewer encrypted
Wireshark > Statistics > Protocol
communications than in traditional IT Hierarchy
sudo tcpreplay --intf1=<nic_for_snort > --mbps=topspeed <ICS-Network_file.pcap>
environments. ICS attacks can abuse
legitimate engineering software and Provides statistics about observed
Alert on communications to PLC that is not HMI:
native industrial control protocols. protocols on the network. Protocols
alert tcp !$Modbus_HMI any -> $Modbus_PLC any (msg:“TCP comms to PLC which are displayed in a tree layout with bar
Information, assets, protocols, files, graphs indicating the percent of the
is not the HMI”;)
and commands from the control network protocol seen in an overall capture. The
Alert on possible recon scan or mapping using ModbusTCP on a network that does not can be discovered and analyzed by the list should be recorded to determine
use it: tools and filters below. which protocols are needed and
Wireshark: expected for operations. Legitimate
alert tcp any any -> any 502 (msg:“Scan or usage of ModbusTCP on network
Wireshark > Statistics > Endpoints protocols could be abused in attack
without it”;)
scenarios, so it is important to record
Provides statistics about logical
Alert on possible TCP connection to known malicious command and control server: and analyze protocol patterns and
addresses on the network, including the
source and investigate and validate
asset IP and MAC addresses. Displays
alert tcp any any -> <evil_C2_ip> any (msg:“Connection attempt to known devices sending commands to field
number of packets, total bytes, bytes
evil C2 IP address”;) devices.
received and transmitted, and attempts
malware analysis sandbox to determine All DNP3 function codes in use and IP addresses using them:
threat behaviors to develop defensive
tshark -n -Y dnp3 -T fields -e ip.src -e ip.dst -e dnp3.al.func -e _ws.col.
countermeasures.
Info -r <ICS-Network_file.pcap> | sort | uniq
General network statistics about logical addresses on the network:
IP addresses of devices using BACnet and BACnet control commands:
tshark -qz ip_hosts,tree -r <ICS-Network_file.pcap>
tshark -Y bacnet -T fields -e ip.src -e ip.dst -e bacnet.control -e _ws.col.
Asset names from NetBIOS communications: Info -r <ICS-Network_file.pcap> | sort | uniq
tshark -Y nbns -T fields -e nbns.name -r <ICS-Network_file.pcap> | sort | Possible HTTP downloads, including filename and uniform resource identifier (URI):
uniq
tshark -n -T fields -e http.request.method -e http.host -e http.request.uri
Asset names from DNS that could be assets performing Internet checks: -r <ICS-Network_file.pcap> | sort | uniq
tshark -T fields -e ip.src -e dns.qry.name -Y ‘dns.flags.response eq 0’ -r Export data for analysis – HTTP downloads, including filename and URI:
<ICS-Network_file.pcap> | sort | uniq tshark -r <ICS-Network_file.pcap> --export-objects http,<OutputDir> | sort |
uniq
Traffic going to external addresses by internal source IP to external IP:
tshark -T fields -e ip.src -e ip.dst -r <ICS-Network_file.pcap> ‘not ip.dst in Export data for analysis – SMB file transfers, including filename and file data:
Encrypted communications, less common in ICS, which could be covert channels: Files transferred via server message block (SMB) with remote hostname, account name,
file(s) accessed:
tshark -Y ssl -T fields -e ip.src -e ip.dst -e tcp.port -e _ws.col.Info -r
<ICS-Network_file.pcap> | sort | uniq tshark -n -Y ‘frame.number == 189’ -T fields -e smb2.filename -e smb2.tree -e
smb2.acct -e smb2.host -r <ICS-Network_file.pcap>
Protocols in use on the control network:
ICS security defenders must know what is normal in the ICS environment, which network
tshark -T fields -e frame.protocols -r <ICS-Network_file.pcap> | sort | uniq protocols are expected in different control system states, and what commands inside
| cut -d : -f 2-20 ICS protocols can read and change physical outputs in the field.
NSM Collection
Pros Cons
Method
The NSM collection, detection, and analysis phases should be started and repeated
while the above methods are applied across the three phases to prioritize the safety and Newly registered devices
on the net work
reliability of ICS operations. Deeper engineering knowledge is required for more specific
ICS protection. High confidence indicator of compromise matches and the discovery of
anomalous network patterns will call industrial incident response steps into action. Figure 7: Stage 2- ICS Threat Detection Concepts for Full-Capture Packet Analysis
STAGE 1: ICS THREAT DETECTION CONCEPTS FOR 5-TUPLE STAGE 3: ICS THREAT DETECTION BASED ON ICS BEHAVIOR
Top talker IP
addresses 3 Unexpected remote
access to HMI
1 Devices talking
that should not be
Connection at tempts to
Internet addresses
Matches on known
malicious IPs
The repeatable active cyber defense strategies specific to industrial control system networks and environments. It is common
for professionals working or looking to work or consult in these areas to earn their GRID
cycle guides a team through proactive
certification.
monitoring as a best practice in today’s
ICS threat landscape. The cycle has five
phases, as shown in Figure 9.
NSM excels in control system ICS NSM is especially important in
1. Threat Intelligence Consumption: environments due to the more static the case of adversaries living off the
Cyber threat intel is refined nature of ICS networks compared with land, where it is unlikely that antivirus
information with context on cyber IT enterprise networks. The active agents, even allowing for listing features
threats and threat groups that cyber defense cycle makes clear the designed specifically for ICS, would
defenders can leverage to detect, benefits of ICS NSM in today’s threat detect the abuse of legitimate control
Figure 9: The Active Cyber Defense Cycle
scope, or prevent the same or https://www.sans.org/white-papers/36240/ landscape. It leverages ICS NSM by system functions, including the abuse of
similar attacks previously observed. increasing knowledge of the control legitimate ICS/OT network protocols and
system, collecting data, analyzing engineering software.
2. Visibility: Increasing visibility can enhance technical and situational awareness of
data for threats, and executing ICS-
control system traffic and security. This means having a formal asset inventory and
specific incident response. However,
at least a passive view of the ICS network, and using technology that can dissect
the active cyber defense cycle and ICS
and properly interpret specific industrial protocols in network traffic streams.
NSM – what can be called the “network
3. Threat Detection: Detecting threats requires the capability to leverage technology visibility” of control environments –
that sifts through data for malicious signs of attack attempts or intruder entry. are not only about security. They also
directly support engineering tasks
Career Development
4. Incident Response: Successful incident response requires being prepared to such as communication, command, and
Opportunity - ICS418
execute quick triage and adapt incident response steps specific to control systems integration troubleshooting, all of which
while maintaining safety. support safety for facilities and their The SANS ICS418: ICS Security Essentials
for Managers course empowers new and
workers.
5. Threat and Environment Manipulation: To make the environment less habitable established ICS security managers from
all areas to understand the differences
for threat actors, defenders need to know how to change the threat during the
between IT and ICS/OT, prioritize safety,
attack or change the control system. A threat is defined as a malware capability
build and maintain strong relationships,
introduced by a threat actor or as human threat actors using legitimate operational build teams, and effectively manage ICS/
software or protocols with malicious intent to cause negative impacts. OT cyber risk.
Epilogue to Volume 2
Adversaries continue to evolve their ICS security managers looking to ICS security defenders looking to improve tactical ICS security must obtain and
attack tradecraft using traditional IT improve ICS risk management and the continue to grow their knowledge of cybersecurity and engineering operations
malware and extending the attack range resilience of their ICS security program (including protocols and commands), while prioritizing safety and administrating
with knowledge of how to abuse ICS must first establish an official asset modern security tools specifically designed or tuned for ICS environments. A main
systems. This “living off the land” attack inventory with the methodologies focus should be on performing the repeatable steps of the active cyber defense cycle
approach has them abusing native described in this manual. They must while leveraging ICS network visibility, packet captures and analysis, and hunt for
commands and software. In their wake, then leverage and mature the program threats proactively in the network.
adversaries leave serious financial, to an active defense position. The
ICS facilities owners and operators will do well to consider these top takeaways to
brand, and operational impacts, with objective is to ensure that security
kick-start or mature their ICS cybersecurity program:
potential catastrophic consequences controls are in place specifically
for operating environments, the safety for industrial control systems, with
of people, cities, regions, and countries ICS network and engineering device Continue to prioritize safety as #1
who run and rely on them. visibility. Security team members must
possess the ICS-knowledge required Embrace ICS and IT security differences
for rapid ICS incident triage and the
recovery of engineering devices to Establish a secure and searchable ICS asset Inventory
trusted restore points.
Enable ICS network security monitoring
Career Development Deploy the Active Cyber Defense Cycle for technical teams
Opportunity - ICS418
Align priorities against the Sliding Scale of Cyber Security
The SANS ICS418: ICS Security Essentials
for Managers two-day course prepares new
and experienced managers and leaders
responsible for ICS/OT security. Students
complete many in-class leadership drills and
real-world management-level ICS security
scenarios in an online leadership simulation
game across both days.