INCIDENT RESPONSE

WHAT CONSTITUTES AN INCIDENT?

The Computer Security Resource Center (CSRC) of the National Institute of Standards and Technology
(NIST) defines both events and incidents in Special Publication 800- 61, the Computer Security Incident
Handling Guide.

• An event is described simply as “any observable occurrence in a system or network,”

• An incident is defined as “violation or threat of violation of computer security policies, acceptable use
policies, or standard security practices.”

A great number of government agencies, partner organizations, contractors, and

supporting businesses have IT Security departments that rely on NIST guidelines for
internal policy.
• A computer security incident is “any unlawful, unauthorized, or unacceptable action that involves a
computer system or computer network.”
• Expand this definition to “any unlawful, unauthorized, or unacceptable action that involves a
computer system, cell phone, tablet, and any other electronic device with an operating system or that
operates on a computer network.”

 It is important to understand that different organizations will likely have their own definition of what
constitutes an “incident.”
 There will be different tiers of incidents with varying responses based on the tier.
 Defining an incident and expected response is part of a mature security organization.
 An occurrence that results in actual or potential jeopardy to the confidentiality, integrity, or availability
of an information system or the information the system processes, stores, or transmits or that constitutes
a violation or imminent threat of violation of security policies, security procedures, or acceptable use
policies.

 Anomalous or unexpected event, set of events, condition, or situation at any time during the life cycle of
a project, product, service, or system.

 An occurrence that actually or potentially jeopardizes, without lawful authority, the confidentiality,
integrity, or availability of information or an information system; or constitutes a violation or imminent
threat of violation of security policies, security procedures, or acceptable use policies.
 WHAT IS INCIDENT RESPONSE?
Incident response is a coordinated and structured approach to go from incident detection
to resolution.

Incident response may include activities that:

• Confirm whether or not an incident occurred
• Provide rapid detection and containment
• Determine and document the scope of the incident
• Prevent a disjointed, noncohesive response
• Determine and promote facts and actual information
• Minimize disruption to business and network operations
• Minimize the damage to the compromised organization
• Restore normal operations
• Manage the public perception of the incident
• Allow for criminal or civil actions against perpetrators
• Educate senior management
• Enhance the security posture of a compromised entity against future incidents
 The activities and team members that are a part of the incident response will vary based on the goals of the
incident response.

 The goals of an incident response may vary depending on factors such as the severity of the incident, the
victim’s needs, the victim’s incident response methodology, the timing of the incident, the intent of the attack
group involved in the compromise (if known), the industry or customers impacted, and executive support for
the incident response.

 In general, an incident response consists of an investigation team that determines what happened and
performs a damage assessment, a remediation team that removes the attacker from the environment and
enhances the victim’s security posture, and some form of public relations (to senior level management,
internal employees, business partners, or the public).
 THE INCIDENT RESPONSE PROCESS
 An incident response process is a collection of procedures aimed at identifying, investigating and responding
to potential security incidents in a way that minimizes impact and supports rapid recovery.

 The incident response process consists of all the activities necessary to accomplish the goals of incident
response.

 The overall process and the activities should be well documented and understood by your response team, as
well as by stakeholders throughout your organization.

The process consists of three main activities, and we have found that it is ideal to have dedicated staff for each:

• Initial response

• Investigation

• Remediation
 Initial response is an activity that typically begins the entire IR process.
 Once the team confirms that an incident is under way and performs the initial collection and
response steps, the investigation and remediation efforts are usually executed concurrently.

 The investigative team’s purpose is solely to perform investigatory tasks.

 During the investigation, this team continually generates lists of what we call “leads.”
 Leads are actionable items about stolen data, network indicators, identities of potential subjects, or
issues that led to the compromise or security incident.

 These items are immediately useful to the remediation team, whose own processes take a significant
amount of time to coordinate and plan.
 In many cases, the activity that your team witnesses may compel you to take immediate action to halt
further progress of an intrusion.
 Initial Response

 The main objectives in this step include assembling the response team, reviewing network-based and other
readily available data, determining the type of incident, and assessing the potential impact.
 The goal is to gather enough initial information to allow the team to determine the appropriate response.

• Typically, this step will not involve collecting data directly from the affected system.
• The data examined during this phase usually involves network, log, and other historical and contextual
evidence. This information will provide you the context necessary to help decide the appropriate response.

For example, if a banking trojan is found on the CFO’s laptop, your response will probably be quite different
than if it is found on a receptionist’s system.

Also, if a full investigation is required, this information will be part of the initial leads.
Some common tasks you may perform during this step are:

• Interview the person(s) who reported the incident. Gather all the relevant details they can provide.

• Interview IT staff who might have insight into the technical details of an incident.

• Interview business unit personnel who might have insight into business events that may provide a context
for the incident.

• Review network and security logs to identify data that would support that an incident has occurred.

• Document all information collected from your sources.

 Investigation

 The goal of an investigation is to determine facts that describe what happened, how it happened, and in some
cases, who was responsible.

 As a commercial IR team, the “who” element may not be attainable, but knowing when to engage external
help or law enforcement is important.

 Without knowing facts such as how the attacker gained access to your network in the first place, or what the
attacker did, you are not in a good position to remediate.

 A five-step process, that promotes an effective investigation.

 Initial Leads
The collection of initial leads is a critical step in any investigation.

A common investigative misstep in many organizations: focus only on finding malware.

It is unlikely that an attacker’s only goal is to install malware. The attacker probably has another objective in
mind, perhaps to steal e-mail or documents, capture passwords, disrupt the network, or alter data.
Once the attacker is on your network and has valid credentials, they do not need to use
malware to access additional systems. Focusing only on malware will likely cause you
to miss critical findings.

• Remember, the focus of any investigation should be on leads.

• There have been many incidents where other teams completed investigations that uncovered few findings.
• In many cases, the failure of prior investigations was due to the fact that the teams did not focus on developing
valid leads. Instead, they focused on irrelevant “shiny objects” that did not contribute to solving the case.
• There are numerous incidents where we found significant additional findings, such as substantial data loss, or
access to sensitive computer systems, simply by following good leads.
• Often overlooked is the process of evaluating new leads to ensure they are sensible.
• The extra time you spend evaluating leads will help the investigation stay focused.

There are three common characteristics of good leads:

1. Relevant: The lead pertains to the current incident. This may seem obvious, but is often
overlooked. A common trap organizations fall into is categorizing anything that seems suspicious as part
of the current incident. Also, an incident prompts many organizations to look at their environment in
ways they never have before, uncovering many “suspicious activities” that are actually normal. This
quickly overwhelms the team with work and derails the investigation.

2. Detailed: The lead has specifics regarding a potential course of investigation. For example, an
external party may provide you with a lead that indicates a computer in your environment
communicated with an external website that was hosting malware. Although it was nice of them to
inform you, this lead is not very specific. In this case, you need to pry for more details. Ask about the
date and time of the event and IP addresses—think who, what, when, where, why, and how. Without
these details, you may waste time.
3 Actionable: The lead contains information that you can use, and your organization possesses the
means necessary to follow the lead. Consider a lead that indicates a large amount of data was
transferred to an external website associated with a botnet. You have the exact date, time, and
destination IP address. However, your organization does not have network flow or firewall log data
available to identify the internal resource that was the source of the data. In this case, the lead is not
very actionable because there is no way to trace the activity to a specific computer on your network.
 Indicators of Compromise (IOC) Creation

• IOC creation is the process of documenting characteristics and artifacts of an incident in a structured
manner.

• This includes everything from both a host and network perspective—things beyond just malware. Think
about items such as working directory names, output file names, login events, persistence mechanisms,
IP addresses, domain names, and even malware network protocol signatures.

• The goal of IOCs is to help you effectively describe, communicate, and find artifacts related to an
incident.

• An important consideration in choosing how to represent IOCs is your ability to use the format within
your organization.
Network indicators of compromise are most commonly represented as Snort rules, and there are both free
and commercial enterprise-grade products that can use them.

From a host perspective, some of the IOC formats available are:

• Mandiant’s OpenIOC (www.openioc.org)
• Mitre’s CybOX (cybox.mitre.org)
• YARA (code.google.com/p/yara-project)

 For two of these formats, OpenIOC and YARA, there are free tools available to create IOCs.
 Mandiant created a Windows-based tool named IOC Editor that will create and edit IOCs based on
the OpenIOC standard.
 For YARA, there are a number of tools to create and edit rules, or even automatically create rules
based on a piece of malware.
 IOC Deployment

• Using IOCs to document indicators is great, but their real power is in enabling IR teams to find evil in
an automated fashion, either through an enterprise IR platform or through visual basic (VB) and
Windows Management Instrumentation (WMI) scripting.
• The success of an investigation depends on your ability to search for IOCs across the enterprise and
report on them in an automated way—this is what we mean by “IOC deployment.”
• Therefore, your organization must possess some capability to implement IOCs, or they are not much
use.

• For network-based IOCs, the approach is straightforward—most solutions support Snort rules.
• However, there is not yet an accepted standard for host-based IOCs. Because of this, effectively using
host-based IOCs in your investigative process may be challenging.

• You don’t need a large budget to use IOCs, although your ability to effectively use them across an
enterprise will likely require significant funding. There are both free and commercial tools that can use
the YARA and OpenIOC standards to search for IOCs.
• On the free side, the YARA project provides tools to search for YARA rules. Also, a number of open
source projects can use YARA rules, some of which are listed in the YARA tools link.

• Mandiant provides a free tool, Redline, that you can use to search for OpenIOCs on systems.

These free tools are quite effective on a small number of systems, but do not scale up very well.

• To effectively search for an IOC in an enterprise, you will need to invest in a large-scale solution.
• For example, FireEye can use YARA rules, and Mandiant’s commercial software offerings can use
OpenIOCs. Keep in mind, though, that the software and processes that support using IOCs are still
not very mature.
 Identify Systems of Interest

• After deploying IOCs, you will begin to get “hits.” Hits are when an IOC tool finds a match for a given rule or
IOC.
• Prior to taking action on a hit, you should review the matching information to determine if the hit is valid.
This is normally required because some hits have a low confidence because they are very generic, or because
of unexpected false positives. Sometimes we retrieve a small amount of additional data to help put the hit in
context.
• Unless the hit is high confidence, at this point it’s still unknown whether or not the system is really part of the
incident.

• We take a number of steps to determine if the system is really of interest.

• As systems are identified, you should perform an initial triage on the new information.
• These steps will help to ensure you spend time on relevant tasks, and keep the investigation focused:

 Validate: Examine the initial details of the items that matched and determine if they are reliable. For
example, if an IOC matches only on a file’s name, could it be a false positive? Are the new details
consistent with the known time frame of the current investigation?

 Categorize: Assign the identified system to one or more categories that keep the investigation
organized. Labeling a system as “Compromised” is too vague. It is more helpful to use categories that
more clearly indicate the type of finding and the attacker’s activities, such as “Backdoor Installed,”
“Access With Valid Credentials,” “SQL Injection,” “Credential Harvesting,” or “Data Theft.”

 Prioritize: Assign a relative priority for further action on the identified system. A common practice
within many organizations is to prioritize based on business-related factors such as the primary user or the
type of information processed. However, this approach is missing a critical point in that it does not
consider other investigative factors. For example, if the initial details of the identified system’s
compromise are consistent with findings from other systems, further examination of this system may not
provide any new investigative leads and could be given a lower priority. On the other hand, if the details
suggest something new, such as different backdoor malware, it may be beneficial to assign a higher priority
for analysis, regardless of other factors.
 Preserve Evidence

• Once systems are identified and have active indicators of compromise, the next step is to collect additional
data for analysis.
• We must create a plan for collecting and preserving evidence, whether the capability is in house or
outsourced.
• The primary goals when preserving evidence are to use a process that minimizes changes to a system,
minimizes interaction time with a system, and creates the appropriate documentation.

• You may collect evidence from the running system or decide to take the system down for imaging.
• Because any team has limited resources, it does not make sense to collect large volumes of data that may
never be examined (unless there is good reason). So, for each new system identified, you must make a
decision about what type of evidence to collect.
• Always consider the circumstances for each system, including if there is anything different about the
system or if a review of live response data results in new findings.

• The typical evidence preservation categories are live response, memory collection, and forensic disk image.
 Live response: Live response is the most common evidence collection process we perform on an incident
response. A live response is the process of using an automated tool to collect a standard set of data about a
running system. The data includes both volatile and nonvolatile information that will rapidly provide answers
to investigative questions. The typical information collected includes items such as a process listing, active
network connections, event logs, a list of objects in a file system, and the contents of the registry. We may also
collect the content of specific files, such as log files or suspected malware. Because the process is automated
and the size of the data is not too large, we perform a live response on most systems of interest. A live
response analysis will usually be able to further confirm a compromise, provide additional detail about what
the attacker did on the system, and reveal additional leads to investigate.

 Memory collection: Memory collection is most useful in cases where you suspect the attacker is using a
mechanism to hide their activities, such as a rootkit, and you cannot obtain a disk image. Memory is also
useful for cases where the malicious activity is only memory-resident, or leaves very few artifacts on disk. In
the majority of systems we respond to, memory is not collected, however. It is found that analyzing memory
has limited benefits to an investigation because it does not provide enough data to answer high-level questions.
Although you may be able to identify that malware is running on a system, you will likely not be able to
explain how it got there, or what the attacker has been doing on that system.
 Forensic disk image: Forensic disk images are complete duplications of the hard drives in a system.
During an incident response, it is common for us to collect images in a “live” mode, where the system is not
taken offline and we create an image on external media. Because disk images are large and can take a long
time to analyze, we normally collect them only for situations where we believe a disk image is necessary to
provide benefit to the investigation. Forensic disk images are useful in cases where an attacker performed
many actions over a long time, when there are unanswered questions that other evidence is not helping with,
or where we hope to recover additional information that we believe will only be available from a disk image.
In incidents that do not involve a suspected intrusion, full disk image collection is the norm.
 Analyze Data
• Analyzing data is the process of taking the evidence preserved in the previous step and performing an
examination that is focused on answering the investigative questions.
• The results of the analysis are normally documented in a formal report.
• This step in the incident response lifecycle is where we usually spend most of our time. Your organization
must decide what analysis you will perform on your own, and what portions, if any, you will outsource.

There are three major data analysis areas:

 Malware analysis: During most investigations, we come across files that are suspected malware. We have
a dedicated team of malware analysts that examines these files. They produce reports that include indicators of
compromise and a detailed description of the functionality.

 Live response analysis: Examining the collected live response data is one of the more critical analysis
steps during an investigation. If you are looking at live response data, it’s normally because there is some
indication of suspicious activity on a system, but there are limited details. During the analysis, you will try to
find more leads and explain what happened. If details are missed at this stage, the result could be that you
overlook a portion of the attacker’s activity or you dismiss a system entirely. The results of live response
analysis should help you understand the impact of the unauthorized access to the system and will directly
impact your next steps.
 Forensic examination: A forensic examination performed on disk images during an incident response
is a very focused and time-sensitive task. We normally write down a handful of realistic questions we want
answered, decide on an approach that is likely to uncover information to answer them, and then execute. If
we don’t get answers, we may use a different approach, but that depends on how much time there is and
what we expect to gain from it. If the incident you are responding to is more traditional, such as an internal
investigation that does not involve an intrusion, this is the analysis on which you will spend most of your
time. There is a degree of thoroughness applied to traditional media forensic examinations that most IR
personnel and firms have no experience in.
 Remediation
• Remediation plans will vary greatly, depending on the circumstances of the incident and the potential impact.
The plan should take into account factors from all aspects of the situation, including legal, business, political,
and technical. The plan should also include a communication protocol that defines who in the organization will
say what, and when.

• The timing of remediation is critical. Remediate too soon, and you may fail to account for new or yet-to-be-
discovered information. Remediate too late, and considerable damage could occur, or the attacker could change
tactics. The best time to begin remediation is when the detection methods that are in place enter a steady state.
That is, the instrumentation you have configured with the IOCs stop alerting you to new unique events.

• It is recommend starting remediation planning as early in the incident response process as possible so that you
can avoid overtasking your team and making mistakes.
• Some incidents require significantly more effort on remediation activities than the actual investigation. The
approach we take is to define the appropriate activities to perform for each of the following three areas:
1. Posturing
2. Tactical (short term)
3. Strategic (long term)
 Posturing is the process of taking steps that will help ensure the success of remediation. Activities such as
establishing protocol, exchanging contact information, designating responsibilities, increasing visibility,
scheduling resources, and coordinating timelines are all a part of the posturing step.

 Tactical consists of taking the actions deemed appropriate to address the current incident. Activities may
include rebuilding compromised systems, changing passwords, blocking IP addresses, informing customers
of a breach, making an internal or public announcement, and changing a business process.

 Finally, throughout an investigation, organizations will typically notice areas they can improve upon.
However, you should not attempt to fix every security problem that you uncover during an incident; make a
to-do list and address them after the incident is over.
 The Strategic portion of remediation addresses these areas, which are commonly long-term improvements
that may require significant changes within an organization.
THE
END