WHITE PAPER

USING SPLUNK TO DEVELOP AN INCIDENT

RESPONSE PLAN
 WHITE PAPER

Security incidents can happen without warning and Other key capabilities of any effective security
they often go undetected for long periods of time. platform, includes the ability to either manually
Organizations struggle to identify incidents because or automatically aggregate events, support
they often work in silos or because the amount of application programming interfaces (APIs) that can
alerts is overwhelming and hard to determine the be used to pull data from or push information to
signals among the noise. third-party systems.

Security teams in charge of dealing with these threats These platforms must also provide the ability to
are also often over burdened with false alerts. All of gather legally admissible forensics evidence, and
this slows and impacts an organization’s effectiveness these platforms must also have playbooks that
with incident response. provide organizations with guidance on how to
respond to specific types of incidents.
Although preventive measures can help diminish the
overall incident, and remains vital in a CISO’s overall Most importantly, an analytics-driven security
security strategy, it’s impossible to impede every platform needs to include auto-response capabilities
attack. In fact, organizations must assume a network that can disrupt cyberattacks already in progress.
will be compromised at some point. What can
The security platform also needs to be the hub
become more important than preventing a breach
around which a customizable workflow for managing
is minimizing the resulting damage by detecting,
incidents can be crafted. Of course, not every incident
containing and controlling the incidents.
has the same level of urgency attached to it.
Incident response specifically refers to the process
An analytics-driven security platform provides IT
and activities associated with investigating,
organizations with the means to categorize the
containing and remediating a security incident or
severity of any potential threat via dashboards that
breach after it has already happened. The goal of
can be used to triage new notable events, assign
any organization with an effective incident response
events to analysts for review, and examine notable
plan is to minimize damage by managing security
event details for investigative leads, an analytics-
events in near real time and by leveraging information
driven SIEM arms IT organizations with the contextual
uncovered during the investigation of the attack. This
insight needed to determine the appropriate response
helps to make decisions to contain and eliminate the
to any event.
impact of an attack.

Those response capabilities include the ability to

At the core of any effective incident response
identify notable events and their status, indicate the
strategy is a robust analytics-driven security platform
severity of events, start a remediation process, and
that makes it possible not only to identify incidents
provide an audit of the entire process surrounding
and their severity, but also provide the means to track
that incident.
and re-assign tasks associated with responding to the
alert, as well as, add annotations. Finally, the IT team is empowered with dashboards
where they can intuitively apply filters to enable
Responding With Confidence
the team to expand or reduce the scope of analysis
It is important to understand how an analytics-driven
during an investigation with only a few clicks. This
security platform can actually help with incident
eliminates a more manual process.
response and improving an organization’s security
posture. It is critical that any security platform enable The end goal should be nothing less than enabling
IT teams with the ability to provide other members of any security team member to place events, actions
an organization with varying levels of access based and annotations into a timeline that makes it simple
on their roles. for other members of the team to easily comprehend
what is occurring. Those timelines can then be

Using Splunk to Develop an Incident Response Plan 2

 WHITE PAPER

included in a journal that makes it simple to review of near real-time snapshots of the state of risk to
attacks and to implement a repeatable kill chain their security, data, the network, endpoints, as well
methodology to deal with specific types events. as, cloud devices, systems and applications. It proves
difficult and time consuming to not only categorize,
What does all of this mean in the real world? Perhaps
authorize and monitor systems, but assess, select and
the best way to understand the impact of an analytics-
implement the correct security controls.
driven security platform in developing an incident
response plan is to look at the real-world use cases. Using Splunk for security monitoring, customers can
more easily identify potential attacks, compromised
We will look at the following use cases:
systems or both. By detecting and monitoring for
• Security monitoring vulnerabilities that lead to attacks, customers can

• Incident investigation and forensics better protect their critical data to quickly understand
and remediate against attacks, such as phishing.
• SOC automation

• Fraud detection Incident Investigation and Forensics

Incident investigation seeks to determine the
• Insider threat
circumstances of the incident. Depending on the
• Malware detection severity and scope of the incident, it may warrant
or require an investigation. Security incidents occur
Security Monitoring
without warning and can go undetected long
Our world is filled with data. And with so much data
enough to pose a serious threat to your business.
collection, organizations find it difficult to scale and
Investigations can prove challenging and time
determine what needs to be monitored. This is why
consuming—typically by the time a security team
security monitoring can be difficult, expensive and
is aware of an incident, there is a good chance the
burdensome. Understanding how to implement proper
organization has already been negatively impacted.
security controls, keep them current, and create a
baseline to start the discovery and to determine With today’s threat landscape, it’s critical that
critical data can be tedious and time-consuming. organizations react quickly to identify and respond to
any type of threat, especially advanced threats that
But security monitoring is a critical component
continue to increase in complexity. It’s important that
of risk mitigation. It enables information security
teams quickly know how to identify a real incident in
professionals and others to see a continuous stream

A dashboard in Splunk that helps

organizations visualize their
security posture.

Using Splunk to Develop an Incident Response Plan 3

 WHITE PAPER

a sea of alerts and determine false positives to act at The teams are dealing with organizations that often
the speed of the business and deter the incident from have heterogeneous environments, lacking end-
becoming a breach. By implementing best practices to-end visibility or integrated tools to automate
organizations not only protect their customers and responses, making the organization challenging to
partners, but also can help with addressing critical protect. They also often work across silos in order to
customer problems. manage and orchestrate the response to malicious
activities, incidents and breaches.
For teams to effectively understand the incidents that
are occuring they need the visibility from across data Organizations need to leverage analytics and machine
silos, which gives them the ability to be proactive or learning, as well as known best practices, to free up
react quickly to determining the scope of the incident analysts to do what they do best--make remediation
and quickly remediate. If a team cannot effectively decisions. A high level security solution, such as the
identify the scope of the incident the incident can Splunk platform, enables organizations operationalize
quickly turn into a breach, impact the business and analytics-driven security practices in their SOC to
spread to other areas causing more damage and speed investigation and automate response.
making it harder to remediate.
Security automation with Splunk provides analysts
Forensic analysis is typically performed as part with all of the data they need to effectively
of a scheduled compliance, legal discovery, or diagnose and triage breaches freeing analysts to
law enforcement investigation. Forensic analysis focus on priority investigations, while automating
is focused on a full understanding and thorough common and less critical actions and tasks. This
remediation of a breach. The teams typically require provides a framework to speed investigations by
much longer lived logs, files, and interactions with gathering all relevant context into one place and
a much broader set of departments, including it also speeds up the response response process
operations, legal, HR, and compliance. A thorough because of automation.
forensic investigation allows the remediation of all
Fraud Detection
threats with the careful analysis of an entire attack
chain of events. Fraud is a growing problem for organizations from
healthcare providers to banks. Specifically, fraud,
Traditionally, organization use monitoring theft, and abuse detection and prevention is a big
tools, which just tell you when something isn’t data challenge as business continue to move online
working. It’s important to proactively manage and into the cloud.
operations and respond before an outage occurs
or service erodes. The ability to search across The problem has been accelerated with the
all security relevant data in both IT and security digitalization of our lives, and the fact that
domains increase the end-to-end visibility of the organizations are creating new online services at a
organization. The more Operational Intelligence the faster pace than ever. Fraud can have a significant
quicker and easier it is to investigate and resolve impact to organizations both financially and
incidents that occur an infrastructure. operationally. In addition to massive monetary losses,
fraud leaves businesses vulnerable to damaged
SOC Automation reputation and customer relation strain.
Security teams have an onslaught of alerts making it
Many Fraud related incidents begin as an IT and
difficult to determine signals from the noise. These
security incident, which is an event that disrupts the
teams are in charge of timely, important decisions to
day-to-day of IT or security operations. If you believe
prevent breaches that can impact the business. Often
an incident is fraud related best practices is to follow
they lack the skills, experience, collaboration tools
a incident response plan. You will want to investigate
and process needed to quickly investigate, remediate
to determine what happened, decide what to do
threats resulting in major risks to the business.

Using Splunk to Develop an Incident Response Plan 4

 WHITE PAPER

about it and if in fact the incident is fraud related. If • Flexibility to integrate and export data to other
it is a fraudulent incident organizations must have systems via scripting, alert actions, and dynamic
the ability to perform advanced data analytics in forms or drilldowns.
order to recognize and respond to patterns of fraud.
• The ability to pull historical reports for compliance
Simply stated, quicker fraud detection is essential to
requirements and to assist in financial crime
minimizing business risk. Fraudulent patterns (both
investigations.
internal and external) are often lurking in the vast,
unstructured machine data generated by a company’s Insider Threat
applications and IT systems. An insider threats comes from people within the
organization, such as employees, former employees,
Although every organization has their own unique
contractors or business associates, who have inside
fraud challenges to address, there are some common
information concerning the organization’s security
use cases across banking, financial, healthcare
practices, data and computer systems. Whether the
and other industries. One common use case is an
insider is acting maliciously or unaware of the actions
account takeover and dealing with either fraudulent
the impact to the business is one of the greatest
transactions, which is any single event from a
threats to a business.
monetary device - when the action of fraudulent
transaction or intent occurs, or fraudulent behaviors - Often by the time a security team is aware of an
actions that string together diverse data sets. insider threat, there is a good chance that the insider
threat has already cause damaged.
Beyond use case development, Splunk can also be
leveraged to provide additional features and tools to And the impact of insider threats on business can
assist with fraud detection, analytics, investigation, be catastrophic. The FBI estimated that the average
and response. Here are few bonus benefits that cost per insider threat incident is $412,000, and the
Splunk provides in the fight against fraud. average loss per industry is $15 million over ten years.
• Customized, form-based dashboards with drill-
An analytics-driven security solution, such as Splunk,
downs to provide analysts easy access to data that is
can help organizations deal with insider threats by
targeted to their investigative needs. This can allow
successfully implementing a incident response plan.
lower-tier analysts to perform searches over relevant
Splunk Enterprise Security (ES) and Splunk UBA
data without learning the Splunk search language.
provide ready to use dashboards to help detect
• Lookup tables and GUI-based editors to allow abnormal employee behaviors.
teams to easily manage and update their lookups
that are used to enrich data or to maintain Splunk software also provides examples of insider
blacklists and whitelists. threats, a multi-pass machine learning architecture
that can help with automated correlations of
• Summary dashboards to provide high-level
anomalous behavior into high fidelity threats. The
overviews, trend analysis statistics and workflow
Splunk platform also provides dynamic and recurring
based reports.
content to help with identifying latest threats without
• Ease and flexibility in the onboarding of a wide any operational downtime.
variety of data sources. Regardless of whether
the data is structured (database tables) or
unstructured proprietary data sources, Splunk can
be configured to ingest the data.

• Correlation of otherwise disjoined data sources.

Splunk provides the ability to join distinct data
sources together to provide insight into sequence-
based transactions.

Using Splunk to Develop an Incident Response Plan 5

 WHITE PAPER

Malware Detection The Federal Bureau of Investigation reported $209

Malware attacks are a growing problem for million was paid to ransomware criminals in Q1 2016.
organizations of every size with the numbers of The FBI also projected the takeover attacks to be a $1
attacks and the money spent to clean up the damage billion source of income for cybercriminals by the end
on the rise. The malware attack was once a consumer- of 2016. And this number doesn’t include the revenue
oriented threat, then a minor corporate issue, but now loss and the opportunity cost from the business
attacks regularly steal headlines. And because it is a disruption caused by an attack.
type of malware attack, ransomware can often sit on
Another study found that nearly 70 percent of
networks undetected for a long time.
executives hit by a ransomware attack in 2016 paid
In fact, malware accounts for 82 percent of security to unlock their data. And those numbers are likely
incidents, according to a SANS survey. even higher because they do not include smaller
organizations and consumers who are also paying a
Ransomware, a type of malware attack, typically ransom to get their data back.
target vulnerabilities on endpoints, preying on
organizations that may not be fully up to date in their
“security hygiene.” This translates to basic security
practices, such as patches and antivirus, which is
especially important in smaller IT and security teams
with a less formal process.

THE EVOLUTION OF RANSOMWARE

Ransomlock Cryptodefense Teslacrypt 73V3N

Urausy Crypowall CTB-Locker Locky
Cryptolocker Reveton Lockscreen Samsam
Lockdroid Tox Keranger
2013
Virlock Teslacrypt 2.0 Powerware
Torrentlocker Petya
2014
DMAlock Teslacrypt 3&4
Chimera Cerber
Jigsaw
2015
Rokku
Hydracrypt

2016

Using Splunk to Develop an Incident Response Plan 6

 WHITE PAPER

So how do organizations defend against ransomware? follow a process and plan to detect, investigate
A large part of getting better at ransomware prevention and remediate should the incident turn out to be
is to do the fundamentals better, which is where a breach. These more mature incident response
incident response come in. Organizations need to take teams achieve greater success in detection and
a step back and ask how well they are instrumented to containment by making use of proactive continuous
do three key security basics extremely well: monitoring and response strategy, rather than a
reactive intermittent response processes.
1. Overall security posture assessment

2. Efficiently investigate with the right amount of Achieving this level of security maturity is not
context to verify threats impossible, or difficult even. But it requires the
ability and the knowledge to know where to focus.
3. Respond appropriately, quickly and effectively
It requires the ability to investigate and analyze your
Organizations who can leverage an additional layer system to identify the signature of malware and
of visibility can learn to better gain control over the attack behaviors and other possible compromises
environment – this is of utmost importance – to be within the network.
able to adapt. The requirement is to, quite simply, get
Organizations also need to be sure they are able
better at detection, investigation, and response. That
to capture the full scope of an incident and have
is, rather than relying wholly on “these tools do this
knowledge of previous incidents. To reduce the
for you,” take the approach of “use the information
potential damage of an incident, it is important to
from these tools to do the fundamentals better.”
disable specific applications or services that are
Taking a risk-based approach allows organizations to most vulnerable or compromised, which can greatly
respond quickly and appropriately based on analytics- minimize the impact of an incident. Many organizations
driven decisions, which is a preferable alternative to are also seeing greater value and the need for more
learning about it via major news headlines. automation and integration with an analytics-driven
SIEM solution for better incident response.
Fighting Back
With so many breaches make headlines recently, With these steps, along with improved integration
many organizations are being pushed to develop of a next-gen SIEM, organizations will gain better
incident response capabilities to keep from visibility, which will empower their security teams
becoming international news. The organizations to focus on more quickly and efficiently detecting
that are most successful at dealing with incidents anomalous behaviors for improved incident response.

Are you interested in learning how machine data can improve your organization’s incident response and strengthen its
security posture? See how Splunk customers are leveraging an analytics-driven security platform for incident response.

Learn more: www.splunk.com/asksales www.splunk.com

© 2018 Splunk Inc. All rights reserved. Splunk, Splunk>, Listen to Your Data, The Engine for Machine Data, Splunk Cloud, Splunk Light
and SPL are trademarks and registered trademarks of Splunk Inc. in the United States and other countries. All other brand names,
product names, or trademarks belong to their respective owners. WP-Splunk-Incident-Response-101