Paloalto - Cortex Xsoar Threat Intel Management Guide

Cortex XSOAR Threat Intel Management
Guide
5.5
paloaltonetworks.com/documentation
Contact Information
Corporate Headquarters:
Palo Alto Networks
3000 Tannery Way
Santa Clara, CA 95054
www.paloaltonetworks.com/company/contact-support
About the Documentation

• For the most recent version of this guide or for access to related documentation, visit the Technical
Documentation portal www.paloaltonetworks.com/documentation.
• To search for a specific topic, go to our search page www.paloaltonetworks.com/documentation/
document-search.html.
• Have feedback or questions for us? Leave a comment on any page in the portal, or write to us at
documentation@paloaltonetworks.com.
Copyright
Palo Alto Networks, Inc.
www.paloaltonetworks.com
© 2020-2020 Palo Alto Networks, Inc. Palo Alto Networks is a registered trademark of Palo
Alto Networks. A list of our trademarks can be found at www.paloaltonetworks.com/company/
trademarks.html. All other marks mentioned herein may be trademarks of their respective companies.
Last Revised
March 26, 2020
2 CORTEX XSOAR THREAT INTEL MANAGEMENT GUIDE |

Table of Contents
Threat Intel Management Overview.............................................................. 5
Threat Intel Concepts.................................................................................................................................7
Fetch indicators...............................................................................................................................7
Exclusion list.....................................................................................................................................7
Indicator expiration........................................................................................................................ 7
Indicator smart merge................................................................................................................... 7
Indicator timeline............................................................................................................................ 7
Common indicator data model....................................................................................................8
Feed-based job................................................................................................................................ 8
Manage Indicators...............................................................................................9
Understand Indicators..............................................................................................................................11
Feed Integrations..........................................................................................................................11
Indicators Page..............................................................................................................................13
Indicator Reputation.................................................................................................................... 14
Indicator Types............................................................................................................................. 16
Indicator Fields..............................................................................................................................19
Exclusion List................................................................................................................................. 21
Create a Feed-Triggered Job.....................................................................................................21
Manage the Indicator Timeline.................................................................................................22
Auto Extract Indicators............................................................................................................................23
Auto Extract Modes.................................................................................................................... 23
How to Define Auto Extract..................................................................................................... 24
Configure What Auto Extract Executes................................................................................. 24
Disable Auto Extract for Scripts and Integrations............................................................... 24
Auto Extract Indicators from a Phishing Email..................................................................... 25
Threat Intel Feeds............................................................................................ 29

Feed Integrations...................................................................................................................................... 31
Configure the PAN-OS EDL Service Integration.............................................................................. 33
Set the Source Reliability of Enrichment Integrations.....................................................................35
Configure the Export Indicators Integration...................................................................................... 36
Export Indicators...............................................................................................39
Manually Export Indicators.....................................................................................................................41
Export Indicators Integrations............................................................................................................... 42
Configure the Export Indicators Integration..........................................................................42
Configure the PAN-OS EDL Service Integration..................................................................43
Export Indicators Playbooks...................................................................................................................45
Migrate Indicators to Elasticsearch.............................................................. 47

Indicator Migration Overview................................................................................................................49
Elasticsearch Security Guidelines..........................................................................................................50
How it works.................................................................................................................................50
Enable security features in Elasticsearch............................................................................... 50
Disable security features............................................................................................................50
TABLE OF CONTENTS iii

Elasticsearch Sizing Requirements........................................................................................................51
Maximum indicator capacity and disk usage comparison.................................................. 51
Maximum number of indicators in a single fetch.................................................................51
Single feed fetch comparison....................................................................................................51
Elasticsearch server..................................................................................................................... 52
Additional configurations........................................................................................................... 52
Migrate Indicators to Elasticsearch for a Single Server...................................................................53
Migrate Indicators to Elasticsearch for Multi-Tenant...................................................................... 55
Backup Indicators Stored in Elasticsearch.......................................................................................... 57
Restore Indicators Stored in Elasticsearch......................................................................................... 58
Troubleshoot Elasticsearch.....................................................................................................................59
Troubleshoot Elasticsearch Memory Issues...........................................................................59
Troubleshoot Elasticsearch Feed Ingestion Issues...............................................................59
iv TABLE OF CONTENTS
Threat Intel Management Overview
The Cortex XSOAR native threat intel management capabilities provide you with the ability
to unify the core components of threat intel, including threat intel aggregation, scoring, and
sharing. Cortex XSOAR automates threat intel management by ingesting and processing
indicator sources, such as feeds and lists, and exporting the enriched intelligence data to
the SIEMs, firewalls, and any other system that can benefit from the data. These capabilities
enable you to sort through millions of indicators daily and take automated steps to make those
indicators actionable in your security posture.
> Threat Intel Concepts
5
6 CORTEX XSOAR THREAT INTEL MANAGEMENT GUIDE | Threat Intel Management Overview
© 2020 Palo Alto Networks, Inc.
Threat Intel Concepts
These are the key concepts associated with threat intel management in Cortex XSOAR.
• Fetch indicators
• Export indicators
• Exclusion list
• Indicator expiration
• Indicator smart merge
• Indicator timeline
• Common indicator data model
• Feed-based job
Fetch indicators
Cortex XSOAR includes integrations that fetch indicators from either a vendor-specific source, such as
AutoFocus, or from a generic source, such as a CSV or JSON file.
Export indicators
You can export indicators as a hosted list, an EDL, or a TAXII collection. This enables your SIEM or firewall
to ingest or pull the indicator list to update policy rules. The supported list file types are JSON, CSV, and
TXT.
Exclusion list
Indicators added to the exclusion list are ignored by the system and are not considered indicators.
Indicator expiration
When ingesting and processing millions of indicators on a daily basis, it’s important to control whether or
not they are active or expired, and to define how and when indicators are expired.
The indicator field Expiration Status displays the indicator status, Active or Expired.
The indicator field Expiration displays the method by which and when that indicator is expired. Indicator
expiration is applied at the indicator type level. Indicators assigned to a specific indicator type inherit the
indicator type’s expiration method.
Indicator smart merge

The same indicator can originate from multiple sources and be enriched with multiple methods (integrations,
scripts, playbooks, and so on). Cortex XSOAR employs smart merge logic to make sure indicators are
accurately scored and aggregated.
Indicator timeline
The indicator timeline is the default section in the indicator summary layout. The timeline is in table format
and displays an indicator’s complete history, including the first seen and last seen timestamp, changes made
to indicator fields, and more.
CORTEX XSOAR THREAT INTEL MANAGEMENT GUIDE | Threat Intel Management Overview 7
Common indicator data model
When indicators are ingested, regardless of their source, they have a unified, common set of indicator fields,
including traffic light protocol (TLP), expiration, and tags.
Feed-based job
You can define a job to trigger a playbook when the specified feed or feeds finish a fetch operation that
included a modification to the list. The modification can be a new indicator, a modified indicator, or a
removed indicator.
8 CORTEX XSOAR THREAT INTEL MANAGEMENT GUIDE | Threat Intel Management Overview
Manage Indicators
> Understand Indicators
> Indicators Page
> Indicator Reputation
> Indicator Types
> Indicator Fields
> Exclusion List
> Create a Feed-Triggered Job
> Manage the Indicator Timeline
> Auto Extract Indicators
9
10 CORTEX XSOAR THREAT INTEL MANAGEMENT GUIDE | Manage Indicators
Understand Indicators
Indicators are artifacts associated with incidents, and are an essential part of the incident management and
remediation process.
They help to correlate incidents, create hunting operations, and enable you to easily analyze incidents and
reduce MTTR.
Cortex XSOAR includes an Indicator repository, which collects and correlates indicators across all incidents,
alerts, and feeds flowing into Cortex XSOAR.
• Indicators Page
• Indicator Reputation
• Indicator Types
• Indicator Fields
Detect and ingest indicators
There are several methods by which indicators are detected and ingested in Cortex XSOAR.
Method Description
Integration • Feed: integrations that fetch indicators from a feed, for example TAXII,
AutoFocus, Office 365, and so on.
• Mail: integrations that consume emails with STIX or CSV files and add
the indicators to the indicator repository.
Incident • Manual: user marks a piece of data as an indicator.

• Auto-extract: indicators are extracted from every incident that flows
into Cortex XSOAR, for example from a SIEM integration.
Regex query A query that identifies indicators in the War Room.
STIX file Manually upload a STIX file on the Indicators page.
Script • FetchIndicatorsFromFile: accepts a file from which it extracts indicators

and processes them in Cortex XSOAR.
• CreateIndicatorsFromSTIX: extracts indicators from a STIX file and
processes them in Cortex XSOAR.
Feed Integrations
Cortex XSOAR has several out-of-the-box threat intelligence feed integrations.
• AutoFocus
• AWS
• Microsoft Azure
• Bambenek Consulting
• Blocklist_de
• Microsoft Office 365
• Palo Alto Networks PAN-OS EDL Service
• Proofpoint
• Recorded Future RiskList
CORTEX XSOAR THREAT INTEL MANAGEMENT GUIDE | Manage Indicators 11

• Spamhaus
• TAXII
Common feed integration parameters
This is a non-exhaustive list of the most common feed integration parameters. Each feed integration
might have parameters unique to that integration. Make sure to read the documentation for specific feed
integrations.
Parameter Description
Name A meaningful name for the integration instance. For example, if you
have separate instances to fetch indicator types, you can include the
name of the indicator type that the instance fetches.
Fetch indicators Select this option for the integration instance to fetch indicators.
Some integrations can fetch indicators or incidents. Make sure you
select the relevant option for what you need to fetch in the instance.
Sub-Feeds Some feeds might have several lists or files that provide indicators.
The sub-feeds parameter enables you to select the specific list or file
from which to fetch indicators. For example, Bambenek Consulting
provides different lists for IPs and domains. Each of the Bambenek lists
are available as sub-feeds.
URL The URL of the feed.
Fetch Interval How often the integration instance should fetch indicators from the
feed.
Indicator Reputation The indicator reputation to apply to all indicators fetched from this
integration instance.
Source Reliability The reliability of the source providing the threat intelligence data.
Indicator Expiration Method The method by which to expire indicators from this integration instance.
The default expiration method is the interval configured for the indicator
type to which this indicator belongs.
• Indicator Type: the expiration method defined for the indicator type
to which this indicator belongs (interval or never).
• Time Interval: expires indicators from this instance after the specified
time interval, in days or hours.
• Never Expire: indicators from this instance never expire.
• When removed from the feed: when the indicators are removed from
the feed they are expired in the system.
Bypass exclusion list When selected, the exclusion list is ignored for indicators from this feed.
This means that if an indicator from this feed is on the exclusion list, the
indicator might still be added to the system.
Trust any certificate When selected, certificates are not checked.

Use system proxy settings Runs the integration instance using the proxy server (HTTP or HTTPS)
that you defined in the server configuration.
Do not use by default Excludes this integration instance when running a generic command that
uses all available integrations.
Indicators Page
The Indicators page displays indicator dashboards, a table or summary view of all indicators, and enables
you to perform several indicator actions.
Indicator actions
You can perform the following actions on the Indicators page.
Action Description
Create incident Creates an incident from the selected indicators and populates relevant incident
fields with indicator data.
Edit You can edit a single indicator or select multiple indicators to perform a bulk
edit.
Delete and Exclude You can select to delete and exclude one on or more indicators from all
indicator types or from a subset of indicator types.
If you select the Do not add to exclusion list check box, the selected indicators
are only deleted.
Export Exports the selected indicators to a CSV file.
Export (STIX) Exports the selected indicators to a STIX file.
Upload a STIX file Uploads a STIX file and adds the indicators from the file to the system.
Create a new Manually creates a new indicator in the system.

indicator
Indicator query
You can search for indicators using any of the available search fields, but there are several fields specific to
indicators that you can use to search for indicators.
Field Description
sourceBrands Indicator feed or enrichment integrations.
sourceInstances A specific instance of an indicator feed or enrichment integration.
expirationSource The source of the indicator having expired status.

Field Description
isShared Whether the indicator is shared to tenant accounts (multi-tenant only).
tags Tags applied to indicators.
comments Search for keywords within indicators’ comments.
Indicator Reputation
An indicator’s reputation is assigned according to the reputation returned by the source with the highest
reliability. In cases where multiple sources with the same reliability score return a different reputation for
the indicator, the worst reputation is taken.
Indicator reputations
Indicators are assigned a reputation on a scale of 0 to 3.
Score Reputation Color
0 None No color
1 Good Green
2 Suspicious Orange
3 Bad Red
Example 1
In this example, two 3rd-party integrations, VirusTotal and AlienVault, return a different reputation for the
same indicator. VirusTotal returns a reputation of Good, and AlienVault returns a reputation of Bad. The
indicator’s reputation will be Bad.
Example 2
In this example, two sources with different reliability scores return a different reputation for the same
indicator. The first source is a TAXII feed with a reliability score of C - Fairly reliable, and the second source
is a CSV feed with a reliability score of B - Usually reliable. The TAXII feed returns a reputation of Bad
and the CSV feed returns a reputation of Good. The indicator’s reputation will be Good because the CSV
reliability score is higher than that of the TAXII feed.
Source reliability
The reliability of an intelligence-data source influences the reputation of an indicator and the values for
indicator fields when merging indicators.
Indicator fields are merged according to the source reliability hierarchy. This means that when there are two
different values for a single indicator field, the field will be populated with the value provided by the source
with the highest reliability score.
In rare cases, two sources with the same reliability score might return different values for the same indicator
field. In these cases, the field will be populated with the most recently provided source.
For the field types Tags and Multi-select, all values are appended, nothing is overridden.

Source Reliability Score Notes
User (manual) A+++ A user manually updates the

reputation of an indicator.
Reputation script A++ A script with the reputation tag,

which calculates the reputation
of an indicator. For example, the
DataDomainReputation script
evaluates the reputation of a
URL or domain.
3rd-party enrichment A+ An integration or service that

evaluates the reputation of
an indicator. For example, the
urlscan.io integration evaluates
the reputation of a URL.
Feed reliability A: Completely reliable The feed reliability is applied at

the integration instance level.
B: Usually reliable
C: Fairly reliable
D: Not usually reliable
E: Unreliable
F: Reliability cannot be judged
Indicator expiration
Indicators can have the status Active or Expired, which is determined by the expirationStatus field. When
indicators expire, they still exist in Cortex XSOAR, meaning they are still displayed and you can still search
for them. A job runs every hour to check for newly expired indicators.
By default, indicators are expired according to either the expiration interval configured for the indicator
type to which the indicator belongs, or to never expire.
This is the hierarchy by which indicators are expired.
Method Description
Manual A user manually expires an indicator. This method overrides all other
methods.
Feed integration The expiration method configured for an integration instance, which
overrides the method defined for the indicator type.
Indicator type The expiration method defined for the indicator type to which this indicator
belongs (interval or never). This is the default expiration method for an
indicator.

Customize the Dbot Reputation Score Logic
Cortex XSOAR calculates the reputation score of an entity (IP, URL or hash file) by considering the score
received by third-party integrations used (such as VirusTotal and X-Force). The Dbot score is determined by
the data reputation scripts. Amending the following scripts changes the way Dbot handles the reputation.
• DataIPReputation
• DataHashReputation
• DataURLReputation
You can customize the Dbot reputation score according to your own logic. For information about how
Cortex XSOAR handles reputation scores, see Indicator Reputation.
STEP 1 | Go to the Automation page and locate the script you want to edit.
STEP 2 | Click Copy Automation and modify an existing reputation script, such as DataURLReputation.
In the following example, we redefine the values for each reputation:
Ensure that the Reputation tag is selected.
STEP 3 | Click Save.
STEP 4 | To add the script to the indicator, go to Settings > Indicator Types.
STEP 5 | Select the indicator type that you want to add the script and click Edit.
STEP 6 | In the Reputation Script field, select the script you modified in step 2.
Indicator Types
The indicators are categorized by indicator type, which determines the indicator layout (fields) that are
displayed and which scripts are run on indicators of that type.
There are several system-level indicator types.
• IP Address
• Registry Path Reputation
• File
• Email
• Username
• Hostname
• Domain
• File Enhancement Scripts

• CVE CVSS Score
Create an Indicator Type

When you create a custom indicator type, you configure numerous fields and settings that impact how
indicators of that type are enriched, expired, how the reputation is calculated, among others.
Before you create a custom indicator type, you should familiarize yourself with the indicator type profile.
STEP 1 | Go to Settings > Advanced > Indicator Type.
STEP 2 | Click the Add Indicator Type button.
STEP 3 | Configure the indicator type as needed.
Indicator Type Profile

In addition to configuring the standard indicator type fields, you can map custom indicator fields to context
data.
There are a number of configuration options and fields that you must complete when creating a new
indicator type.
Table 1: Settings
Field Description
Name A meaningful name for the indicator type.
Regex The regular expression (regex) by which to identify indicators for this
indicator type.
Formatting Script The script to run on and modify how the indicator displays in Cortex
XSOAR, such as in the War Room, reports, and so on. For example,
the UnescapeURLs script extracts URLs that are redirected by security
tools or unescapes URLs that are escaped for safety (e.g., hxxps://
www[.]CortexXSOAR[.]com.
Enhancement Scripts A script to run on an identified indicator. For example, an enrichment

script, a script that runs a search in a SIEM for the indicator, and so on.
After indicators are identified, you can go to the indicator quick view,
click the Actions button and run an enhancement script directly on an
indicator. In order for these scripts to be available in the drop-down
menu, they need the enhancement tag.
Reputation Command The command to run to calculate the reputation of indicators of

this type. The result (reputation) is only associated with the specific
indicator on which it’s run (not the indicator type).
Excluded Integrations Integrations to exclude when calculating the reputation, evaluating,

and enriching indicators of this indicator type.
Reputation Script User-created scripts that either override the Cortex XSOAR command
algorithm or run on top of the data returned from the command. In
order for these scripts to be available in the drop-down menu, they

Field Description
require the reputation tag. The output of this script is a reputation
score, which is used as the basis for the indicator reputation.
Indicator Expiration Method The method by which to expire indicators of this type. The expiration
method that you select is the default expiration method for indicators
of this indicator type.
The expiration can also be assigned when configuring a feed
integration instance, which overrides the default method.
• Never Expire: indicators of this type never expire.
• Time Interval: indicators of this type expire after the specified
number of days or hours.
Context path for reputation When an indicator is auto-extracted, the entry data from the
value (Advanced) command is mapped to the incident context. This path defines the
context key that the indicator reputation is mapped to.
Context value of reputation The value of this field defines the actual data that is mapped to the
(Advanced)) context path.
Cache expiration in minutes The amount of time (in minutes) after which the cache for indicators
(Advanced) of this type expire. The default is 4,320 minutes (three days).
Formatting scripts for out-of-the-box indicator types are now system level. This means that
the formatting scripts for these indicator types are not configurable. To create a formatting
script for an out-of-the-box indicator type, you need to disable the existing indicator type and
create a new (custom) indicator type. If you configured a formatting script before this change
and updated your content, this configuration will revert to content settings (empty).
File Indicators
Cortex XSOAR uses a single File indicator for file objects. As a result, files appear with their SHA256 hash
and all other hashes associated with the file, (MD5, SHA1, and SSDeep) are listed as properties of the same
indicator. In addition, when ingesting an incident through an integration, all file information is presented as
one object.
For example, when looking at an incident, there is a file indicator with a Bad Reputation value:
When clicking at the indicator, you can see additional information for that indicator, including all of the
other known hashes associated with this file:

If the file appears in a different incident with a different name, and has any of the same hash values, it
automatically associates with the original indicator.
The new File indicator only affects new indicators ingested to the Cortex XSOAR platform.
Indicators that were already in Cortex XSOAR continue to appear as their respective hash-
related indicators.
If you want to have each file hash appear as its own indicator, do the following:
1. Go to Settings > Advanced > Indicator Types.
2. Select the File indicator and click Disable.
3. Select the following required hashes:
• File SHA-256
• File SHA-1
• File MD5
• SSDeep
4. Click Enable.
Indicator Fields
After you create a custom indicator field, you can add it to the indicator layout for the indicator types to
which you assicated the field.
• Create a Custom Indicator Field
• Map Custom Indicator Fields
Create a Custom Indicator Field

Indicator Fields are used to add specific indicator information to incidents.When you create an indicator
field, you can associate the field to a specific incident type or to all incident types.
STEP 1 | Go to Settings > Advanced > Fields.
STEP 2 | From the drop-down menu, select Indicator.

STEP 3 | Click New Field.
STEP 4 | Configure the basic settings.
Field Description
Field Type Determines the acceptable values for the field.
Case Sensitive If selected, the field is case sensitive, which affects searching for
the field in Cortex XSOAR.
Mandatory If selected, this field is mandatory when used in a form.
Field Name A meaningful display name for the field. After you type a name,
you will see below the field that the Machine name is automatically
populated. The field’s machine name is applicable for searching and
the CLI.
Tooltip An optional tooltip for the field.
Placeholder Optional text to display in the field when it is empty.
STEP 5 | Configure the attributes.
Field Description
Add to indicator types By default, the Associate to all option is selected, which means this
field will be available to use in all incident types.
Clear the check box to associate this field to a subset of indicator
types.
Make data available for search The values for this field can be returned in searches.
Map Custom Indicator Fields

You map custom fields for an indicator type. Before you can map custom indicator fields, you need to
Create a Custom Indicator Field and associate the field with the necessary indicator types.
STEP 1 | Go to Settings > Advanced > Indicator Types.
STEP 2 | Select the check box for the indicator for which to map the custom fields.
STEP 3 | Click the Edit button.
STEP 4 | Click the Custom Fields tab.

The custom fields associated with this incident type are listed in the table. If you do not see a custom
field in the list, verify that you associated the custom field to this incident type.
STEP 5 | (Optional) In the Indicator Sample panel, enter an indicator relevant to the indicator type to load
sample data.

You can map a field to a context key from the indicator’s returned context data.
STEP 6 | Click Choose data path to map the custom field to a data path.
1. (Optional) Click the curly brackets to map the field to a context path.
2. (Optional) From the Indicator Sample panel, select a context key to map to the field.
Exclusion List
Indicators added to the exclusion list are ignored by the system and are not considered indicators. You can
still manually enrich IP addresses and URLs that are on the exclusion list, but the results are not posted to
the War Room.
There are several methods by which to add indicators to the exclusion list.
Delete and exclude

You can select one or more indicator from the Indicators table and click the Delete and Exclude button.
The indicators are deleted from the Indicators table and added to the exclusion list. You can associate these
indicators with one or more indicator types.
Manually add indicators to the exclusion list

From the Exclusion List page, you can manually add a single indicator or define indicators using a regular
expression (regex) or CIDR.
Regex
A regular expression enables you to identify a sequence of characters in an unknown string. The following
example would identify www.demisto.com: [A-Za-z0-9!@#$%\.&]*demisto[A-Za-z0-9!@#$%
\.&]*.
CIDR
Classless inter-domain routing (CIDR) enables you to define a range of IP addresses. For example,
192.168.100.14/24 represents the IPv4 address 192.168.100.14 and its associated routing prefix
192.168.100.0, or equivalently, its subnet mask 255.255.255.0, which has 24 leading 1-bits. The IPv4 block
192.168.100.0/22 represents the 1024 IPv4 addresses from 192.168.100.0 to 192.168.103.255.
Create a Feed-Triggered Job

You can define a job to trigger a playbook when the specified feed or feeds finish a fetch operation that
included a modification to the feed. The modification can be a new indicator, a modified indicator, or a
removed indicator.
For example, let’s say you want to update your firewall every time a URL is added to, modified, or
removed from the Office 365 feed. You can configure a job that triggers that playbook to run whenever a
modification is made to that feed.
You can customize the new job form by editing the Indicator Feed incident type.
If you want to trigger a job after a feed completes a fetch operation, and the feed does
not change frequently, you can select the Reset last seen option in the feed integration
instance. The next time the feed fetches indicators, it will process them as new indicators in
the system.
STEP 1 | Go to the Jobs section.

STEP 2 | Click the New Job button.
STEP 3 | Configure the job parameters.
Job type Select the Feed triggered option.
Trigger Define the trigger for the playbook.

• All feeds: the playbook will run when a modification is made to
any feed.
• Specific feeds: select the feed instances that will trigger the
playbook to run when a modification is made to the specified
feed instances.
Name A meaningful name for the job.
Playbook The playbook that will run when the conditions for the job are met.
Tags Add tags to apply to the job, which you can use as a search
parameter in the system.
Manage the Indicator Timeline

A large number of indicators can affect performance of the indicator timeline. There are several advanced
server configurations you can implement to manage the indicator timeline performance.
STEP 1 | Go to Settings > About > Troubleshooting.
STEP 2 | In the Server Configuration section, click Add Server Configuration.
Key Value Description
indicator.timeline.enabled true or Enables the indicator timeline in all flows. The

false default is true.
true or Enables the indicator timeline for a specific

indicator.timeline.enabled.type.<indicatorType>
false indicator type. This configuration overrides
the indicator.timeline.enabled
configuration.
For example:
indicator.timeline.enabled.type.ip
true or Enables the indicator timeline in the auto-extract

indicator.timeline.auto.extract.enabled
false flow. The default is true.
indicator.timeline.max.sizeNumber The maximum number of indicator comments

(timeline and regular). The default is 100.
true or
indicator.timeline.worker.enabled Enables you to add timeline comments through
false content integrations.

Auto Extract Indicators
The Auto Extract feature extracts indicators and enriches their reputations using commands and scripts
defined for the indicator type. You can automatically extract indicators in the following scenarios:
• When fetching incidents
• In a playbook task
• Using the command line
By default, Auto Extract is enabled to help you get up and running as you set up your environment. As your
system matures and you start ingesting more events and have more integrations configured, using Auto
Extract can adversely affect system performance.
As a result, Cortex XSOAR recommends that you turn off Auto Extract using the server configurations for
the different Auto Extract options and only turn it on for those specific scenarios where it is necessary.
Auto Extract Modes

Auto Extract supports the following modes:
• None - Indicators are not automatically extracted. Use this option when you do not want to further
evaluate the indicators.
• Inline - Indicators are extracted and enriched within the context that Auto Extract runs, and the findings
are added to the Context Data. For example, if you define Auto Extract for the Phishing incident type
as inline, all of the indicators for incident classified as Phishing will be extracted and enriched before
anything else happens. The playbook you defined to run by default will not run until the indicators have
been fully processed. Use this option when you need to have the most robust information available per
indicator. Unless otherwise configured in a system configuration, this is the default mode in which Auto
Extract executes.
This configuration will slow down your system performance.
• Out of band - Indicators are enriched in parallel (or asynchronously) to other actions. The enriched data
is available within the incident, however, it is not available for immediate use in task inputs or outputs
since the information is not available in real time.
Global Server Configurations for Auto Extract

You can control the default behavior for auto extract using the following server configurations:
Component Key
Incident ingestion reputation.calc.algorithm
Tasks reputation.calc.algorithm.tasks
Manual reputation.calc.algorithm.manual
Each configuration can accept one of the following values:

• 1 = None
• 2 = Inline. This is the default behavior
• 3 = Out of Band

How to Define Auto Extract
Incident Types
To define auto extract for a default incident type, perform the following steps. The default auto extract
value for incident types is inline.
1. Navigate to Settings > Advanced > Incident Types.
2. Select the incident you want to edit by clicking the checkbox and then clicking the Edit button.
3. In the auto extract drop down menu, select the mode you want to use.
4. Click Save.
Playbook Tasks
To define auto extract for a playbook task, perform the following steps. The default auto extract value for
playbook tasks is none.
1. In the playbook click, a task to open the Edit Task window.
2. Click the Advanced tab.
3. In the auto extract drop down menu, select the mode you want to use.
4. Click OK.
Cortex XSOAR CLI
To define auto extract using the Cortex XSOAR CLI, use the command auto-extract= with the
script and the mode for which you are setting up auto-extract. For example, !EmailReputation
email=email@email.com auto-extract=inline, filling in the script and mode you want to define.
Configure What Auto Extract Executes

When Auto Extract is used, it extracts all indicators that match the regex defined in an indicator type, and
enriches those indicators using its commands. For example, out-of-the-box, the URL indicator is enriched
using the !url command. You can decide to further enrich IP indicators by using a script that calls multiple
integrations, such as urlscan.io and URLhaus.
STEP 1 | Navigate to Settings > Advanced > Indicator Types.
STEP 2 | Select the indicator type for which you want to configure the command or script and click Edit.
For out of the box indicators, the Name and Regex fields are disabled.
STEP 3 | Under Reputation command, enter the command to execute when auto extracting indicators
of this type.
STEP 4 | Under Exclude these integrations for the reputation command, select which integrations
should not be used when executing the reputation command.
STEP 5 | Under Reputation Script, select the script to run when enriching indicators of this indicator
type. The scripts override the reputation command.
Disable Auto Extract for Scripts and Integrations

You can disable auto-extract for a specific automation or integration.
STEP 1 | Disable for an Automation

To disable Auto Extract for an automation, add the "IgnoreAutoExtract": True value to the entry return.
entry = { 'Type': entryTypes['note'],

'Contents': { 'Echo' : demisto.args()['echo'] },
'ContentsFormat': formats['json'],
'ReadableContentsFormat': formats['markdown'],
'HumanReadable': hr, 'IgnoreAutoExtract' : True
}
STEP 2 | Disable for Integrations

To disable Auto Extract for a specific integration, add the "IgnoreAutoExtract": True value to the
integration configuration.
Auto Extract Indicators from a Phishing Email

The following scenario shows how Auto Extract is used in the Process Email - Generic playbook to
automatically extract and enrich a very specific group of indicators.
STEP 1 | Navigate to the Playbookspage and search for the Process Email - Generic playbook.
This playbook parses the headers in the original email used in a phishing attack. It is important to parse
the original email used in the Phishing attack and not the email that was forwarded to make sure that
you are only extracting and enriching the email headers from the malicious email and not the one your
organization uses to report phishing attacks.
STEP 2 | Open the Add original email attachments to context task.

Under the Outputs tab you can see all of the different data that the task extracts.

STEP 3 | Navigate to the Advanced tab.
Under Auto extract indicators, ensure that the Inlineoption is selected. This indicates that all of the
outputs will be processed before the playbook moves ahead to the next task.

STEP 4 | Open the Set incident with the Email object data task. This task receives the data from the
Add original email attachments to context task and sets the various data points to context.
Under the Advancedtab, ensure that Auto extract indicators is set to None because the indicators have
already been enriched and there is no need to do it again.

In the above example, had we set the reputation.calc.algorithm.tasks server configuration to 1, we would
not have had to go into the Advanced tab of the Set incident with the Email object data task and manually
tell the task not to extract the indicators. It would use the system default.

Threat Intel Feeds
> Feed Integrations
> Configure the PAN-OS EDL Service Integration
> Configure the Export Indicators Integration
> Set the Source Reliability of Enrichment Integrations
29
30 CORTEX XSOAR THREAT INTEL MANAGEMENT GUIDE | Threat Intel Feeds
Feed Integrations
Cortex XSOAR has several out-of-the-box threat intelligence feed integrations.
• AutoFocus
• AWS
• Microsoft Azure
• Bambenek Consulting
• Blocklist_de
• Microsoft Office 365
• Palo Alto Networks PAN-OS EDL Service
• Proofpoint
• Recorded Future RiskList
• Spamhaus
• TAXII
Common feed integration parameters
This is a non-exhaustive list of the most common feed integration parameters. Each feed integration
might have parameters unique to that integration. Make sure to read the documentation for specific feed
integrations.
Name A meaningful name for the integration instance. For example, if you
have separate instances to fetch indicator types, you can include the
name of the indicator type that the instance fetches.
Fetch indicators Select this option for the integration instance to fetch indicators.
Some integrations can fetch indicators or incidents. Make sure you
select the relevant option for what you need to fetch in the instance.
Sub-Feeds Some feeds might have several lists or files that provide indicators.
The sub-feeds parameter enables you to select the specific list or file
from which to fetch indicators. For example, Bambenek Consulting
provides different lists for IPs and domains. Each of the Bambenek lists
are available as sub-feeds.
URL The URL of the feed.
Fetch Interval How often the integration instance should fetch indicators from the
feed.
Indicator Reputation The indicator reputation to apply to all indicators fetched from this
Source Reliability The reliability of the source providing the threat intelligence data.
Indicator Expiration Method The method by which to expire indicators from this integration instance.
The default expiration method is the interval configured for the indicator
type to which this indicator belongs.
CORTEX XSOAR THREAT INTEL MANAGEMENT GUIDE | Threat Intel Feeds 31

• Indicator Type: the expiration method defined for the indicator type
to which this indicator belongs (interval or never).
• Time Interval: expires indicators from this instance after the specified
time interval, in days or hours.
• Never Expire: indicators from this instance never expire.
• When removed from the feed: when the indicators are removed from
the feed they are expired in the system.
Bypass exclusion list When selected, the exclusion list is ignored for indicators from this feed.
This means that if an indicator from this feed is on the exclusion list, the
indicator might still be added to the system.
Trust any certificate When selected, certificates are not checked.
Use system proxy settings Runs the integration instance using the proxy server (HTTP or HTTPS)
that you defined in the server configuration.
Do not use by default Excludes this integration instance when running a generic command that
uses all available integrations.

Configure the PAN-OS EDL Service Integration
You should create a separate instance for each indicator type. Each instance should utilize a unique port.
STEP 1 | Go to Settings > Integrations > Servers & Services.
STEP 2 | Search for Palo Alto Networks PAN-OS EDL Service.
STEP 3 | Configure the integration instance for a specific indicator type.
Name A meaningful name for the instance. For example, name the
instance name according to the indicator type this instance fetches:
EDL_instance_IPs.
Indicator Query The query that defines which indicators to fetch for this instance.
EDL Size The maximum number of items in the EDL.
Update EDL On-Demand Only If you select this check box, the EDL is not automatically updated
(refreshed).
Refresh Rate How often the EDL is updated. You can specify the rate in natural
language, for example, 5 minutes, 1 hour, and so on.
Listen Port The port on which to run the EDL service from within Cortex
XSOAR.
Certificate (Required for HTTPS) Full text of the certificate including the header
and footer.
Private Key (Required for HTTPS) Full text of the private key including the
header and footer.
STEP 4 | Click Done.
You can access the EDL using your using Cortex XSOAR address and the specified port (HTTP) or the
integration instance name (HTTPS).
HTTP
In a web browser, go to http://<Cortex XSOAR_address>:listen_port, where Cortex
XSOAR_address is the URL of your Cortex XSOAR instance and listen_port is the port you specified in the
HTTPS
To access the EDL using the integration instance name, you have to add a server configuration.
• Go to Settings > About > Troubleshooting.
• In the Server Configuration section, verify that the instance.execute.external key is set to true.
If this key does not exist, add it.

• Go to https://<Cortex XSOAR_address>/instance/execute/instance_name, where Cortex
XSOAR_address is the URL of your Cortex XSOAR instance and instance_name is the name of the

Set the Source Reliability of Enrichment
Integrations
By default, the source reliability of enrichment integrations is A+. You set the source reliability of
enrichment integrations by adding a server configuration that specifies the integration and the new source
reliability score.
STEP 1 | Remove the current source reliability score from the enrichment integration.
1. Go to Settings > Integrations > Servers & Services.
2. Search for the integration instance that you want to set the source reliability score for and click
access the instance parameters..
3. In the Source Reliability field click the x next to the source reliability score.
4. Click Done.
STEP 2 | Go to Settings > About > Troubleshooting.
STEP 3 | In the Server Configuration section click Add Server Configuration.
STEP 4 | In the Key field enter integrations.enrichment.reliability.<integration_name>,

where integration_name is the name of the enrichment integration for which to set the
reliability score.
STEP 5 | In the Value field enter the source reliability score for the enrichment integration.
• A - Completely reliable
• B - Usually reliable
• C - Fairly reliable
• D - Not usually reliable
• E - Unreliable
• F - Reliability cannot be judged

Configure the Export Indicators Integration
You should create a separate instance for each indicator type. You should configure a unique listening port
for each integration instance.
Cortex XSOAR recommends that you use HTTPS when accessing the indicator service through the URL and
port. To do so, you must provide a certificate and private key, in the respective fields.
In addition, whether you are accessing the integration instance through the listening port or through the
instance name, Cortex XSOAR recommends that you provide a username and password by which the
service is accessed.
STEP 2 | Search for Export Indicators Service.
ExportIndicators_instance_IPs.
Outbound Format The file format for the indicator list. Supports text, JSON, JSON-
SEQ, and CSV.
List Size The maximum number of items in the list.
Update On-Demand Only If you select this check box, the list is not automatically updated
(refreshed).
Refresh Rate How often the list is updated. You can specify the rate in natural
Listen Port The port on which the Export Indicator integration from within
Cortex XSOAR. If you have multiple Export Indicators Service
integration instances, make sure to use different listening ports to
separate the outbound feeds.
and footer.
header and footer.
HTTP Server If you select this check box the certificate and private key are
ignored, and the integration runs in HTTP (Not recommended).
Incident Type Not applicable for this integration.

STEP 4 | (Optional) Ensure that you have configured connection credentials for accessing the integration.
You can access the list using your Cortex XSOAR address and the specified port or the integration instance
name (HTTPS).
Listening Port
Instance
To access the list using the integration instance name, you have to add a server configuration.

Export Indicators
> Manually Export Indicators
> Export Indicators Integrations
> Export Indicators Playbooks
39
40 CORTEX XSOAR THREAT INTEL MANAGEMENT GUIDE | Export Indicators
Manually Export Indicators
You can select one or more indicators from the Indicators table and export them as a CSV file or STIX file.
The file can then be sent to or pulled by a SIEM or firewall, or as the input for a playbook that processes
indicators.
STEP 1 | Go to the Indicators page.
STEP 2 | Select the check box for one or more indicators that you want to export to a file.
STEP 3 | Export the selected indicators.

• (Optional) Click the Export button to export the indicators to a CSV file.
• (Optional) Click the Export (STIX) button to export the indicators to a STIX file.
CORTEX XSOAR THREAT INTEL MANAGEMENT GUIDE | Export Indicators 41

Export Indicators Integrations
You specify an indicator query for the outbound-feed integration to determine which indicators are
exported. You can send the resulting list or file to a SIEM or firewall to update policy rules.
Export Indicators Service Integration
The Export Indicator Service integration fetches indicators based on the query that you define and exports
them as a CSV file to a URL.
PAN-OS EDL Service integration
The PAN-OS EDL Service integration enables you to define an indicator query for the indicators you want
to send from Cortex XSOAR to update an external dynamic list. An external dynamic list is a text file that
is hosted on an external web server so that the firewall can import objects—IP addresses, URLs, domains—
included in the list and enforce policy.
Configure the Export Indicators Integration

You should create a separate instance for each indicator type. You should configure a unique listening port
for each integration instance.
Cortex XSOAR recommends that you use HTTPS when accessing the indicator service through the URL and
port. To do so, you must provide a certificate and private key, in the respective fields.
In addition, whether you are accessing the integration instance through the listening port or through the
instance name, Cortex XSOAR recommends that you provide a username and password by which the
service is accessed.
STEP 2 | Search for Export Indicators Service.
ExportIndicators_instance_IPs.
Outbound Format The file format for the indicator list. Supports text, JSON, JSON-
SEQ, and CSV.
List Size The maximum number of items in the list.
Update On-Demand Only If you select this check box, the list is not automatically updated
(refreshed).
Refresh Rate How often the list is updated. You can specify the rate in natural

Listen Port The port on which the Export Indicator integration from within
Cortex XSOAR. If you have multiple Export Indicators Service
integration instances, make sure to use different listening ports to
separate the outbound feeds.
and footer.
header and footer.
HTTP Server If you select this check box the certificate and private key are
ignored, and the integration runs in HTTP (Not recommended).
Incident Type Not applicable for this integration.
STEP 4 | (Optional) Ensure that you have configured connection credentials for accessing the integration.
You can access the list using your Cortex XSOAR address and the specified port or the integration instance
name (HTTPS).
Listening Port
Instance
To access the list using the integration instance name, you have to add a server configuration.
Configure the PAN-OS EDL Service Integration

You should create a separate instance for each indicator type. Each instance should utilize a unique port.
STEP 2 | Search for Palo Alto Networks PAN-OS EDL Service.

EDL_instance_IPs.
EDL Size The maximum number of items in the EDL.
Update EDL On-Demand Only If you select this check box, the EDL is not automatically updated
(refreshed).
Refresh Rate How often the EDL is updated. You can specify the rate in natural
Listen Port The port on which to run the EDL service from within Cortex
XSOAR.
and footer.
header and footer.
You can access the EDL using your using Cortex XSOAR address and the specified port (HTTP) or the
integration instance name (HTTPS).
HTTP
HTTPS
To access the EDL using the integration instance name, you have to add a server configuration.

Export Indicators Playbooks
When you run a playbook or playbook task in Quiet Mode the task or playbook information
is not written to the War Room, and inputs and outputs are not displayed in the playbook.
However, errors and warnings are still written to the War Room. If you define a playbook task
input that pulls from indicators, the entire playbook runs in Quiet Mode.
You should not run a query on a field that you might change in the playbook flow. For
example, you shouldn’t have playbook with query Score:Bad and then change the indicator
score as a part of the playbook.
Generic playbooks
Each generic playbook is dedicated to processing a single indicator type.
• Process Domain Indicators
• Process Files Indicators
• Process IP Indicators
• Process URL Indicators
QRadar playbooks
There is a separate QRadar playbook for each indicator type, which adds indicators of that type to QRadar,
and a QRadar playbook that adds all indicators to QRadar.
• QRadar Add Hash Indicators
• QRadar Add IP Indicators
• QRadar Add Domain Indicators
• QRadar Add URL Indicators
• QRadar Add All Indicator Types
ArcSight playbooks
There is a separate ArcSight playbook for each indicator type, which adds indicators of that type to
ArcSight, and an ArcSight playbook that adds all indicators to ArcSight.
• ArcSight Add Hash Indicators
• ArcSight Add IP Indicators
• ArcSight Add Domain Indicators
• ArcSight Add URL Indicators
• ArcSight Add All Indicator Types

Migrate Indicators to Elasticsearch
> Indicator Migration Overview
> Elasticsearch Sizing Requirements
> Migrate Indicators to Elasticsearch for a Single Server
> Migrate Indicators to Elasticsearch for Multi-Tenant
> Backup Indicators Stored in Elasticsearch
> Restore Indicators Stored in Elasticsearch
> Troubleshoot Elasticsearch
47
48 CORTEX XSOAR THREAT INTEL MANAGEMENT GUIDE | Migrate Indicators to Elasticsearch
Indicator Migration Overview
When you migrate all indicators that exist in Cortex XSOAR to Elasticsearch, they are moved to a
designated index used by a specific Elasticsearch instance.
You perform the migration by running the migration tool, which is a standalone binary file. The binary file
must be run with either sudo or admin permission. The migration tool uses the demisto.conf file to read
the following information:
• Database location
• Partitions data
• Elasticsearch configuration (elastic.conf, if it exists)
You must stop the Cortex XSOAR server before you run the migration tool. This enables the
tool to safely access the database and required configurations.
The migration tool begins by reading the Cortex XSOAR database to identify existing partitions and custom
fields. It then creates the index (if it does not already exist) based on the Elasticsearch configuration in the
demisto.conf file, or based on the Elasticsearch default configurations set in the Elasticsearch cluster
setting. After Elasticsearch is successfully configured, the tool reads each partition, from older to newer, and
copies all indicators to the index. Duplicate indicators are overridden, taking only the latest version of the
indicator.
CORTEX XSOAR THREAT INTEL MANAGEMENT GUIDE | Migrate Indicators to Elasticsearch 49

Elasticsearch Security Guidelines
It is recommend that you implement these suggested best practices to secure tenant accounts that use
Elasticsearch indexes.
How it works
The tenant inherits the credential method from the main account/host configuration, either using an API
key or username, role, and password.
API key
If an API key is set in the main account/host configuration, when you create or restart a tenant account,
Cortex XSOAR checks if a key for the tenant already exists (based on the tenant name). If a key doesn't
exist, one is created.
The API key is encrypted afterwards using the route /encrypt/ in the configuration file.
User name, role, and password
If an API key is not set in the main account/host configuration, when you create or restart a tenant account,
Cortex XSOAR checks if the role and user for the tenant already exists (based on the tenant name). If the
role and user don't exist, they are created. The user is created with a 32-character password that contains
capital letters, lower-case letters, numbers, and special characters.
The password is encrypted afterwards using the route /encrypt/ in the configuration file.
Enable security features in Elasticsearch

In order to automatically generate unique credentials for each tenant account's index, in your
elasticsearch.yml file, you need to add the following key: xpack.security.enabled: "true",.
If you do not enable XPack security the tenant accounts will inherit the main account credentials.You can
still create or restart a tenant account but will receive the following warning:
security (xpack) is not active. Will not set account user. Enable security by
setting [xpack.security.enabled] to [true] in the elasticsearch.yml file and
restart the node
Disable security features

If you enabled security features in Elasticsearch, you can create a server configuration in Cortex XSOAR
that will override and disable the security features.
1. Go to Settings > About > Troubleshooting.
2. In the Server Configurations section click Add server configuration.
Security.elasticsearch.account: false

Elasticsearch Sizing Requirements
We recommend using Elasticsearch if you plan to exceed at least one of the following maximum capacities
for BoltDB.
The indicators used to test the sizing requirements did not contain a significant number
of additional fields or custom fields. If you plan to have additional and custom fields for
indicators, the maximum numbers should be reduced.
Maximum indicator capacity and disk usage comparison

The following table compares the maximum total indicator capacity, and disk usage for BoltDB and
Elasticsearch. The maximum indicator capacity value was determined when testing the system.
Benchmark BoltDB Elasticsearch
Maximum indicator capacity 5-7 million 100 million

(total)
(Requires up to 10 seconds for a (Requires approximately 40
complex query) seconds for a complex query)
Disk usage 23 million (~ 69 GB) 100 million (~ 70 GB)
If performance is poor, or you know in advance that you will need more than the maximum number of
indicators, you should consider scaling BoltDB or moving to Elasticsearch. If you are already in Elasticsearch,
you can scale it as well. For both BoltDB and Ealsticsearch, you can scale by either adding engines for one
or more feed integrations or increasing the resources (CPU,RAM,Disk IOPS) of the Cortex XSOAR server.
For Elasticsearch, you can also increase the Elasticsearch cluster size from 1 server to 2 or more servers.
Maximum number of indicators in a single fetch

The following table compares the maximum number of indicators in a single fetch for BoltDB and
Elasticsearch.
Benchmark BoltDB Elasticsearch
Maximum number of indicators 1 million 3 million

per single fetch (one-time)
Single feed fetch comparison

The following table compares the number of indicators, time to ingestion, and disk usage for BoltDB and
Elasticsearch.
Number of Indicators Database Time to Ingestion Disk Usage
30k BoltDB 15s 0.5 GB
Elasticsearch 14s 254 MB + 145.5 MB

(Elasticsearch index)

Number of Indicators Database Time to Ingestion Disk Usage
Elasticsearch 23s 285 MB + 145.5 MB

Elasticsearch 45s 353 MB + 291 MB

1M BoltDB 10m42s 20 GB
Elasticsearch 8m19s 1.3 GB + 955 MB

2M BoltDB 27m48s 34 GB
Elasticsearch 18m30s 1.9 GB + 2.18 GB

Elasticsearch server
Component Dev Environment Minimum Production Minimum
CPU 8 CPU Cores 16 CPU cores
Memory 16 GB RAM 32 GB RAM
Storage 500 GB SSD 1 TB SSD with minimum 3k dedicated

IOPS
Additional configurations
It is recommended that you implement the following Elasticsearch configurations in Cortex XSOAR.
Set the number of shards for an index
This server configuration enables you to set the number of shards for a specific index upon creation, where
<common-indicator> is the name of the index. You should set the value to the number of CPU cores in the
Elasticsearch dedicated server. The default is 1.
elasticsearch.shards.<common-indicator>
Set the number of replica shards for an index
This server configuration enables you to set the number of replica shards for a specific index upon creation,
where <common-indicator> is the name of the index. You should set the value to the number of backups
you require. The default is 0.
elasticsearch.replicas.<common-indicator>

Migrate Indicators to Elasticsearch for a Single
Server
When you run the migration tool, parameter values specified in the demisto.conf file override values
supplied for tool flags and default values. If no value exists in the demisto.conf file, values supplied in the
tool flags override default values, but do not write the values to the demisto.config file. For example,
if the db-path is identified in the configuration file, the tool will use the value in that file, not the value
supplied or the default value, when running the tool.
Download the migration tool
To download the migration tool, append &downloadName=elasticsearch_migration_tool to the
end of the download link that you received.
Configuration file parameters
The elasticsearch object should be a top-level object in the demisto.config file (within the main curly
brackets). We recommend that you specify the following keys. You can use either username/password or
API key.
• enabled
• url
• username
• password
• apiKey
“elasticsearch”: {
“enabled”: true,
“url”: “http://localhost:9200”,
“username”: “user”,// Username in elastic
“password”: “password”,
“apiKey”: “”, // an API key can be specified instead of username and
password
“shards”: {
“common-indicator”: 1
},
“replicas”: {
}
},
Tool flags
Flag Type Description Required
-accounts string A comma-separated list of accounts to migrate. Optional

If not specified, all accounts are migrated.
Multi-tenant deployments only.

-config-path string The path to the configuration file for the server. Optional
The default path is /etc/demisto.conf.
-db-path string The path to the database directory. Optional

The default path is /var/lib/demisto.
-elastic-batch- integer The number of indicators per batch to write to Optional

size Elasticsearch indexes.
The default value is 500.
-elastic-index- string The index prefix used in Elasticsearch. Optional

prefix
-elastic-key string The API key to connect to Elasticsearch. Optional
-elastic- string The password to connect to Elasticsearch. Optional

password
-elastic-url string The URL of your Elasticsearch environment. Optional

The default is http://localhost:9200.
-elastic- string The username to connect to Elasticsearch. Optional

username
-log-level string The log level to display. Optional

The default is info.
Prerequisites
Run all commands from the Cortex XSOAR server machine.
STEP 1 | Stop the Cortex XSOAR server.

• CentOS: sudo systemctl stop demisto
• Ubuntu: sudo service demisto stop
STEP 2 | Edit the demisto.conf as needed.
STEP 3 | Run the ./elasticMigrator command with either demisto or sudo permissions.
./elasticMigrator -elastic-url "http://localhost:9200" -elastic-username
user -elastic-password passwd
STEP 4 | Start the Cortex XSOAR service.

• CentOS: sudo systemctl start demisto
• Ubuntu: sudo service demisto start

Migrate Indicators to Elasticsearch for Multi-
Tenant
The tool migrates indicators to an Elasticsearch database. When you run the tool, the contents of the
Cortex XSOAR database are read, and a corresponding object is created in Elasticsearch. The tool is run
from the main machine and each host machine.
When you run the migration tool, parameter values identified in the demisto.conf file override values
supplied for tool flags and default values. If no value exists in the demisto.conf file, values supplied in the
tool flags override default values, but do not write the values to the demisto.config file. For example,
if the db-path is identified in the configuration file, the tool will use the value in that file, not the value
supplied or the default value, when running the tool.
Configuration file parameters
The elasticsearch object should be a top-level object in the demisto.config file (within the main curly
brackets). Make sure the configurations match on the demisto.config files for the main machine and
each host machine. This is an example of the Elasticsearch object to add to the demisto.config file.
“elasticsearch”: {
“enabled”: true,
“url”: “http://localhost:9200”,
“username”: “user”,// Username in elastic
“password”: “password”,
“apiKey”: “”, // an API key can be specified instead of username and
password
“shards”: {
},
“replicas”: {
}
},
Tool flags
-accounts string A comma-separated list of accounts to migrate. Optional

If not specified, all accounts are migrated.
Multi-tenant deployments only.
-config-path string The path to the configuration file for the server. Required
The default path is /etc/demisto.cong.
-db-path string The path to the database directory. Optional

The default path is /var/lib/demisto.

-elastic-batch- integer The number of indicators per batch to write to Optional

size Elasticsearch indexes.
The default value is 500.
-elastic-index- string The index prefix used in Elasticsearch. Optional

prefix
-elastic-key string The API key to connect to Elasticsearch. Optional
-elastic- string The password to connect to Elasticsearch. Optional

password
-elastic-url string The URL of your Elasticsearch environment. Required

The default is http://localhost:9200.
-elastic- string The username to connect to Elasticsearch. Optional

username
-log-level string The log level to display. Optional

The default is info.
STEP 1 | On the main machine, stop the Cortex XSOAR service.

STEP 2 | On the host machines, stop the Cortex XSOAR service.

STEP 3 | Edit the demisto.config as needed.
STEP 4 | From the main server machine, run the ./elasticMigrator command with either demisto or
sudo permissions.
STEP 5 | From each host machine, run the ./elasticMigrator command with either demisto or sudo
permissions.
./elasticMigrator -elastic-url "http://localhost:9200" -elastic-username
user -elastic-password passwd
STEP 6 | On the main machine, start the Cortex XSOAR service.

STEP 7 | On the host machines, start the Cortex XSOAR service.


Backup Indicators Stored in Elasticsearch
When you use Elasticsearch to store your indicators, you need to specify the path where Cortex XSOAR
saves the Elasticsearch snapshots.
Elasticsearch is backed up at the same time that Cortex XSOAR is backed up.
STEP 1 | Open the elasticsearch.yml file.
STEP 2 | Add the following key-value pair: path.repo:

<path>/<to>/<elasticsearch>/<snapshots>.

Restore Indicators Stored in Elasticsearch
Follow these steps to restore an Elasticsearch indicators snapshot.
STEP 1 | Stop the Cortex XSOAR service with the sudo service demisto stop command.
STEP 2 | Close the relevant Elasticsearch index with the curl -XPOST
“http://<serveradress:port>/dmst-common-indicator/_all/_close” command.
STEP 3 | Restore the snapshot with the "http://<serveradress:port>/_snapshot/

DemistoBackupRepository/snapshot_name_to_restore/_restore" command.
STEP 4 | Start the Cortex XSOAR service with the sudo service demisto start command.

Troubleshoot Elasticsearch
• Troubleshoot Elasticsearch Memory Issues
• Troubleshoot Elasticsearch Feed Ingestion Issues
Troubleshoot Elasticsearch Memory Issues

JVM memory
The default JVM memory is 1 GB. If this is insufficient, you might need to increase the JVM memory.
Batch size
You can modify the batch size in your Elasticsearch platform.
You should set the JVM memory to a maximum of 50% of the server RAM.
Term query size

The term query size is used by the bulk edit. The default term query size is 65536. If this is insufficient, you
might need to increase the term query size.
Bulk size
The bulk size depends on the available JVM memory, and affects the amount of data that Cortex XSOAR
can send and process in Elasticsearch.
Heap size
The recommended maximum heap size is 50% of the entire server, as long as the other 50% are free.
Disable swapping
It is recommended to disable swapping in Elasticsearch in order to improve performance.
Troubleshoot Elasticsearch Feed Ingestion Issues

We recommend that you use the following search query syntax: field:(a,b,c …).
Workaround
To fix this issue, you can increase the maximum clause count and the maximum total field count in the
elasticsearch.yml file.
Table 2: Maximum clause count
Elasticsearch Version Key Value
6.0 and later index.query.bool.max_clause_count A number larger than the

default of 1,024.
5.x and earlier indeces.query.bool.max_clause_count A number larger than the

default of 1,024.

Table 3: Maximum total field count
The maximum total field count includes nested fields. The default value is 1,000.
Key Value
index.mapping.total_fields.limit A number larger than the default of 1,000.

Paloalto - Cortex Xsoar Threat Intel Management Guide

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Paloalto - Cortex Xsoar Threat Intel Management Guide

Uploaded by

Copyright:

Available Formats

Cortex XSOAR Threat Intel Management

About the Documentation

2 CORTEX XSOAR THREAT INTEL MANAGEMENT GUIDE |

Threat Intel Feeds............................................................................................ 29

Migrate Indicators to Elasticsearch.............................................................. 47

TABLE OF CONTENTS iii

> Threat Intel Concepts

Indicator smart merge

Incident • Manual: user marks a piece of data as an indicator.

Regex query A query that identifies indicators in the War Room.

STIX file Manually upload a STIX file on the Indicators page.

Script • FetchIndicatorsFromFile: accepts a file from which it extracts indicators

CORTEX XSOAR THREAT INTEL MANAGEMENT GUIDE | Manage Indicators 11

URL The URL of the feed.

Trust any certificate When selected, certificates are not checked.

12 CORTEX XSOAR THREAT INTEL MANAGEMENT GUIDE | Manage Indicators

Export Exports the selected indicators to a CSV file.

Export (STIX) Exports the selected indicators to a STIX file.

Create a new Manually creates a new indicator in the system.

sourceBrands Indicator feed or enrichment integrations.

sourceInstances A specific instance of an indicator feed or enrichment integration.

expirationSource The source of the indicator having expired status.

CORTEX XSOAR THREAT INTEL MANAGEMENT GUIDE | Manage Indicators 13

isShared Whether the indicator is shared to tenant accounts (multi-tenant only).

tags Tags applied to indicators.

comments Search for keywords within indicators’ comments.

Score Reputation Color

14 CORTEX XSOAR THREAT INTEL MANAGEMENT GUIDE | Manage Indicators

User (manual) A+++ A user manually updates the

Reputation script A++ A script with the reputation tag,

3rd-party enrichment A+ An integration or service that

Feed reliability A: Completely reliable The feed reliability is applied at

D: Not usually reliable

F: Reliability cannot be judged

CORTEX XSOAR THREAT INTEL MANAGEMENT GUIDE | Manage Indicators 15

Ensure that the Reputation tag is selected.

STEP 3 | Click Save.

STEP 7 | Click Save.

16 CORTEX XSOAR THREAT INTEL MANAGEMENT GUIDE | Manage Indicators

Create an Indicator Type

STEP 1 | Go to Settings > Advanced > Indicator Type.

STEP 2 | Click the Add Indicator Type button.

STEP 3 | Configure the indicator type as needed.

Indicator Type Profile

Name A meaningful name for the indicator type.

Enhancement Scripts A script to run on an identified indicator. For example, an enrichment

Reputation Command The command to run to calculate the reputation of indicators of

Excluded Integrations Integrations to exclude when calculating the reputation, evaluating,

CORTEX XSOAR THREAT INTEL MANAGEMENT GUIDE | Manage Indicators 17

18 CORTEX XSOAR THREAT INTEL MANAGEMENT GUIDE | Manage Indicators

Create a Custom Indicator Field

STEP 1 | Go to Settings > Advanced > Fields.

STEP 2 | From the drop-down menu, select Indicator.

CORTEX XSOAR THREAT INTEL MANAGEMENT GUIDE | Manage Indicators 19

STEP 4 | Configure the basic settings.

Field Type Determines the acceptable values for the field.

Mandatory If selected, this field is mandatory when used in a form.

Tooltip An optional tooltip for the field.

Placeholder Optional text to display in the field when it is empty.

STEP 5 | Configure the attributes.

STEP 6 | Click Save.

Map Custom Indicator Fields

STEP 1 | Go to Settings > Advanced > Indicator Types.