Forward- This presentation may contain forward-looking statements regarding future events, plans or the

expected financial performance of our company, including our expectations regarding our products,
technology, strategy, customers, markets, acquisitions and investments. These statements reflect

Looking management’s current expectations, estimates and assumptions based on the information currently
available to us. These forward-looking statements are not guarantees of future performance and involve
significant risks, uncertainties
involve significant and other
risks, uncertainties factors
and other that maythat
factors cause
mayour actual
cause ourresults, performance or
actual results,
Statements achievements
performance ortoachievements
by
be materially different
the forward-looking
expressed or implied bystatements
from results,
to be materially
containedstatements
the forward-looking
performance
different
in this presentation.
or achievements
from results, performance orexpressed
contained in this presentation.
or implied
achievements

For additional information about factors that could cause actual results to differ materially from those
described in the forward-looking statements made in this presentation, please refer to our periodic
reports and other filings with the SEC, including the risk factors identified in our most recent quarterly
reports on Form 10-Q and annual reports on Form 10-K, copies of which may be obtained by visiting the
Splunk Investor
the Splunk Relations
Investor website
Relations at www.investors.splunk.com
website at www.investors.splunk.com or the
or SEC's website
the SEC's at www.sec.gov.
website at
The forward-looking
www.sec.gov. statements made
The forward-looking in this presentation
statements made in thisare made as ofare
presentation themade
time and
as ofdate of this
the time and
presentation. If reviewed after
date of this presentation. the initial
If reviewed presentation,
after even if made
the initial presentation, available
even if madebyavailable
us, on ourbywebsite or
us, on our
otherwise,
website or itotherwise,
may not contain
it may notcurrent or accurate
contain current orinformation. We disclaim
accurate information. Weany obligation
disclaim anyto update or
obligation to
revise
updateany forward-looking
or revise statement statement
any forward-looking based on newbased information, future events
on new information, or otherwise,
future except as
events or otherwise,
required
except asbyrequired
applicable law.
by applicable law.
In addition, any information about our roadmap outlines our general product direction and is subject to
change at any time without notice. It is for informational purposes only and shall not be incorporated into
any contract
into any or other
contract commitment.
or other We undertake
commitment. no obligation
We undertake either
no obligation to develop
either the features
to develop or or
the features
functionalities described or to include any such feature or functionality in a future release.

Splunk, Splunk>, Data-to-Everything, D2E and Turn Data Into Doing are trademarks and registered trademarks of Splunk Inc. in the United States and other
countries. All other
other countries. brandbrand
All other names, product
names, namesnames
product or trademarks belongbelong
or trademarks to theirtorespective owners.
their respective © 2021
owners. © Splunk Inc. AllInc.
2021 Splunk rights reserved.
All rights reserved.
 © 2021 SPLUNK INC.

Application
Integration
Monitoring Using
Splunk ITSI
ITO1256B

Jin Chen
AIOps Foundation Senior Manager | Lenovo
Jeremy Wang
Data Analytics Director | Lenovo
 © 2021 SPLUNK INC.

Jin Chen Jeremy Wang

IT Operation Senior Mgr. | Lenovo Data Analytics Director | Lenovo
 © 2021 SPLUNK INC.

1) About Lenovo
Agenda
2) Background and Challenges

3) Monitoring Solutions

4) Solution Demo

5) Benefits and Take Away

 © 2021 SPLUNK INC.

About Lenovo

$60B 180
Revenue Areas

16 159
Gartner 2020 Fortune 500
Supply Chain 25
 © 2021 SPLUNK INC.

Lenovo AIOps Platform

Data Layer and Source

Service Operation
Ops

IT service tickets from

Incidents, SR,
ITSM, Jira, etc. ITSM System
Changes, Configurations
IT Operation
Sales, logistic, Data Platform
Business
Biz

materials, etc from ERP,

Volume, Process Data CRM, PLM, etc.

Events, Logs,
Application
App

Configuration, Metrics Automation

Usage, Logs, Experience,
from applications and Platform
Performance, Interface
monitor tools

Middleware/Database
Infrastructure

Availability, Traffic, Failures,

Performance, Capacity
Server/Storage/Network
Availability, Performance,
Capacity, Failures, Logs
Events, Logs, ….
Configuration, Metrics,
Topology from monitor ….
tools and infra devices
 © 2021 SPLUNK INC.

Complex Application Integration Environment

● Hybrid cloud environment
Public Cloud 1
● Multiple application integration
SaaS
APPs
technologies APP1 APP2 APP3

● Business process running cross

multiple business applications

● Business/IT need to understand
Private Cloud
real-time integration status Public Cloud 2

For one project:

SFTP APP6 APP7

APP4 APP5
>30 >150 >200K
Applications Interfaces Messages
per day
 © 2021 SPLUNK INC.

Challenges with Application Integration Monitoring

Environment Organization Timely Knowledge

Complicated hybrid Multiple Support Near real-time Knowledge Gap

Cloud Environment Organization monitoring and alert Between Biz and IT
 © 2021 SPLUNK INC.

Integration Monitoring Model

Application Integration Process Sample:

SFTP

APP1 APP2 APP3 APP4 APP5

Minimal Monitoring Unit:

KPIs:
● Kafka ● Availability
● WSO2 ● Performance
● SFTP Source Middleware Destination ● Traffic
● ... APP APP ● Failures
● ...
 © 2021 SPLUNK INC.

Data Collection Standard

• Data collection channels
– Kakfa
– HEC
– Uni-Forwarder
– DB connect
– Splunk Add-On
• Log data format
– Content: Timestamp, Log Level, IP, Username, Object, System, Operation, Status, ….
– Time Format: YYYY-mm-dd HH:MM:SS.SSS+0000,YYYY-mm-dd HH:MM:SS.SSSZ
– ….
• Development Code Sample
– Java: Logback, Log4j
– Python: logging
– ….
 © 2021 SPLUNK INC.

Data Preparation & Data Model

Step1: Field Alias - standard Step2: Calculated Fields - Step3: Create Event types &
fields name standard fields format Tags
Timestamp:
TOPIC
yyyy-mm-dd hh:mm:ss.z Eventtype Integration_Monitoring_<Kafka>_<App1
topic Topic >_<producer>
yyyymmddhhmmss Unix Format
topic_name Tag Integration_Monitoring_<Kafka>
yyyy/mm/dd hh:mm:ss

Step4: Create Data Model: Step5: Create Data Model:

root event child- Three Layer

APP 1

APP 1_Consumer

APP 1_Producer
 © 2021 SPLUNK INC.

Service & KPI

Step 1 Create Services by
upstream - downstream systems Step 3 Create KPI for key indicators

Key indicators:
Build services based on minimum integration
– Average Latency for each message type
unit
– Long pending message amount
Example:
– Sent/Received message amount
AppName_Integration_Source-destination
– Failure rate
Threshold & Severity: consistent with alert requirements
Step 2 Create & assign entities by message type

Step 4 Involve integration services

Message types:
into application service models
Kafka - topic
FTP - file
WSO2 - API name

Build entities for every message types

Example:
Source_MessageName_Destination
 © 2021 SPLUNK INC.

Alerting

Correlation Automation
Aggregation
Service & KPI Search Notable Events Episodes Action Rules Emails
Policies
Incidents

Step 1： Step 2： Step 3:

Configure service health
monitoring correlation search
● Fully aligned with both IT &
Biz teams
● Prioritize events based on
business impact
Configure notable event
aggregation policies
● Aggregation policies are
according to integration
process (source to
destination)
Configure episode action rules
● Trigger ITSM incidents,
email notification or
automation process
according to scenarios
● Provide all the necessary
information into alerts
(service impact, drill down
dashboard, etc.)
 © 2021 SPLUNK INC.

Anomaly Detection
Step 1: Identify scenarios need using Anomaly Dynamic Threshold
Detection
• 80% issues can be well detected by rules, only 20% issues
need to use anomaly detection

Step 2: Identify corresponding KPIs

• Message Amount, Ave Message Latency, Message Failure
Rate, etc... Trend Analysis

Step 3: Build Dynamic Threshold

Step 4: Using Trend Analysis for key KPI

• Add trend as an indicator for your KPI to identify critical issue
before issue happened. Multiple Indicator Alert

Step 5: Build alert based on Multiple Indicator

• Using multiple indicator to identify critical issue and decrease
unvalid alert
 © 2021 SPLUNK INC.

Visualization for Mgmt and Biz

Who will use:

Management and Business users

Why they use:

• Know the latest progress when critical issue happened
• Know the latest Biz and application status

Key Comments
• Involve both IT and Business into discussion and design
• Using language both IT and Business can understand
• Build overview for users
 © 2021 SPLUNK INC.

Visualization for IT Engineer

Who will use:

IT Engineer

Why they use:

• Issue and risk Identification
• Root cause analysis

Key Comments
• Only show Key KPI status
• Set up baseline and show in the dashboard
• Verification in daily operation and iteration to improve
 © 2021 SPLUNK INC.

Solution Demo
 © 2021 SPLUNK INC.

Key Benefit
In Integration Monitoring Area

50% 90% 75% 80%

Shorten MTTR for Reduced failed Saved manual effort Reduced business
critical IT issues business transaction for issue auto-fix complaints compared
caused by integration with previous
issue situation
 © 2021 SPLUNK INC.

Take Away
• Build a cross-function virtual monitoring team
• Engage business in monitoring design
• Always thinking to bring value to business
• Define effective data standard as early as possible
• Focus on the most important services or pain points
• Start from MVP（Minimum Viable Product）and iterate fast
 © 2021 SPLUNK INC.

Thank You Please provide feedback via the

SESSION SURVEY