|






Cristian Levcovici
Windows Client Performance
j



 ntroduction
 Overview of xperf
 Overview of xperfinfo
 Extensibility
 References

  



 What is xperfinfo/xperf?
 What is ETW?
ntroduction





 Extensible performance analysis toolset

 Based on ETW instrumentation
 Performs high-
high-level decoding

 Cross-platform (XPSP1+, W2K3, LH)

Cross-
 Cross--architecture (x86, x64, ia64)
Cross

 Capture--anywhere process-
Capture process-anywhere
ntroduction

0



 High level control and decoding of a large number
of ETW events built into the NT kernel
ƛ process lifetime ƛ disk /O
ƛ thread lifetime ƛ file /O
ƛ image lifetime ƛ registry
ƛ sample profile ƛ hardfault
ƛ context switch ƛ pagefault
ƛ DPC (Deferred Procedure Call) ƛ virtual allocation
ƛ SR (nterrupt Service Routine) ƛ heap
ƛ driver delay ƛ TCP/UDP

Stackwalking support (LH only) Ʀ and many others

ntroduction

R

 R
 Event Tracing for Windows
ƛ High performance, low overhead, highly scalable tracing
facility provided by the Windows OS (Win2K+)
 Extensively used by the NT kernel for self-
self-instrumentation

ƛ On a typical server setup, TraceEvent() takes 1,500 to

2,000 cycles
 <2% CPU overhead for a sustained rate of 20,000
events/sec on a 2GHz processor

ƛ Uses an efficient buffering and logging mechanism

 Non-blocking and uses per-
Non- per-processor buffers that are written
to disk by a separate writer thread
ntroduction

R

 R  

 Event Tracing for Windows
ƛ Fast, reliable, and versatile set of features for logging
events raised by user-
user-mode applications and kernel-
kernel-mode
drivers

ƛ Turn tracing on/off dynamically without requiring reboots

or application restarts

ƛ The disk log is a binary file

 Event format is encoded by GUD + type + version
 Trace must be post processed
ƛ events must be correlated with context events and domain
specific knowledge for high-
high-level decoding
ntroduction

R

 R  

 Debug application bugs including hangs,

crashes, or unexpected behavior

 Diagnose performance problems

 Track computing resource consumption at

application transaction level for capacity
planning
ntroduction

R



 Provider
ƛ Provides event traces. Can be user-
user-mode app,
kernel--mode driver, or the kernel itself
kernel
ƛ Providers are instrumented with ETW APs to
register with the ETW framework to send event
traces from various points in the code.
ƛ When enabled dynamically by the trace
controller application, the provider sends event
traces to a specific trace session designated by
the controller.
 Controller
ƛ Assists in starting, stopping or updating trace
sessions in the kernel as well as enabling or
disabling providers
ƛ Used to set trace session properties such as
sequential or circular file logging or direct
delivery to consumers
 Consumer
ƛ Application that reads trace files or listens to
active trace sessions and processes logged
events
ƛ Not aware of the Providers
ƛ Only receive event traces from the trace sessions
or log files
 Event Trace Session infrastructure
ƛ Brokers the event traces from the provider to
consumer and in the process adds valuable data
to each event such as TimeStamp, Thread,
Process, CPU
j
 

 Detailed interactive analysis of ETW

traces
ƛ Emphasis on kernel events
ƛ Support for 3rd party events, primarily in
conjunction with kernel events
Overview of xperf




Overview of xperf




Selection
Overview of xperf




Context-Menu Summary Table
Overview of xperf

CP||   

Selected
% Total
Time
Time % of Time excluding DPC and SR
nterval

Close
Summary
Table

Status Bar Report

Overview of xperf





Sidebar
Overview of xperf





Sidebar
Overview of xperf





Sidebar Scrollbar
Overview of xperf





Scrollbar
Overview of xperf





Scrollbar
Overview of xperf





Scrollbar

Selection
Overview of xperf





Context-Menu Summary Table

Overview of xperf

Œ

j  


Expand
Overview of xperf

Œ

j  


Expand
Overview of xperf

Œ

j  
 Close
Summary
Table
Disk Service Time /O Size /O Priority

Expand
ndividual /Os
Overview of xperf




Overview of xperf





Selection
Overview of xperf





Context-Menu Detail Graph

Overview of xperf

Œ
Œ



Change Disk
Overview of xperf

Œ
Œ



Change Disk
Overview of xperf

Œ
Œ



Selection
Overview of xperf

Œ
Œ

  

Disk Service Time Disk Queue Depth File Path

ndividual /Os
in time order
by completion
time

Status Bar Report

Overview of xperf




Overview of xperf




Context-Menu
Overview of xperf




Context-Menu Summary Table
Overview of xperf

CP|  


Expand
Overview of xperf

CP|  


Expand
Overview of xperf




Overview of xperf




Overview of xperf





Load Symbols
Overview of xperf




Context-Menu
Overview of xperf




Context-Menu Summary Table
Overview of xperf

CP|  




Π
 


Expand
Overview of xperf

CP|  




Π
 


Expand
Overview of xperf

CP|  




Π
 


Expand
Overview of xperf



 

 xperf provides many graphs and

summary tables
ƛ Sample Profile ƛ Registry counts
ƛ CPU Availability ƛ Driver Delay
ƛ CPU Scheduling ƛ Hardfault
ƛ Disk Counts ƛ Pagefault
ƛ Disk Utilization ƛ Services
ƛ Disk Detail ƛ Plug ƞnƞ Play
ƛ Process Lifetime ƛ Marks
ƛ DPC (Deferred Procedure Call) ƛ Generic
ƛ SR (nterrupt Service Routine)
j
 


 High level control and decoding

 Dumping of ETW traces
 Command line analysis of ETW traces
ƛ emphasis on kernel events
ƛ support for 3rd party events
Overview of xperfinfo

 
 
 

 Start kernel trace

C:\analysis> xperfinfo ±on base+cswitch

 Run scenario
C:\analysis> MyTestApp.exe

 Stop and merge kernel trace

C:\analysis> xperfinfo ±d trace.etl
Merged Etl: trace.etl

ë

üü You can retrieve all known kernel flags and groups with
ë


C:\analysis> xperfinfo ±help providers
Overview of xperfinfo

 
  
 

 Start user trace

C:\analysis> xperfinfo ±start MySession ±on Kerberos+MRxSmb ±f kerberos.etl

 Run scenario
C:\analysis> MyTestApp.exe

 Stop user trace

C:\analysis> xperfinfo ±stop MySession

ë

üü You can retrieve all known providers with

ë


C:\analysis> xperfinfo ±help providers
Overview of xperfinfo

 
  
 

 Start kernel and user traces

C:\analysis> xperfinfo ±on base+cswitch
C:\analysis> xperfinfo ±start MySession ±on Kerberos+MRxSmb ±f kerberos.etl

 Run scenario
C:\analysis> MyTestApp.exe

 Stop user trace Stopping kernel trace

C:\analysis> xperfinfo ±stop MySession ±stop ±d trace.etl

Merged Etl: trace.etl
Overview of xperfinfo

x


 ë 
C:\analysis> xperfinfo -i trace.etl -a tracestats

Number of Processors : 4
CPU Speed : 2372 MHz
OS Version : 05.01.01.00
OS Build Number : 2600
Clock type : PerfCounter
Boot time : 2005/10/13:16:05:14.5000000
Native Pointer Size : 4 (32bit)
Start time : 2005/10/14:04:03:14.3388906
End time : 2005/10/14:04:03:23.8073376 (+ 0:00:00:09.4684470)
Total # Lost Buffers : 0
Total # Lost Events : 0

Number of Traces : 1

Trace name: trace.etl

Log file mode : Relogged
Pointer size : 4 (32bit)
Start time : 2005/10/14:04:03:14.3388906
End time : 2005/10/14:04:03:23.8073376 (+ 0:00:00:09.4684470)
# Lost Buffers : 0
# Lost Events : 0
Overview of xperfinfo

x


   
C:\analysis> xperfinfo -i trace.etl -a tracestats -detail

Classic EventGuid TotalCount TotalSize Name

====================== ========== =============== ===========================
66838 1703946 <All>

{01853a65-418f-4f36-aefc-dc0f1d2fd235}

58 39376 SysConfig

Type Level Version Count TotalSize Name

---- ----- ------- ---------- --------------- ---------------------------
0x0a 0x00 0x0001 1 952 SysConfig: CPUs
0x0b 0x00 0x0001 2 1224 SysConfig: Physical Disks
0x0c 0x00 0x0001 4 448 SysConfig: Logical Disks
0x0d 0x00 0x0001 1 744 SysConfig: Network Cards
0x0e 0x00 0x0001 2 5168 SysConfig: Video Adapters
0x0f 0x00 0x0001 47 30832 SysConfig: Services
0x10 0x00 0x0001 1 8 SysConfig: Power Management

...
Overview of xperfinfo

x




ë

C:\analysis> xperfinfo -i trace.etl -a tracestats /?

Action invocation:

xperfinfo -i <trace file> ... [-o <output>] -a tracestats ...

Action help:

tracestats [-timespan [actual]] [-detail]

-timespan [actual] Show information about session and traces. [default]

Without parameters, -timespan requires inspection of trace
headers only; no pass is performed through the traces in
the session.
When the parameter "actual" is specified, the actual
times of the first event and the last event in the session
are added to the report. In this case, a pass through the
traces in the session is required.

-detail Show detailed information about providers, ids, tasks,

opcodes, versions, channels and levels of events in the
session along with provider and opcode friendly names.
Requires a full pass through the traces in the session.







üü C:\analysis> xperfinfo ±help tracestats
Overview of xperfinfo

Π
  
C:\analysis> xperfinfo -i trace.etl ±o trace.txt
[1/2] 100.0%
[2/2] 100.0%
C:\analysis> notepad trace.txt
Overview of xperfinfo

Π
  



Π

C:\analysis> set _NT_SYMBOL_PATH=srv*C:\symbols*\\symbols\symbols
C:\analysis> xperfinfo -i trace.etl ±o trace_symbols.txt -symbols
[1/2] 100.0%
[2/2] 100.0%
C:\analysis> notepad trace_symbols.txt



ëü C:\analysis> xperfinfo ±help symbols
Overview of xperfinfo







 xperfinfo provides many actions

ƛ dumper ƛ diskio
ƛ tracestats ƛ filename
ƛ sysconfig ƛ hardfault
ƛ marks ƛ pagefault
ƛ process ƛ drvdelay
ƛ perfctrs ƛ reference_set
ƛ profile ƛ boot
ƛ cswitch ƛ suspend
ƛ dpcisr ƛ shutdown
ƛ registry Ʀ and many others
Overview of xperfinfo

j
ë

 Your best friend is the online help

C:\analysis> xperfinfo -help

Usage: xperfinfo options ...

Lists all xperfinfo -help start for logger start options
available xperfinfo -help providers for known tracing flags
xperfinfo -help stop for logger stop options
processing xperfinfo -help merge for merge multiple trace files
actions xperfinfo -help processing for trace processing options
xperfinfo -help symbols for symbol decoding configuration
xperfinfo -help query for query options
xperfinfo -help mark for mark and mark-flush
xperfinfo -help format for time and timespan formats on
the command line
xperfinfo -help advanced for advanced options (things you
don't need to know :-))
 






 xperfinfo and xperf are built on top of an extensible
 ,, that supports 3rd party binary
core,  
addins providing custom
ƛ Event dumpers
ƛ nfosources
ƛ Event name databases
ƛ Graphs & summary tables
ƛ Application drivers
ƛ xperfinfo Actions

 For details, please see the ƠExtending xperfcoreơ

presentation
x
 Homepage
ƛ httpü//toolbox/sites/xperf

 Tools distribution point

ƛ \\ntperformance\
ntperformance\tools\
tools\xperf\
xperf\x86\
x86\latest
ƛ \\ntperformance\
ntperformance\tools\
tools\xperf\
xperf\amd64\
amd64\latest
ƛ \\ntperformance\
ntperformance\tools\
tools\xperf\
xperf\ia64\
ia64\latest

 README.TXT
ƛ \\ntperformance\
ntperformance\tools\
tools\xperf\
xperf\x86\
x86\latest\
latest\README.TXT
C