Professional Documents
Culture Documents
DougClifton - DB2 Utilities Update 2011-1
DougClifton - DB2 Utilities Update 2011-1
DougClifton - DB2 Utilities Update 2011-1
Doug Clifton
Executive IT Specialist
cliftonw@us.ibm.com
1
Disclaimer
THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL
PURPOSES ONLY.
WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE
INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED AS IS, WITHOUT
WARRANTY OF ANY KIND, EXPRESS OR IMPLIED.
IN ADDITION, THIS INFORMATION IS BASED ON IBMS CURRENT PRODUCT PLANS AND STRATEGY,
WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE.
IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR
OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.
NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF:
CREATING ANY WARRANTY OR REPRESENTATION FROM IBM (OR ITS AFFILIATES OR ITS OR
THEIR SUPPLIERS AND/OR LICENSORS); OR
Agenda
Recent DB2 9 enhancements
Current DB2 9 work
DB2 10 utility enhancements
Summary
SORTNUM Elimination
CHECK INDEX, REBUILD INDEX, REORG, RUNSTATS
PK45916 (V8) & PK41899 (V9)
Better performance, more robust, simpler
SORTNUM no longer required
Difficult to estimate: failure if too low, excessive sort work allocation if too high
New zparms UTSORTAL & IGNSORTN (online changeable)
UTSORTAL YES|NO
Use RTS data to estimate number of rows to sort
DB2 will dynamically allocate sort work datasets
If SORTWK DD cards not hard coded
IGNSORTN YES|NO
Override utility job setting of SORTNUM
Recommendation
LOAD COPYDICTIONARY
PK63324/PK63325 (V9)
REORG avoidance prime empty partitions with compression dictionary
Avoid FRVs for LOAD/UNLOAD of zero length LOBs
PM12286 (V9)
Performance
7
12
LOB
Part 1
Part 2
14
LOB
V10: LOAD/UNLOAD
Remove MAX_UTIL_PARTS zparm
Restriction removed for REORG in
V9
S e c o n d s e la p s e d
700
600
500
400
PDSE
VBS
300
200
100
0
5000 10000 20000 30000 40000
LOBs unloaded
15
V10: COPY
Dataset-level Flashcopy support
COPY, RECOVER, REORG, LOAD, REBUILD INDEX, REORG INDEX
New zparms & utility parms to govern
Significant CPU & elapsed time reduction for large pagesets
Create transaction-consistent image copies from SHRLEVEL CHANGE
3
2.5
2
1.5
1
0.5
0
IC
FC
25
50
75
100
Seconds
S eco n d s
IC
FC
25
50
75
100
V10: COPY
Post-GA Flashcopy enhancements
Support for REORG AUX YES to FlashCopy aux objects
Remove requirement for sequential inline copy for REORG
Additional zparms with APAR PM26762
3
2.5
2
1.5
1
0.5
0
IC
FC
25
50
75
100
Seconds
S eco n d s
IC
FC
25
50
75
100
V10: COPY
Improved dataset management & performance
CHANGELIMIT will not allocate copy dataset unless copy taken
&ICTYPE now matches actual image copy type
18
V10: RECOVER
New BACKOUT YES option for point in time recovery
True rollback, not run of generated SQL undo statements
Requires COPY YES for indexes
VERIFYSET option to fail PIT recovery if entire set not included
Base, LOB, XML, history not RI
ENFORCE option to avoid CHKP/ACHKP when subset of set recovered
Improved performance due to avoidance of set checking (RI, aux)
19
V10: Stats
RUNSTATS PROFILE support for simplification
RUNSTATS on views
Autonomic features through new stored procedures & catalog
tables
All catalog statistics columns made updatable
RUNSTATS SHRLEVEL REFERENCE updates RTS
TOTALROWS & TOTALENTRIES columns
zIIP-enablement for RUNSTATS
Auto sampling rates & page sampling instead of row sampling
Significant CPU & ET savings
TABLESAMPLE SYSTEM AUTO
20
V10: CHECK
CHECK utilities will no longer set CHKP/ACHKP
CHECK SHRLEVEL CHANGE default changed to fail if Flashcopy not
available
ZPARM to govern
CHECK DATA enhanced for XML support
Document validation
Schema validation
Automated exception table processing for XML documents
21
V10: Other
Removed UTSERIAL lock for greater utility concurrency
SQL SELECT on SYSLGRNX
Expect delivery post-GA
LISTDEF & TEMPLATE enhancements
LISTDEF support for CHECK DATA
LISTDEF support for multiple part ranges on REORG
LISTDEF support for DEFINED YES|NO|ALL
Improved utility performance since unnecessary to build & then discard
structures for undefined objects
Default changed to DEFINED YES & empty lists result in RC4
REPORT RECOVERY support for SLBs
DSNACCOX enhancements
Support hashed pagesets
New RTS columns for SSD, other
22
Summary
Eliminate outages
Improve performance
Reduce resource consumption
Reduce complexity & improve automation
Day 1 utility support for core DB2 10 function
Continued delivery of performance improvements & features of
real value
23
24
Agenda
General recommendations
Backup
Recovery/QUIESCE
MODIFY RECOVERY
LOAD/UNLOAD
REORG
CHECK utilities
DSN1COPY
Summary
25
General recommendations
26
Summary
Newer releases are better than older releases
Performance
Function
But be mindful of part level REORG in V9
Newer maintenance levels are better than older maintenance
levels
Performance
Function
Go to Utility Update talks at conferences to get the
latest information
27
Sort processing
Improved utility sort processing
CHECK INDEX, REBUILD INDEX, REORG, RUNSTATS
PK45916 (V8) & PK41899 (V9)
Better performance, more robust, simpler
SORTNUM no longer required
Correct value hard to determine, resulting in utility failure if too low or excessive
sort work allocation if too high
New zparms UTSORTAL & IGNSORTN (online changeable)
UTSORTAL YES|NO
Use RTS data to estimate number of rows to sort
DB2 will dynamically allocate sort work datasets
If SORTWK DD cards not hard coded
IGNSORTN YES|NO
Override utility job setting of SORTNUM
Recommendation
Turn on UTSORTAL, test it, then consider turning on IGNSORTN
DSNU3340I 168 08:13:52.66 DSNUGLSR - UTILITY PERFORMS DYNAMIC ALLOCATION OF SORT DISK SPACE
28
Sort processing
Sort work data set allocation
If assigning a data class though ACS routine, ensure its defined as
single volume only
Turn off space constraint relief in assigned data class
Use large volumes for sort work data sets
>64K tracks supported since z/OS 1.7
DDs for ACS routines:
SORTWKnn, SWxxWKnn, DATAWKnn, DAxxWKnn, STATWKnn,
STxxWKnn
See infoapars II14047 & II14213 for sort/DFSORT recommendations
Add //SORTDIAG DD DUMMY to utility job JCL
Negligible cost
Adds 10 lines of DFSORT diagnostic output required in unlikely event
of problem
29
PBGs
No LOAD or REORG parallelism
No pruning of partitions
No LOAD at partition level
Rows can flow from one part to another within part range
LOAD / REORG at table space level will grow new parts as
needed
REORG cannot grow new parts if LOB column exists
REORG cannot move rows between parts if LOB column exists
REORG of single part or subset of parts will not grow new
parts
Rows must fit back into part, but may not!
PCTFREE/FREEPAGE may cause REORG to fail
PBG
Part 1
UTRW/UTRO
Part 2
UTRW
Part 3
UTRW
RRF
RRF row format introduced in V9 NFM to remove overhead of processing rows
with variable length columns
Some concern about conversion to RRF from BRF on REORG or LOAD in V9 NFM
Primarily concern is worse compression ratio for rows with many var length columns
Currently PK78958 disables RRF conversion for compressed page sets
Conversion is also blocked for tables with editprocs
UTS table spaces are expected to be in RRF format
Hidden zparm can govern conversion/creation in BRF/RRF format
Recommendation: Leave hidden zparm alone
unless cannot tolerate RRF for new page sets in V9 NFM
Compressed page sets will not get converted
New page sets, including UTS page sets, will be RRF
If hidden zparm is turned off, UTS page sets created in BRF format run a higher risk of
defects
New developments
PK87348/PK85881
Externalise zparm to govern BRF/RRF conversion, provide external options on
31LOAD/REORG
2011
IBM Corporation
BACKUP
33
COPY
Exploit parallelism
Consider OPTIONS EVENT(ITEMERROR,SKIP)
Sets UTRW state only for duration of copy of individual
page set
But increases COPY overhead
Serialisation required for each pageset on the fly
34
Backup solutions
Multiple options
BACKUP SYSTEM
Significant DASD investment required
Can be complex to set up & administer, but invocation simple
Must understand any limitations that currently exist
E.g. DASD mirroring issues, dataset movement issues, etc.
Sequential image copies
Tried and trusted solution since V1.1
Other external backups, such as volume-level backups
Outside of DB2s control
Requires careful management and co-ordination
Expect further options in future
Choice is dependent on environment and requirements, all options
will continue to be supported
35
RECOVERY/QUIESCE
36
Recovery/QUIESCE
Maximise exploitation of parallel restore and Fast Log Apply
Recover multiple objects in a list in parallel but ideally <100
Never run more than 10 RECOVER jobs per member
Split off page sets that are not updated and recover separately
For PIT recovery, include whole RI set in same RECOVER
statement
For PIT recovery, include base and aux objects in same RECOVER
statement
Vnext will enforce this
If you want object level recovery from system level backup in V8
then use Recovery Expert tool
QUIESCE
Do you still need it in V9 with PIT recovery with consistency?
Use WRITE NO unless you absolutely have to have pages written
out
37
MODIFY RECOVERY
38
MODIFY RECOVERY
Base your MODIFY strategy on your backup strategy
and not vice versa
Consider running every time a backup is taken or at
least weekly
REORG SYSLGRNX regularly for optimal performance
and minimal MODIFY impact on system
DB2 9 has RETAIN LAST n, GDGLIMIT and BSDS
options
Careful with GDGLIMIT if you use multiple GDGs for a
single object
Will not clean up orphan entries in V9
Run MODIFY to delete recovery information from prior
to a REORG that materialises row alterations
Makes subsequent REORGs more efficient
39
LOAD/UNLOAD
40
LOAD/UNLOAD
Run with LOG NO, REUSE, KEEPDICTIONARY if
possible
Allocate inline copy data sets to DASD
Split up input dataset and drive LOAD partition
parallelism in a single LOAD
Use SORTNUM elimination
Specify SORTKEYS if input is on tape or variable
length
For LOAD REPLACE, consider loading into a clone
then renaming tables or datasets
Consider using USS named pipes
Refer to PK70269 for V8 and V9
41
LOAD/UNLOAD
43
REORG
44
REORG
LOAD COPYDICTIONARY can avoid need to run REORG
If an NPI exists then concurrent REORGs of parts in the same table space is
not permitted in V9
Due to shadowing of NPIs, due to removal of BUILD2 phase
Expect to provide ability to REORG disparate parts via APAR in V9
Be aware that REORG of a single part with an NPI can take longer in V9
Need to shadow NPI
If NPI is disorganized then it can take longer since unloading keys in order
But parallelism in V9 on unload/reload/logapply means multi-part REORG
should be faster and the log phase is much better at keeping up with
logging rates
So REORG multiple (adjacent) parts in the same REORG statement
Inline statistics can increase application outage duration
Catalog statistics updated after switch but whilst data is drained
45
REORG
SHRLEVEL CHANGE main recommendations
Use DRAIN ALL to minimize application impact
(DRAIN_WAIT+MAXRO)<(IRLMRWT -5 or 10 seconds)
Avoid application timeouts
But dont set MAXRO too low
46
47
CHECK utilities
48
CHECK
If no applications are impacted and you just want to check
consistency, then:
Either run CHECK SHRLEVEL CHANGE
Or run standard CHECK utility but be ready with REPAIR to reset
CHKP/ACHKP states
Or could consider running SQL ISO(UR) instead
If running SHRLEVEL CHANGE:
It will not reset CHKP or ACHKP states, nor will it set them
Make sure the page set is on FlashCopy-enabled DASD
If not then youll get a slow copy with the data in UTRO
If using DASD mirroring or BACKUP SYSTEM then use ZPARM
UTIL_TEMP_STORCLAS to prevent impacting either
Refer to PK41711
49
SCOPE AUXONLY
AUXERROR REPORT
AUXERROR INVALIDATE
CHECK INDEX
Aux index
CHECK DATA
Row
LOB
CHECK LOB
Row
LOB
50
SCOPE AUXONLY
AUXERROR REPORT
AUXERROR INVALIDATE
CHECK INDEX(1)
DOCID
Index
CHECK DATA(3)
Cols:
Cols:
DOCID
XMLCOL
DOCID
NODEID
XML Record Value
XML
Index
CHECK INDEX(2)
51
DSN1COPY
52
DSN1COPY
DSN1COPY is an essential part of the utilities portfolio
DSN1COPY runs standalone and cannot ensure that data matches
definition at target
All target datasets must be pre-allocated for multi-piece table spaces
Areas to watch out for:
BRF-RRF mismatch
Tolerated by SQL, but not REORG
Convert page sets to ensure copy is RRF-RRF or BRF-BRF
If that isnt possible, e.g. if image copy is BRF, then UNLOAD from BRF
image copy and LOAD into RRF page set
Data definition changes, e.g. columns added
Use REPAIR VERSIONS at target site
For alterations prior to V8, REORG at source before DSN1COPY
Different table space types or different segsizes
Not policed, abends will occur
53
DSN1COPY
Areas to watch out for:
XML
Data-dependent information kept in catalog table XMLSTRINGS
Cannot DSN1COPY XML table space from one subsystem/group to
another
DSN1COPY within a subsystem/group is fine
Solution is UNLOAD/LOAD/CROSSLOADER
54
Summary
Stay reasonably current on versions and maintenance
Understand what this gives you in terms of utility capability
Revisit your existing utility jobs to benefit from new options
55