SAP Note

    1971776 - High Usage of Paging Space on AIX  

Version   2     Validity: 11.02.2014 - active   Language   English (Master)

Header Data
Released On 18.02.2014 09:25:25
Release Status Released for Customer
Component BC-OP-AIX IBM AIX
Priority Recommendations / Additional Info
Category Help for error analysis

Symptom
Your AIX LPAR continues to allocate an increasing amount of paging space. However, from the application configuration (SAP and DB instance),
you would expect a limited amount of paging space usage, unlike the high increasing growth observed.

Other Terms
Paging Space, Swap Space, High Usage, High Allocation, AIX optimization features

Reason and Prerequisites

AIX uses a paging space related optimization mechanism, which is meant to avoid expensive disk writes during page out operations. When a
process page that has been paged out earlier from RAM to paging space is later getting paged back into RAM due to a READ request, it is
copied from paging space to a page in main memory, but it's corresponding page in paging space will NOT be cleaned up.

The purpose is that if the same page needs to be paged out again, and if the corresponding page in memory hasn't changed in the meantime,
nothing needs to be written onto the paging space except that the already existing 'duplicate page' there needs to be marked as active. This helps
to avoid disk write operations and to improve performance. Duplicates will be cleaned up if the owning process terminates or if the paging space 
hits a low watermark.

Unfortunately currently there exists no direct means to distinguish between the paging space allocation due to 'really paged out' pages versus
'duplicate' pages. The 'lsps -s' command only shows the overall allocation.

Solution
In case of unexpected high paging space allocation and its subsequent growth for no apparent reason, the following may be evaluated to get clarity
whether these are 'real demands' or whether the apparent growth can be attributed to 'duplicates' pages.

l Identify the actual demand of applications/processes and of the operating system: The value 'active virtual memory', shown e.g. by 'vmstat' 
command under 'avm' or by 'svmon -G' command under 'memory/virtual'
l Identify the paging space used: 'svmon -G' command under 'pg space/inuse'
l Identify the size of your main memory / RAM-size: 'svmon -G' command under 'memory/size'
l Take into account typically 3% (precise value being vmo parameter minperm%)  of main memory for file cache. 

=> The formula 'avm - (0,97 * RAM-size)' gives you the 'real paging space demands'.

=> The paging space allocation beyond that gives roughly the amount of 'duplicate pages'. Thereby take into account that a small amount of 1-
2% of paging space will always be used for AIX administrative purposes.

If you want to clean up all current duplicates, giving up the potential performance benefits, to get 100% clarity about your current real paging space
demands, following procedure would do it.

Please be aware that the procedure implies a performance risk to running systems, so use with care, especially for productive environments.

Procedure:

l Add another paging space additional to the existing, with same size, and activating, via 'mkps' and 'swapon'. 
l Deactivate the original paging-space via 'swapoff', so that only the new one remains; this leads to all 'real-required pages' be moved over, 
and all 'duplicate pages' cleaned up.
l Activate the original paging space again, deactivate the additional, which will move-over back the 'real-required pages'.
l Remove the additional paging-space.
l In case that your paging-space is comprised of multiple paging-space files, the procedure goes quite analog, only you create multiple
additional paging-spaces with corresponding sizes.

Validity
This document is not restricted to a software component or software component version

References
This document is referenced by:
SAP Notes (1)
1972803   SAP on AIX: Recommendations