Professional Documents
Culture Documents
RAC Cheatsheet
RAC Cheatsheet
This is a quick and dirty cheatsheet on Oracle RAC 10g, as my experience with RAC grows I will update this section, below is a beginners
guide on the commands and information that you will require to administer Oracle RAC.
Acronyms
Acronyms
GCS
in memory database containing current locks and awaiting locks, also known as PCM
GES
coordinates the requests of all global enqueues uses the GCS, also known as non-PCM
GRD
all resources available to the cluster (formed and managed by GCS and GES), see GRD
for more details
GRM
helps to coordinate and communicate the locks requests between Oracle processes
GSD
runs on each node with one GSD process per node. The GSD coordinates with the
cluster manager to receive requests from clients such as the DBCA, EM, and the
SRVCTL utility to execute administrative job tasks such as instance startup or
shutdown. The GSD is not an Oracle instance background process and is therefore not
started with the Oracle instance
PCM (IDLM)
formly know as (integrated) Distributed Lock Manager, its another name for GCS
Resource
n/a
Resource (Global)
n/a
LVB
TRFC
Traffic Controller
$ORA_HOME/log/<hostname>/client/ocrconfig_<pid>.log
$ORA_CRS_HOME/crs/log
$ORA_CRS_HOME/crs/init
contains trace files for the CRS daemon during startup, a good place to start
$ORA_CRS_HOME/css/log
contains cluster reconfigurations, missed check-ins, connects and disconnects from the
client CSS listener. Look here to obtain when reboots occur
$ORA_CRS_HOME/css/init
contains core dumps from the cluster synchronization service daemon (OCSd)
$ORA_CRS_HOME/evm/log
$ORA_CRS_HOME/evm/init
$ORA_CRS_HOME/srvm/log
$ORA_CRS_HOME/log
log fles for Oracle clusterware which contains diagnostic messages at the Oracle cluster
level
Useful Views/Tables
GCS and Cache Fusion Diagnostics
v$cache
v$cache_transfer
contains information from the block headers in SGA that have been pinged at least once
v$instance_cache_transfer
contains information about the transfer of cache blocks through the interconnect
v$cr_block_server
v$current_block_server
v$gc_element
contains one-to-one information for each global cache resource used by the buffer cache
v$lock
contains information about locks held within a database and outstanding requests for locks and latches
v$ges_blocking_enqueue
contains information about locks that are being blocked or blocking others and locks that are known to the lock manager
v$enqueue_statistics
v$resource_limits
v$locked_object
contains information about DML locks acquired by different transactions in databases with their mode held
v$ges_statistics
v$ges_enqueue
GES diagnostics
v$ges_convert_local
v$ges_convert_remote
v$ges_resource
v$ges_misc
v$ges_traffic_controller
v$hvmaster_info
contains information about current and previous master instances of GES resources in relation to hash value ID of resource
v$gcshvmaster_info
v$gcspfmaster_info
conatins information about current and previous masters about GCS resources belonging to files mapped to a particular
master, including the number of times the resource has remastered
v$cluster_interconnects
contains information about interconnects that are being used for cluster communication
v$configured_interconnects
same as above but also contains interconnects that AC is aware off that are not being used
Cluster Interconnect
Miscellanous
v$service
x$kjmsdp
x$kjmddp
Useful Parameters
Parameters
cluster_interconnects
_gcs_fast_config
_lm_master_weight
controls which instance will hold or (re)master more resources than others
_gcs_resources
_lm_tickets
_lm_ticket_active_sendback
_db_block_max_cr_dba
limits the number of CR copies per DBA on the buffer cache (see grd)
_fairness_threshold
used when too many CR requested arrive for a particular buffer and the block becomes disowned (see grd)
_gc_affinity_time
_gc_affinity_limit
defines the number of times a instance access the resource before remastering
_gc_affinity_minimum
defines the minimum number of times a instance access the resource before remastering
_lm_file_affinity
_lm_dynamic_remastering
_gc_defer_time
define the time by which an instance deferred downgrading a lock (see Cache Fusion)
_lgwr_async_broadcast
Processes
Process Monitor
EVMd
OCSSd
Event Management
Cluster Synchronization Services
CRSd
this is the cache fusion part, it handles the consistent copies of blocks that are tranferred
between instances. It receives requests from LMD to perform lock requests. I rools back
any uncommitted transactions. There can be upto ten LMS processes running and can be
started dynamically if demand requires it.
LMSn
they manage lock manager service requests for GCS resources and send them to a service
queue to be handled by the LMSn process. It also handles global deadlock detection and
monitors for lock conversion timeouts.
LMON
this process manages the GES, it maintains consistency of GCS memory in case of process
death. It is also responsible for cluster reconfiguration and locks reconfiguration (node
joining or leaving), it checks for instance deaths and listens for local messaging.
A detailed log file is created that tracks any reconfigurations that have happened.
LMD
this manages the enqueue manager service requests for the GCS. It also handles deadlock
detention and remote resource requests from other instances.
LCK0
manages instance resource requests and cross-instance call operations for shared
resources. It builds a list of invalid lock elements and validates lock elements during
recovery.
DIAG
Diagnostic Daemon
This is a lightweight process, it uses the DIAG framework to monitor the healt of the
cluster. It captures information for later diagnosis in the event of failures. It will perform
any neccessary recovery if an operational hang is detected.
General Administration
Managing the Cluster
starting
stopping
/etc/init.d/init.crs start
crsctl start crs
/etc/init.d/init.crs stop
crsctl stop crs
/etc/init.d/init.crs enable
/etc/init.d/init.crs disable
crsctl enable crs
crsctl disable crs
Note: starts listeners if not already running, you can use the -o option to specify
startup/shutdown options
force
open
mount
nomount
srvctl stop database -d <database> -o <option>
Note: the listeners are not stopped, you can use the -o option to specify startup/shutdown
options
immediate
abort
normal
transactional
status
srvctl
srvctl
srvctl
srvctl
srvctl
status
status
status
status
status
srvctl
srvctl
srvctl
srvctl
srvctl
stop
stop
stop
stop
stop
srvctl
srvctl
srvctl
srvctl
srvctl
start
start
start
start
start
stopping/starting
database -d <database>
instance -d <database> -i <instance>,<instance>
service -d <database>
nodeapps -n <node>
asm -n <node>
database -d <database>
instance -d <database> -i <instance>,<instance>
service -d <database> -s <service>,<service> -i <instance>,<instance>
nodeapps -n <node>
asm -n <node>
database -d <database>
instance -d <database> -i <instance>,<instance>
service -d <database> -s <service>,<service> -i <instance>,<instance>
nodeapps -n <node>
asm -n <node>
adding/removing
srvctl
srvctl
srvctl
srvctl
add
add
add
add
srvctl
srvctl
srvctl
srvctl
srvctl
OCR utilities
log file
$ORA_HOME/log/<hostname>/client/ocrconfig_<pid>.log
ocrcheck
checking
dump contents
export/import
Note: will return the OCR version, total space allocated, space used, free space, location of
each device and the result of the integrity check
ocrdump -backupfile <file>
Note: by default it dumps the contents into a file named OCRDUMP in the current directory
ocrconfig -export <file>
ocrconfig -restore <file>
# show backups
ocrconfig -showbackup
# to change the location of the backup, you can even specify a ASM disk
ocrconfig -backuploc <path|+asm>
backup/restore
# perform a backup, will use the location specified by the -backuploc location
ocrconfig -manualbackup
# perform a restore
ocrconfig -restore <file>
# delete a backup
orcconfig -delete <file>
Note: there are many more option so see the ocrconfig man page
## add/relocate the ocrmirror file to the specified location
ocrconfig -replace ocrmirror '/ocfs2/ocr2.dbf'
add/remove/replace
CRS Administration
CRS Administration
starting
stopping
disabling/enabling
## Oracle 10g R1
/etc/init.d/init.crs [disable|enable]
## Oracle 10g R2
$ORA_CRS_HOME/bin/crsctl [disable|enable] crs
checking
$ORA_CRS_HOME/bin/crsctl
$ORA_CRS_HOME/bin/crsctl
$ORA_CRS_HOME/bin/crsctl
$ORA_CRS_HOME/bin/crsctl
$ORA_CRS_HOME/bin/crsctl
check
check
check
check
check
crs
evmd
cssd
crsd
install -wait 600
status
$ORA_CRS_HOME/bin/crs_stat
create profile
$ORA_CRS_HOME/bin/crs_profile
register/unregister application
$ORA_CRS_HOME/bin/crs_register
$ORA_CRS_HOME/bin/crs_unregister
Start/Stop an application
$ORA_CRS_HOME/bin/crs_start
$ORA_CRS_HOME/bin/crs_stop
Resource permissions
$ORA_CRS_HOME/bin/crs_getparam
$ORA_CRS_HOME/bin/crs_setparam
$ORA_CRS_HOME/bin/crs_relocate
Relocate a resource
Nodes
member number/name
olsnodes -n
olsnodes -l
activates logging
olsnodes -g
display
oifcfg getif
delete
set
starting
gsdctl start
stopping
gsdctl stop
status
gsdctl status
clscfg -install
Oracle Interfaces
clscfg -add
clscfg -delete
clscfg -local
clscfg -concepts
clscfg -trace
help
clscfg -h
cemulto -n
Note: in Oracle 9i the ulity was called "cemutls"
cemulto -w
Note: in Oracle 9i the ulity was called "cemutls"
Node Scripts
Add Node
Delete Node
addnode.sh
Note: see adding and deleting nodes
deletenode.sh
Note: see adding and deleting nodes
Enqueues
displaying statistics
SQL>
SQL>
SQL>
SQL>
column
column
column
column
Messaging (tickets)
ticket usage
select local_nid local, remote_nid remote, tckt_avail avail, tckt_limit limit, snd_q_len
send_queue, tckt_wait waiting from v$ges_traffic_controller;
downconvert
Note: lower the _fairness_threshold if the ratio goes above 40%, set to 0 if the instance is a
query only instance.
Remastering
## Obtain the OBJECT_ID form the below table
SQL> select * from v$gcspfmaster_info;
$ export SRVM_TRACE=true
Disable tracing
$ export SRVM_TRACE=""
Voting Disk
adding
deleting
querying