4/4/24, 4:12 AM Internal Workings of the Oracle RAC Systems

sponsored by (http://www.solarwinds.com/)

Internal Workings of the Oracle RAC Systems

By Richard Niemiec (https://logicalread.com/author/richard-niemiec/) on April
21, 2013

Oracle 11g uses Global Cache Services to coordinate activity. A lock is treated as a held resource.
RAC is a multi-instance database. Multiple instances access the same database concurrently. In
terms of structure, the difference between a RAC instance and a stand-alone Oracle instance is
miniscule. Besides all the usual Oracle background processes, many special processes are
spawned to coordinate interinstance communication and to facilitate resource sharing among
nodes in a cluster. The Oracle documentation goes through all of the processes if you are
interested in knowing more. Here is a brief description of some of the main ones:

ACMS The Atomic Controlfile to Memory Service (ACMS) is an agent on a per-instance basis
that helps to ensure a distributed SGA memory update is globally committed on success
and globally aborted on failure.
LMON The Global Enqueue Service Monitor (LMON) monitors the entire cluster to manage
global enqueues and resources. LMON manages instance and process expirations and the
associated recovery for the Global Cache Service.
LMD The Global Enqueue Service Daemon (LMD) is the lock agent process that manages
enqueue manager service requests for Global Cache Service enqueues to control access to
global enqueues and resources. The LMD process also handles deadlock detection and
remote enqueue requests.
LMSn These Global Cache Service processes (LMSn) are processes for the Global Cache
Service (GCS). RAC software provides for up to ten Global Cache Service processes. The
number of LMSn processes varies depending on the amount of messaging traffic among
nodes in the cluster. The LMSn processes do these things:
Handle blocking interrupts from the remote instance for Global Cache Service
resources.
Manage resource requests and cross-instance call operations for shared resources.
Build a list of invalid lock elements and validate lock elements during recovery.
Handle global lock deadlock detection and monitor lock conversion timeouts.
LCK0 process The Instance Enqueue Process manages global enqueue requests and cross-
instance broadcast. Manages non-cache fusion and library/row cache requests.
Cookie Settings
SolarWindsRMSn RAC management
uses cookies to enhance site processes include
navigation, analyze sitetasks
usagelike
and the
assistcreation of resources as nodes are
added. efforts. By clicking "Accept All Cookies", you agree to our use of
with our marketing
cookies onRSMN
your device in accordance
Remote with our (RSMN)
Slave Monitor Cookie Policy. For more
performs information
remote Accept
instance tasks forAll Cookies
a coordinating
on how we process your data, refer to our SolarWinds Privacy Notice
process.
(https://www.solarwinds.com/privacy)
GTX0-j The Global Transaction Process supports global XA transactions.
Global Cache Service (GCS) and Global Enqueue Service (GES)
https://logicalread.com/oracle-11g-rac-systems-overview-mc02/ 1/6
4/4/24, 4:12 AM Internal Workings of the Oracle RAC Systems

GCS and GES (which are basically RAC processes) play the key role in implementing Cache Fusion.
GCS ensures a single system image of the data even though the data is accessed by multiple
instances. The GCS and GES are integrated components of Real Application Clusters that
coordinate simultaneous access to the shared database and to shared resources within the
database and database cache. GES and GCS together maintain a Global Resource Directory
(GRD) to record information about resources and enqueues. GRD remains in memory and is
stored on all the instances. Each instance manages a portion of the directory. This distributed
nature is a key point for fault tolerance of the RAC.

The coordination of concurrent tasks within a shared cache server is called synchronization.
Synchronization uses the private interconnect and heavy message transfers. The following types
of resources require synchronization: data blocks and enqueues. GCS maintains the modes for
blocks in the global role and is responsible for block transfers between instances. LMS processes
handle the GCS messages and do the bulk of the GCS processing.

An enqueue is a shared memory structure that serializes access to database resources. It can be
local or global. Oracle uses enqueues in three modes: null (N) mode, share (S) mode, and
exclusive (X) mode. Blocks are the primary structures for reading and writing into and out of
buffers. An enqueue is often the most requested resource.

GES maintains or handles the synchronization of the dictionary cache, library cache, transaction
locks, and DDL locks. In other words, GES manages enqueues other than data blocks. To
synchronize access to the data dictionary cache, latches are used in exclusive (X) mode and in
single-node cluster databases. Global enqueues are used in cluster database mode.

Cache Fusion and Resource Coordination

Because each node in a Real Application Cluster has its own memory (cache) that is not shared
with other nodes, RAC must coordinate the buffer caches of different nodes while minimizing
additional disk I/O that could reduce performance. Cache Fusion is the technology that uses
high-speed interconnects to provide cache-to-cache transfers of data blocks between instances
in a cluster. Cache Fusion functionality allows direct memory writes of dirty blocks to alleviate the
need to force a disk write and reread (or ping) of the committed blocks. This is not to say that
disk writes do not occur; disk writes are still required for cache replacement and when a
checkpoint occurs. Cache Fusion addresses the issues involved in concurrency between
instances: concurrent reads on multiple nodes, concurrent reads and writes on different nodes,
and concurrent writes on different nodes.

Oracle only reads data blocks from disk if they are not already present in the buffer caches of
any instance. Because data block writes are deferred, they often contain modifications from
multiple transactions. The modified data blocks are written to disk only when a checkpoint
SolarWinds
occurs.uses cookies
Before I gotofurther,
enhance you
site navigation,
need to be analyze site usage
familiar withand assist of concepts introduced in
a couple
with our marketing efforts. By clicking "Accept All Cookies", you agree to our use of
Oracle 9i RAC: resource modes and resource roles. Because the same data blocks can
cookies on your device in accordance with our Cookie Policy. For more information
on concurrently
how we process exist in multiple
your data, instances,
refer to our SolarWindsthere areNotice
Privacy two identifiers that help to coordinate these
blocks:
(https://www.solarwinds.com/privacy)

https://logicalread.com/oracle-11g-rac-systems-overview-mc02/ 2/6
4/4/24, 4:12 AM Internal Workings of the Oracle RAC Systems

Resource mode The modes are null, shared, and exclusive. The block can be held in
different modes, depending on whether a resource holder intends to modify data or merely
read it.
Resource role The roles are locally managed and globally managed.
Global Resource Directory (GRD) is not a database. It is a collection of internal structures and is
used to find the current status of the data blocks. Whenever a block is transferred out of a local
cache to another instance’s cache, the GRD is updated. The following information about a
resource is available in GRD:

Data Block Identifiers (DBA)

Location of most current versions
Data blocks modes (N, S, X)
Data block roles (local or global)
Past Image

To maintain data integrity, a new concept of past image was introduced in the 9i version of RAC.
A past image (PI) of a block is kept in memory before the block is sent and serves as an indication
of whether it is a dirty block. In the event of failure, GCS can reconstruct the current version of
the block by reading PIs. This PI is different from a CR block, which is needed to reconstruct read-
consistent images. The CR version of a block represents a consistent snapshot of the data at a
point in time.

For example, Transaction-A of Instance-A has updated row-2 on block-5, and later another
Transaction-B of Instance-B has updated row-6 on the same block-5. Block-5 has been
transferred from Instance-A to B. At this time, the past image (PI) for block-5 is created on
Instance-A.

Related Posts
Oracle RAC Cache Fusion Reads and Writes (https://logicalread.com/oracle-11g-rac-cache-
fusion-reads-writes-mc02/)

Monitoring Cache Coherency in an Oracle RAC Environment

(https://logicalread.com/monitoring-cache-coherency-in-oracle-rac-mc04/)

About Oracle RAC Global Cache CR Request Wait Event (https://logicalread.com/oracle-rac-

global-cache-cr-request-wait-event-mc01/)

SolarWinds uses cookies to enhance site navigation, analyze site usage and assist
with our marketing efforts. By clicking "Accept All Cookies", you agree to our use of
Comments
cookies on your device in accordance with our Cookie Policy. For more information
on how we process your data, refer to our SolarWinds Privacy Notice
(https://www.solarwinds.com/privacy)
1. Sumeet says

https://logicalread.com/oracle-11g-rac-systems-overview-mc02/ 3/6
4/4/24, 4:12 AM Internal Workings of the Oracle RAC Systems

January 11, 2018 at 6:06 am (https://logicalread.com/oracle-11g-rac-systems-overview-

mc02/#comment-11086)
A very well written article indeed . Very concise and objective in its approach ..

Reply

Leave a Reply
Comment *

Name *

Email *

Website

Save my name, email, and website in this browser for the next time I comment.

Post Comment

The Experts
Featured Contributor
Ronald Bradford (https://logicalread.com/author/ronald-bradford/)
Founder and CEO, Effective MySQL

SolarWinds uses cookies to enhance site navigation, analyze site usage and assist
with our marketing efforts. By clicking "Accept All Cookies", you agree to our use of
cookies on your device in accordance with our Cookie Policy. For more information
on how we process your data, refer to our SolarWinds Privacy Notice
(https://www.solarwinds.com/privacy)

Follow SolarWinds

https://logicalread.com/oracle-11g-rac-systems-overview-mc02/ 4/6
4/4/24, 4:12 AM Internal Workings of the Oracle RAC Systems

 (https://twitter.com/SolarWinds)  (https://www.facebook.com/SolarWinds) 
(https://www.linkedin.com/company/solarwinds/)  (https://www.youtube.com/user/solarwindsinc)

RSS Feeds
 All (http://logicalread.com/feed)
 Databases (http://logicalread.com/category/databases/feed)
 Systems (http://logicalread.com/category/systems/feed)
 Security (http://logicalread.com/category/security/feed)
 Cloud (http://logicalread.com/category/cloud/feed)
 ITSM (http://logicalread.com/category/itsm/feed)

The Experts
Featured Contributor
Ronald Bradford (https://logicalread.com/author/ronald-bradford/)
Founder and CEO, Effective MySQL

Free Trial – Improve Database Performance by 65%!

Free 14-day trial for Oracle, SQL Server, DB2, Sybase, and databases on VMware
Fast—installs in 20 minutes or less
No agent and no load on monitored servers

Download Free Trial

Follow Us
 
SolarWinds uses cookies to enhance site navigation, analyze site usage and assist
(https://twitter.com/SolarWinds) (https://www.facebook.com/SolarWinds)
with our marketing efforts. By clicking "Accept All Cookies", you agree to our use of

(https://www.linkedin.com/company/solarwinds/) 
cookies on your device in accordance with our Cookie Policy. For more information
(https://www.youtube.com/user/solarwindsinc)
on how we process your data, refer to our SolarWinds Privacy Notice
(https://www.solarwinds.com/privacy)

https://logicalread.com/oracle-11g-rac-systems-overview-mc02/ 5/6
4/4/24, 4:12 AM Internal Workings of the Oracle RAC Systems

Subscribe to
LogicalRead

Sponsored By Contribute articles LogicalRead About SolarWinds

(mailto:logicalread@solarwinds.com?(https://logicalread.com/) (http://www.solarwinds.com/comp
subject=I’d%20like%20to%20contribute%20content!)
SQL Server Privacy Statement
Give feedback (https://logicalread.com/category/databases/sqlserver/)
(http://www.solarwinds.com/legal/
(mailto:logicalread@solarwinds.com?
Oracle Terms of Use
 subject=I’d%20like%20to%20share%20feedback%20on%20LogicalRead)
(https://logicalread.com/category/databases/oracle/)
(https://www.solarwinds.com/lega
Contact us
(https://twitter.com/SolarWinds) (mailto:logicalread@solarwinds.com) DB2
(https://logicalread.com/category/databases/db2/)
SAP ASE
 (https://logicalread.com/category/databases/sybase/)

(https://www.facebook.com/SolarWinds)

(https://www.linkedin.com/company/solarwinds/)

(https://www.youtube.com/user/solarwindsinc)

Facebook

SolarWinds uses cookies to enhance site navigation, analyze site usage and assist
with our marketing efforts. By clicking "Accept All Cookies", you agree to our use of
cookies on your device in accordance with our Cookie Policy. For more information
on how we process your data, refer to our SolarWinds Privacy Notice
(https://www.solarwinds.com/privacy)

https://logicalread.com/oracle-11g-rac-systems-overview-mc02/ 6/6