Oracle Database Appliance High-Availability Bare Metal System

Oracle Database Appliance High-Availability Bare Metal System https://docs.oracle.com/en/engineered-systems/oracle-database-applianc...
Oracle Database Appliance

Technical Architecture
January 2024
Copyright © 2022, 2024 Oracle and/or its a�liates
1 of 27 5/16/2024, 3:33 PM
Copyright Ⓒ 2022, 2024 Oracle and/or its a�liates.
This software and related documentation are provided under a license agreement containing restrictions on use and
disclosure and are protected by intellectual property laws. Except as expressly permi�ed in your license agreement or
allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit,
perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or
decompilation of this software, unless required by law for interoperability, is prohibited.
The information contained herein is subject to change without notice and is not warranted to be error-free. If you �nd
any errors, please report them to us in writing.
If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf
of the U.S. Government, then the following notice is applicable:
U.S. GOVERNMENT END USERS: Oracle programs, including any operating system, integrated software, any
programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial
computer software" pursuant to the applicable Federal Acquisition Regulation and agency-speci�c supplemental
regulations. As such, use, duplication, disclosure, modi�cation, and adaptation of the programs, including any
operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be
subject to license terms and license restrictions applicable to the programs. No other rights are granted to the U.S.
Government.
This software or hardware is developed for general use in a variety of information management applications. It is not
developed or intended for use in any inherently dangerous applications, including applications that may create a risk
of personal injury. If you use this software or hardware in dangerous applications, then you shall be responsible to
take all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle Corporation and
its a�liates disclaim any liability for any damages caused by use of this software or hardware in dangerous
applications.
Oracle and Java are registered trademarks of Oracle and/or its a�liates. Other names may be trademarks of their
respective owners.
Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used
under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo,
and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered
trademark of The Open Group.
This software or hardware and documentation may provide access to or information about content, products, and
services from third parties. Oracle Corporation and its a�liates are not responsible for and expressly disclaim all
warranties of any kind with respect to third-party content, products, and services unless otherwise set forth in an
applicable agreement between you and Oracle. Oracle Corporation and its a�liates will not be responsible for any
loss, costs, or damages incurred due to your access to or use of third-party content, products, or services, except as
set forth in an applicable agreement between you and Oracle.
2 of 27 5/16/2024, 3:33 PM
Oracle Database Appliance High-Availability Bare Metal System
Node 1 Node 2
BUI SDK CLI dcs-admin odaadmcli odaadmcli dcs-admin CLI SDK BUI
oakd oakd
dcs-agent Oracle HAMI sync Oracle HAMI dcs-agent
Repository Repository
dcs-controller MySQL MySQL dcs-controller
VM dbsystems
VM dbsystems VM dbsystems
Oracle Oracle
MySQL CDB HAMI sync HAMI CDB MySQL
DCS GI Repository Repository GI DCS

DBs VMs VMs DBs
OS/Virtual Disks/AFD OS/Virtual Disks/AFD
ACFS ACFS
interconnect
GI/ASM DiskGroups GI/ASM DiskGroups
BM CPU Pools CPU Pools VM CPU Pools VM CPU Pools CPU Pools BM CPU Pools
AFD AFD
OS and KVM OS and KVM
Shared Storage
Notes
Oracle Database Appliance saves time and money by simplifying deployment, maintenance, and support of
database solutions for organizations of every size. Oracle Database Appliance hardware models are optimized to
run Oracle Database Standard Edition and Enterprise Edition.
Following are the components of an Oracle Database Appliance deployment:
UI (User Interface): A web-based application to administer and manage an appliance. It interacts with the DCS
agent application to perform tasks on the appliance.
CLI: Command Line Interface to manage Oracle Database Appliance. It interacts with the DCS agent application to
perform tasks on the appliance.
SDK (Software Development Kit): Oracle Database Appliance Java SDK works with the Oracle Database
Appliance software to interact with the DCS agent. You can develop custom client application using REST APIs to
manage the appliance.
3 of 27 5/16/2024, 3:33 PM
DCS agent: The DCS agent is a REST application that performs various operations on the appliance, such as
provisioning, database lifecycle management, backup and recovery, and storage management.
DCS admin: This application performs lifecycle management operations on the DCS agent, DCS controller,
ODACLI, and other components.
oakd: This process manages the storage components of Oracle Database Appliance.
odaadmcli: The CLI that manages the storage components.
dcs-controller: The application on which the Browser User Interface (BUI) is deployed.
Oracle HAMI : Oracle High Availability Metadata Infrastructure service providing distributed services required by
DCS including locking and synchronizing con�guration details in the cluster.
repository: The �le system that holds the images and clone �les used by both bare metal Oracle Database
Appliance system and VM DB system.
MySQL: The metadata store used by DCS agent.
DBs: Oracle Databases managed by Oracle Database Appliance software.
VMs: The KVM guest machines, also known as application VMs that are managed partially by Oracle Database
Appliance. The Oracle Database Appliance software manages basic operation of the VM, such as start, stop, create,
delete, and failover. Oracle Database Appliance does not manage the applications in the VM.
ACFS: Oracle ACFS �le system used for VM storage, Oracle Database homes, and Oracle Database storage.
GI/ASM disk groups: Oracle Grid Infrastructure and ASM disk groups, such as DATA and RECO.
BM CPU Pools: Use the BM CPU pool to ensure that the database on a bare metal host runs exclusively on the CPU
lists speci�ed in the CPU pool. All Oracle processes are bound to the CPUs in this CPU pool and run only on these
CPUs. The bare metal CPU pool is assigned to the database by updating the init.ora parameter
processor_group_name.
VM CPU Pools: Use the VM CPU pools to cage and manage CPU resource allocations to workloads for virtual
machines. Workloads are isolated by creating CPU pools and assigning virtual machines to a speci�c CPU pool.
When you pin a virtual machine to a CPU pool, you ensure that the virtual machine uses CPUs in only that CPU
pool.
AFD: Oracle ASM Filter Driver, a kernel module that resides in the I/O path of the Oracle ASM disks. Oracle ASM
uses the �lter driver to validate write I/O requests to Oracle ASM disks.
OS and KVM: This represents the operating system and KVM Hypervisor.
VM dbsystems: The fully-managed KVM DB System. The DB system contains its own DCS agent, MySQL
metadata, Oracle HAMI, Oracle Grid Infrastructure, and Oracle Database. There is no Oracle ASM instance on DB
system. The DB system repository is an NFS mount from the repository on the bare metal system. Oracle Database
software fully automates the creation of the DB system, including the VM, Oracle Grid Infrastructure, and database
creation, patching of the system and database, and deletion of the system.
Shared Storage: The HDD and SSD storage are shared by both nodes on an Oracle Database Appliance high-
4 of 27 5/16/2024, 3:33 PM
availability model. On an Oracle Database Appliance single-node system, NVME drives provide the storages. These
storages are used by Oracle ASM. The shared storages are passed to the VM DB systems using the virt-scsi
protocol.
Related Resources
• Oracle Database Appliance Documentation (h�ps://docs.oracle.com/en/engineered-systems/oracle-

database-appliance/index.html)
5 of 27 5/16/2024, 3:33 PM
Oracle Database Appliance ODACLI
REST over HTTPS DCS Agent
Credentials
Request Validator
odacli
Response
Process
request
Read/store server certi�cates Read credentials/token Validate credentials/token
Java Truststore Oracle PKI wallet
Notes
Use the Oracle Database Appliance command line interface (ODACLI) to perform operations on the DCS agent.
Oracle Database Appliance establishes communication to the DCS agent is established using REST over HTTPS
with the local DCS agent. The credentials and tokens are stored on an Oracle PKI wallet and the public certi�cates
are stored on a Java (PKCS12) TrustStore. Once the secured communication is established correctly, the operation
is invoked and the results are printed to the standard output.
Use the --json/-j option for any command to print the output in JSON format, which is in a machine-readable
form. For advanced interaction with the DCS agent, use Oracle Database Appliance REST APIs. In case of any issue
during the operation, either on server or client side, use the respective logs to view the issues.
To query available operations, use odacli -h. To view available operations on certain resource types, use odacli
resource -h.
Related Resources
• Oracle Database Appliance Command-Line Interface (h�ps://www.oracle.com/pls/topic/lookup?ctx=en/

engineered-systems/oracle-database-appliance&id=CMTXN-GUID-80C91680-BAE4-4101-9E02-
DF0783FE29F1)
• Oracle Database Appliance Command-Line Interface for KVM (h�ps://www.oracle.com/pls/topic/lookup?

ctx=en/engineered-systems/oracle-database-appliance&id=CMTXN-GUID-A2869D8B-15DA-48CC-
A4C0-55014ECEFDCE)
6 of 27 5/16/2024, 3:33 PM
Oracle Database Appliance Browser User Interface (BUI)
Frontend Backend
Collect Data Request
Display Results Response

Users
What the User Sees DCS Controller
and Interacts with
Rest API call

Request
Response
HTML, CSS, JavaScript DCS Agent
JSON data
Notes
The Oracle Database Appliance Browser User Interface (BUI) is a web-based application to administer and manage
the appliance. The BUI uses Oracle Javascript Extension Toolkit (JET) framework (h�p://oraclejet.org (h�p://
oraclejet.org) ) which supports the Model-View-ViewModel (MVVM) architectural design pa�ern.
The Oracle JET framework provides a set of UI components with built-in accessibility support. To support the
MVVM design, Oracle JET is built upon a modular framework that includes a collection of third-party libraries and
Oracle-provided �les, scripts, and libraries.
DCS Controller: When you access the BUI URL, the browser sends an HTTP request to the DCS controller, to send
a copy of the web application. The request consists of a line indicating a method such as GET or POST, a Uniform
Resource Identi�er (URI) indicating which resource is being requested, and an HTTP protocol version.
The index.html page is the �rst page to download and becomes the blueprint by which the web application is
constructed. As the browser reads the HTML code, it sends out more requests to the DCS controller to send more
pieces of the page. The browser caches static assets, that is, the parts of a website that do not change from visit to
visit, such as images, HTML, CSS, and JavaScript. Once the request is processed successfully, the login page is
rendered on the browser.
The BUI code that runs on the client side makes a REST API call to the DCS agent for all requests and processes the
results that are sent back. The DCS agent is managed outside of the DCS controller and runs on a separate web
7 of 27 5/16/2024, 3:33 PM
server. Each request to the DCS agent is made as an asynchronous call. The DCS agent generates a response
based on the REST API call and includes the HTTP protocol version and the HTTP status code. The agent sends
back a response to the browser for the request made. The client-side code then processes the result sent back by
the DCS agent and presents it on the BUI. The status code and reason phrase tell the client how the server handled
the request.
Generally, the status code 200 is returned, indicating that the request was handled successfully and the body data
contains the requested item. Other result codes indicate HTTP errors, such as 404 Not Found.
Related Resources
• Managing Oracle Databases (h�ps://www.oracle.com/pls/topic/lookup?ctx=en/engineered-systems/oracle-

database-appliance&id=CMTXN-GUID-4EAD207D-41FF-4FD7-8804-DE1B2EB7BE90)
• Patching Oracle Database Appliance (h�ps://www.oracle.com/pls/topic/lookup?ctx=en/engineered-

systems/oracle-database-appliance&id=CMTXN-GUID-24D9EEC6-447C-49DF-BDC0-D44CE7940367)
• About the Browser User Interface (h�ps://www.oracle.com/pls/topic/lookup?ctx=en/engineered-systems/

oracle-database-appliance&id=CMTXN-GUID-375A7CE4-4537-4082-88C0-9AFE5DB8D791)
8 of 27 5/16/2024, 3:33 PM
Oracle Database Appliance KVM and DB System
9 of 27 5/16/2024, 3:33 PM
VM (vm1) VM Storage (vms1)

Applications
Guest OS VM VDisk VM VDisk VM VDisk

(vm1_1) (vm1_2) (vm1_N)
Guest Kernel
vDevices
VM CPU Pool VDisk VDisk VDisk
Virtual CPUs (pool1) (vdisk1_1) (vdisk1_2) (vdisk1_N)
vCpu_0 vCpu_1 vCpu_N 1 33 17 49
2 34 18 50 ACFS �lesystem
VNC/Console
Virtual NICs
ASM/ADVM volume
Bridge vNetwork vNIC_1 vNIC_2 vNIC_N
(vnet1)
Virtual Storage
brvnet1
VM VDisk VDisk VDisk
(vm1_1) (vdisk1_1) (vdisk2_1)
Bridge vNetwork VM (vm2) VM CPU Pool VM Storage (vms2)

(vnet2) (pool2)
Applications
3 35 19 51
Guest OS 4 36 20 52 VM VDisk VM VDisk VM VDisk
(vm2_1) (vm2_2) (vm2_N)
brvnet1
Guest Kernel
VM CPU Pool
Bridge-VLAN vNetwork (poolN)
(vnet3_vlan103) vDevices
5 37 21 53 VDisk VDisk VDisk
Virtual CPUs (vdisk2_1) (vdisk2_2) (vdisk2_N)
6 38 22 54
vCpu_0 vCpu_1 vCpu_N
brvnet3 bond0.103 ACFS �lesystem
Virtual NICs
ASM/ADVM volume
vNIC_1 vNIC_2 vNIC_N
Virtual Storage
Bond VM (vmN)
Physical CPUs VM Storage (vms2)
Applications
Socket 0
Guest OS
NIC1 NIC2 0 1 2 3 32 33 34 35 VM VDisk VM VDisk VM VDisk
(vmN_1) (vmN_2) (vmN_N)
Guest Kernel 4 5 6 7 36 37 38 39
vDevices 8 9 10 11 40 41 42 43
12 13 14 15 44 45 46 47 VDisk VDisk VDisk
Virtual CPUs
(vdiskN_1) (vdiskN_2) (vdiskN_N)
vCpu_0 vCpu_1 vCpu_N
ACFS �lesystem
Virtual NICs Socket 0
vNIC_1 vNIC_2 vNIC_N ASM/ADVM volume
16 17 18 19 48 49 50 51
20 21 22 23 52 53 54 55
Virtual Storage
24 25 26 27 56 57 58 59
28 29 30 31 60 61 62 63
libvirtd/virsh
KVM Hypervisor CRS
Host OS
Host Kernel
Hardware
10 of 27 5/16/2024, 3:33 PM
Notes
Following are the components of an Oracle Database Appliance KVM and DB system:
VM Storage: A VM storage is a central location for storing resources that are essential to create and manage
virtual machines. These resources include ISO �les, virtual machine con�guration �les, and virtual disks. Virtual
storage is con�gured on Oracle ACFS. Oracle Database Appliance KVM uses virtual storage to optimize available
disk space usage in the environment, and for easy reallocation of virtual machines if a physical server fails. Create
the VM storage �rst before creating the VM.
vdisk: Virtual disks provide additional storage options for virtual machines by enabling you to a�ach additional
block storage to your virtual machines. Similarly, you can detach the disk if you no longer need the additional
space.
VM CPU Pool: VM CPU Pools include VM and DB system CPU pools. VM CPU pools are for application VMs, and
DB system CPU pools are for DB systems. Use the VM CPU pools to cage and manage CPU resource for virtual
machines. Workloads are isolated by creating CPU pools and assigning virtual machines to a speci�c CPU pool.
When you pin a virtual machine to a CPU pool, you ensure that the virtual machine uses CPUs from only that CPU
pool. CPU pools cannot share CPUs. But one CPU pool can be assigned to multiple VMs or DB systems.
VMs: The KVM guest machines or application VMs on Oracle Database Appliance. The Oracle Database Appliance
software manages basic operations of the VM such as start, stop, create, delete, and failover, but it does not
manage applications in the VM. Oracle Database Appliance supports all VM operating systems �avors that Oracle
KVM supports.
VM DB system: The fully-managed KVM DB System. Oracle Database Appliance software fully automates the
creation of the DB system including the VM, Oracle Grid Infrastructure and Oracle Database creation, patching of
the system and database, and deletion of the system. KVM database systems enable hard partitioning for Oracle
Database licensing, where each KVM database system has its own CPU pool that is automatically assigned during
KVM database system creation.
vnetwork: Oracle Database Appliance KVM virtual network supports two type of networks, bridged and bridged-
vlan. In a bridged network, a Linux bridge is created and the network interface or bond interface, is a�ached to the
bridge. In a bridged-vlan network, a Linux bridge is created and a VLAN is a�ached to the bridge. VLAN can be
created on all available public interfaces, including the interface where public network is already con�gured.
Related Resources
• Managing an Oracle Database Appliance KVM Deployment (h�ps://www.oracle.com/pls/topic/lookup?

ctx=en/engineered-systems/oracle-database-appliance&id=CMTXN-GUID-7123ADEA-7134-42E8-95CC-
E2F444DBAC27)
• Migrating from Virtualized Platform to KVM on Oracle Database Appliance (h�ps://www.oracle.com/pls/

topic/lookup?ctx=en/engineered-systems/oracle-database-appliance&id=CMTXN-GUID-B5084F8C-
C5ED-4B38-8FA9-9DAB3B5780ED)
• Oracle Database Appliance Command-Line Interface for KVM (h�ps://www.oracle.com/pls/topic/lookup?

ctx=en/engineered-systems/oracle-database-appliance&id=CMTXN-GUID-A2869D8B-15DA-48CC-
A4C0-55014ECEFDCE)
• Managing CPU Pools on Oracle Database Appliance (h�ps://www.oracle.com/pls/topic/lookup?ctx=en/
11 of 27 5/16/2024, 3:33 PM
engineered-systems/oracle-database-appliance&id=CMTXN-
GUID-185A01BF-5A4B-4B38-8EFF-8FA635253DC1)
12 of 27 5/16/2024, 3:33 PM
Oracle Database Appliance Backup and Recovery
Recovery
Recovery Restore from backup
Backup Backup
Backups at Backups at
Objectstore NFS
ODA ODA
Backup Backupreport
Recovery
-dbname
-pitrscn
Backup
Databases Databases at FRA Databases -backuplocation
Objectstore Disk NFS

Backupcon�g Backupcon�g Backupcon�g
Copy Restore
-destination -destination -destination backupreport from backup
-recovery window -recovery window -recovery window
-container
Objectstoreswift Backup Backupreport

-tenancy Schedule
-endpoint URL -id -dbname
-user name -name -pitrscn
-password -cron expression -backuplocation
MySQL DB Database
Notes
Oracle Database Appliance supports three types of backup con�guration:
• Backup to an Oracle Fast Recovery Area (FRA) disk (Internal FRA)
• Backup to Oracle Cloud Infrastructure Object Storage (Oracle Object Storage)
• Backup to a Network File System (NFS) location (External FRA)
You can use the backup, restore, and recovery features with databases that have backup policy a�ached to them in
the system. You can choose to manage your backups, and set up your backup, restore, and recovery options.
ObjectStoreSwift object must be created before creating the Object Store backup con�guration. When a database
is modi�ed with either Object Store or Disk or NFS backup con�guration, Database backup and archive log backup
schedules are created for that database. The database can be restored only from Object Store and NFS backups.
Corresponding backup report must be copied manually to the system where the database needs to be restored.
Related Resources
• Backup, Restore and Recover Databases (h�ps://www.oracle.com/pls/topic/lookup?ctx=en/engineered-

systems/oracle-database-appliance&id=CMTXN-GUID-D18FA32D-14A7-4C33-BA9D-2CB04452A3B4)
• odacli Backup and Recovery Commands (h�ps://www.oracle.com/pls/topic/lookup?ctx=en/engineered-

systems/oracle-database-appliance&id=CMTXN-GUID-802FDC91-CEFD-4C6C-BF08-D06664CF58EF)
13 of 27 5/16/2024, 3:33 PM
Oracle Database Appliance Oracle Data Guard
Primary Oracle Database Standby Oracle Database

Appliance machine Appliance machine
Data Guard Con�guration
Oracle Net
Primary Oracle Data Oracle Data Primary

Database Guard Broker Guard Broker Database
Log transport
DCS Agent DCS Agent
DCS CLI DCS CLI
Notes
Oracle Database Appliance provides client interface through ODACLI commands for easy con�guration and
management of Oracle Data Guard for high availability, data protection, and disaster recovery. Oracle Data Guard
Broker is a distributed management framework that automates the creation, maintenance, and monitoring of
Oracle Data Guard con�gurations.
An Oracle Data Guard con�guration set up with ODACLI commands consists of one primary database and one
standby database. The primary system is the Oracle Database Appliance system that hosts the primary database.
The standby system is the Oracle Database Appliance system that hosts the standby database.
Each system runs its own ODACLI which can call operations on the DCS agent. The primary DCS CLI can call
operations on primary and standby DCS agent for the odacli configure-dataguard command. It
orchestrates a sequence of jobs on two hosts to complete Oracle Data Guard con�guration on Oracle Database
Appliance. The DCS agent connects with Oracle Data Guard Broker and databases to manage Oracle Data Guard
con�guration on Oracle Database Appliance.
14 of 27 5/16/2024, 3:33 PM
Related Resources
• Using Oracle Data Guard for Disaster Management and Recovery on Oracle Database Appliance (h�ps://
www.oracle.com/pls/topic/lookup?ctx=en/engineered-systems/oracle-database-appliance&id=CMTXN-
GUID-5AEABC5A-7FCE-4F49-B80A-D09FCA21D3F6)
• odacli Oracle Data Guard Commands (h�ps://www.oracle.com/pls/topic/lookup?ctx=en/engineered-

systems/oracle-database-appliance&id=CMTXN-GUID-C48EE862-7037-4950-BB40-A1DDC3058CDA)
15 of 27 5/16/2024, 3:33 PM
Oracle Database Appliance Network
Public Network Switch
Oracle Database Appliance Oracle Database Appliance

Node 0 Node 1
Port 1 bond bond Port 1
Public Public
Network vlan vlan Network
Port 2 Port 2
Port 1 bond pubnet pubnet bond Port 1
Public Public
Network vlan vlan Network
Port 2 Port 2
Port 1 bond Private Interconnect bond Port 1
Private Private
Network Network
Port 2 Port 2
Notes
Oracle Database Appliance has a �xed network con�guration on Oracle Database Appliance X7-2 and earlier. The
public network ports are from the onboard network card. All network bonding setup are con�gured during
imaging.
On Oracle Database Appliance X8-2 and later, the network set up is not �xed. The public network ports are from
PCI network card. Each PCI slot can be installed with either two ports 25G or four ports 10G network cards.
Additional network cards can be added into the system after the system deployment.
Public network: Public bonding interfaces are created during imaging of the system.
Private network: For Oracle Database Appliance high-availability systems, there is a private network card for the
interconnection between two nodes. This network card can be Ethernet ports or In�niBand ports on Oracle
Database Appliance X6-2 and X5-2. A bonding interface is created over the two ports for communcation between
the two nodes. When the node number for the system is con�rmed, the private IP address for the node is assigned
automatically.
For DB system support on the Oracle Database Appliance system, create a network bridge on the public network
so that it can be shared by multiple KVM systems. During the deployment of the system, a network bridge
16 of 27 5/16/2024, 3:33 PM
(pubnet) is created on the selected public network interface. You can use the pubnet to connect to the appliance.
Related Resources
• Managing Networks (h�ps://www.oracle.com/pls/topic/lookup?ctx=en/engineered-systems/oracle-

database-appliance&id=CMTXN-GUID-460BC374-FE8C-4F16-9C47-C6D44DE95DAD)
• Readying Oracle Database Appliance for Deployment (h�ps://www.oracle.com/pls/topic/lookup?ctx=en/

engineered-systems/oracle-database-appliance&id=CMTXN-GUID-5F28BB08-9051-469F-9354-
D53D6B69C416)
17 of 27 5/16/2024, 3:33 PM
Oracle Database Appliance High Availability Storage
Node 0 Node 1
EM Plugin ODAADMCLI System, Processor, Memory, CSSD Inter-node communication CSSD

Network, Power, Enclosure, FAN
XML Con�g Cluster Log Cluster

DCS Agent Files ILOM RE con�g ASR RE con�g
Yes
Con�g Resource Cluster Disk Failure

Server Monitor Sync Code Check
OAKLIB OAKD
(Follower)
h�ps REST API CALLS
Interface
Cluster
Sync
OAKD (Leader)
Storage asmappl.con�g
Storage DiskEvent
Module Monitor
Disk O�ine/Online Event
Disk Poor Disk

Storage Kernel Performance Predictive
Event Events (diskstats) Failure
Check
Storage and
ASM ASM Disk Hardware
Action Management
Library
Host Operating System (OS)
. . . . . . . . . . .
Shared Storage (JBOD)
Notes
OAKD is the storage management system for Oracle Database Appliance. It provides storage discovery, auto
con�guration of storage, and disk management.
It has three modes: non-cluster, upgrade, and foreground. OAKD starts in non-cluster when Oracle Grid
Infrastructure stack is not yet installed. After installation of Oracle Grid Infrastructure, OAKD starts in cluster mode.
Use odaadmcli commands to query information from OAKD or provide information to OAKD. It has various
18 of 27 5/16/2024, 3:33 PM
modules such as server con�guration, storage management, resource monitoring, cluster synchronization, and
disk events monitoring. When the appliance is started, OAKD uses the storage and hardware management library
to discover storage. Con�guration servers read XML con�guration �les describing various properties of various
OAKD resources. OAKD provides storage con�guration details from its metadata �le to Oracle ASM though an
interface library oaklib which also makes secure REST call to DCS agent. The resource management module
monitors various hardware components at regular intervals. Cluster communication modules waits for cluster
recon�guration events to take appropriate actions. OAKD monitors storage events for disk operations such as
insert or removal and completes the action in OAKD and Oracle ASM.
OAKD monitors for predictive failure information on disk. OAKD also reads system statistics periodically for disk
performance matrix. OAKD monitor for Oracle ASM action events such as disk o�ine or online events. It also
performs auto online operation for transient failures which can cause Oracle ASM to o�ine a disk. Other modules
such as DCS and Oracle Enterprise Manager plugin for Oracle Database Appliance use the odaadmcli interface to
query storage information.
Related Resources
• Managing Storage (h�ps://www.oracle.com/pls/topic/lookup?ctx=en/engineered-systems/oracle-database-

appliance&id=CMTXN-GUID-1EEE82F1-BEED-4FE7-AC4C-3891441483A9)
• odacli Storage Commands (h�ps://www.oracle.com/pls/topic/lookup?ctx=en/engineered-systems/oracle-

database-appliance&id=CMTXN-GUID-DD417B37-DA21-404E-9A5B-D8C54A85B627)

D53D6B69C416)
19 of 27 5/16/2024, 3:33 PM
Oracle Database Appliance Disk Partnership on High-Availability

Storage
Oracle Database Appliance High-Availability Disk Partnership
Node 0 Node 1
HBA 0 HBA 1 HBA 0 HBA 1
P0 P1 P0 P1 P0 P1 P0 P1
P0 P1 P2 P3 P0 P1 P2 P3
IOM 0 (Expander) IOM 1 (Expander)
S20 S21 S22 S23
S16 S17 S18 S19
S12 S13 S14 S15
S8 S9 S10 S11
S4 S5 S6 S7
S0 S1 S2 S3
Notes
Oracle ASM supports �xed partnership for Oracle ASM disks. Oracle ASM performs spli�ing and mirroring of data
across all disk partners. Oracle ASM supports either four or eight disk partners for Oracle Database Appliance
high-availability. Oracle Database Appliance X6-2 and X5-2 support four disks partnership.
Oracle Database Appliance X7-2 and later models have an eight disk partnership, if there are a minimum of eight
disks. If there are not enough disks, then each disk partners with all others. For example, Oracle Database
Appliance X8-2 with six SSDs does not have eight disks to maintain eight disks partnership, hence all six disks
partner among themselves.
For two JBOD systems, disk partnerships are across the JBOD boundary. It means that any disk in the �rst JBOD
partners with eight disks from second JBOD and vice versa.
20 of 27 5/16/2024, 3:33 PM
Disk partnership information is a part of the asmappl.con�g �le.
The disk partnership on Oracle Database Appliance high-availability system is as follows:
DATA Disk S0 partners with S1, S2, S3, S4, S11, S12, S13, S14
RECO Disk S0 partners with S4, S5, S6, S7, S8, S9, S10, S11
If disk in slot 0 is lost for any reason, then Oracle ASM restores and rebalances data on lost disks among the
partners of the lost disk.
If disk in slot 0 is lost for any reason, Oracle ASM restores and rebalances data on lost disks among the partners of
lost disk.
Related Resources



D53D6B69C416)
21 of 27 5/16/2024, 3:33 PM
Oracle Database Appliance Single-Node Storage
System, Processor, Memory, Log

EM Plugin ODAADMCLI CSSD
Network, Power, Enclosure, FAN ASR
XML Con�g Cluster Yes

DCS Agent ILOM RE con�g
�les
Con�g Resource Cluster Critical

Server Monitor Sync Warning
OAKLIB Set?
h�ps REST API CALLS

Interface
OAKD (Leader)
Storage asmappl.con�g
Storage DiskEvent
Module Monitor
Disk O�ine/Online Event
Storage and
ASM DISK Kernel Disk Disk Poor Critical
ASM ACTION Hardware Udev performance Warning
(SQL*Plus) Management Event (diskstats) Check
Library
Host Operating System (OS)
. . . . . . . . . . .
NVMe Disk NVMe Disk NVMe Disk
NVMe Disk NVMe Disk
Notes
On an Oracle Database Appliance single-node server, OAKD is always in the leader role. Oracle Database Appliance
single-node systems do not have SAS storage, hence hardware library is not used. Storage and Hardware
Management Library is used for storage discovery.
There are no SAS HBA (Controller) and hence hardware events noti�cation is available. UDEV events noti�cation
are used for disk hot plugs. Instead of predictive failure checks, NVMe disks uses critical warning check using OAK
NVMe interface.
Related Resources
22 of 27 5/16/2024, 3:33 PM



D53D6B69C416)
23 of 27 5/16/2024, 3:33 PM
Oracle Database Appliance Disk Partnership on Single-Node Storage
Notes
Oracle ASM supports �xed partnership for disks. Oracle ASM performs spli�ing and mirroring of data across all
disk partners. All Oracle Database Appliance single-node systems support 8 disks partnership provided there are
enough disks to support it. If the number of NVMe disks are less than eight then all disks partner among
themselves.
NVMe disks are populated across the PCI switch card for equal distribution of IO bandwidth across IO sub-systems.
QRM1 and QRM2 have 1GB partitioned carved out from the operating system disk. They are used as DATA and
RECO quorum disk respectively in Oracle ASM. These disk are used if the number of NVMe disks is less than 5.
They are used for storing PST and voting �les. They are used when a disk with voting �le or PST get lost. Their
content helps to decide the majority without which the Oracle ASM disk group cannot be mounted.
The disk partnership information is part of the asmappl.con�g �le.
The disk partnership on Oracle Database Appliance single-node system is as follows:
DATA Disk S0 partners with S2, S3, S4, S5, S8, S9, S10, S11
RECO Disk S0 partners with S2, S3, S4, S5, S8, S9, S10, S11
If disk in slot 0 is lost for any reason, Oracle ASM restores and rebalances data on lost disks among the partners of
lost disk.
Related Resources
24 of 27 5/16/2024, 3:33 PM



D53D6B69C416)
25 of 27 5/16/2024, 3:33 PM
Oracle Database Appliance Multi-User Access Architecture
Oracle Database Appliance Multi-User Access Architecture
ODACLI AUTHENTICATION SSO WALLET
BUI REST API AUTHORIZATION MySQL DB

INTERFACE
User-Resource
Mappings
User1 -
DB Storage1,
DB Home1, DB1
User2 -
RESOURCE DB Storage2,
ODA JAVA SDK OWNERSHIP & DB Home2, DB2
ACCESS check
Notes
On an Oracle Database Appliance multi user access-enabled system, an Oracle Database Appliance user runs a
REST call on the DCS agent using an ODACLI command, or a Browser User Interface (BUI) operation, or an Oracle
Database Appliance Java SDK call.
Within the DCS agent, the REST API layer receives this request and makes a call to the authentication module of
the Oracle Database Appliance multi-user access framework. The authentication module of the multi-user access
framework authenticates the credentials of the REST caller against the SSO wallet.
After successful authentication, the REST API layer makes a call to the authorization module of the Oracle
Database Appliance multi-user access framework. The authorization module of the multi-user access framework
checks if the user is authorized to perform this operation by checking against the user role and entitlements-
related information stored in the MySQL metadata of the multi-user access framework.
After successful authorization, the REST API layer makes a call to the Resource Ownership and Access module of
the multi-user access framework. The Resource Ownership and Access module checks against the resource
ownership information stored in the MySQL metadata of the multi-user access framework to determine if the user
has the required access privileges to perform lifecycle management operations on the speci�ed resource. If the
Resource Ownership and Access check is successful, then the REST API layer runs the REST endpoint and displays
the results. Otherwise, an error message to this e�ect is displayed.
26 of 27 5/16/2024, 3:33 PM
Related Resources
• Implementing Multi-User Access on Oracle Database Appliance (h�ps://www.oracle.com/pls/topic/lookup?

ctx=en/engineered-systems/oracle-database-appliance&id=CMTXN-GUID-D3A80446-
BA06-4E8B-9254-24D974616000)
• Oracle Database Appliance X9-2-HA Hardware Model (h�ps://docs-uat.us.oracle.com/en/engineered-

systems/oracle-database-appliance/19.15/cmtnz/F42007_01.pdf)
• Oracle Database Appliance X9-2S/X9-2L Hardware Model (h�ps://docs-uat.us.oracle.com/en/engineered-

systems/oracle-database-appliance/19.15/cmtnf/F42005_01.pdf)
27 of 27 5/16/2024, 3:33 PM

Oracle Database Appliance High-Availability Bare Metal System

Uploaded by

Copyright:

Available Formats

You might also like

Oracle Database Appliance High-Availability Bare Metal System

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Oracle Database Appliance High-Availability Bare Metal System

Uploaded by

Copyright:

Available Formats

Oracle Database Appliance High-Availability Bare Metal System https://docs.oracle.com/en/engineered-systems/oracle-database-applianc...

Oracle Database Appliance

Copyright Ⓒ 2022, 2024 Oracle and/or its a�liates.

Oracle Database Appliance High-Availability Bare Metal System

dcs-agent Oracle HAMI sync Oracle HAMI dcs-agent

DCS GI Repository Repository GI DCS

OS and KVM OS and KVM

Following are the components of an Oracle Database Appliance deployment:

odaadmcli: The CLI that manages the storage components.

MySQL: The metadata store used by DCS agent.

DBs: Oracle Databases managed by Oracle Database Appliance software.

• Oracle Database Appliance Documentation (h�ps://docs.oracle.com/en/engineered-systems/oracle-

Oracle Database Appliance ODACLI

REST over HTTPS DCS Agent

Read/store server certi�cates Read credentials/token Validate credentials/token

Java Truststore Oracle PKI wallet

• Oracle Database Appliance Command-Line Interface (h�ps://www.oracle.com/pls/topic/lookup?ctx=en/

• Oracle Database Appliance Command-Line Interface for KVM (h�ps://www.oracle.com/pls/topic/lookup?

Oracle Database Appliance Browser User Interface (BUI)

Collect Data Request

Display Results Response

Rest API call

• Managing Oracle Databases (h�ps://www.oracle.com/pls/topic/lookup?ctx=en/engineered-systems/oracle-

• Patching Oracle Database Appliance (h�ps://www.oracle.com/pls/topic/lookup?ctx=en/engineered-

• About the Browser User Interface (h�ps://www.oracle.com/pls/topic/lookup?ctx=en/engineered-systems/

Oracle Database Appliance KVM and DB System

VM (vm1) VM Storage (vms1)

Guest OS VM VDisk VM VDisk VM VDisk

Bridge vNetwork VM (vm2) VM CPU Pool VM Storage (vms2)

VM VDisk VDisk VDisk

KVM Hypervisor CRS

• Managing an Oracle Database Appliance KVM Deployment (h�ps://www.oracle.com/pls/topic/lookup?

• Migrating from Virtualized Platform to KVM on Oracle Database Appliance (h�ps://www.oracle.com/pls/

• Oracle Database Appliance Command-Line Interface for KVM (h�ps://www.oracle.com/pls/topic/lookup?

• Managing CPU Pools on Oracle Database Appliance (h�ps://www.oracle.com/pls/topic/lookup?ctx=en/

Oracle Database Appliance Backup and Recovery

Objectstore Disk NFS

Objectstoreswift Backup Backupreport

Oracle Database Appliance supports three types of backup con�guration:

• Backup to an Oracle Fast Recovery Area (FRA) disk (Internal FRA)

• Backup to Oracle Cloud Infrastructure Object Storage (Oracle Object Storage)

• Backup to a Network File System (NFS) location (External FRA)

• Backup, Restore and Recover Databases (h�ps://www.oracle.com/pls/topic/lookup?ctx=en/engineered-

• odacli Backup and Recovery Commands (h�ps://www.oracle.com/pls/topic/lookup?ctx=en/engineered-

Oracle Database Appliance Oracle Data Guard

Primary Oracle Database Standby Oracle Database

Data Guard Con�guration

Primary Oracle Data Oracle Data Primary

DCS Agent DCS Agent

DCS CLI DCS CLI

• odacli Oracle Data Guard Commands (h�ps://www.oracle.com/pls/topic/lookup?ctx=en/engineered-

Oracle Database Appliance Network

Public Network Switch

Oracle Database Appliance Oracle Database Appliance

Port 1 bond bond Port 1

Port 1 bond pubnet pubnet bond Port 1

Port 1 bond Private Interconnect bond Port 1

• Managing Networks (h�ps://www.oracle.com/pls/topic/lookup?ctx=en/engineered-systems/oracle-

• Readying Oracle Database Appliance for Deployment (h�ps://www.oracle.com/pls/topic/lookup?ctx=en/