11/22/2021

Oracle Exadata Database Machine

ORACLE DATABASE ARCHITECTURE

Computing Architecture
• Oracle Database is a leading Database product
• The architecture of an Oracle Database is built
on basic Computing Architecture
• Every Computer has many components, out of
them, the 3 main resources are
– CPU
– Memory ( RAM)
– Storage

Computing Architecture
• The 3 main resources of COMPUTER
Computing are shown in
the diagram MEMORY (RAM)
• The characteristics of
these components are CPU
discussed in the next
few slides
STORAGE

1
 11/22/2021

Computing Architecture
STORAGE COMPUTER

• This is the place where files

MEMORY (RAM)
are stored and PERSISTED
• Can be Hard Disks or Flash
based Storage CPU
• Any data to be processed
has to be read into Memory
from Storage STORAGE

Computing Architecture
MEMORY COMPUTER

• To enable the CPU to do

MEMORY (RAM)
processing data has to
be brought into Memory
• The data has to be read CPU
from STORAGE and
brought to MEMORY
STORAGE

Computing Architecture
CPU COMPUTER

• This is the place

MEMORY (RAM)
processing happens
• To process, the data has
to be available in the CPU
Memory

STORAGE

2
 11/22/2021

Computing Architecture
MEMORY COMPUTER

• Changes made in
MEMORY (RAM)
MEMORY have to be
written back to
STORAGE to persist the CPU
changes ( Make them
permanent)
STORAGE

Oracle Database Home

Oracle Home COMPUTER
• A filesystem location or
directory, where Oracle
Software is Installed
• The location is a path in the STORAGE
Oracle_Home
storage filesystem

• e.g. /u01/app/oracle/product/12.1.0/db_home

Oracle Database Home

Multiple Oracle Homes COMPUTER
• Multiple Oracle Homes can
exist in a Computer
STORAGE
• Each Home could be a 12c Oracle_Home
different Version of Oracle
18c Oracle_Home
Software and has a unique
path / location
• e.g. /u01/app/oracle/product/12.1.0/db_home
/u01/app/oracle/product/18.1.0/db_home

3
 11/22/2021

Oracle Database Home

Multiple Databases COMPUTER
• In general, an Oracle Home
can have multiple Databases
created STORAGE
• Each Database has to point DB1 DB2
to an Oracle Home for it to 12c Oracle_Home
know the binaries it has to
DB3 DB4
use
18c Oracle_Home

Oracle Database Architecture

DATABASE COMPUTER
• An Oracle Database is a
collection of files kept in MEMORY (RAM)
Storage
• Important Files are
– Datafiles CPU
– Control Files
DB1 DB2
– Redo Log Files
STORAGE
– Parameter File etc.

Oracle Database Architecture

STORAGE COMPUTER

• Nowadays Database Files MEMORY (RAM)

are not kept in the Local
disks on a Computer CPU
• Specialized storage
servers are used for better Local Storage
Performance and
DB1 DB2
Availability
STORAGE SERVER

4
 11/22/2021

Oracle Database Architecture

STORAGE COMPUTER

• The Database Server is MEMORY (RAM)

connected to the Storage
Server using High Speed CPU
Network connectivity
– Fibre Channels Local Storage
– Infiniband etc.
DB1 DB2
STORAGE SERVER

Oracle Database Architecture

INSTANCE COMPUTER

• The Memory and CPU MEMORY (RAM)

components together is
called as an Instance CPU
• Has 2 components
– SGA – Memory Local Storage
– Processes - CPU
DB1 DB2
STORAGE SERVER

Oracle Database Architecture

INSTANCE - SGA COMPUTER

• The SGA is a memory MEMORY (RAM)

allocation where the data
from the files are brought CPU
into for processing
• It has different Local Storage
components for different
DB1 DB2
types of data
STORAGE SERVER

5
 11/22/2021

Oracle Database Architecture

INSTANCE - Processes COMPUTER
• Various Processes are MEMORY (RAM)
started for each
Database CPU
• The Processes work with
the Data in the Memory
and interact with Storage Local Storage
and Users who are
DB1 DB2
connected STORAGE SERVER

Tablespace and Data Files

• Each Database can have multiple Tablespaces
• Tablespaces are logical Storage Structures
• Each Tablespace can have 1 or more Datafiles
• Each Datafile belongs to one Tablespace only
Database
Tbs1 Tbs2 Tbs3 Tablespaces

DF1 DF2 DF3 DF4 DF5 DF6 Datafiles

CDB vs NON-CDB Architecture

• In Oracle Database 12c, the new Architecture of
CDB – Container Database was introduced
• Starting 12c, Databases can be created with this
new CDB Architecture or the traditional non-CDB
Architecture
• Till 11g R2, the traditional Architecture of non-CDB
was only available

6
 11/22/2021

Non-CDB Architecture
• Each Database has a SYSTEM tablespace which
has all the Oracle Supplied Metadata / Code
• The SYSTEM Tablespace also has User Metadata
for Objects created by the users
• Data(rows) for Tables created by Non-CDB
System
users typically sits in User created Tablespace
Tablespaces
• PL/SQL is in SYSTEM Tbs1 Tbs2 Tbs3
• Each Database has its Instance User Created Tablespaces

CDB Architecture
• There is a ROOT database with System Metadata
and code supplied with Oracle Database Software
• Additionally there are Pluggable Databases used to
contain user / application metadata & code and data
for the objects created by users / applications
CDB
ROOT

PDB1 PDB2 PDB3 PDB4 PDB5

CDB Architecture
• There is a SYSTEM tablespace in ROOT database
with all the Oracle Supplied Metadata / Code
• SYSTEM Tablespace in Pluggable Databases
contain user / application metadata and code for the
objects created by the user / application
System CDB
ROOT Tablespace
System System System
Tablespace Tablespace Tablespace
PDB1 PDB2 PDB3

7
 11/22/2021

CDB Architecture
• Typically each Application has its own PDB
• User Created Tablespaces in each PDB has the
Tables and Data for the Application
• The Metadata for these objects is in the System
Tablespace of the respective PDB
CDB
System
ROOT Tablespace
System System System
Tablespace Tablespace Tablespace
PDB1 Tbs1 PDB2 PDB3
Tbs1 Tbs1

CDB Architecture
• Pluggable Databases refer to the ROOT for the
Oracle Supplied Code and Metadata
• Oracle Database patching happens only in ROOT
and it automatically patches the PDBs also
CDB
ROOT

PDB1 PDB2 PDB3 PDB4 PDB5

CDB Architecture
• The Entire CDB shares the Same Instance
Instance

CDB
ROOT

PDB1 PDB2 PDB3 PDB4 PDB5

8
 11/22/2021

Data Guard Overview

• Data Guard is a Feature of Oracle Database where
a Primary Database can have one or more Standby
Database created

Standby
Database 1

Primary
Database Standby
Database 2

Data Guard Overview

• A Standby Database is an exact Copy of the
Primary Database
• It can be used as a Disaster Recovery mechanism
for the Primary Database

Primary Standby
Database Database 1

Data Guard Overview

• The Primary Database is available for Read and
Write, Applications make changes to the Primary
Application

Primary Standby
Database Database 1

9
 11/22/2021

Data Guard Overview

• The Primary sends the change log to the standby
database and thus the Standby is in Sync
Application

Primary Standby
Database Database 1

Data Guard Overview

• The Standby can be opened Read Only, but by
default the sync stops when it is opened Read Only
Application Read
Application

Primary Standby
Database Database 1

Active Data Guard

• With Active Data Guard, the Standby can be opened
Read Only and be in Sync with the Primary Database
Application Read
Application

Primary Standby
Database Database 1

10
 11/22/2021

Role Transition - Switchover

• Initiated at the Primary database during a planned
maintenance to make the Standby the new Primary

Application No Data Loss

New Standby New Primary

Old Primary Old Standby

Role Transition - Failover

• Initiated at the Standby database to make it Primary if
the original Primary has an Unplanned Downtime
Application Possible
Data Loss

Primary New Primary

Database Old Standby

Protection Modes
• Data Guard configuration has 3 Protection Modes
– Maximum Performance (ASYNC)
– Maximum Availability (SYNC)
– Maximum Protection (SYNC No Data Loss Configuration)
• Configuration will take care of how the Standby is in
Sync with Primary ( and the impact on Failover)

11
 11/22/2021

RAC Overview
• A Database Server has an
Instance for the Database Application
• Applications have access to the
instance only when the DATABASE SERVER
Database server is available
• Also, the Instance is INSTANCE
constrained by the CPU
resources on the Database
Server
STORAGE SERVER

RAC Overview
• RAC takes care of 2 important
Application
aspects
• Scalability
– Ability to take up more load by DATABASE SERVER
having more CPU resources
• High Availability INSTANCE
– Even if a Database Server is
down another server is available
STORAGE SERVER
With Database Files

RAC Overview
• RAC is a Cluster setup of 2 or more Database
servers that connect to the same Database
Application

DATABASE SERVER 1 DATABASE SERVER 2

INSTANCE 1 INSTANCE 2

Database Files

12
 11/22/2021

RAC Overview
• To manage the Cluster, there is a Cluster software
called Grid Infrastructure ( ASM + Clusterware)
Application

DATABASE SERVER 1 DATABASE SERVER 2

INSTANCE 1 INSTANCE 2

Database Files

RAC Overview
• A separate Network called Interconnect is required
between the Database Servers
Application

DATABASE SERVER 1 DATABASE SERVER 2

INSTANCE 1 Interconnect INSTANCE 2

Database Files

RAC - Scalability
• Now with a Cluster of DB Instances available, an
Application can perform more work with the Cluster
Application

DATABASE SERVER 1 DATABASE SERVER 2

INSTANCE 1 INSTANCE 2

Database Files

13
 11/22/2021

RAC - Scalability
• Alternatively, more applications can be deployed to
work with the Cluster of Instances
Application 1 Application 2

DATABASE SERVER 1 DATABASE SERVER 2

INSTANCE 1 INSTANCE 2

Database Files

RAC – High Availability

• Even if one of the Database Servers is down, the
Applications still have access to the Database
Application 1 Application 2

DATABASE SERVER 1 DATABASE SERVER 2

INSTANCE 1 INSTANCE 2

Database Files

RAC – Multiple Databases

• One Cluster can have instances of Multiple
Databases also
Application 1 Application 2

DATABASE SERVER 1 DATABASE SERVER 2

INSTANCE 1 INSTANCE 2

Database Files

14
 11/22/2021

Additional learning
• Oracle Database Architecture (non-CDB) Overview -
https://www.youtube.com/playlist?list=PL3Hc8S1SG2AQnk6Nu57m2JLkWxorMAsmE

• Oracle Database Multi-Tenant (CDB) Architecture

Overview -
• https://www.youtube.com/playlist?list=PL3Hc8S1SG2AQysApdyORoO8AmBTkdhxat

• Oracle ASM -
https://www.youtube.com/playlist?list=PL3Hc8S1SG2ARyNRoWvp7q8SSFFHthSy9j

• Oracle Grid Infrastructure Concepts -

https://www.youtube.com/playlist?list=PL3Hc8S1SG2ASA5pRscvXUMYk6eqXjqYB2

Oracle Exadata Database Machine

STORAGE & ASM FUNDAMENTALS

Moore’s Law

https://en.wikipedia.org/wiki/Moore%27s_law

15
 11/22/2021

Evolution of Storage
• 1956 – IBM RAMAC 305 – 5MB
– Size of a refrigerator
• 1992 – Seagate – 7200 RPM – 2.1 GB
• 1996 – Seagate – 10000 RPM
• 2000 – Seagate – 15000 RPM

• Today the talk is about SSDs / Flash

Understanding Storage
• All data needs to be READ into COMPUTER
Memory to Process
MEMORY (RAM)
• The speed of reading depends on
the Speed of the Disk Subsystem
• Understand your Storage CPU
Specifications before you use…

STORAGE

What do you ask for?

• Space in GB / TB
– This is Storage Space
• How soon you want

• Do you know the performance specifications

– MBPS
– IOPS

16
 11/22/2021

Space vs IOPS vs MBPS

• IOPS
– HDD ~ 250 IOPS
– SSD start at 8600 IOPS and go upto Million IOPS
• MBPS
– HDD MBPS – 260 mbps ( Maximum)
– SSD – 1700MBPS PCIe even more

• Larger the SPACE in a disk … you need to read all

the data with the same speed
• Do you know what your storage LUN is made up of?

Storage
• Used to store data permanently in a
computer
• Also called Secondary Storage
• Today there are 2 types popular
– Hard Disks
– Flash / Non-Volatile Memory

Evolution of Storage – Hard Disks

• For many decades, Storage
meant Hard Disks, which is a
mechanical device whose
performance is determined by
the RPM (Rotations)
• The Threshold for maximum
Rotations has reached… thus
the speed / IO performance of
Hard disk also has peaked

17
 11/22/2021

Hard Disk Connectivity

• To Connect Hard Disks to the
Mother Board the connectivity
options have been

• IDE
• SATA ( 600 MB/sec)
• Which met the I/O capability of
Hard Disks

Evolution of Storage - Flash

• In the Past Decade a new form of
Storage – Flash Storage or NVM
( Non-Volatile Memory) has
become extremely popular
• This new form of storage provides much higher
speeds of I/O capability in comparison to Hard Disks
• They needed a much better interface beyond SATA to
exploit the I/O capabilities

Flash Connectivity
• Flash is available as SSD and as
PCIe slots
• These provides much better I/O
capabilities than Hard Disk
• NVMe is a device specification to
connect Flash Storage using PCIe
• It can provide 2000 MB/sec and
more which enables to exploit the
I/O capabilities of Flash storage

18
 11/22/2021

Storage Components
• Hard Disks
• Flash DRAM

Higher Cost Per GB

• PMEM – Persistent Memory Persistent Mem

Faster
FLASH

PMEM
• PMEM is a new Storage device
• It is Much Faster than Flash
– Comparable to DRAM/RAM/Memory
• Unlike DRAM which is volatile, PMEM is
persistent

Hot and Cold Area ( Hard Disks)

• The Outer Regions of the Disk are
Hot Areas
– Keep frequently accessed data
• The Inner Regions of the Disk are
Cold Areas
– Keep less frequently accessed data

• In Flash there is no such distinction

19
 11/22/2021

What is your LUN made up of

• How many disks / spindles behind it
• Is it from HOT or COLD area ( in a Hard Disk)

LUN from 3 Disks LUN from 6 Disks

What is ASM?
• A software provided by Oracle – Optimized for
Database I/O
• Removes the requirement for a Volume Manager
– typically provided with Storage Solutions
• Can be used with JBOD (Just a Bunch of Disks)
• Disk Group in ASM used for keeping Database files
• 3 Important Aspects of ASM to be understood
– Striping
– Redundancy
– Failure Groups

What is a Disk Group?

• A Group of Disks
– Similar to a LUN in traditional Storage Servers
DG1 DG2 DG3

20
 11/22/2021

What is Striping?
• Consider a Disk Group with all 9 Disks
– This is just an example to understand the concept

DG1

D1 D4 D7

D2 D5 D8

D3 D6 D9

What is Striping?
• A file is created in the Disk Group

Datafile01.dbf

DG1

D1 D4 D7

D2 D5 D8

D3 D6 D9

What is Striping?
• The file is split into chunks ( stripes ) and written /
distributed to all disks in the Disk Group
Datafile01.dbf

DG1

S1 D1 S5 D4 S9 D7

S7 D2 S2 D5 D8
S3

S4 D3 S8 D6 S6 D9

21
 11/22/2021

What is Striping?
• The file in the Disk Group will perform better than
if the file had been written to just a single Disk
Datafile01.dbf
Datafile01.dbf

DG1

D1 D1 D4 D7
S1 S5 S9

S7 D2 S2 D5 D8
S3

S4 D3 S8 D6 S6 D9

What is Redundancy?
• What if one of the disks fails…
• Will all the data be still accessible?
Datafile01.dbf

DG1

S1 D1 S5 D4 S9 D7

S7 D2 S2 D5 D8
S3

S4 D3 S8 D6 S6 D9

What is Redundancy?
• ASM offers 3 Redundancy settings
– External – Only one strip is stored
– Normal – Two copies ( Primary and Secondary)
– High – Three Copies ( Primary , Secondary & Tertiary)
• These are discussed in detail in the next few slides
• Use the one most appropriate for your requirement

22
 11/22/2021

External Redundancy
• Used when there is Redundancy at Storage Level
• ASM need not worry about data availability
Datafile01.dbf

DG1

S1 D1 S5 D4 S9 D7

S7 D2 S2 D5 D8
S3

S4 D3 S8 D6 S6 D9

Normal Redundancy
• ASM stores one Primary Stripe and a Secondary
(copy) in another disk
Datafile01.dbf

DG1

S1 S9 D1 S5 S3 D4 S9 S5 D7

S7 S6 D2 S2 S7 D5 S3 S2 D8

S4 S8 D3 S8 S4 D6 S6 S1 D9

Normal Redundancy
• Any one Disk failure can be tolerated and data is
still available/accessible
Datafile01.dbf

DG1

S1 S9 D1 S5 S3 D4 S9 S5 D7

S7 S6 D2 S2 S7 D5 S3 S2 D8

S4 S8 D3 S8 S4 D6 S6 S1 D9

23
 11/22/2021

High Redundancy
• ASM stores one Primary Stripe a Secondary (copy)
in another disk and a Tertiary(copy) in another disk
Datafile01.dbf

DG1

S1S9S2D1 S5S3S6D4 S9S5S7D7

S7S6S4D2 S2S7S8D5 S3S2S9D8

S4S8S1D3 S8S4S3D6 S6S1S5D9

High Redundancy
• Any 2 failures can be tolerated and the data is still
available
Datafile01.dbf

DG1

S1S9S2D1 S5S3S6D4 S9S5S7D7

S7S6S4D2 S2S7S8D5 S3S2S9D8

S4S8S1D3 S8S4S3D6 S6S1S5D9

FailGroup / Failure Group

• By Default in ASM each Disk in a Disk Group is a
Failure Group
• When Redundancy is implemented, a copy of a
Stripe is written to another Disk (Failure Group)
• That ensures a disk failure will have a copy of the
data in that disk in various other disks

24
 11/22/2021

FailGroup / Failure Group

• When creating a Disk Group, you can configure
Failure Group to be a collection of Disks so that
even if that entire collection of disks fail, data is still
available
– Could be a Controller failure that you want to tolerate
– Could be a Storage Server failure to be tolerated

FailGroup of Storage Server

• With Normal Redundancy, each Primary copy has
its Secondary copy in another Failure Group

Datafile01.dbf

FG1 DG1 FG2

S1 D1 S5 D4 S9 D7 S9 D1 S3 D4 S5 D7

S7 D2 S2 D5 D8 S6 D2 S7 D5 S2 D8
S3

S4 D3 S8 D6 S6 D9 S8 D3 S4 D6 S1 D9

FailGroup of Storage Server

• With High Redundancy, each Primary copy has its
Secondary copy in another Failure Group and
Tertiary copy in another Failure Group
Datafile01.dbf

FG1 DG1 FG2 FG2

S1 D1 S5 D4 S9 D7 S9 D1 S3 D4 S5 D7 S9 D1 S3 D4 S5 D7

S7 D2 S2 D5 D8 S6 D2 S7 D5 S2 D8 S6 D2 S7 D5 S2 D8
S3

S4 D3 S8 D6 S6 D9 S8 D3 S4 D6 S1 D9 S8 D3 S4 D6 S1 D9

25
 11/22/2021

Failure Group
• In Generic ASM, by default each DISK in a Disk
Group is a Failgroup

• In Exadata, by Default, each Storage Server (and all

storage devices in it) is a separate Failgroup…

Redundancy vs FailGroup
• Redundancy specifies how many Faults can be
tolerated
– High ( 2 faults), Normal ( 1 fault)
• FailGroup configuration specifies what Fault needs
to be tolerated
– Disk, Controller or Storage Server etc…

• You configure based on your Business Needs

26