Oracle Exadata

Bulk Data Loading

/company/k21academy /k21academy /k21academy /k21academy contact@k21academy.com

 Atul Kumar

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

2
 Making Best of Your Training
➢ Get Support: support@k21academy.com (New Mail for Different Issue)
➢ Add Yourself in WhatsApp Group
➢ Unlimited Retake for Next 3 Years
➢ Before Coming to Class: View Presentation & Look at Activity Guide
➢ Ask as Many Questions as you can & make session interactive
➢ Remind Us to Record the Session
➢ Feedback Good/Bad to Trainer for improvement

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

3
 Getting Help

support@k21academy.com

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

4
 Agenda: Module

➢ Staging Data Files Using DBFS

➢ Staging Data Files Using ACFS

➢ Staging Data Files Using External File Systems

➢ Comparison of Staging Area Options

➢ Preparing the Data Files

➢ Configuring a DBFS Staging Area

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

5
 Agenda: Module

➢ Configuring the Target Database

➢ Loading the Target Database

➢ Summary

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

6
 Bulk Data Loading Architectures for Exadata

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

7
 Staging Data Files Using DBFS

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

8
 Staging Data Files Using DBFS

➢ DBFS enables the database to be used as a file system:

➢ DBFS provides shared storage for staging or storing data files, scripts,
reports, and other application files.
➢ Files are stored as SecureFiles LOBs in database tables that are stored
in Exadata.
➢ Files are exposed to the database servers by using a POSIX-
compatible file system interface.

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

9
 Staging Data Files Using DBFS

➢ Files are transactionally consistent with database data.

➢ Files can leverage Oracle Database availability features:
➢ ASM mirroring, RMAN, Data Guard, Flashback, and so on.

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

10
 Staging Data Files Using ACFS

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

11
 Staging Data Files Using ACFS

➢ ASM Cluster File System (ACFS):

➢ Fully-featured cluster file system built on ASM:
➢ Snapshots
➢ Replication
➢ Advanced auditing
➢ Realm-based security and encryption
➢ Tagging API for metadata

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

12
 Staging Data Files Using ACFS

➢ Comparable performance with DBFS for bulk loads

➢ Separate from all databases:
➢ Files are stored outside the Oracle databases
➢ Files can only be accessed through file system APIs
➢ Cannot leverage database features such as RMAN and Data Guard
➢ Requires Oracle Grid Infrastructure version 12.1.0.2 or later
➢ Otherwise, no Exadata-specific implementation requirements

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

13
 Staging Data Files Using External
File Systems

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

14
 Staging Data Files Using External File Systems

➢ Oracle Sun ZFS storage provides a high-performance and cost-effective

alternative that:
➢ Is tested and validated with Exadata
➢ Can connect directly to the Exadata InfiniBand network
➢ Other NAS and NFS-based options are feasible.
➢ Thorough testing is required to ensure acceptable performance.

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

15
 Staging Data Files Using External File Systems

➢ Connection to a SAN requires an intermediate server.

➢ Exadata does not support direct SAN connection.
➢ Performance may be limited by the intermediate server.

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

16
 Comparison of Staging Area
Options

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

17
 Comparison of Staging Area Options

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

18
 Comparison of Staging Area Options

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

19
 Bulk Data Loading Using Oracle DBFS Overview

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

20
 Preparing the Data Files

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

21
 Preparing the Data Files
➢ Data files can be in any format supported by the external table feature.
➢ To facilitate high-performance parallel access:
➢ Oracle automatically divides the files into 10 MB granules.
➢ Exceptions include compressed files, or serial devices.
➢ If granules cannot be created:
➢ Each file is treated as a granule
➢ The number of files determines the maximum degree of parallelism
➢ You must use multiple files to manually enable parallelism

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

22
 Preparing the Data Files
➢ General rules of thumb:
➢ If using multiple files, try to keep them similar in size.
➢ If the file sizes vary significantly, list them in order from largest to
➢ smallest in the external table definition.

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

23
 Configuring a DBFS Staging Area

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

24
 Configuring a DBFS Staging Area

➢ DBFS can be configured in a separate staging database:

➢ Use DBCA to create the staging database.
➢ See My Oracle Support note 1191144.1 for recommendations.
➢ Create a bigfile tablespace for DBFS storage:

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

25
 Configuring a DBFS Staging Area

➢ Create a DBFS user account:

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

26
 Configuring a DBFS Staging Area

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

27
 Configuring a DBFS Staging Area

➢ Additional database server operating system configuration:

➢ Add the user that will mount the DBFS file system to the fuse group:

➢ As root, create /etc/fuse.conf containing the entry user_allow_other:

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

28
 Configuring a DBFS Staging Area

➢ Create a mount point for DBFS with ownership and group permissions set to the
user that will mount the DBFS file system:

➢ Creating the DBFS store:

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

29
 Configuring a DBFS Staging Area

➢ Mounting DBFS:

➢ See My Oracle Support note 1054431.1 for automatic mounting

configuration details.
➢ Using DBFS:
➢ Access DBFS through the mount directory.
➢ Copy files to DBFS by using methods such as FTP and NFS.
www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

30
 Configuring the Target Database

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

31
 Configuring the Target Database

➢ Prerequisites for data file access using external tables:

➢ Create an Oracle directory object that references the DBFS staging area
directory.
➢ Grant the required permissions on the Oracle directory object.
➢ Create the required external tables.
➢ Ensure efficient space management.
➢ Use bigfile tablespaces.

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

32
 Configuring the Target Database

➢ Use 8 MB initial extents for large segments.

➢ Set the INITIAL storage parameter to 8 MB.
➢ Set the CELL_PARTITION_LARGE_EXTENTS initialization parameter to TRUE
or ALWAYS.
➢ Use unlimited quotas to bypass quota management.
➢ Use the parallel clause to set the default degree of parallelism
➢ for the target tables.

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

33
 Loading the Target Database

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

34
 Loading the Target Database

➢ The recommended approach uses external tables:

➢ Parallel direct path loading for high performance
➢ In-flight processing using SQL
➢ Transformations using SQL functions
➢ Sort data while loading to optimize Exadata storage indexes
➢ No need to restage the data

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

35
 Loading the Target Database

➢ Other advanced features

➢ Example: Input file preprocessing
➢ SQL*Loader can also be used:
➢ Parallel direct path loading can also be achieved.
➢ There is less processing flexibility compared with external tables.
➢ Existing SQL*Loader scripts can be easily reused.

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

36
 Loading the Target Database

➢ Parallel loading using external tables:

➢ A CREATE TABLE ... AS SELECT statement automatically uses the default
degree of parallelism.
➢ An INSERT ... SELECT statement needs parallel DML to be enabled:

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

37
 Loading the Target Database

➢ Direct path loading using external tables

➢ A CREATE TABLE ... AS SELECT statement automatically uses direct path loading.
➢ An INSERT ... SELECT statement needs an APPEND hint to enable direct path
loading:

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

38
 Summary
➢ Staging Data Files Using DBFS
➢ Staging Data Files Using ACFS
➢ Staging Data Files Using External File Systems
➢ Comparison of Staging Area Options
➢ Preparing the Data Files
➢ Configuring a DBFS Staging Area
➢ Configuring the Target Database
➢ Loading the Target Database

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

39
 Find Us

https://www.facebook.com/K21Academy

http://twitter.com/k21Academy

https://www.linkedin.com/company/k21academy

https://www.youtube.com/k21academy

www.k21academy.com/joinfb © Copyright 2018 | K21 Academy | All Rights Reserved

40