DATA WAREHOUSE

COLLECTION OF HUGE AMOUNT OF DATA STORED IN ONE PLACE IS CALLED AS DATA WAREHOUSE.

FOR EXAMPLE:
WE CAN TAKE INTELLIPAAT ORGANIZATION WHICH HAS MANY DEPARTMENTS SUCH AS SALES
DEPARTMENT, FINANCE DEPARTMENT,
HR DEPARTMENT, TECHNICAL ANALYST DEPARTMENT IN WHICH ALL THESE DEPARTMENT DATAS ARE
STORED IN ONE PLACE IS CALLED
DATA HOUSE. IT WILL STORE THE DATA SINCE THE ORGANIZATION TILL THE PRESENT TIME.

ALL THE HISTORICAL AND REAL TIME DATA STORED IN DATA WAREHOUSE.

DATA WAREHOUSING IS AN APPLICANCE FOR ALSO STORING, ANALYZING DATA AND REPORTING.
IT IS ALSO A TYPE OF DATABASE MANAGEMENT SYSTEM THAT IS DESIGNED TO ENABLE ANS
SUPPORT BUSINESS INTELLIGENCE
ACTIVITIES.

BUSINESS INTELLIGENCE IS A TECHNOLOGY. WE COLLECT AND ANALYZE AND PRESENT IN A WAY

THAT STAKEHOLDERS CAN TAKE THE
DECISION BY SEEING THE REPORT OR DATA.

REAL TIME DATA WAREHOUSE:

INFORMATION STORED IN A REAL TIME DATA WAREHOUSE REPRESENT MUCH MORE ACCURATE
PICTURE OF THE ACTUAL
SITUATION OF THR ORGANIZATION OR COMPANY AT THE TIME DATA IS REQUESTED AND
ANALYSED.

ETL PROCESS:

DATA WAREHOUSE ARCHITECTURE

SQL---------

ORACLE------
STAGING AREA---------- DATA WAREHOUSE
TERADATA---- (ETL PROCESS)

FLAT FILE---

DATA COMING FROM DIFFERENT RESOURCES AND THAT FROM DIFFERENT RESOURCES SUC AS SQL,
ORACLE, EXCEL, FLAT FILE
ARE EXTRACTED. THE EXTRACTED DATA FROM RESOURCES IS NOISY AND DIRTY ( BECAUSE THE
MIGHT CONTAINS NULL VALUES,
BLANK VALUES AND SPECIAL CHARACTERS THAT MIGHT NOT ALLOW IN DATA WAREHOUSE) AND
THEN IT WILL CONVERT OUR DATA
FROM UNSTRUCTERED FORMAT TO STRUCTURED DATA BY USING TRANSFORMATION. FOR
TRANSFORMATION WE WILL DO FILTERING,
CLEANING AND SPLITING.
IN FLITERING WE EXTRUDE THE UNNECESSARY DATA THAT IS NOT REQUIRED.
IN CLEANING WE FILL THE NULL VALUES BY DEFAULT VALUES.
IN SPLITING WE WILL SPLIT SINGLE ATTRIBUTE TO MULTIPLE ATTRIBUTE. IT WILL LOAD YOU
TO DATA WAREHOUSE.

CHARACTERISTICS OF DATA WAREHOUSE:

SUBJECT ORIENTED-- IT CAN BE USED TO ANALYZE A PARTICULAR SUBJECT AREA OR
DEPARTMENT.
INTEGRATION--IT INTEGRATES DATA FROM MULTIPLE DATA SOURCES.
TIME VARIANT-- HISTORICAL DATA IS KEPT IN DATA WAREHOUSE.
NON-VOLATILE-- OLDER DATA WILL NOT GET DELETED, IF WE ADD NEW DATA.(IT SHOULD NOT
CHANGED).

ACCORDING TO DATA WAREHOUSE FORMAT WE WANT ONE VALUE IN ONE ATTRIBUTE.(ATTRIBUTE

MEANS COLUMN)

DATA WAREHOUSING PROCESSES

DATA CLEANING--IT INCLUDES FILLING IN MISSING VALUES.

INTEGRATION-- IT INCLUDES INTEGRATION OF MULTIPLE DATABASE.
DATA TRANSFORMATION--CONVERTS DATA FROM HOST FORMAT TO WAREHOUSE FORMAT.
DATA LOADING--SORT, SUMMARIZE,COMPUTE, VIEWS, CHECK INTEGRITY.
DATA REFRESHING--PROPOGATES THE UPDATE FROM THE DATA SOURCES TO THE WAREHOUSE.

DATA WAREHOUSE TOOLS:

1.AMAZON REDSHIFT
2.MICROSOFT AZURE
3.TERADATA
4.GOOGLE BIGQUERY
5.SNOWFLAKE
6.AMAZON DYMANODB
7.INFORMATICA OR MANY MORE.

COMPONENTS OF DATA WAREHOUSE:

CENTRAL DATABASE-- A DATABASE SERVES AS RHE FOUNDATION OF YOUR DATA WAREHOUSE.
DATA INTEGRATION-- DATA IS PULLED FROM SOURCE SYSTEM AND MODOFIED.
META DATABASE-- IT IS INFORMATION ABOUT YOUR DATA. IT SPECIFIES THE
SOURCE,USAGE,VALUES AND OTHER FEATURES OF DATASET.
FOR EXAMPLE-- IN OUR PHONE WE HAVE PHOTOS ANC CLICK THREE DOTS WE WENT TO MORE
OPTION ABLE TO SEE ALL DETAILS SUCH TIME, DATE, LOCATION OF YOUR PHOTO.
ACCESS TOOLS-- IT ALLOW USERS TO INTERACT WITH THE DATA IN DATA WAREHOUSE.
EXAMPLE--OLAP TOOLS ONLINE ANALYTICAL PROCESSING TOOL(POWER BI, TABLEAU).

APPLICATIONS OF DATA WAREHOUSE:

1. BANKING
2. FINANCE
3. EDUCATION
4. HEALTHCARE
5. MANUFACTURING
6. RETAILING
7. INSURANCE
8. SERVICES

DATABASE MANANGEMENT SYSTEM: | DATA WAREHOUSE

1. IT IS TRANSACTION ORIENTED
2. CONTAINS DETAILED DATA
3. IT CAPTURES DATA
4. APPLICATION RELATED
WANTED OR REQUIRED DATA)
| 1.IT IS SUBJECT ORIENTED
| 2. CONTAINS HISTORIC DATA
| 3. IT ANALYZE DATA
| 4. SUBJECT ORIENTED(WE CHECK ONLY

ONLINE ANALYTICAL PROCESSING(OLAP)

OLAP TOOLS HELP TO INTERACT WITH DATA WAREHOUSE SUCH AS POWER BI, TABLEAU.
DIFFERENT DATA SOURCES----> DATA WAREHOUSE----> OLAP CUBE <--------------------END
USERS (APP QUERIES)
(ETL PROCESS)
(POWER BI, TABLEAU)
IT IS A COMPUTING METHOD THAT ENABLE THE USERS TO EASILY AND SELECTIVELY EXTRACT
DATA.
IT IS USED TO QUERY DATA IN ORDER TO ANALYZE IT FROM DIFFERENT POINT OF VIEW
OLAP BUSINESS INTELLIGENCE OFTEN HELPS IN TREND ANALYSIS, FINANCIAL REPORTING,
SALES FORECASTING, BUDGETING, AND OTHER PLANNING
PROCESS.
POWER BI AND TABLEAU IS A APPLICATION OF OLAP.

OLTP(ONLINE ANALYTICAL PROCESSING)

IT FACILITATE AND MANAGE ONLINE TRANSACTION PROCESS.
OLTP ENABLES LARGE NUMBER OF DATABASE TRANSACTION MADE BY LARGE NUMBER OF PEOPLE
OVER THE INTERNET.
A DATABASE TRANSACTION IS A CHANGE, INSERTION, DELETION OR QUERY OF DATA IN
DATABASE.
OLTP SYSTEM DRIVES MANY OF THE FINANCIAL TRANSACTIONS MADE EVERYTHING INCLUDING
ONLINE TRANSACTION, ATM TRANSACTION,
E-COMMERCE,AND IN-STORE PURCHASE.