Data management &

storage

T.Abdulmalik Alsarori
 Data
Data are collected observation or measurement
Represented as text – number- multimedia
Data is not only numbers :
1. Notes
2.documents .
3.Books .
4.Transcripts .
5.Audio&music.
6.Videoes.
7.Photographs .
 Data Categories
Data Categories :
1. Quantitative Data.
2. 1.Qualitative Data.
1. Quantitative Data: any type of data can be presented
as number or counted or numerically analyzed
2.Qualitative Data: any type of data that is used to
present ,describe or categorize thing
Data <> statistics
Data

Data
analysis

Statistics
 DIKW Pyramid
Wisdom What do we need to do ?

Knowledge What is going to happen ?

Information What has happened ?

Data
Data solution main operations

Apply
collect Store & organize present

Govern

Secure
 Data collected
-Collect data requirement .
-Identify data sources .
-Identify data types & structure
-data connection method.
-Data ingestion technique .
-Validation rules .
-Transformation rules.
-…..etc
 Data sources (examples)
-Flat files
-Application & database.
-surveys
-Iot devices.
-Web scrapping
Much more ….
 Type of collected data
There are three type of collected data :
1.structured data .
Tabular with rows & columns such as sheets.
Like table or csv files
2. semi –structured data .
mostly large textural data with no fixed structure such as emails
,webpages and xml files .
3. unstructured data.
Can,t have a predefined model multimedia files
(Audio – Picture -Videos)
 Data ingestion and ingestion
techniques
Data ingestion :the process of extracting data from data sources,
transformation it to fit the data requirements and load it in the
data stores making ready for use .
Ingestion techniques :
1.ETL
2.ELT
 Store And Organize
Store and organize :
1.Technology selection
2.Platform architecture
3.Data zones & models
4.Operations performance
5.Hosting options
 1.Technology selection

Data lake
Data lake House
Data 2011 2020
warehouses
Database
Files
1980 Big Data
1960
Sql No-sql

Sql Structured data

Semi-Structured data
Structured data Unstructured data
Data architecture
 Apply & Represent
Data application :
1.Data analysis.
2.Statisttics .
3.Dashboards & reports .
4.Application interfaces.
5.Data monetization .
6.Data science & ML.
7.Autonomous decisions
8.AI CHATGPT &GenAI
 Governance operation
Focus areas:
- Business glossaries
- Data catalogs
- Data lineage
- Master data management MDM
- Data quality
 Secure Data
Requirements of database security :
1.physical database integrity
2.Logical database integrity
3.Element integrity
4.Auditability.