Professional Documents
Culture Documents
Class Notes 1701619372
Class Notes 1701619372
1) https://drive.google.com/file/d/14x0PPLtFfghQuzgD8VsKd99
5lpAAz6uw/view?usp=sharing
2) https://drive.google.com/file/d/15TkpAZMHGjNQBgZaZmdD
C6YGt0FHVOb-/view?usp=sharing
What is a database?
Collection of data , storage of data , past data , current updated in transaction database system
** Similar to database is data warehouse and data lake and delta lake
1. Mysql server
2. Mysql workbench
Download community
Company storing every information in their different databases
1 database will have multiple tables
Database/ emp info- employees hr manager
Databse/product/- tables : product info, customers , marketing data, sales
#structure storing of data
—DATA WAREHOUSE/DATA LAKE
Multiple databases
Diff b/w data warehouse and data lake
——DATABASE
———TABLES
———-COLUMNS
#Database:
1. Relational database (RDBMS) : table information relate /connect to a second or other tables- all
the information that is required by the company is going to be split into diff table for better data
management purpose , it connect information from 1 table to another (using foreign key)
2. Non relational database(Nosql): documents - marriage, divorce, criminal law - no relationship
since there are no tables : Cassandra, mongoldb
#DATA PIPELINE :
Sources : API : fb ——> database/datawarehouse -> extract (sql) -> data analysis (data cleaning
/preprocessing and transformations ) -> customer use views (table), spreadsheets, dashboard (Analytical
tool) -tableau, sigma, power , looker [business consumer for the data ]
Raw data (sources) -> <data engineers data tranformations> api -> data warehouse -> data analyst (data
analysis and transform) -> dashboards /consumer use
Data provider -> data consumer
Fb, shopify -> vero moda
Fb raw data - marketing
1. Like /shares
2. Save /purchase[customer]/browse[lead generate] - geo location, name, emailed, phone , age
3. Browsing info - photo viewing time [browsing time for higher time means more interested
customer]
**target customer
Retail/banking /product - customer
Product design/market/sales and revenue
Facebook (marketing apps) : market our product to customers
Inventory /amazon fulfilment / - store products - qty, items, $cost price , aisle
Fb - > Vero moda/apple -> website -> purchase /browse - customer information -> vero moda(BRAND)
Fb source 100 tables and 50-100 columns -> data before ingesting (data warehouse) data engineers
select what imp tables and what imp column that are needed -> ingesting the data into the data
warehouse -> selective table selection and extract sql -> data analysis (python) -> find answers to certain
question -> reflecting in your dashboards
● Customers
**prepare dashboards
Sales dashboards
KPI :
*** Max profit and Min cost - without hampering the Quality
Trade off Quality and Quantity
Customer Dashboard:
Structured database - tables - rows and columns - via SQL
Table. -
1. Primary key : column in the table : UNIQUE TO EVERY RECORD/ non null: Transaction_id,
emp_id, student_id: give us unique information about the records
2. Foreign key: column relate one table to another : HR.emp_id = employees.id =
emp_personal.id_number: relational database management system
<database.tbname.column>
To relate 1 table information with another table information we
use the foreign key column
In transaction database :
Data time stamp keeping a record of your transactions /changes that are made to the data :
Insertion of the data
Deletion of the data
Update the data
Real time streaming data capture
(Oltp)
Banking
Customer
Employee
employee data who have joined and left the company - delete any record , insert record
Start up company:
Employer will ask some question onbaorded - name, dob, pan card, previous salary,
Database manager
Employee table - create
Insert new employee info - database has to be updated
Product - Nykaa - customer table is blank
Software engineer will be build API(application platform interface) : connecting your shoplift /website ->
database : transfer the data collected into data warehouse
1. 3 rd party data - competitor data plum is direct competitor to Nykaa study the customer
behaviour , products , selling option
show databases;
Use mysql;
show tables;
select * from component;
-- create table tablename (column names datatypes(charachter /integer length))
CREATE table Employees(
id int,
FirstName varchar(255),
Lastname varchar(255),
city varchar(255));