COURSE OUTLINE

DATA
ENGINEER

www.kotaksakti.com
 Preparation Week
WEEK 1 - Introduction to course requirements - Meet your trainer
- Get to know your peers

Data Engineering Fundamentals

WEEK 2 - Introduction to Data Engineering
- Data Engineering vs other data roles
- Keys to being a successful data engineer
- Overview of essential tools for data engineer
- Data architecture designs

Python for Data Engineering

- Integrated Development Environment (IDE) for Python
and basics
- Libraries, packages, and modules
- Data types, functions, and methods
- Loops, lambda, maps
- If Else
- Try Except

Consultation on Assessment

Python for Data Engineering

WEEK 3 - Pandas DataFrame for data manipulation
- Importing data from external sources
- Exploratory analysis
- Data Granularity
- Exporting data
- File formats and storage types
- Answering questions using data

Introduction to Version Control with Git

- Basics of Git and version control
- Creating and managing repositories
- Branching and merging strategies
- Collaborative workflows
- Integrating Git into data engineering projects

Consultation on Assessment
www.kotaksakti.com
 SQL Mastery for Data Engineering
WEEK 4 - SQL vs NoSQL overview
- Basics query syntax
- Advanced query syntax
- Combining data using joins
- Tables and views in SQL database

SQL Mastery for Data Engineering

- Data modelling using star schema
- Automating queries using stored procedure
- SQL best practices
- Running your own SQL database

Consultation on Assessment

ETL Orchestration
WEEK 5 - ETL and its flow
- Introduction to Airflow
- DAGs
- Scheduling and automating ETL jobs using Airflow
- Tools other than Airflow
- Docker
- Installing Airflow on Docker

Python & SQL Data Project

- Performing ETL
- Joining multiple data sources incremental load
- Scheduling with Airflow
- Integrating Python and SQL skills in real projects

Consultation on Final Projcect

www.kotaksakti.com
 Cloud Computing for Data Engineering
WEEK 6 - Cloud concepts in data engineering
- Parallel computing, data lake, and clusters
- Introduction to Spark & Databricks
- Setting up Databricks workspace
- Spark Dataframe
- Performing data manipulation using PySpark and
Spark SQL

Cloud computing for Data Engineering

- Performing ETL from multiple data sources using
Databricks
- Orchestration using Databricks Workflow
- Creating cloud data warehouse in Databricks
- Connecting to PowerBI

Consultation on Assessment

Microsoft Azure for Data Engineering

WEEK 7 - Introduction to Microsoft Azure
- Azure Data Factory (ADF)
- Azure SQL Server
- ETL from multiple data sources using ADF into SQL
Server

Microsoft Azure for Data Engineering

- Advanced ETL using ADF and Databricks
- Common design pattern in the industry
- ETL project assignment

Consultation on Assessment

www.kotaksakti.com