Professional Documents
Culture Documents
BY K Madhavi Data Architect
BY K Madhavi Data Architect
K MADHAVI
DATA ARCHITECT
1
Key
capabilities
2
How to create one or more data
pipelines
3
Usage
4
Brief introduction to event
hubs
5 Demo
6
a journey of a thousand miles begins with a single step
2
Introduction to Azure Data Factory Service, a data integration service in the
cloud
You can create data integration solutions using the Data Factory service that can ingest
data from various data stores, transform/process the data, and publish the result data to
the data stores.
4
5
Use it to ingest data from multiple on-premises and cloud sources.
You can lift & shift existing SSIS packages to azure cloud and run
them with fully compatibility in azure data factory.
6
Azure Office 365
On- premise
DLS C
S
DW AA PBI
S
SERVER AD DS
F
L
DLA
8
2
A pipeline is a logical grouping of activities. They are used
to
group activities into a unit that together perform a task.
10
Activities define the actions to perform on your data. For
example, you may use a Copy activity to copy data from one
data store to another data store. Similarly, you may use a Hive
activity, which runs a Hive query on an Azure HDInsight
cluster to transform or analyze your data. You may also
choose to create a custom .NET activity to run your own
code.
11
PIPELIN
E
12
Data Factory supports two types of activities: and.
13
Data Movement Activates
NoSQL
Sybase
Teradata
14
Amazon S3 Generic HTTP
HDFS Salesforce
18
Linked services are used for two purposes in Data Factory:
including, but not limited to, an on-premises that can host the execution of an activity. For
SQL Server, Oracle database, file share, or an example, the HDInsight Hive activity runs on
Azure Blob Storage account. an HDInsight Hadoop cluster.
19
Linked Services
AD DL
AB F S
20
3
Linked services link data stores to an Azure data factory.
Datasets represent data structures with in the data stores.
For example, an Azure Storage linked service provides
connection information for Data Factory to connect to an
Azure Storage account. An Azure Blob dataset specifies the
blob container and folder in the Azure Blob Storage from
which the pipeline should read the data. Similarly, an Azure
SQL linked service provides connection information for an
Azure SQL database and an Azure SQL dataset specifies the
table that contains the data.
22
DEMO
23