Extract, Transform, Load: Inmon Bill

You might also like

Download as docx, pdf, or txt
Download as docx, pdf, or txt
You are on page 1of 11

I.

Data may be:

DATA WAREHOUSING - is a process for collecting and 1. Structured Data - data that is collected from a
managing data from varied sources to provide known method and can be neatly arranged.
meaningful business insights.  Info that is highly organized, factual,
and to the point.
- a blend of technologies and components which
 Quanti
allows the strategic use of data.
 Data warehouses/ relational databases
- It is a process of transforming data into
 Several pre-determined formats
information and making it available to users in a
2. Semi-structured -
timely manner to make a difference.
3. Unstructured data - that has no pre-defined
- the core of the BI system which is built for data
format or organization, making analysis more
analysis and reporting.
difficult
ETL - extract, transform, load (ETL) is the general  Doesn’t have any pre-determined
procedure of copying data from one or more sources structured to it and comes in all its
into a destination system which represents the data diversity of form
differently from the source(s) or in a different context  Quali
than the source(s)  Data lakes/ Non-relational databases
 Huge array of formats

A data warehouse merges information coming from


HISTORY OF DATA WAREHOUSE
different sources into one comprehensive database. By
 1960- Dartmouth and General Mills in a joint merging all of this information in one place, an
research project, develop the terms dimensions organization can analyze its employees and clientele
and facts. more holistically. This helps to ensure that it has
 1970- A Nielsen and IRI introduces dimensional considered all the information available. Data
data marts for retail sales.  warehousing makes data mining possible.
 1983- Tera Data Corporation introduces a
Data mining is looking for patterns in the data that may
database management system which is
lead to higher sales, profits and data analysis.
specifically designed for decision support 
 Data warehousing started in the late 1980s
when IBM worker Paul Murphy and Barry Devlin
3 MAIN TYPES OF DATA WAREHOUSES:
developed the Business Data Warehouse.
 However, the real concept was given by Inmon 1. Enterprise Data Warehouse - is a centralized
Bill. He was considered as a father of data warehouse. It provides decision support service
warehouse. He had written about a variety of across the enterprise. It offers a unified
topics for building, usage, and maintenance of approach for organizing and representing data.
the warehouse & the Corporate Information It also provide the ability to classify data
Factory. according to the subject and give access
according to those divisions.
HOW DATA WAREHOUSE WORKS?

A Data Warehouse works as a central repository where 2. Operational Data Store- which is also called
information arrives from one or more data sources. ODS.
Data flows into a data warehouse from the transactional - widely preferred for routine activities like
system and other relational databases. storing records of the Employees
- (ODS) is a powerful tool to help provide any
company or organization operational reports.
3. Data Mart - is a subset of the data warehouse.
7 PRINCIPLES
- specially designed for a particular line of
business, such as sales, finance, sales or finance. 1. GOVERNANCE – develop a program to ensure
- typically summarized data. compliance of key steps in data life cycle.
- It is a subject-oriented database and is also 2. DATA CONTENT – gather info that generates
known as High-Performance Query Structures value for business helping to achieve goals.
(HPQS). 3. DATA QUALITY – consider the quality and
consistency of your data as defined by
 Kimball Model – from varied sources to data standardized processes.
marts then ETL to data warehouse. 4. DATA ACCESS – ensure data is readily/
 Inmon Model – from Varies sources to Data available and understood by all influence in
Warehouse extract to Data Mart. your org.
5. DATA MANAGEMENT – develop and enforce
company-wide standards for technology and
UNDERSTANDING THE TYPES OF DATA IN BUSINESS
proper data usage.
ORGANIZATION
6. DATA INFORMED/DECISION MAKING –
1. Transactional Data - this type of data describes leverage your data as predictive intelligence
your core business activities. when making decision that relate to clients,
2. Master Data - it consists of key information that competitors, funders, donors, society and
make up the transactional data. partners.
3. Reference Data - is a subset of master data. It is 7. ANALYTICS – use data as competitive advantage
usually a standardized data that governed by by exploring patterns and predictive insights –
certain codification. to discover new patterns and perspective for
4. Reporting Data - It’s an aggregated data the business.
compile for the purpose of analytic and
reporting.
HR ANALYTICS - is the process of collecting and
5. Metadata - It’s a data about data. Metadata is
analyzing Human Resource data in order to improve an
giving you the answer to any question that you
organization’s workforce performance. HR analytics
cannot answer by just looking at the data
provides data-backed insight on what is working well
and what is not so that organizations can make
improvements and plan more effectively for future.
ORGANIZATIONAL DATA

- Describes the central characteristics of


organizations, their internal structures and HR DATA SOURCES
processes as well as their behavior as corporate
 Recruitment
actors in different social and economic contexts.
 Demographic Profile
- Refers to the method of classifying and
 Performance Management
organizing data sets to make them more useful.
 Learning Management
- also called firmographic or firm demographic
 Job Architecture
data — is data related to the fundamental
characteristics of organizations  Compensation and Benefits
- cases: most notably market research as well as  Succession Planning
the segmentation of sales and marketing  Talent Development
prospects.  Exit Interview
- can also be used to augment data of other sorts  Learning Management
 Travel unlawfully obtained or unauthorized use of
 Mentoring personal data, considering any violation of your
 Employee Survey Data rights and freedoms as data subject.
 Wellbeing and Wellness  Right to data portability - right assures that
 Absence Data YOU remain in full control of YOUR data. Data
 Financial Data portability allows you to obtain and
 Production Management Data electronically move, copy or transfer your data
 Sales Data in a secure manner, for further use.
 Customer Relationship Management

DATA MANAGEMENT CONCEPTS


RIGHTS AS A DATA SUBJECT •Metadata Management
 Right to be informed - your personal data will •Data Security
be, are being, or were, collected and processed.
 Right to access - right to find out whether an •Document Management and;
organization holds any personal data about you •Content Management
and if so, gain “reasonable access” to them.
Through this right, you may also ask them to 1. METADATA MANAGEMENT - it enables a
provide you with a written description of the business user to search and identify the
kind of information they have about you as well information on the key attributes in webbased
as their purpose/s for holding them. user interface.
 Right to object - if the personal data processing
involved is based on consent or on legitimate 2. DATA SECURITY? - is a set of standards and
interest technologies that protect data from intentional
 Right to erasure - you have the right to or accidental destruction, modification or
suspend, withdraw or order the blocking, disclosure.
removal or destruction of your personal data What are the main elements of data security?
such as your personal data is incomplete,
outdated, false, or unlawfully obtained. Or it is CIA TRIAD
being used for purposes you did not authorize.  Confidentiality ensures that data is accessed
 Right to rectify - right to dispute and have only by authorized individuals;
corrected any inaccuracy or error in the data a  Integrity ensures that information is reliable as
personal information controller (PIC) hold about well as accurate; and
you. The PIC should act on it immediately and  Availability ensures that data is both available
accordingly, unless the request is vexatious or and accessible to satisfy business needs.
unreasonable
 Right to file a complaint - If you feel that your What are data security considerations?
personal information has been misused, •Where is your sensitive data located?
maliciously disclosed, or improperly disposed,
or that any of your data privacy rights have •Who has access to your data?
been violated, you have a right to file a •Have you implemented continuous monitoring and
complaint with the NPC. real-time alerting on your data?
 Right to damages - You may claim
compensation if you suffered damages due to
inaccurate, incomplete, outdated, false,
3. DOCUMENT MANAGEMENT SYSTEM OR DMS? RESPONSIBLE AGENCY:
- is a system or software that allows the
 Department of Information and Communication
creation, storage, managing, indexing,
Technology
protection, and retrieval of digital documents.
 National Privacy Commission
How does a document management system work?
BACKGROUND
- It captures documents. It stores documents.
- The Philippines has a growing and important business
And it distributes documents.
process management and health information
technology industry.
4. CONTENT MANAGEMENT SYSTEM OR CMS? - is
an application that is used to manage web - Total IT spending reached $4.4 billion in 2016, and
content, allowing multiple contributors to expected to more than double by 2020.
create, edit and publish.
- The country is also in the process of enabling free
EXAMPLES: public Wi-Fi.

a. Drupal  rapid growth of the digital economy and increasing


b. Joomla international trade of data
c. Squarespace
- Filipinos are heavy social media users
d. Weebly
e. Wordpress  67M internet users – world’s #1 in terms of social
f. Magento media usage (Digital 2018 by Hootsuite, and We Are
g. Wix Social Ltd.)
h. Mod X
 Facebook users – 30M in 2013 to 67M in 2017

TIPS AND TAKEAWAYS FOR AN ENHANCED DATA


DRIVEN DECISION MAKING STRATEGY THE JOURNEY OF THE DPA
 Analyze and understand the data - European Union’s 1995 Data Protection Directive
 Don’t be afraid to revisit and re-evaluate the
data - Electronic Commerce Act of 2000 (R.A. No. 8792) –
 Present the data in a meaningful way recognition and use of electronic commercial and non-
 Set measurable goals for decision making commercial transactions and documents
 Continue to evolve your data driven business - membership in the Asia-Pacific Economic Cooperation
decisions (APEC) -- Privacy Framework in 2005

- DTI Administrative Order No. 8 in 2006 -- which


II. prescribed guidelines for a local data protection
certification system
DATA PRIVACY ACT
- The DPA was signed into law in 2012, with the local
REPUBLIC ACT 10173 DATA PRIVACY ACT OF 2012 BPO sector as its most visible endorser
(DPA) “An act protecting individual personal
information and communications systems in the - Creation of the Dept. of Information and
government and the private sector, creating for this Communications Technology (DITC) in 2015 (R.A. No.
purpose a National Privacy Commission, and for other 10844)
purposes”
- The activation of the National Privacy Commission  the data subject must be informed about the
(NPC) in 2016  DPA’s Implementing Rules and extent and purpose of processing
Regulations was put in effect on September 9, 2016  for the “automated processing of his or her
personal data for profiling, or processing for
direct marketing, and data sharing”
PROVISION OF THE DPA  for sharing information with affiliates or even
mother companies
1. Chapter I – General Provisions
 must be “freely given, specific, informed,” and
2. Chapter II – The National Privacy
must be evidenced by recorded means
Commission
3. Chapter III – Processing of Personal - Consent is not required for processing where the data
Information subject is party to a contractual agreement, for
4. Chapter IV – Rights of the Data Subject purposes of fulfilling that contract.
5. Chapter V – Security of Personal
 for protection of the vital interests of the data
Information
subject
6. Chapter VI – Accountability for Transfer of
 to response to a national emergency
Personal Information
 for the legitimate interests of the data
7. Chapter VII – Security of Sensitive Personal
controller
Information in Government
8. Chapter VIII – Penalties
9. Chapter IX – Miscellaneous Provisions
AGREEMENT

- “The law requires that when sharing data, the sharing


APPROACH OF THE GOVERNMENT be covered by an agreement that provides adequate
safeguards for the rights of data subjects, and that
- The processing of personal data shall be allowed
these agreements are subject to review by the National
subject to adherence to the principles of:
Privacy Commission”
 transparency
 legitimate purpose
 Proportionality - processing of information SENSITIVE PERSONAL INFORMATION
shall be adequate, relevant, suitable, necessary
- The law defines sensitive personal information as
and not excessive in relation to a declared and
being:
specified purpose.
 About an individual’s race, ethnic origin, marital
status, age, color, and religious, philosophical or
DATA PROCESSING AND CONSENT political affiliations;
 About an individual’s health, education, genetic
- Collection of personal data must be:
or sexual life of a person, or to any proceeding
 Declared or any offense committed or alleged to have
 Specified committed;
 Legitimate purpose  Issued by government agencies “peculiar”
(unique) to an individual, such as social security
number;
- Consent is required prior to the collection of all
 Marked as classified by executive order or act
personal data.
of Congress.
SENSITIVE PERSONAL INFORMATION - The National Privacy Commission, which was created
to enforce RA 10173, will check whether companies are
- All processing of sensitive and personal information is
compliant based on a company having 5 elements:
prohibited EXCEPT in certain circumstances.
 Appointing a Data Protection Officer 
 Consent of the data subject;
 Conducting a privacy impact assessment
 Pursuant to law that does not require consent;
 Creating a privacy knowledge management
 Necessity to protect life and health of a
program
person;
 Implementing a privacy and data protection
 Necessity for medical treatment;
policy
 Necessity to protect the lawful rights of data
 Exercising a breach reporting procedure
subjects in court proceedings, legal
proceedings, or regulation.

3 PEOPLE INVOLVED IN DPA


 data subject
PENALTIES
 data information controller
 Ranging from P100,000 to P5,000,000  data information processor
(approximately US$2,000 to US$100,000)
 Imprisonment of 1 year up to 6 years
 Unauthorized Processing of Personal CONFIDENTIALITY - refers to protecting information
Information and Sensitive Personal Information from being accessed by unauthorized parties. In other
 Accessing Personal Information and Sensitive words, only the people who are authorized to do so can
Personal Information Due to Negligence. gain access to sensitive data.
 Improper Disposal of Personal Information and
Sensitive Personal Information
 Processing of Personal Information and NON-DISCLOSURE AGREEMENT (NDA) - a legal contract
Sensitive Personal Information for Unauthorized or part of a contract between at least two parties that
Purposes outlines confidential material, knowledge, or
 Unauthorized Access or Intentional Breach. information that the parties wish to share with one
 Concealment of Security Breaches Involving another for certain purposes, but wish to restrict access
Sensitive Personal Information. to or by third parties.
 Malicious Disclosure.
 Unauthorized Disclosure

III.
WHO NEEDS TO REGISTER?
BUSINESS INTELLIGENCE - is a set of processes,
- Companies with at least 250 employees or access to
architectures, and technologies that convert raw data
the personal and identifiable information of at least
into meaningful information that drives profitable
1,000 people are required to register with the National
business actions.
Privacy Commission and comply with the Data Privacy
Act of 2012 - It is a suite of software and services to
transform data into actionable intelligence and
knowledge.
COMPLIANCE OF THE DATA PRIVACY ACT - It has a direct impact on organization’s
strategic, tactical and operational business
decisions.
- BI supports fact-based decision making using patterns, spot trends, and uncover insights that foster
historical data rather than assumptions and gut growth and improvement.
feeling.
- BI tools perform data analysis and create
reports, summaries, dashboards, maps, graphs, Importance of Data Gathering
and charts to provide users with detailed
 Data - A collection of facts that has been
intelligence about the nature of the business.
translated into a form that computers can
- It help businesses to identify market trends and
process.
spot business problems that need to be
 Data Gathering is the systematic approach to
addressed.
collecting and measuring information from a
- BI technology can be used by Data analyst, IT
variety of sources to get a complete and
people, business users and head of the
accurate picture of an area of interest.
company.
- BI system helps organization to improve Importance
visibility, productivity and fix accountability.
- Gathering data allows us to store and analyze
important information about our existing and
potential customers.
CONCEPTS OF BUSINESS INTELLIGENCE
- Gathering data can also save money by building
The 4 basic components within business intelligence a database of customers for future marketing.
are: - Allows organizations to visualize relationships
between what is happening in different
1. The data itself (raw data) -The first component of an
locations, departments, and systems.
integral BI solution is the data itself. This data could be
- You no longer need to cast a “wide net” to
anything like sales records for the year, the keywords
reach potential consumers within your target
implemented in your latest advertising program, salary
audience.
and benefit tables, or profit and loss statements.
- You can use the information to build segmented
2. The data warehouse -The data storage warehouse is audience profiles for future marketing efforts,
the logistics platform that connects all of your different making your marketing spend more efficient
databases together and allows you to create and effective.
relationships between them. This is an area that has
seen great advances recently with the introduction of
IMPORTANCE OF STORING DATA
cloud-based BI tools.
 DATA STORAGE- refers to conserving your data
3. Data access, analytics, and presentation -Once all of
files in a secured location and in a way such that
your data is connected and can ‘talk to each other,’ one
it can be accessed readily
of the next key business intelligence concepts is to
 DATA ANALYZING - Process of cleaning,
make use of that data. This involves accessing the data,
transforming and modeling data to discover
analyzing it for important trends, and presenting it in a
useful information for business decision-
way that is immediately understandable.
making.
4. Data dashboarding and reporting -The fourth and  DATA ANALYSIS TOOLS
perhaps most pivotal component of an interactive
• R-PROGRAMMING
dashboard is the ability to continuously track, monitor,
and report your data. • PLOTLY

-By having access to a flexible, customizable, data- • TALEND


driven online dashboard, you can to set targets, identify
 TYPES OF DATA ANALYSIS: TECHNIQUES AND Importance of Accessing Data
METHODS
 Make better decisions, faster.
• Text Analysis  It helps you solve problems.
 You have the ability to remove, edit and add
• Statistical Analysis
data.
• Diagnostic Analysis
How to Make Data More Accessible
• Predictive Analysis
 Mobile Applications
• Prescriptive Analysis  Automated Email Delivery
 Integration With Existing Tools
 Export Functionality
 TEXT ANALYSIS - used to transform raw data  Using Sisense Pulse to Automate Access
into business information.
Two Important Aspects of Data Access Control:
 STATISTICAL ANALYSIS - shows "What happen"
by using past data in the form of dashboard.  Assigning employees proper access rights, and
determining when to revoke them.
TWO CATEGORIES:
 Conducting the internal systems audit.
• Descriptive Analysis

• Inferential Analysis
IV.
 DIAGNOSTIC ANALYSIS - shows "why did it
WHAT IS A RECORD?
happen?" by finding the cause from the insight
found in statistical analysis. - is an output that contains valuable information
 PREDICTIVE ANALYSIS - "What is likely to for the business and reflects details of
happen" by using previous data. transactions of an institution, its staff and its
 PRESCRIPTIVE ANALYSIS - Combines insights stakeholders
from all previous analysis to determine which - is an evidence of an activity or transaction
action to take in a current problem or decision. performed by the organization which shall
require retention for a specific period of time
- is information created, received and maintained
DATA ANALYSIS PHASES as information by the organization

 Data Requirement Gathering


 Data Collection TYPES OF RECORDS
 Data Cleaning
According to medium
 Data Analysis
 Data Interpretation  ELECTRONIC
 Data Visualization  PAPER
 MICROFILM

According to the nature of the document


Importance of Accessing Data
 HISTORICAL
 DATA ACCESS - It refers to a user's ability to access
 LEGAL
or retrieve data stored within a database or other
 PERSONNEL
repository.
 FINANCIAL
 COMPANY POLICIES & PROCEDURES
process which can be admissible in any court of
law.
According to their importance/future usefulness;
WHAT ARE PROCESSES INCLUDED IN RECORDS
VITAL
MANAGEMENT?
 Necessary for the continued
 CREATION of record
 operation of the business
 RETENTION/ARCHIVING of record
 cannot be replaced
 DISPOSAL of record
 should not be destroyed
 examples: contracts/deeds

IMPORTANT RECORDS OF CREATION

 Necessary for the continued operation of the - GUIDELINES IN CREATING RECORD


business but could be replaced with additional - EXERCISE ON CREATION OF RECORDS
cost and effort
 can be transferred to inactive storage
GUIDELINES IN CREATING RECORD
 should not be destroyed
 examples: financial records/ reports of  Records should be reliable.
operations/ board meeting minutes - contents are complete and accurate
representation of the business transaction
USEFULLNESS
- can be depended upon in the course of
 Records which assist business operations in succeeding transactions
running smoothly on a day to day basis - Record should be created at the time of the
 replaceable but may cause inconvenience transaction or soon afterwards
 can be moved to inactive storage/or destroyed  Records should provide data which will translate to
after a - period of time information, which in turn would be used in sound
 examples: letters/reports decision making.
- At a minimum should include the date and time
NON-ESSENTIAL
of transaction, relevant names, nature of
 Records which do not provide any future value transaction and pertinent terms describing the
to the cooperative may be destroyed after they transaction
have served their purpose
 examples: memos/notices/ - newsletters
WHAT IS RECORDS ADMINISTRATION?

ADMINISTRATION of records is the HEART and SOUL of


RECORDS MANAGEMENT - It is the efficient and
the records management system
systematic control of the creation, receipt,
maintenance, use and disposition of records - It is the process by which records are stored,
and circulated or tracked
WHY SHOULD RECORDS BE MAINTAINED?
It involves determination of the following:
 Records should be maintained to satisfy an
organization’s internal business needs a. an effective record keeping strategy which can
 Records should be maintained to satisfy be achieved by proper file maintenance (record
compliance to regulatory requirements preservation)
 Records should be maintained because they b. identifying which records have to stored and
provide evidence of a transaction or business the length of time/duration for their storage or
usefulness to the organization (record - save financial resources as this reduces cost
retention) caused by staff workload and hours, equipment
c. a method by which records in circulation will be and office supplies
tracked (record transfer)
Things to consider in storing acting records

- Active paper files should be kept as close to the


RECORDS PRESERVATION
user as possible as this will allow for easy access
How can this be done? - File storage equipment and room should be
appropriately secured
1. GROUPING OF RECORDS or thinking of records
- Equipment and supplies should be appropriate
in terms of their series
for the volume and size of records to be stored
Examples : and for the storage location
- Active files should be weeded often to keep
a. Meeting Records
them current and manageable
b. Correspondence Files - Inactive records, duplicates, and non-record
material should be removed according to
c. Purchasing Files
standards set.
d. Vouchers

2. FILING IN TERMS OF A TIME PERIOD or TYPES OF FILING STRATEGIES


organizing a filing system which closes at a
Depending on the type of records you work with and
particular date This involves creation of a new
the way you need to retrieve them, the following filing
folder for each new year for a particular series
strategies may be used:
and sending the past year’s file to archives for
safekeeping  simple alphabetic filing
3. Maintain a CONSISTENT FILING SYSTEM across
Reference : www.filingmate.com
all levels of the organization.
 filing by subject
Inconsistency may be a result of employee turnover.
 numeric classification systems
Solution: Train new employees in the procedure being
used and require to follow the procedure religiously
RECORDS TRANSFER (CIRCULATION AND TRACKING)
4. FILING GUIDELINES Production of a simple
printed guide which contains in detail all the CIRCULATION - This refers to the access to records,
records series produced, and the guidelines to both internally (within the organization) and externally
be followed for maintaining them in a (outside the organization)
consistent manner.
- This should take into consideration balancing
the need for confidentiality & privacy against
FILING SYSTEMS allowing public access to the records
- This also involves proper recording of release of
FOR ACTIVE RECORDS:
documents and their subsequent return
Active records are those which we use on a frequent
TRANSFER - This refers to the process of transferring
basis. Why do we need to properly manage active files?
records from active files to archives.
- to save time in locating records
- make best use of space
- Does it perhaps pose a liability, even after the
administrative value has disappeared?

F ISCAL

- Is the information necessary for tax or audit


purposes?
- Is it helpful in future budget and fiscal
planning?
RECORDS RETENTION

How long should we retain our records? What is their


common useability span? PROTECTIVE PREVENTIVE MEASURES

RECORD RETENTION POLICY  Involves identification of measures which will help


protect records and prevent their wrongful or
- should clearly specify RETENTION PERIODS
untimely disposal/destruction/analysis or risks
- should define the STORAGE STANDARDS and
associated
LOCATION RECORDS RETENTION
 Risk assessment of storage site -
- There is no universal guide to record retention
description of the facility, type and
periods and records disposition schedules
condition of the facility, size of the room
- Retention period may be determined/defined
and locality risk
depending upon particular usage patterns,
 Climate - temperature range, air
department needs, document historical value,
conditioning, humidity
and legal issues
 Lighting - natural lighting, sunlight,
fluorescent
 Security - entry alarms, doors, windows,
RECORDS RETENTION (HALF-LIFE Model)
motion sensors, fire alarm for heat and
H ISTORIC smoke, extinguishers, insurance
 Other Vulnerabilities - electrical,
- Does the record document an important part
equipment, plumbing, moisture
of the cooperative’s history?
accumulation, flood, building leaks evidence
- Will it be a valuable resource to researchers
of insects, humidity changes, vandalism,
many years from now?
insects (cockroaches), rats, temperature
A DMINISTRATIVE changes and mold/mildew

- How often do you actually use the material? PREVENTIVE/PROTECTIVE MEASURES


- What is the value of the information to your - DISASTER RECOVERY KIT
daily operation? - SALVAGE OF WATER DAMAGED BUSINESS
- At what point would not having the material RECORDS
available cease to be a serious hardship?

L EGAL

- Are there any external requirements to


maintain the information, possibly from
governmental agencies?
- Does the record help to prevent a legal
liability?

You might also like