Professional Documents
Culture Documents
Data Warehousing: L.Ramanathan Asst. Prof. Scse VIT University
Data Warehousing: L.Ramanathan Asst. Prof. Scse VIT University
Data Warehousing: L.Ramanathan Asst. Prof. Scse VIT University
Key Issues
Value and Expectations- To identify whether a data warehouse is needed for the company. If needed, what are the expectations you wish to achieve. Risk Assessment- Planners have to assess the risk factors. If the project fails how much will be the loss, whether direct or indirect etc.
Top-Down or Bottom-Up
Once we have decided to build a data warehouse, we should decide on the approach.
Top-down From enterprise level move down to data marts Bottom-up - From several data marts move up to a single large data warehouse. Each has its own merits and demerits. Depending upon our requirements and resources available ,we should decide on either one of these approaches.
Develop the DW as an in house project or buy the tools that are available in the market Large companies will have a blend of these two Need to find the balance between in-house and vendor software.
Single Vendor or Best of breed- Either use the products of a single vendor or use products from more than one vendor.
High Level integration Seamless cooperation among components Centrally Managed information exchange Overall price Customized environment Select products best suited for the function No need to compromise between database and support tools
Need to focus on what information is needed and not on how to provide information Conduct a preliminary survey of requirements Prioritize and determine the plan for individual data marts
The Project should have the support of the top management right from day one Whenever the project encounters major setbacks, the top official is expected to resolve the issues.
Justifying the data warehouse- Justifying the total cost required for developing a DW by balancing the risks against the benefits is not that easy. If the top management of the company is able to understand the benefits of having a DW then there is no need for justification. But if the top management doesnt understand its importance then a formal justification is needed. One such approach is to find the ratio between current costs and proposed costs and highlighting that to the management.
The Overall Plan- An overall plan sets the direction, tone and goals of the project.
Outline for Overall Plan Introduction Mission Statement Scope Goals and Objectives Key Issues and Options Values and Expectations Justification Executive Sponsorship Implementation Strategy Tentative Schedule Project Authorization
THE DATA WARE HOUSE PROJECTDW DIFFERENT FROM OLTP PROJECT INFORMATION DATA DATA
ACQUISITION
Large number of sources Different Computing Platforms Outside sources Huge initial load Data Replication Complex data integration and transformation
STORAGE
DELIVERY
Storage of large data Several user types Parallel processing Data storage in staging area Several index files Storage of newer data types Archival of old data and new tool compatibility Multiple query types Multidimensional analysis OLAP functionality Meta data management Interface to DSS and feed to data mining
A data warehouse life cycle is entirely different from that of a SDLC. A DW life cycle should include repetitive tasks going through cycles of refinement. This is because of the complexity and broad scope of the project. The functional components of a DW are data acquisition, data storage and information delivery. The phases of the development should be based in these functional components.
P R O J. P L A N
DATA STORAGE
INFORMATION DELIVERY
Success of a project depends on the project team. The project team minimizes the complexity by assigning the right task to the right person. Each person in a team is assigned with a specific role and responsibility. Organizing the Project Team
Putting right person in the right area List all the project challenges and specialized skills that are needed Prepare a list of team roles needed to develop the project Assign the roles to individual or group as per the requirement. Skills, experience and Knowledge are important for team members.
Roles
Executive Sponsor Project Manager User Liaison Manager Infrastructure Specialist Business Analyst Lead Architect
Responsibilities
Direction,support,Arbitration Assignments,Monitoring,Control Coordination with user groups Infra. Design or Construction Requirements Definition Architecture Design
Responsibilities
Relational and Dimensional modeling
DBA functions Data Extraction, Integration and Transformation
Quality Control
Program, System & tools testing Confirm data meaning/relation In house projects
Lead Trainer
Business Analyst
Data Modeler
THE DATA WARE HOUSE PROJECTTHE PROJECT TEAM- SKILLS & EXPERIENCE
Roles
DW administrator Data Transformation Specialist Quality Assurance Analyst Testing Coordinator End User appln specialist Development Programmer Lead Trainer
User Participation
Project Planning
Requirements Definition
Identify all source systems Define metrics for measuring business success Define information needed from DW.
Review dimensional data model Data extraction and transformation Review architectural design
Design
Construction
Participate in user acceptance testing Validate data extraction and transformation functions Test usage of meta data Verify audit trails Confirm initial data load Arrange and participate in user training Provide input for enhancements Test and accept enhancements
Deployment
Maintenance