Professional Documents
Culture Documents
K17406C Group1 Production PDF
K17406C Group1 Production PDF
----------
FINAL REPORT
Students:
2
Design data warehouse and analyze production data in business
3
Design data warehouse and analyze production data in business
4
Design data warehouse and analyze production data in business
• Data repository
- The phases of data warehousing are reflected in the typical structure, the so-
called reference architecture of data warehouse systems. Of course, the system
architecture of a DW changes depending on the specific product and provider.
Fundamentally they orientate themselves around a technical structure based
on a modular model, made up of three levels:
• Data collection
• Data repository
5
Design data warehouse and analyze production data in business
• Data provision
6
Design data warehouse and analyze production data in business
- The facts table contains so-called facts: the performance and result figures of
the company that need to be continuously held onto (e.g. Work order), and
facts in fact tables are additive, which means it can be calculated in order to
achieve higher result that the original data record simply cannot provide.
- The dimension table contains attributes with which the data from the facts
table can be described from. It refers to a collection of reference information
relating to the saved events within the table of facts.
- In a star schema, the facts table is simply connected with all dimension tables
via foreign key relationships. Connections between individual dimension
tables are established.
- In this specific project, we will be using Dimensional Model to build our data
mart in order to store data for further analytics purposes. The Dimensional
Model is the fundamental for the Bottom-up Data Warehouse building method
developed by Ralph Kimball, because this specific methodology can solve
two major and critical problem of a Data Warehouse.
- For a data warehouse to be used at its best, the user must be able to recognize
and familiarize with the new system fast, which means that the data warehouse
must be easy to understand, and the Bottom-up methodology is designed
primarily for this kind of requirement. The second and most crucial in any
information systems is that every data warehouse developed must be
optimized for querying speed, imagine that your company and the rival
company is in an intense competive situation and even one single decision
made is significantly affect the whole result, so the querying process from
your data warehouse must be faster than the enemy for you to take an upper
7
Design data warehouse and analyze production data in business
- In the Bottom-up methodology, each Data Mart must be created first, and then
all created Data Mart will be integrated to become one entity, which is the
Data Warehouse. So what are we going to do exactly in this project is that we
will be building the data mart for the production process based on the OLTP
database of the Adventure Works databased provided by Microsoft.
• Microsoft SQL Server Analysis Services (SSAS): Store and query data
in Cube format for Data Warehouse.
8
Design data warehouse and analyze production data in business
- Step 5: Recognize fact data and build fact tables based on answering the
objectives from step 2. Every data record in this design have to be appropriate
with the fact grains measure as stated at step 3.
- Step 6: Figure out all relationship the fact tables have with each individual
dimension table.
- Step 7: Transforming and normalizing data types and data keys. Dealing with
null, random and noise values.
- Step 8: Choosing ETL tools and transfer data from data sources into Data
Mart, in this case we will be using SSIS provided by Microsoft.
- Step 10: Build reports as required initially with data from Data Warehouse.
Use the BI and SSRS tools to extract data to process or generate reports for
users to view. When extracting data from Cubes we use the MDX language to
query.
2.4 BI Systems
2.4.1 What is BI?
- BI stands for Business Intelligence, which combines products and
technologies and methods to organize the main information that governs in a
reasonable way to improve profitability and performance of an organization.
In a wider manner, BI is business information and business analytics for key
9
Design data warehouse and analyze production data in business
• BI turns data into usable information. Raw data doesn’t tell us what to
do in business all on its own. BI systems allow for comprehensive
analysis of data to identify important trends that can be used to modify
or implement strategic plans and to understand the interconnections
between different functions and facets of your business.
10
Design data warehouse and analyze production data in business
11
Design data warehouse and analyze production data in business
12
Design data warehouse and analyze production data in business
- The process begins with a request for production that is typically triggered by
another process such as fulfillment process, which needs to complete a
customer purchase order (make-to-order strategy), or by material planning,
which has determined that the company needs to increase inventory levels
(make-to-stock strategy).
- In such cases, data about the order are transmitted to the external system. After
the finished goods have been produced, the actual production is confirmed in
the system, signaling that the steps required to manufacture the materials have
been completed. The materials are then moved to storage, and the system
reports that they are now available for consumption by other processes (e.g.,
fulfillment). In addition, several activities are performed periodically during
the process, including overhead allocation, work in process determination, and
order settlement.
13
Design data warehouse and analyze production data in business
14
Design data warehouse and analyze production data in business
- Scrap rate can be used as a KPI (Key Performance Indicator) to measure the
quality of the production output in a specific manufacturing process. In a
production site, a certain amount of scrap is inevitable; it is a natural
consequence of the manufacturing process but should be detected early in the
manufacturing flow.
15
Design data warehouse and analyze production data in business
- The production manager first estimates the future market or demand for the
company goods and services. Then he estimates the Men (human resources),
Money (financial resources), Materials, Machines, and Methods, which will
be required to produce those goods and services. In this part, our group will
focus on the main materials needed to produce finished products of the
Adventure Works company.
16
Design data warehouse and analyze production data in business
- We also using Sales and Purchasing data to support our production process
analysis.
17
Design data warehouse and analyze production data in business
18
Design data warehouse and analyze production data in business
- Product Type:
- Product sell start date and sell end date: These two attribute indicating the date
a specific product is started to be on the market by Adventure Works, and the
end date is the date that the product is no longer sold by Adventure Works
19
Design data warehouse and analyze production data in business
also the storage location for the products associate with each specific work
order routes, so it appears in both ProductInventory and WorkOrderRouting.
20
Design data warehouse and analyze production data in business
- These types indicate where do the goods come from and where it being
exported out to, which will tell us about which direction of goods each
transaction was meant to go.
- But when we created the Inventory Transaction fact table, there are some
records which do not have any transaction, so the Transaction Type attribute
of these records are null, and according to our knowledge about data
warehouse design, we have to deal with these null value, so we created a new
transaction type coded “N”, meaning “No transaction” for these records.
21
Design data warehouse and analyze production data in business
- The range of value of Time dimension table is from year 2010 to year 2014
22
Design data warehouse and analyze production data in business
23
Design data warehouse and analyze production data in business
24
Design data warehouse and analyze production data in business
specific product one location belong to one operation sequence; has planned
manufacturing start date and planned manufacturing end date; has actual start
date and actual end date; has number of manufacturing hours used; has
estimated manufacturing cost; has actual manufacturing cost; has date and
time the record was last updated.
26
Design data warehouse and analyze production data in business
- This fact table is created with references from Dim_Product table and
Dim_Transaction_Type, and some records reference from Fact_Work_Order
(because there are transactions from Work Order).
- This table works as a snapshot for each product in every single date from the
Dim_Date table, which means every record in this table represent a transaction
of a product in a specific day.
27
Design data warehouse and analyze production data in business
- We have 504 products, multiply it for 1826 date we get from Dim_Time table,
then we have more than 900000 lines of transaction, but after checking the
result later, we found out that the transactions have not being recorded until
2011, so we drop every line from 2010 from the table, and now we have only
about 838209 records contain transaction data of every product from 2011 to
2014.
- Originally we want to convert each record in this table into transaction types
such as In movement and Out movement, but we cannot figure out what
movement exactly are from Work Order, so we kept transaction types as
default (W, P,S and N for No Transaction)
28
Design data warehouse and analyze production data in business
29
Design data warehouse and analyze production data in business
4.1.3. Diagram
30
Design data warehouse and analyze production data in business
• Step 2: New SSIS Package > Data Flow Task > Double click on Data
Flow Task > OLE DB Source from SSIS Toolbox > Double click on
OLE DB Source > Select source database > Data access mode select
SQL command
31
Design data warehouse and analyze production data in business
• Step 4: Drag Slowly Changing Dimension from the toolbox out, choose
the destination and check input, output table integrity
• Final result:
• The product dimension table is the most important table for our
production process. Because we are creating this data mart with the star
32
Design data warehouse and analyze production data in business
• The result coming out is so good, there are some dimension which we
were able to stuff them in this dimension table and these are the way
we deal with null values:
33
Design data warehouse and analyze production data in business
35
Design data warehouse and analyze production data in business
• This table is a very special table, this table is crucial for everything we
are doing in this project, without this, nothing could have been done
efficiently. Originally, we figured out 2 methods to build and populate
this table. The first one is by using Analytics Project in Microsoft
Visual Studio, but this method did not work correctly for us, so we
decided to deal with this table manually by using a special T-SQL query
provided by our teacher Pham Van Tuyen to generate a time dimension
table which contains values from January 1st 2010 to December 31st
2014 (this is the time limit provided by Adventure Works database):
36
Design data warehouse and analyze production data in business
• The result:
• Also, the result came our quite good and we were happy by it, but as
we have done for all other date attribute, we have changed the data type
of the Date_Key attribute (primary key) of this table to nvarchar(8) in
the table design interface.
ETL Fact_Inventory_Transaction
- This table were created by specify every transaction of each product in every
single day in the Dim_Time table. As we stated in the previous section, the
37
Design data warehouse and analyze production data in business
table now contains over 800000 records of product multiplied by date, so now
we begin the ETL process of this table to populate it with transaction data.
- We used two tables available in the Adventure Works OLTP database, the
Production.TransactionHistory and Production.TransactionHistoryArchive
because we want to get all the data provided for the most accurate analytic
later.
- Then we “JOIN-ed” these two tables together using T-SQL and now we have
a data table about all the transactions ever recorded in the system:
- As you can see, the table name and the date was not exactly modified, so we
modified them and the result came out with a beautiful and a cleaner look:
38
Design data warehouse and analyze production data in business
- Then we “JOIN-ed” this table with the “Product x Date” we created before,
and now we have the complete transaction table with data from 2011 to 2014.
- Then we use SSIS to populate the table, we also used Sort to sort the record
by Product and then by Date.
ETL Fact_Bill_Of_Materials
39
Design data warehouse and analyze production data in business
- In this table, we use the same method of ETL as before. We also changed the
Componen_Unit into full string instead of just Code like the original table
from Adventure Works.
40
Design data warehouse and analyze production data in business
41
Design data warehouse and analyze production data in business
- Choose data source. After clicking OK. Visual Studio will prompt you to enter
information to authenticate database access. You can enter the User Name and
Password you created in SQL server. Here we use the default account mode
of SQL Server so select Radio Button Use the service account.
42
Design data warehouse and analyze production data in business
- The dialog box will then appear so you can change the name of the Data
source. You can also check the connection string. Finally, click Finish to
finish creating the Data source step for SSAS.
- Step 3: Create Data Source View Right-click Data Source Views, then select
New Data Source View.
43
Design data warehouse and analyze production data in business
44
Design data warehouse and analyze production data in business
- Select the dimension tables and event tables you want to analyze. After
selecting, click Next. Then, Visual Studio displays a dialog box that allows
you to change the name of Data Source View. Finally, click Finish to finish
46 the step of creating the Data Source View to create a view that combines
the dimension tables and event tables.
45
Design data warehouse and analyze production data in business
- Step 4: Create Cube Right click on Cubes select New Cube to create new
Cube.
46
Design data warehouse and analyze production data in business
- Cube Wizard dialog box created for you to choose how to create Cube. Here
we use the method Use existing tables (Use existing tables). Then click Next.
47
Design data warehouse and analyze production data in business
- Select the Fact table you want to analyze. Then click Next.
48
Design data warehouse and analyze production data in business
- Select the dimension table you want to analyze. Click Next and Finish.
- Result:
49
Design data warehouse and analyze production data in business
50
Design data warehouse and analyze production data in business
51
Design data warehouse and analyze production data in business
• Name: ScrapRateKPI
• Goal Expression:
• Status Expression:
52
Design data warehouse and analyze production data in business
• Trend Expression:
- Step 3: After setting KPI, click tab browser, choose Excel symbol to Analysis on
Excel.
53
Design data warehouse and analyze production data in business
• Name: ProductManufacturingCostKPI
• Goal Expression:
• Status Expression:
• Trend Expression:
- Step 3: After setting KPI, click tab browser, choose Excel symbol to Analysis on
Excel.
54
Design data warehouse and analyze production data in business
55
Design data warehouse and analyze production data in business
5.2. Data analysis with the Pivot Table tool in Microsoft Excel
56
Design data warehouse and analyze production data in business
Pivot tables are one of Excel’s most powerful features. A pivot table allows you
to extract the significance from a large, detailed data set. Pivot Table has great
flexibility and can be adjusted quickly according to the need to display the results
we want. Based on the Pivot Table, we can use the Pivot Chart to model more
specific data using charts.
- Step 1: Open Microsoft Excel 2016. Select Data from the Menu, select From
Other Sources and select the connection type From Analysis Services
- Step 2: After selecting the connection type, Excel will display a Form for you
to select the Database Server to connect. Here we enter the "." In server name
to connect to Database Local, then select Next.
57
Design data warehouse and analyze production data in business
- Step 3: In this step, Excel will display a Combobox for you to select the data
block you want to analyze. In this case, we choose to analyze Sales data block,
select Next
58
Design data warehouse and analyze production data in business
- Step 4: In this step, Excel wants you to save the connection string. Here I leave
the default, if you want to change the address, click Browse ... to change the
path for it. You can also add a description in the Description section. Then,
select Finish.
- Step 5: Excel asks you to check the Pivot Tale Report format or you want to
format it and select the data display area
59
Design data warehouse and analyze production data in business
- Step 6: After importing the data into the Pivot Table for analysis, we have
result
60
Design data warehouse and analyze production data in business
61
Design data warehouse and analyze production data in business
62
Design data warehouse and analyze production data in business
- Step 2: Select the Databases to analyze. When you conduct SSAS database
using Visual Studio, the database will create a Cube database in SQL Server
Analysis mode for you to analyze in MDX language.
63
Design data warehouse and analyze production data in business
64
Design data warehouse and analyze production data in business
- There are 504 products divided into 4 categories: raw materials (not salable,
purchase), semi-finished goods (not salable, in-house), trading goods (salable,
purchase), finished goods (salable, in-house). The results in the picture above
have products with no scrap rates, those products are of categories trading
goods and raw materials because they are not manufactured but purchased.
The remaining 2 types of products, finished goods and semi-finished goods,
are manufactured, so there is a scrapped rate. We can see BB Ball Bearing
product has scrap rate by year: 2011 (~3.68), 2012 (~2.09), 2013 (2.13), 2014
(~7.01).
65
Design data warehouse and analyze production data in business
- The above MDX statement results in the listing of transactions for the
products. As mentioned before, there are 3 Transaction Types:
• Work Order
• Purchase Order
• Sales Order
66
Design data warehouse and analyze production data in business
67
Design data warehouse and analyze production data in business
• There are five most common reasons: Paint process failed, Trim length
too long, Thermoform temperature too low, Drill size too small, Wheel
misaligned
• The last is the product with the highest cost and most scrapped rate.
- Recommend Solutions:
• Improve location efficiency to reduce production time or consider
flexibly using alternative locations with lower costs while ensuring
production.
68
Design data warehouse and analyze production data in business
69
Design data warehouse and analyze production data in business
- This figure express the time, the cost planned and in reality of the company
from the Adventure Works Production Data Warehouse that we built. We can
realize that every actual cost completely matched the planned cost.
- This result ensure that every following stage can be execute on time, and on
schedule. The actual production date always have to be planned ahead. The
accuracy goal is achieved.
70
Design data warehouse and analyze production data in business
71
Design data warehouse and analyze production data in business
- As we can see in the table above, the product quantity always above the
reorder point and safety stock level.
72
Design data warehouse and analyze production data in business
- For every record that the figure above that lower than the Reorder Point will
immediately be procured and/ or manufactured to balance out the inventory.
73
Design data warehouse and analyze production data in business
really support the production process, make sure it runs smoothly and sticks
to the plan, limits mistakes and errors to the smallest number.
6.2.2 Forecasting
- In the production process, the trigger of every manufacturing activities is
configured based on two different process, make-to-stock and make-to-order.
The planning stage of the production process is not only helps pushing up the
performance of the make-to-order process, makes it more efficient, it also
makes the execution stage of the production process happens smoothly, avoid
accidental risks.
- In the make-to-stock process, if you have a detailed production plan, it will
undoubtedly ensure the readiness of your stock.
- In order for us to make a good production plan, we have to consider all the
necessary transactional data in the past and other parameters to forecast
accurately, limits risks to minimum level. The data have to be actual data,
detailed and drilled down to the smallest level, ensure the accuracy and the
reliability of the forecasting result.
- We did run analysis on the transactional data of the procurement process,
production process and fulfillment process to derive the information needed
to evaluate the production planning process in the past and also predict the
production process demands in the future.
74
Design data warehouse and analyze production data in business
75
Design data warehouse and analyze production data in business
- Next figure, we can recorgnize that in the time before Q1 2013, the production
process had the largest number, as in two figures above, we can say that in
this period, the most focused products are bikes and other related components.
Other reselling products like trading goods are not really the strength of the
company in this specific period.
- However, after Q1 2013, there is a dramatic change in the figure, it is a burst
in the reselling department while the production department is decreased
apparently.
- This indicates the shifting in the business goals, they no longer focus so much
on the manufactured products and shifted their focus to other trading goods.
- In addition, when looking at the Sales Line, we can easily see that in some last
period of time from 2011 to 2014 of the data, the sales line tends to shifting
to the purchase line.
76
Design data warehouse and analyze production data in business
- From here, the process of forecasting future trend can help we realize the
changes in the way of doing business in the company, for instance, the number
of manufacturing products has the trend to go down very clearly.
77
Design data warehouse and analyze production data in business
78
Design data warehouse and analyze production data in business
79
Design data warehouse and analyze production data in business
CHAPTER 7: CONCLUSION
7.1. Results
- Understand the meaning of implementing BI reports.
• Documenting how to analyze source data, ETL.
• Know how to use the SQL Data Tools toolkit to create BI reports.
• Build a BI reporting system to support administration.
• Increase the ability to self-study and research a problem.
• Improved ability to read English book.
7.3. Development
- Examine more practical needs from customers to have specific requirements,
accumulate knowledge to develop, support ERP projects that need to use BI
reports at work.
- Further processing and intervention in the research of source data and ETL
for better data quality and more reliable results.
- Extend interactive features in reports and nested reports.
80
Design data warehouse and analyze production data in business
CHAPTER 8. REFERENCES
- [1] - https://www.ionos.com/
- [2] - https://searchbusinessanalytics.techtarget.com
- [3]- https://docs.microsoft.com/en-us/sql/analysis
services/multidimensionalmodels/mdx/mdx-query-the-basic-
query?view=sql-server-2017
- [4] - https://powerbi.microsoft.com/en-us/
- [5] - Ho Trung Thanh - Data warehouse analysis in Business
81