Star Schema and Technology review
Musa Sami Ata Abdel-Rahman
Supervisor: Professor Sebastian Link
Postgraduate Project in Computer Science
Abstract— Data warehouse (DW) is a collection of decision
support technologies, aimed for enabling the knowledge worker
to make better and faster decisions. We are living in the Era
of Big data and several evolutions happened in terms of data
resources and data formats for the last decades. That enforces
academia and industry to search deeply about alternatives
and to provide better solutions as a response to these new
factors in the way that can make the DW still valid to work
on the same way or better compared with the past solutions
that were provided through traditional databases. This paper
concentrates on the most important component in DW systems,
Star Schema. The paper explores the Data warehouse concept,
OLAP, and compares Star schema with other data modelling
like 3NF and SnowFlake and shows why we need Star schema.
Within that we make a highlight on the evolution of database
systems and shows several studies that were made before about
converting star schema from traditional DBMS to other types
of DB systems. In the end, we make a review from the industry
and show what the commercial vendors provides to implement
Star schema within the products that they have.
Index Terms— Data warehouse,Star
Schema,OLAP,NoSQL,NewSQL,RDBMS,Business Intelligence
1. I NTRODUCTION
We are living in times that everything is increasing,
especially when we talk about data. The universe is moving
fast, and more data is generated exponentially every day.
With all data coming on your way, you need the right tools
to deal with the growing amounts of data. The growth in the
quantity and diversity of data has led to data sets larger than
is manageable by the conventional, handson management
tools. To manage these new and potentially invaluable data
sets, new methods of data management and new applications
in the form of predictive analytics, have been developed.
According to [1], The most important assets of any company
is its information, which is always kept into two main forms:
operational systems of records and data warehouse. Large
corporations are overwhelmed with data. They invest in data
facilitation to support them in the decision-making process,
including reporting, dashboards, advanced visualisation,end-
user self-service and initiatives strategic to business intelli-
gence. The main concern for these companies is not only
how to keep data, but how to narrow the gap between the
time of the data generation and the time of gaining benefits
hidden within it. in other words, The challenge for companies
today, many beholden to legacy systems and IT bureaucracy
is that this growing demand for analytics often outstrips
their ability service it. That would make dealing with Big
data analytics is another challenge that these corporates are
facing, especially when they have multi-systems and the
wide range of data distribution resources, so the effort will
not be only in unlocking the potentially useful information,
but also in gathering, processing and consolidating these
data before starting the analytics process.Generally, data
management is a set of applications of quantitative and
qualitative methods to solve relevant problems and predict
outcomes. One of the salient revelations of today, with the
vast and growing amount of data, is that domain knowledge
and analysis cannot be separated. It is Mandatory to have a
full background about the situation that we need to analyze,
and that can be done by aligning the technical side and the
business side together to gain results. It is important to the
technical team to be aware of the domain of business, and on
the same time, the business should be aware of the new ideas
and technologies that can be effective for delivering analysis.
To satisfy that, we need to know about technologies that can
support corporates to manage their data and the facilities
that can help the Decision makers to achieve from the data
warehouse since it is an essential element of decision support
process [2]. The evolution of big data enforces to make
Several Types of research on Data warehouse to enhance data
delivery. These researches are done on the concept itself or
go deeply on each data warehouse components, Like data
modelling or ETL (Extract, Transform, and Load) process.
Researchers are not only within academia but also within the
Technology vendors as well. Many commercial products and
services are now available, and all of the principal database
management system vendors now are offering solutions in
these areas. Recently, some of them asked whether the data
warehouse is still alive or not? According to Eckerson and
Cloudera, Data warehousing is not dead, but it is struggling.
Many challenges of Legacy Data Warehousing enforce it to
evolve and adapt to fit with the realities of modern data
management and to overcome the challenges of scalability
and elasticity, data variety, data latency, adaptability, data
silos and data science compatibility. But that does not prevent
as we said before that the Data warehouse can be enhanced
and be a part of solutions for other new data management
concepts, like Data lake. According to Horton networks, both
environments are complementary to one another in modern
business. We can use their features and facilities of both to
cross the big data challenges by having integration between
both of them and break the gaps that they have. This paper
is trying to explore the Data warehouse and focusing on the
most important concept that it provides,star scehma, and how
the current commercial products available are dealing with
star schema and how they support this concept. We need
not forget that the database concept has been changed for
at least the last 10 years. Before we were dealing with the
classically structured databases, Which serves the relational
and structured data, but now we have other types within the
industry, semi-structured and unstructured data. We will take
into consideration these new facts and see how vendors deal
with that to implement star schema, or whether they give a
solution or not to have a star schema.
2. DATA WAREHOUSE OVERVIEW
A. Definition
We have different definitions for Data warehouse. A
widely accepted definition for Data warehouse is a collection
of decision support technologies, aimed at enabling the
knowledge worker (executive, manager, analyst) to make
better and faster decisions [1]. A data warehouse is a subject-
oriented, integrated, time-varying, non-volatile collection of
data that is used primarily in organizational decision making
[2]. According to Ralph Kimball and Margy Ross Typically
[1], Data warehouse and operational systems are corre-
lated, but have different aspects. The operational systems
are optimized to process transactions quickly, on the other
hand, Data warehouse (DW) is responsible to evaluate the
organizations performance over time, by keeping eye on the
operational systems which feed the data warehouse. Different
terms are also used to distinguish between the two concepts.
For a data warehouse, we can use (OLAP), online analytical
processing, and (OLTP), online transaction processing, to
describe the operational systems. Data warehouse has the
main Goals to achieve[1]:-
‚ it must make an organizations information easily acces-
sible
‚ it must present the organizations information consis-
tently
‚ it must be adaptive and resilient to change
‚ it must be a secure bastion that protects our information
assets
‚ it must serve as the foundation for improved decision
making
‚ The business community must accept the data ware-
house if it is to be deemed successful
B. OLAP and OLTP
As an extension to the definition and characteristics of
data warehouse, we need to define the distinction between
the OLAP and OLTP. In general, the main differences are:-
‚ Data contents and system orientation
‚ Database Model Design
These two points are summarizing the differences between
both systems in terms of performance and workload. That
comes from the type of operations that we use on both
systems, the size of data, and the purposes of a design
adapted in each of them. The OLTP mainly uses insert
and update data manipulations (DML), and the database
is designed to deal with a specific objective, which means
distributed facts within several systems. In contrast, OLAP
uses SQL queries for reading data, does not need any kind of
manipulation data operations like OLTP, and that comes from
the design used which mainly guarantee a data consolidation
from heterogeneous sources (i.e. several OLTP sources).
The database modelling in both systems are different and
we can say it is the major factor that controls and lead
the differences between both of them. OLTP is using ER
modelling, 3NF (3rd normal form) or higher. It eliminates
data duplication, provides efficient storage for detailed data,
but not suitable for data analysis. When we look to figure 1
which represents the classic sales system, the ER model has
7 tables comparing with 4 tables that the dimensional model
has. it is clear that ER modelling would have a complex
query to gain a high level of data comparing with the
dimensional model since we need to make 6 join operations
comparing with 3 joins on the other model. We know that
join operations will decrease the performance, especially
when we have a huge number of records [3]. We can notice
that the Dimensional model is simple and clear for any
user, unlike the ER model which needs a technical user to
understand, and that is not helpful in data analytics.
Decision support systems mainly concentrate mainly on hav-
ing multidimensional data, which needs special operations
like roll-up (increasing the level of aggregation) and drill-
down (decreasing the level of aggregation or increasing
detail) along one or more dimension hierarchies, slice and
dice (selection and projection), and pivot (re-orienting the
multidimensional view of data) [2].
Fig. 1. ER Model versus Dimensional model[3]
C. Data Warehouse architecture
Typical architecture is shown in figure 2. As we can see
it is a combination of systems and operations. OLTP system
and external data sources are the main input for the data
warehouse. ETL process which is mainly responsible for
data manipulation, in terms of loading, transformation and
extracting data from its resources. Data cleaning is one of
the main duties of ETL, which includes handling missing
values and parsing into a standard format. You may find in
other resources that another component is added between
OLTP and data warehouse, it is called ”Operational Data
Stage” (ODS) [1]. It is mainly an additional step that can
be added between the OLTP and Data warehouse. Instead
of transferring data directly, we can put ODS in the middle,
which will contain the needed data for DW system, and after
doing the cleaning part, we can after that transfer it to DW

f Dimension tables are integral companions to a fact table.
Fig. 2. Data Warehousing Architecture[4]
system. Another component that you can is Data Mart. It
is derived from the data warehouse, but more high level
and usually oriented to a specific business line or team. it
enables faster rollout, since they do not require enterprise-
wide consensus, but they may lead to complex integration
problems in the long run if a complete business model is not
developed [2]. The main important parts of DW architecture,
which is related to our study are Data modelling, Database
management systems and methods used to fetch our data.
The data modelling within the data warehouse should reflect
the OLAP structure, which means a multidimensional model
that can facilitate complex analyses and visualization. It is an
important technique for making databases simple to Deliver
data thats understandable to the business users and Deliver
fast query performance [1]. We have several Database man-
agement systems that have utilities and characteristics used
to implement the source and target data stores. Basically, it
should satisfy the requirements needed to implement a data
warehouse system. That includes supporting a dimensional
data model and providing a language that can be used to
fetch and process the data [4]. The evolution in database
systems brings other languages besides SQL ( Sequential
Query language) and its extension, like Object-oriented.
Since a data warehouse reflects the business model of an
enterprise, an essential element of a warehousing architecture
is Metadata management. It is divided into three types;
Administrative metadata, Business metadata, and Operational
metadata [2]. Administrative metadata is used to describe
the format, structure, syntax, and semantics of data, such
as justification and spacing. Business metadata includes
business terms and definitions, ownership of the data, and
charging policies. Operational metadata includes information
that is collected during the operation of the warehouse, such
as usage statistics, error reports, and audit trails.
3. D IMENSIONAL M ODELING R EVIEW
A. Basic concept
Dimensional modeling uses two major types of tables:
Fact table and Dimension table. Fact tables hold numeric
data that can be summarized as needed and contain infor-
mation about specific business processes or events within
these processes. Each row in a fact table corresponds to a
measurement event and represents a specific level of detail
which the facts may be defined and summarized, referred to
as the grain[5]. The measures have different types of [6]:
‚ Additive: measures that can be added across all dimen-
sions
‚ Nonadditive: measures that cannot be added across all
dimensions
‚ Partially additive: measures that can be added across
some dimensions and not with others
The dimension tables contain the textual context associated
with a business process measurement event and describe the
who, what, where, when, how, and why associated with that
event [1]. In the dimensional model, the data is described
by the relation between the fact table and dimension table.
The fact table contains the content of the data warehouse
and it is surrounded by several dimension tables contain
the description. Besides measures, Fact table has also keys,
which consist of a group of foreign keys (FK) that point
to the primary keys of dimension tables that are associated
with this fact table to enable business analysis[6]. Figure
Fig. 3. Example of a Fact table with dimension tables (Star Schema)
3 gives an example that clarifies the relationship between
a fact table and dimension tables. The join cardinality of
dimension tables to fact tables is one-to-many because each
record in a dimension table can describe many records in the
fact table. Dimension Table has another important function.
Each dimension table consists of columns that correspond to
attributes of the dimension, and the attributes of a dimension
may be related via a hierarchy of relationships. For example,
in Figure 3 we have a dimension table called (DimDate),
it contains (Year, Quarter, Month, Week, Day), and it is
connected to the fact table (online-Sales) which contains
a measure called (Revenue), then we can summarize the
revenue at different time level by drill-down or roll-up.
The strength of any dimensional modelling comes from the
summarizing that the model can provide. That can be done
with high tuned queries between the fact table and dimension
tables, or building a separated summary tables fact table with
specific dimensions [2].
Materialized view is another approach of summarizing that
combine both physical implementation and queries. It is a
static view that contains the results of an aggregated query
and has the same concept of a summary table. Instead of
executing complex queries, the Materialized view is rep-
resented as a database object. It facilitates access to the
fact table and the desired dimension, and it reflects any
changing in the source data (fact table) without re-execute
the complex query that it represents [2]. Some Data architects
think that this separation of the data into facts and dimensions
would lead to complex queries, and that would affect the
system performance. As an alternative solution, they come
up with Flat Table, which is collapsing all entities in the data
model down to into the terminal entities. The flat table will
minimize the number of tables in the database and therefore
minimize the possibility that joins will be needed in end-
user queries [6]. With this approach, there is no loss in
information that the original data has, but it will lead to other
problems like massive redundancy, and it will not contain any
aggregations. In fact, it may lead to aggregation errors when
there are dependency relationships between kernel entities.
B. Star schema and Snowflake schema
Data warehouses are typically based on relational star
schemas or snowflake schemas, an approach to implementing
multidimensional databases using relational database tech-
nology. When we need to talk about any of them, we
can not neglect the other, especially when we know that
SnowFlake schema can be considered a special type of
extended star schema, since it represents a connected lattice
of star schemas.
Star schema consists of a central fact table surrounded by
dimension tables as shown in 3 [7]. When the model contains
multiple fact tables share dimension tables as shown in
figure 4, then we call it Constellation schema or Galaxy
schema [8]. According to Kimball, The Design Approach
Fig. 4. Galaxy schema
for Star Schema is based on the analysis of business process
requirements, by identifying the grain of the data, defining
the relevant facts that need to be aggregated, defining the
dimensional attributes to aggregate by, and then forming
a star schema based on these [6]. Dimension table in star
schema represents set of attributes which are not normalized,
which means that star schema does not support a direct
attribute hierarchy [2] and that is the main difference be-
tween star model and snowflake model. Snowflake makes
normalization to its dimension tables, and that would give
multi-level hierarchy, unlike star schema which gives a single
level of hierarchy [4]. When we compare between figure 3
and figure 5, we can see that no change happened to the
fact table, but the approach for dimension table is different.
Dependency chains in star schema are collapsed, while in
snowflake schema are expanded out. Snowflake is near to
Fig. 5. Snowflake schema example
(3NF) in normalization, and that is reflected by breaking
dimension tables to avoid redundancy, and that done by
representing each level as a separate table that includes a
key and the descriptive attributes of the level [6].
C. Star schema versus Snowflake schema
Both models can be used for data warehouse systems. In
general, star schema is the most used, but that does not mean
it is free of drawbacks. Star schema has several advantages:
it is simple and easy to understand. It reduces the number
of tables in the database and the number of relationships
between them, so the number of joins required in user queries
is minimized, and that will lead to a higher performance [6].
Finally, the normalized approach that star schema has makes
the aggregations of measures are straightforward and that
makes star schema a prefered choice for business people.
However, the star schema has some drawbacks. For example,
it does not model hierarchies adequately since the hierarchy
structure is not clear and additional information is required
to represent it. Sometimes it is difficult to associate attributes
within their corresponding levels. Moreover, when we have
many levels in hierarchies, the number of attributes is at
least as large as the hierarchy depth, making the hierarchy
structure difficult to understand [9].
Snowflake is prefered for technical people since the Normal-
ized tables are easy to maintain and optimize storage space.
Besides that, it represents hierarchical structures better. Every
level can be easily distinguished and, further, levels can
be reused between different hierarchies. Additionally, in
this representation, specific attributes can be included in
the different levels of a hierarchy [9]. On the other hand,
snowflake schemas are less performant for querying due to
the joins that are needed for combining the data scattered
in the various tables composing a hierarchy. The (3NF)
that snowflake has would bring the same disadvantages that
we discussed before in section 2-A. ETL process these
days uses different techniques like pipelined, partitioned, and
parallelism to avoid sequential loading [2]. with Snowflake,
ETL process would be affected since snowflake is more
complex in design and cannot be parallelised because the
dependency within the model restricts it.
Both models are implemented in real life, and some people
like Charles D.Tupper [6] does not think that one of them
is the best solution, but the optimal one is a compromise
between the two options. Business and use case can be the
main factor to decide. According to Oracle [10], Ralph Kim-
ball proposes three cases where snowflake implementation is
not only acceptable but is also the key to a successful design:
‚ Large customer dimensions where, for example, 80 per
cent of the fact table measurements involve anonymous
visitors about whom you collect little detail and 20
per cent involve reliably registered customers about
whom you collect many detailed data by tracking many
dimensions
‚ Financial product dimensions for banks, brokerage
houses, and insurance companies, because of each of
the individual products have a host of special attributes
not shared by other products
‚ Multienterprise calendar dimensions because each or-
ganization has idiosyncratic fiscal periods, seasons, and
holidays
Furthermore, Kimball usually advises that it is not a good
idea to expose end users to a physical snowflake design,
because it almost always compromises understandability and
performance. In his book (The Data Warehouse Toolkit) [1],
he is encouraging not to use snowflake since it does not
have ease of use like star schema. Even the advantages that
snowflake provides, like saving disk space and ability to
browse within a dimension, Kimball thinks that they would
be minor issues comparing with the primary objectives that
the design should handle, performance, optimization and ease
of use. We can sacrifice these advantages to gain the primary
objectives [1].
Complexity is not only determined by query and loading, but
also the ability of changes that can be done in the future.
For that purpose, Christian Kaas, Torben, and Rasmussen
[11] made a study about schema evolution for stars and
snowflakes. They made an investigation about how both
models response to changes, instance adaption and impact
on existing queries by implementing eight evolution opera-
tions that cover insertion and deletion of dimensions, levels,
dimension attributes, and measure attributes the evolution
properties of star and snowflake schemas. They came up with
a conclusion that In all cases, the star schema was either
superior to the snowflake schema or had the same problems,
and it is considerably more robust towards schema changes
than the snowflake schema.
D. OLAP and Star Sceham
Within the research, I found that in the past there was
a separation between the OLAP concept and star schema.
In Fact, The OLAP was representing the multidimensional
model that the decision makers were looking for and usually,
they presented OLAP logically as a cube, like a figure
6. We can see that it gives highly interactive support to
decision makers by providing the functionality needed for
summarizing, viewing, and consolidating the information
available in data warehouses [1]. At that time, Vendors were
implementing OLAP into 2 types: Relational OLAP (RO-
LAP) and multidimensional OLAP (MOLAP). The physical
implementation for ROLAP is simply a star schema and
MOLAP is an array format. MOLAP can be considered as
the nearest approach for the cube model, and many vendors
like Oracle, Microsoft and IBM provided MOLAP solutions,
but the hard effort to maintain and administrate, especially
with the evolution of the data and its amount, makes vendors
concentrate on ROLAP approach, especially when you know
that MOLAP has 2 storage layers and indexing techniques
that makes a high limitations on DW performance and data
scalability [2]. So with the evolution in database systems,
you would find papers that discuss the implementation of
OLAP on those systems, they would concentrate on ROLPA
(star schema) rather than MOLAP (OLAP cube). According
to Alfredo Cuzzocrea,Ladjel Bellatreche and Il-Yeol Song
[12], The implementation of OLAP cube over Big data is
considered as an open problem and actual research trends in
the field of Data Warehousing. They Highlighted the reasons
and suggested the future research directions that could serve
in this direction.
Fig. 6. OLAP Cube example
E. Query on Star schema
The main idea of star schema is making the data available
in a simple model, which will simplify loading and access the
data. We have seen that the star schema structure distributes
the data into a large fact table connected to several dimension
tables. In general, when we make a query on star schema for
a specific pattern, it does not mean reading the data as it is
but also includes data processing based on business needs. In
fact, it is a reflection to OLAP operations that we discussed
before in 2-B. The procedure of executing query includes
[5]:-
‚ Selection of required facts: first, the end-user decides
which facts should be selected by identifying the fact
tables to be used.
‚ Selection of required dimensions: by joining dimen-
sions to the query, end-users add required context.
‚ Limiting scope of facts : the facts are filtered according
to dimensional values (either from the fact table or from
joined dimensions).
‚ Summarization of facts: the raw facts are summarized
with aggregate functions, such as SUM() or COUNT().
The first thing comes to our mind when we talk about the
query is SQL, but we will see later that different approaches
come later based on the evolution of database management
systems and data warehouse solutions. Query on star schema
is called Star Query, and join tables is called Star join [1].
The art of executing the efficient query is called Optimizaion,
and measuring the speed of the query is called Performance.
These two factors are the main indicators that differentiate
between data warehouse vendors. They are struggling to
provide access to the data with an optimal query that has
a high speed. Optimization is vital to the performance of the
database system. Each vendor has its own query optimizer
that makes an execution plan to give a robust query and
utilizes the system resources to provide a result with a high
performance (speed) [5]. But How Star Query is affecting
performance? simply by three operations, Star joins, filtering,
and aggregations. When you make a query that includes
these actions, you will initiate two implicit operations, data
loading and data scanning. The mechanism of these two
operations differs based on vendor specifications, and it relies
on several factors, like the algorithm used, hardware and
storage techniques. So when we define a query with multiple
joins, with several filter conditions, and ask for aggregations
with several dimension, then it would raise the workload
internally and in terms of loading and scanning, and that can
be considered as a physical definition of complex query[2].
For this reason, providing means for increasing the perfor-
mance of a data warehouse for analytical queries and other
types of data processing is one of the important research
and technological areas [13]. Many solutions were provided
on the past to enhance the performance related to the data
structure, like providing Materialized view, summary tables,
or even Flat Table within star schema [2], which means
transferring the latency cost to the ETL process. providing
indexes is another solution to enhance join operation and
scan operation , like Bitmap indexing (which is not supported
by snowflake according to Kimball [1]) , Join indexing , and
and Bitmap-join indexing [2] . Later, new challenges came to
the data warehouse and business intelligence (BI) systems,
like making these systems near to real-time, which means
make them sync with the original resources, by enhancing
the ETL process and Query process. Query transformation
is another approach that serves in this direction. It is based
on conducting individual tasks such as translating values,
deriving new calculated fields, joining tables at the source,
converting date fields, and reformatting field sizes, table
names and data type [14]. To make that practical, all major
vendors offer variant strategies in parallel processing, data
partitioning, and Query Extensions [2]. These strategies
differ between vendors since the performance area relies
on three DW components: hardware used, data storage, and
database management system. it is important to know how
each vendor uses these components to format its solution.
that will lead us to know how they implement star schema
physically and on which direction they concentrate on to
enhance the star query performance [13].
4. DATABASE MANAGEMENT SYSTEMS
The evolution of database systems through the past years
was mainly related to the data content and responding to
the Big Data phenomenon. Several types were provided in
order to systematically and spontaneously tackle the growing
varieties and vagaries of data structures, schemas, sizes,
speeds, and scopes. It is important to recognize that to
figure out how the consequences of that evolution effect
on star schema implementation. The most commonly used
systems are- Relational database systems (RDBMS), Big
data management with the distributed file system (DFS),
Nonrelational data model (NoSQL), and (NewSQL). a dif-
ferent classification can be done based
A. Relational database systems (RDBMS)
The most commonly used systems, and had the biggest
share of the market at the time object-oriented databases were
created. It came from the relational model which is based
on representing data into tables with rows and columns. A
table must contain keys, Primary key to differentiate between
rows, and Foreign key that refer to other tables to create a link
between their data. These links are known by Integrity Rules,
which is responsible to ensure that the data they contain stay
accurate and are always accessible [15]. RDBMSs typically
use SQL and Stored procedures to manage structured data
and allow JOIN-operations to combine the different tables.
Both of them make RDBMSs able to handle queries and
transactions that ensure efficient, correct, and robust data
processing even in the presence of failures [4] [15]. Key
features of RDBMSs are ACID (Atomicity, Consistency,
Isolation, and Durability) [16], and besides Relational model,
RDBMSs can also handle dimensional model when they deal
with structured data and applicable to some semi-structured
data. However, The evolution of Big data shows that RDBMS
has a shortage. With static schema, no linear query execution
time and unstable query plan, RDBMS is poor in scalability.
It is very expensive and difficult to handle. To scale relational
database it has to be distributed on to multiple servers, that
will make Handling tables across different servers is difficult
and lead to complex queries which will effect on query
performance.
B. Data File distribution and Data Engines
In the past Parallelism was the most significant technique
used for processing massive databases, and all major vendors
of database management systems offered data partitioning
and parallel query processing [2]. Later with the evolution
in data amount and structure, which is known later by Big
Data, The traditional databases were not capable of handling
unstructured data and high volumes of real-time datasets,
and that enforced to come up with solutions that combine
structured and unstructured data, or in other words, to handle
the main attributes of Big Data: volume, velocity, and variety
(3Vs) [17]. Distributed file systems (DFS) came to solve
the shortages in scalability and performance that RDBMS
has. It is responsible to store and share files over a set
of connected nodes reliability, scalability, availability and
efficiency. Google started that approach by providing Google
file system (GFS) and based on that Hadoop offered its own
Distributed File System (HDFS). Basically, it is a framework
(ecosystem) to manage processing and the hardware available
by combining the software platform and the hardware to get
flexibility in storing and processing of large volumes of both
structured and unstructured data by taking data transactions
in whatever form and processing those transactions across
commodity hardware in parallel. The main workload on
this approach relies on the data processing model and data
engines rather than data storing and formatting. It started
with MapReduce, which is using divide and conquer strategy
to make computations [17]. The simplicity in its principle
and basic concept made MapReduce a key of success for
processing, analyzing and managing large datasets in several
implementations, but that did not deny the fact of difficulties
that it has in development, optimization, and maintaining its
functions especially in large-scale projects. Therefore, these
difficulties gave a rise to query platforms based on MapRe-
duce, a layer constructed directly on the top of MapReduce,
designed to translate queries and scripts into executable
native MapReduce jobs and to solve other limitations like
join operation and handling streaming data. Platforms like
Hive, Impala, and Spark not only giving SQL-like query
interface but also enabling to treat DFS systems as schemas.
Later Several Big Data Processing model has been suggested
in combined with MapReduce, or to replace MapReduce,
such as Resilient Distributed Dataset (RDD) and Bulk Syn-
chronous Processing (BSP) [17], and the evolution happened
to HDFS was done to these processes, by having a platform
and SQL-like query interface.
For example, Spark started with RDD to extend Hadoop to
a general-purpose framework and to replace a MapReduce
model with a better solution. It is an immutable distributed
collection of elements of the data, partitioned across nodes
in the cluster that can be operated in parallel with a low-
level API that offers transformations and actions. Later Spark
came up with Data Frame based on RDD. Instead of dealing
in low-level programming, Data Frame is better in terms
of having an interface for RDDs especially when you have
structured data. It has SQL for high-level data processing,
and that can give the ability to treat data within DFS as
similar to the data within the traditional database, as a
schema.
Fig. 7. Evolution of data and big data process engines
C. Nonrelational data Store (NoSQL)
NoSQL can be referred to (No SQL) or (Not only SQL). It
is a distributed, non-relational database-systems, not primar-
ily built on tables since it has many forms, such as document-
based, graph-based, and object-based and generally do not
use SQL for data manipulation. [4]. NoSQL has several
motivations that encourage vendors to adapt to their data
warehouse solutions [18]:-
‚ Simple and flexible. NoSQL data stores offer flexible
schemas or are sometimes completely schema-free and
are designed to handle a wide variety of data structure
‚ Ability to scale horizontally over many commodity
servers. Some data stores provide data scaling, while
others are more concerned with reading and/or write
scaling
‚ Provide high availability. Many NoSQL data stores are
aimed towards highly distributed scenarios, and consider
partition tolerance as unavoidable
‚ Unlike RDBMS, they do not support ACID transac-
tions as provided by RDBMS. NoSQL data stores are
sometimes referred to as BASE systems [19] (Basically
Available, Soft state, Eventually consistent)
NoSQL-databases can be classified into four categories:
Key-value store, Document store, Column-store, and Graph-
store. You can find in other papers that they classify these
categories into : Aggregate Oriented, and Non-aggregate
Oriented [19]. In general, Choosing one of them for data
storing relies mainly on the business needs, and main features
that differ between each of them, which are summarized at
the end of this section in table I.
Key-value stores have a simple data model based on key-
value pairs, which resembles an associative map or a dic-
tionary. The basic concept is that a globally distributed
hash table has keys that lead to different database servers
scattered all over the world. Each data item is converted
into a key using some unique formula which is stored in the
lookup table or directory. When the data are needed, the key
is converted into the location of the data and accordingly
data are retrieved [19]. The value is opaque to the data
store and can be used to store any arbitrary data, including
an integer, a string, an array, or an object, providing a
schema-free data model [18]. Along with being schema-
free, key-value stores are very efficient, fast, and scalable,
but since they have Many data structures, it cannot be
easily modelled, which means that they are not suitable for
scenarios requiring relations or structures. Any functionality
requiring relations, structures, or both must be implemented
in the client application interacting with the key-value store
[18].
Document stores provide another derivative of the key-
value store data model by using keys to locate documents
inside the data store, but they are more complex compared
with key-value stores. Like the key-value store, we have
a key and a value, but the value in Document stores are
semi-structured data, and that gives the ability to include
hundreds of attributes and the number and type of attributes
can differ from row to row, and that makes the value to be
represented as a Document. Most document stores represent
documents using JSON format or some format derived from
it. A set of documents is called collections. Collections
 discarding unwanted data in rows by performing I/O on
the blocks corresponding to columns that are actually being
read [19]. Besides that, a prior definition of columns is
not required, which offers huge flexibility in storing any
data type[18]. Graph-stores are considered to be the best

Fig. 8. Key-value store example by Andr Ribeiro

Fig. 10. Column-store example [16]

choice to store data along with relationships. Graph-stores

originated from graph theory which is is a mathematical
concept used to represent a set of objects, known as vertices
or nodes, and the links (or edges) that interconnectand these
Fig. 9. Document-Store example [16] vertices [18]. Graph-stores use graphs as their data model and
at the top of GD there may be a key/value store, columnar
database, BigTable database or combination of these and
may contain any number of documents which each of them other architectures [19].
should have a unique key [19]. This is the key feature Graph-stores have many features like efficiency in traversing
of document-oriented databases to become schema-less as
well as giving them the advantage of porting and storing
data of different types easily. Moreover, Document stores
give the capability of indexing documents based on the
primary key as well as on the contents of the documents, and
that differentiates this data model from the key-value stores
model, in which the values are opaque to the data store [18].
The ability to contain complex data structures such as nested
objects make Document stores to have different options for
Fig. 11. Graph-store example [16]
data processing, some of them provides MapReduce, other
systems provide SQL processing capabilities like RDBMS,
relationships between different entities. They offer persistent
but that does not mean that they match in the structure, since
storage of objects and relationships and support simple and
rows within a table in RDBMS follow the same schema, but
understandable queries with their own syntax [16], and that
in document stores each document inside can have a different
make them suitable in scenarios such as social network-
structure [19].
ing applications, pattern recognition, dependency analysis,
Column-stores are a hybrid between NoSQL and relational
recommendation systems and solving path finding problems
databases. They provide some row-and-column structure, but
raised in navigation systems. For that reasons, Facebook for
they do not have the strict rules of relational databases. Some
example is implementing every thing in GD [18].
column databases are inspired by key-value databases, but
most of them are inspired by Googles Bigtable, an internal TABLE I
distributed data storage system developed by Google [18]. SALIENT FEATURES OF VARIOUS N O SQL [18]
The approach to store and process data by column instead
of row. Each row is composed of a set of column families, Data Store Performance Scalability Flexibility Complexity
and different rows can have different column families [3]. Key-Value High High High Low
Column High High Moderate Low
Some systems store the data belonging to a row together Document High Variable High Low
on the same node, and other systems separate them on Graph Variable Variable High High
several nodes using composite partition keys, like Cassandra
[18], which also provides additional functionality of super
columns, which are formed by grouping various columns D. NewSQL
together. With this design, Column-stores are more flexible NoSQL databases were designed to address performance
comparing with RDBMS, since they precisely access the and scalability requirements, but they did not concentrate
data they need to answer a query rather than scanning and on ACID transactions as provided by RDBMS. The Idea
of combining both features comes with a new generation
of databases, NewSQL. It is a hybrid solution that aims to
bring the relational data model into the world of NoSQL
[18]. The main focus of NewSQL systems is to address the
issue of scalability of online transaction processing (OLTP)
workloads with the support of programming model cus-
tomizability and reduce management tasks in SQL databases.
NewSQL DBs open a path to migrate from traditional SQL
systems (RDBMS) to modern SQL systems that have the
same NoSQL and Big data characteristics; performance and
scalability [20],by absorbing data models and data manipula-
tion capabilities of NoSQL databases,and extending RDBMS
functionalities, but that does not mean a full replacement for
RDBMS; they are designed for a specific scenario in which
can be successful, like OLAP and Big data analytics [21].
The main Five characteristics of NewSQL are:-
‚ SQL as the primary mechanism for interaction
‚ ACID support for transactions
‚ a shared-nothing distributed architecture
‚ A non-locking concurrency control mechanism so real-
time reads will not conflict with writes, and thus cause
them to stall
‚ an architecture providing much higher per-node perfor-
mance than available from traditional RDBMS solutions
NewSQL can be categorized into 3 categories based on
the different approaches adopted by vendors: New Archi-
tectures, Transparent Sharding Middleware, and Database-
as-a-Service [22].
New Architecture Databases are newly designed from
scratch to achieve performance and scalability. they are de-
signed from a new codebase without any of the architectural
baggage of legacy systems. It can be further categorized such
as General Purpose Databases and In-Memory Databases
[23]. The main advantage of this type is that all parts of
the system can be optimized for multi-node environments.
For example, most NewSQL DBMSs are able to send intra-
query data directly between nodes rather than having to
route them to a central location like with some middleware
systems. Having a custom engine instead of relying on an
off-the-shelf distributed filesystem like (HDFS) or storage
fabric like (Apache Ignite) is another advantage, because it
will give the DBMS the whole responsibility to distribute
the database across its resources and allows the DBMS
to send the query to the data rather than bring the data
to the query, and that leads to less network traffic since
transmitting the queries is typically less than transmitting the
data. Besides that, Managing their own storage also enables
a DBMS to employ more sophisticated replication schemes
than what is possible with the block-based replication scheme
used in HDFS. That allows these DBMSs to achieve better
performance than other systems that are layered on top of
other existing technologies.On the other hand, That type of
new systems can be not attractive to some organizations.
Most of them prefer adopting well-known technologies rather
than using new ones because the old ones have been tested
and has much-experienced people comparing with the new
ones.
Transparent Sharding Middleware systems provide a shard-
ing middleware layer to automatically split databases across
multiple nodes.These nodes run the same DBMS, have
only a portion of the overall database, and cannot be
accessed independently [22]. The centralized middleware
component routes queries coordinates transactions, as well
as manages data placement, replication, and partitioning
across the nodes. It is responsible for executing queries on
behalf of the middleware at its local DBMS instance and
returning results.Altogether, these allow middleware products
to present a single logical database to the application without
needing to modify the underlying DBMS. With that type of
systems Developers do not need to make any changes to
their application to use the new sharded database because
they are often a drop-in replacement for an application
that is already using an existing single-node DBMS. The
middleware approach is a good solution for scaling especially
when you have traditional DBMS (RDBMS), but with these
RDBMS we will not get a strong manager that is optimized
for memory-oriented storage like in some of the NewSQL
systems, it will enforce us to apply a local optimization on
each node for each query.
Database-as-a-Service offerings from cloud computing
providers that are also based on new architectures. We
know that with this services organizations do not have to
maintain the DBMS on either their own private hardware
or on a cloud-hosted virtual machine. Instead, the DBaaS
provider is responsible for maintaining the physical configu-
ration of the database, including system tuning, replication,
and backups. Although Cloud was implemented before on
RDBMS and NoSQL, they used the same underlying disk-
oriented DBMS, here we are considering only cloud systems
that has NewSQL architecture, which uses a log-structured
storage manager to improve I/O parallelism [22]. The main
advantage that it provides its own custom DBaaS which can
be deployed on all of the major cloud platforms. It gives the
ability for distributing the database across different providers
in the same geographical region to avoid downtimes due to
service outages[24].
5. R ELATED WORK
Main studies on Star schema were done into 2 directions;
implementing star schema from RDBMS to other database
management systems and Star query performance. Through
those studies, the main idea was whether these systems
could support the purposes of OLAP, like multidimensional
modelling, data roll-up and roll-down, hierarchies, and slice
and dice operations. That is compatible with Golfarelli and
Rizzi [25], who made a paper that surveyed more than
20 years of research on data warehouse systems. They
concluded that most of the work papers were done on 4
main subjects; Query processing, Multidimensional mod-
elling, Performance and tuning, and architecture. Sen, Arun
and Sinha [26] made a comparison between Data warehouse
systems based on of infrastructure-based for 15 vendors pro-
viding data warehousing systems. Although they came with
a conclusion that methodologies used by these companies
differ in details, all focus on the techniques of capturing
and modelling user requirements in a meaningful way, They
used several attributes to check the differences. Two of these
attributes were related to data modelling and the ability
to support for Normalization/Denormalization. Within the
data modelling, they differentiated between star schema and
relational modelling, and they found that 2 out of 15 were not
supporting star schema; Teradata/NCR and SAS. Supporting
for Normalization/Denormalization is important for OLAP
queries, and that was important for measuring performance
and how can improve that. They found that some systems
supported that explicitly but some of them were not reporting
anything regarding that. Using the MapReduce technique to
use for star schema data warehouses solve Big data problems
related to aggregations and joins was another attractive sub-
ject. Barkhordari, Mohammadhossein and Niamanesh [27]
proposed a method called Chabok that based on two-phased
Map-Reduce and implemented Hadoop, and TPC-DS queries
(http://www.tpc.org/tpcds/) were executed for benchmarking.
They used two-phased MapReduce since one of them repre-
sents the Fact table and the other one represents a dimension
table. They made a query language called MHBQL which
has the same functionality of SQL in terms of selection,
joins and aggregations.They compared their queries with
SPARK and Hbase queries and found that they have a high
performance since they adapted independency in hardware
nodes, which enables higher speeds of query execution,
which omits joins and removes network congestion. Instead
of using data replication to solve the performance issue,
they made it optional and used it only for maintenance
and backup issues. Chevalier, El Malki, Kopliku, Teste,
and Tournier [28] Discussed investigated the use of NoSQL
systems for implementing OLAP systems, mainly the RO-
LAP model (Star Schema). they defined a set of rules to
map star schemas into two NoSQL models: column-oriented
(HBase) and document-oriented (MongoDB), and the process
included data transformation, data loading and aggregate
computation. That process allowed them to compare the
different approaches with each other. They were successful
not only in implementing their use case on both systems
but also they measured the performance on both systems.
In general, They recommended Hbase (column-oriented),
since it showed better using for space and memory, and the
aggregation time was reasonable. When they compared in
terms of queries that demand multiple attributes of a relation,
Document-oriented (MongoDB) showed better results, since
it is supporting nested fields, unlike column-oriented, it might
take longer because data would not be available in one
place. Similar studies were done by Yangui, Nabli, and
Gargouri [29]. They were interested in transforming the mul-
tidimensional model into NoSQL systems, mainly Casandra
(column-oriented) and MongoDB (Document-oriented). The
conclusion that they had was similar to [28], the queries on
Document-oriented not only better than column-oriented but
also better in explaining hierarchies. Maity, Soumya Sen, and
Debnath [30] were looking for challenges of implementing
OLAP using MongoDB, which is classified as a Document-
oriented store. They found that they could implement star
schema, but they recommended to have a new mechanism
in joining operation that can be equivalent to natural join
in RDBMS. That was required not only for selection but
also for creating a fact table. Besides that, they found that
not all OLAP operations can be performed directly, and
that because of technical issues related to the MongoDB
environment. Kovaevi and I. Mekterovi [31] were providing
Alternative business intelligence engines based on NoSQL
and NewSQL instead of RDBMS. It could be similar to
[26] provided before, but here they did not but explicitly
whether these systems can have star schema or not. They
mainly looked to those systems that could provide similar
properties of relational databases, like ACID, SQL and
a relational schema. They made a comparison among 18
features, 4 of them related to the star schema, Data model,
OLAP operations, SQL/ACID and Query language. So we
considered that the system that could provide those 4 features
it could also give the Star schema explicitly, otherwise,
it needs a workaround method to implement. there were
14 systems used in the study, 6 of them NoSQL based,
and the other 8 were on NewSQL based. it was clear that
NewSQL system would support the star schema, and they
address that before making the comparison. They explored
the features of NewSQL and found that it was matching
with the features that RDBMS has. The systems they used
for NewSQL are Apache Spark, Apache Drill, Vertica, SAP
Hana, Oracle Times Ten, SQL Server Column indexes,
VoltDb, and NuoDb. For NoSQL, it was clear that not all
of them were supporting implement star schema inexplicit
way, but we knew from previous that it could be done
especially with column-oriented type. Many studies have
been don Star Query Performance, we mention them as a part
of Star schema subject and to emphasize that Star schema
is still important for the implement. We know that in 2006
Stonebraker’s Vertica developed Schema Benchmark (SSB)
to measure star schema query performance.It is an extension
of the TPC-H benchmark. Later in 2009, it was extended to
(SSB+) by providing a new approach; Adjoined Dimension
Columns (ADC) to speed up the star query [32]. That was
based on traditional databases (RDBMS), So the evolution
of Databases enforced to have a benchmark for star schema
that is implemented on other types of database store. Malki,
Arlind, Essaid, and Olivier [33] summarized that on their
paper; Benchmarking Big Data OLAP NoSQL Databases.
They were not only introduced DSS benchmarks related to
NoSQL and big data like KoalaBench, but also implemented
several models like star schema, snowflake, Falt table and
Flat flexible on 4 NoSQL databases, and they conclude
that Star model had better performance compared with the
others. in fact the research also can be considered as one
of the implementations of star schema on NoSQL databases.
Enhance the join operation is also one of the factors related
to star query performance, like what Zhu, Navneet, Saurabh,
and Patel did in [34], They introduced a new strategy in query
executing based on star schema; lookahead Information Pass-
ing (LIP), which speeds up star queries and as they said gives
an optimal query plan. Makreshanski and others introduced
many-query join (MQJoin) [35], a method for sharing the
execution of a join that can efficiently deal with hundreds
of concurrent queries by minimizing redundant work and
making efficient use of main-memory bandwidth and multi-
core architectures.
6. V ENDOR R EVIEW
In this part, we will make a review of commercial products
available and provided by some main vendors. It was hard to
figure out what are vendors providing especially when you
look to their white papers. They explain the implementation
of data warehouse and OLAP by their systems, some of
them clarify that directly, and the others describe the general
approach that they have, but as we said before most of
them consider mainly star schema as a first model for their
implementation.
A. SAP HANA
SAP one of the software corporations that makes enter-
prise software related to Data warehouse and business intel-
ligence. SAP cares about dynamic dimensional modelling,
so Their solution SAP Business Warehouse powered by SAP
HANA is deploying a model called Dynamic Star Schema,
which is a new modelling pattern partitioning Dimensions
(master data) of fact-table InfoProvider using Composite
Providers or Open ODS Views of type fact [36]. At first, they
were extending Star Schema on row-store RDBMS to Flat
Star Schemas on SAP HANA column-store. But later they
found limitations with that approach, in terms of Flexibility,
adaptability, Maintenance and Support of modern federated
DWH architectures. The Dynamic Star Schema is modelled
using:-
‚ CompositeProvider on SAP BW fact-table InfoProvider
of Integrated DWH/ Propagation Layer
‚ Union-CompositeProvider on InfoProvider if you only
want to define the Star Schema
‚ Open ODS view of type fact on DB-table/ -view and
field-based DSO (advanced) of the Raw DWH / Open
ODS Layer that serve as fact tables
Fig. 12. HANA Dynamic Star Schema
The main components are CompositeProvider, InfoObjects
and InfoProvider, used to define and associate tables. Table
Partitioning, Federation, and Layered Scalable Architecture
(LSA) are features provided to simplify the data warehouse.
Dynamic Star Schema allows combining fact data and master
data across the layer borders; within SAP BW and outside
SAP BW (federation), and combining partitioned master data
(dimensions).
B. Microsoft - SQL Server 2017 and Power BI
Microsoft supoorts data warehouse and star schema
through many products that they provided for many years.
For physcial implementation the latest product that they have
is SQL Server 2017. Microsoft provides many features on
this product; it is Cloud, in-memory and Columnstore and
using Star Schema. It uses Azure for ETL, and according
to [37], one of the data pipeline stages is transforming data
into star schema and that gives the end-user to make queries
using both traditional relational data and new data types with
common T-SQL commands and PolyBase bridge to translate
to non-SQL Server repositories. The query processing is
optimized for smaller result sets typical of data warehouses,
such as performing star joins and aggregating, It may effect
on the performance, but features like Columnstore indices,
Massively Parallel Processing (MPP) and B-Tree indices
guarantee a high performance.
Another option that Microsoft provides is implementing Star
Schema logically through Power BI, the latest Business in-
telligence framework that Microsoft spends their investments
and efforts to provide on that domain [38]. It gives the ability
of connecting to several data sources and making the logical
modelling, like Star Schema without having that physically.
With that virtual modelling, many operations like joining
tables, aggreagtions and selections can be done through
that , wwithout effecting the physical implementation, and
that is common used in other BI systems, But when the
implementation is done on a physical star schema, it will en-
able auto-detection feature, which will activate automatically
the properties that related to the star schema,like creating
relationships between tables, setting Cardinality, and Cross
filter direction.
C. Amazon Redshift
Redshift is a distributed column-oriented data store, based
on the relational row-based PostgreSQL DBMS and a
petabyte-scale cloud DW. It uses MPP architecture, break-
ing up large data sets into chunks which are assigned to
slices within each node. Redshift cluster consists of several
compute nodes that contain the data, a leader node, which
receives queries and manages their execution in the cluster,
and Redshift Spectrum enables running SQL queries on
semi-structured data from the S3 distributed file system [39].
Redshift is Highly supporting Star schema, in fact, when
you look to the Developer guide that they have, you will
see that the first modelling it would be with a star schema.
They believe that most of the current implementations on
DW are on Star schema, so that would encourage migration
from the current systems that the companies have to Amazon
Redshift. The amazing part is that Redshift supports the Star
Schema Benchmark (SSB). which means not only they care

Fig. 13. Redshift Amazon Redshift Architecture-[39]
about modelling but also about performance. This feature
can support you to test the design and other factors like
load time, storage use, and query performance. Besides
that, Redshift provides several Distribution Styles for the
rows of the table like Key distribution, All Ditribution and
EVEN distribution.That would help the query optimizer to
redistribute the rows to the compute nodes as needed to
perform any joins and aggregations. Redistribution might
involve either sending specific rows to nodes for joining
or broadcasting an entire table to all of the nodes. Data
Compression is another feature that supports the optimiza-
tion functionality that Redshift provides. Because Amazon
Redshift uses a columnar design, it can choose optimal
compression algorithms for each column individually based
upon the data contained in that column, typically yielding
significantly better compression than a row-based database.
Compression helps queries to perform faster and minimizes
the amount of physical storage your data consumes.
We know that one of the physical implementations on Star
schema is defining the relations between tables, which means
define primary keys and foreign keys (Constraints). Redshift
uses this information to optimize the query by eliminating
redundant joins and establishing a join order. But some
designers do not follow that procedure of relation definition,
so as an alternative, The query optimizer detects redundant
joins without constraints defined if we keep statistics up to
date, That can be done by using the Analyze operation Which
is responsible for that, or enabling other options within Copy
operation in the way that can update statistics while loading
data. Work load managment and Cursors are other features
that Redshift provides. Although there is nothing specific to
Sar schema, both of them are supporting the performance
of the data warehouse. Workload management can separate
longer running queries from shorter running queries. This
helps keep applications that require responsive queries from
being backed up by long-running queries. It allows allocating
more memory to queue handling more intensive queries, and
to boost their performance. Cursors are used to reduce the
impact on memory when the results are too big. Instead of
reflecting the wholes results into memory, Cursors chunk the
data at a time and control the maximum size to be loaded.
D. Oracle
It was a challenge to summarize the approach that Oracle
had to Star Schema. Once you make a search on Star
schema through Oracle.com, you will find that Star schema
is covered in many articles, blogs, technical documenta-
tion,academic reviews, and use cases. Besides that, Oracle
supports the whole DBMS types, and their products, systems,
and frameworks can be integrated with other solutions and
platforms. You can feel that Oracle makes a full adoption to
Star Schema. For example:
‚ The OLAP Cube option which they have on Oracle 12c
is represented through Star schema [40]
‚ The Oracle Advanced Analytics (OAA), which is an
option of Oracle Database Enterprise for uncovering
hidden relationships in the data, is using data mining
tools for mining data into Star schema for delivering
real-time results and predictions [41]
‚ Oracle usually use bitmap index and star transformation
for implementing Star queries, but it was not applicable
to implement that within their In-memomry DB and
cloud DW since they use column-oriented DB, So
they introduced Vecotr transformation or Vector Group
by , which is doing the same functionality of Star
transformation to work on Column-oriented DB [42]
In general, Oracle Supports Star Schema implementation
from a different perspective, physically through the databases
and logically through the BI systems.
E. IBM
IBM has provided many solutions and platforms that
supporting data warehouse, and all of them were supporting
Star schema modelling,like Cognos BI, DB2 database, Db2
Warehouse and InfoSphereWarehouse. That supports come
through several forms, like visualization tools to implement
Star schema, or enhance performance and optimization, or
providing new algorithms for star joins. For example, the
latest product, Db2 Warehouse, includes features that are not
specific for star schema, but it is important to have [43]:-
‚ Embedded Apache Spark Engine which is used in-
memory SQL columnar processing with a MPP cluster
architecture to speed up complex queries and predictive
model building, testing, and deployment (we will see in
6-H Spark role in a star schema)
‚ Automatic schema generation and data loading, and that
includes Star schema
‚ BLU Acceleration, IBMs in-memory columnar process-
ing technology. Entire datasets and decompression are
not needed in-memory. Expect pre-fetching of data and
data skipping
IBM not only put their efforts on the industry side but
also in Academia as well. You can find easily many white
papers provided by them related to database management
and star schema. For example, They introduced Zigzag join
algorithm for DB2 star schema queries [44], which improves
consistency in performance as well as reduces the execution
time of queries in data warehouse or data mart environments
with large volumes of potentially partitioned data, complex
ad hoc queries, and where database design uses a star
schema.
IBM Business intelligence and reporting tools are simi-
lar to other tools that give that gives the ability to de-
sign logical modelling even if that model does not exist
physically.Cognos, for example, has a component called
Impromptu Catalog, which is a business intelligence tool
repository that handles a logical design for the database, and
similar in it’s functionality to SAP business object universe
builder. Through that component, you can implement any
data modelling, including Star schema.
IBM recognize this book Dimensional Modeling: In a
Business Intelligence Environment as a reference for data
modelling, and within this book, you can find that they
identify Star schema, multi-star schema, and Snowflake as
main modelling design for IBM products.
F. Hive
Hive is a query engine developed at Facebook, brings
the concepts of tables, columns, partitions, and SQL to
the Hadoop architecture by executing internally MapReduce
jobs, keeping the extensibility and flexibility of Hadoop
[7]. HiveQL is the SQL that Hive provides. It has the
same operations that SQL has; selection, Join operations and
aggregations. That allows implementing the typical relational
operations like Star Schema.
G. Atscale
Atscale is a BI platform for Big Data, enabling the usage
of popular Business Intelligence tools on Big Data and
modern data platforms. It aims to provide an OLAP engine
for Hadoop, avoiding ETL and building a virtual ROLAP
(vROLAP) directly above the original data storage. The Cube
that AtScale provides is virtual since it does not contain
data, It describes only how to process and optimize the
data at query runtime. That gives the ability to apply the
desired logical model that you want, even without having
that physically. So when you need to define a star schema,
you can easily define measures, dimensions, hierarchies, and
joins through Atscale, whether your data has data warehouse-
style format or not, it does not matter. AtScale will allow
you to create any logical model you need, regardless of
what the physical model is on disk [45]. Besides modelling,
the vROLAP provides Smart Aggregates, to solve one of
the biggest drawbacks of ROLAP, Maintaining aggregate
tables. Instead of building and maintaining summary tables
up front, a vROLAP engine dynamically builds and main-
tains aggregates on-demand based on what BI users need.
For performance and optimization issues, vROLAP engine
optimizes the queries to work with the data in its current
form. It uses information about the data to get optimal
performance from existing Hadoop resources. vROLAP not
only optimizes query performance, but it also shortens the
entire time to insight lifecycle. It removes the bottlenecks
and complexity that have been a barrier to the widespread
adoption of OLAP on Hadoop [46].
H. Spark and SparkLineData
Spark becomes an accepted Architecture for an Enterprise
Data Warehouse. It started with the resilient distributed
dataset (RDD) to solve data processing, by partitioning
data and executes various parallel operations on a cluster,
and that mainly for unstructured data. After that, Spark
started to provide SQL and Dataframe modules to handle
structured data based on RDD, but with a better interface.
The data frame has a typical structure like any table in the
traditional database, consists of rows and columns, it can
be connected to several resources, like traditional DB or
NoSQL or files, and it can be used easily to represent the Star
schema components. The Spark SQL not only covering the
operations that the traditional SQL has but also has Catalyst
Optimizer, Which is a query optimizer that has similar
functionalities like other query optimizers, detecting join
operations and execute a query plan for better performance.
Catalyst Optimizer converts queries into logical and physical
trees corresponding to the different phases of optimizations.
Within each stage, these trees are manipulated by rules as
shown in figure 14. In Spark, star schema joins is detected
using table and column statistics. With this information, the
Optimizer joins the tables together in a star schema in an
optimal way [47]. Spark now is a part of many solutions
Fig. 14. Catalyst Optimizer
and used by many vendors as a part of the ETL process,
but The most recent product related to OLAP and Business
intelligence based on Apache spark is SparkLineData.
SparkLineData comes to Solve One of the gaps in this
stack is the ability to support fast Slice-and-Dice ad-hoc
workloads; such a workload is fairly common, for example,
browsing and doing discovery through large datasets. It
is a native business intelligence platform geared towards
providing fast ad-hoc querying over a Logical Cube (i.e.
Star-Schema). Its based on an Apache Druid OLAP Indexing
technology; the technique of using OLAP Indexing has the
additional benefit of simplifying the ETL processes and the
Data Management layer [48]. SparkLineData brings sub-
second query response times that can be deployed over enter-
prise data warehouses, data lakes and IoT analytics and pro-
vides sub-second response times for analytics queries using
in-memory multi-dimensional indexes and advanced query
optimization on Apache Spark. Recently, Oracle completed
the acquisition of SparklineData to augment Oracle PaaS
services for enterprise customers transitioning to Big Data
solutions Oracle - Sparklinedata. The key components for
getting Star schema are Druid index, Catalyst optimizer, and
DataSource. SparkLineData orchestrates execution between
Druid and Spark and the Druid Planner that generates optimal
Physical Plans to take advantage of both Spark and Druid
capabilities. The documents related to SparklineData are rare,
and even the previous website that they had before was
closed, but Vincent Smith [48] built a star using Sparkline-
Data and showed with several queries how it was easy to
gain one with a better performance.
Fig. 15. SparkLineData BI Stack
I. SAS - Data Integration Server
In the past, SAS was not supporting Star schema mod-
elling [26], But now it does. SAS introduced before LASR
Star Schema, which is similar to a typical star schema
in a relational database, but it is limited to a single join
condition. It does not support multiple key columns between
fact and dimension tables. Later, SAS Provided Data In-
tegration Server, the next generation product to the SAS
ETL Server [49]. The visual design component of the SAS
Data Integration Server, SAS Data Integration Studio, can be
used to build the star schema or import data models directly
from standard data modelling tools, such as CA ERwin or
Rational Rose. With Data integration Studio you can easily
identify the fact and dimension tables, enable features like
slowly changing dimensions (SCD) for tracking changes
in dimension tables through SCD Type 2 Loader, manage
changes and updates on star schema, and apply several join
algorithms as SORT-MERGE join, index join, Hash join, and
a Multi-way join. For optimization and performance issues,
SAS provides another component with Data Integration,
the Scalable Performance Data Server. It is responsible for
performing, validating and optimizing Star joins with the best
performance. When the star join is utilized, the join requires
only three steps to complete the join, regardless of the
number of dimension tables. This can improve performance
when working with large dimension and fact tables.
7. CONCLUSION
Data modelling still the most interested area related to
Business intelligence and data warehouse environments.
Through this paper, we made a review about data warehouse
and the main focus was on Star Schema. We have seen
why we need to have Star schema and why it differs
comparing with other data modelling. We showed the most
attractive areas related to Star schema that attracted people
to make research such as implementation, performance and
optimization. it was important to have a look at the evolution
of database management structure which was a response to
the amount of data that increased through the past decades.
We have seen that with each type of database management
there a was a study or a use case that a star schema was
implemented on that type of DBMS, which means that Star
schema is still considered as a core part of data warehouse
concept and later for Big data solutions. We made a review
on the current applications, frameworks and BI systems that
are provided by several commercial vendors. We have seen
that these vendors were insisting to adopt star schema on
their solutions, not only to absorb the previous solutions
that the organizations have but also to give the ability
to implement star schema for their future solutions. The
implementation of star schema can be physical or logical,
which means even these solutions that do not implement Star
schema within the database structure, Star schema can be
implemented through the BI and reporting interface. Many
factors that can influence the decision on implementing star
schema, like performance, real-time data and having an
optimal approach to the decision makers, but that does not
mean that star schema cannot handle these. Many studies
through academia and industry had proved that star schema
still applicable to handle several scenarios and use cases, and
even for new concepts like Data lake, they used star schema
as one of its components to implement. As a conclusion, the
new challenges that facing the star schema concept does not
make both academia and industry to give up, but enforced
them to think deeply to solve the new situations , such as new
algorithms or approaches in impelementaions from academia
side, or combine several components and products in one
package to solve and handle the new situations, like most of
the commercial vendors do these days.
R EFERENCES
[1] Kimball, R. and Ross, M., The Data Warehouse Toolkit: The
Definitive Guide to Dimensional Modeling, 2nd ed. Wiley
Publishing, 2013. [Online]. Available: https://books.google.co.
nz/books?hl=en&lr=&id=XoS2oy1IcB4C&oi=fnd&pg=PA1&ots=
1CIdmyiPbD&sig=u Yqnw3V88kgWLGeGM9YZPnfp3I&redir esc=
y#v=onepage&q&f=false
[2] Chaudhuri, S. and Dayal, U., “An overview of data warehousing and
olap technology,” SIGMOD Rec., vol. 26, no. 1, pp. 65–74, Mar.
1997. [Online]. Available: http://doi.acm.org.ezproxy.auckland.ac.nz/
10.1145/248603.248616
[3] Celko, J., “Chapter 2 - columnar databases,” in Joe Celkos Complete
Guide to NoSQL, Celko, J., Ed. Boston: Morgan Kaufmann,
2014, pp. 15 – 25. [Online]. Available: http://www.sciencedirect.com/
science/article/pii/B9780124071926000029
[4] Homayouni, H., Ghosh, S., and Ray, I., “Data warehouse testing,”
ser. Advances in Computers. Elsevier, 2018. [Online]. Available:
http://www.sciencedirect.com/science/article/pii/S0065245817300578
[5] Linstedt, D. and Olschimke, M., “Chapter 7 - dimensional modeling,”
in Data Vault 2.0, Linstedt, D. and Olschimke, M., Eds. Boston:
Morgan Kaufmann, 2016, pp. 171 – 193. [Online]. Available: http:
//www.sciencedirect.com/science/article/pii/B9780128025109000076
[6] Tupper, C. D., “19 - dimensional warehouses from enterprise
models,” in Data Architecture, Tupper, C. D., Ed. Boston:
Morgan Kaufmann, 2011, pp. 337 – 356. [Online]. Available: http:
//www.sciencedirect.com/science/article/pii/B978012385126000019X
[7] Vaisman, A. and Zimányi, E., Logical Data Warehouse Design.
Berlin, Heidelberg: Springer Berlin Heidelberg, 2014, pp. 121–178.
[Online]. Available: https://doi.org/10.1007/978-3-642-54655-6 5
[8] Han, J., Kamber, M., and Pei, J., “4 - data warehousing and
online analytical processing,” in Data Mining (Third Edition), third
edition ed., ser. The Morgan Kaufmann Series in Data Management
Systems, Han, J., Kamber, M., and Pei, J., Eds. Boston:
 Morgan Kaufmann, 2012, pp. 125 – 185. [Online]. Available: http:
//www.sciencedirect.com/science/article/pii/B9780123814791000046
[9] Malinowski, E. and Zimányi, E., Conventional Data Warehouses.
Berlin, Heidelberg: Springer Berlin Heidelberg, 2008, pp. 77–135.
[Online]. Available: https://doi.org/10.1007/978-3-540-74405-4 3
[10] “Star and snowflake schemas,” https://www.oracle.com/webfolder/
technetwork/tutorials/obe/db/10g/r2/owb/owb10gr2 gs/owb/lesson3/
starandsnowflake.htm, accessed: 2018-12-24.
[11] Kaas, C., Pedersen, T. B., and Rasmussen, B., “Schema
evolution for stars and snowflakes,” in ICEIS, 2004.
[Online]. Available: https://pdfs.semanticscholar.org/5fed/
155cd5c6187300a33f241f5b3383a2028aee.pdf? ga=2.161767292.
1152849063.1545621207-1091939017.1544844686
[12] Cuzzocrea, A., Bellatreche, L., and Song, I.-Y., “Data warehousing
and olap over big data: Current challenges and future research
directions,” in Proceedings of the Sixteenth International Workshop
on Data Warehousing and OLAP, ser. DOLAP ’13. New
York, NY, USA: ACM, 2013, pp. 67–70. [Online]. Available:
http://doi.acm.org.ezproxy.auckland.ac.nz/10.1145/2513190.2517828
[13] Wrembel, R., Data Warehouse Performance: Selected Techniques
and Data Structures. Berlin, Heidelberg: Springer Berlin
Heidelberg, 2012, pp. 27–62. [Online]. Available: https:
//doi.org/10.1007/978-3-642-27358-2 2
[14] Vaisman, A. and Zimányi, E., Data Warehouses: Next Challenges.
Berlin, Heidelberg: Springer Berlin Heidelberg, 2012, pp. 1–26.
[Online]. Available: https://doi.org/10.1007/978-3-642-27358-2 1
[15] “A relational database overview,” https://docs.oracle.com/javase/
tutorial/jdbc/overview/database.html, accessed: 2018-12-24.
[16] Siddiqa, A., Karim, A., and Gani, A., “Big data storage technologies:
a survey,” Frontiers of Information Technology & Electronic
Engineering, vol. 18, no. 8, pp. 1040–1070, Aug 2017. [Online].
Available: https://doi.org/10.1631/FITEE.1500441
[17] Wu, C., Buyya, R., and Ramamohanarao, K., “Chapter 1 - big
data analytics = machine learning + cloud computing,” in Big
Data, Buyya, R., Calheiros, R. N., and Dastjerdi, A. V., Eds.
Morgan Kaufmann, 2016, pp. 3 – 38. [Online]. Available: http:
//www.sciencedirect.com/science/article/pii/B9780128053942000015
[18] Grolinger, K., Higashino, W. A., Tiwari, A., and Capretz, M. A.,
“Data management in cloud environments: Nosql and newsql data
stores,” Journal of Cloud Computing: Advances, Systems and
Applications, vol. 2, no. 1, p. 22, Dec 2013. [Online]. Available:
https://doi.org/10.1186/2192-113X-2-22
[19] Chandra, D. G., “Base analysis of nosql database,” Future Generation
Computer Systems, vol. 52, pp. 13 – 21, 2015, special Section:
Cloud Computing: Security, Privacy and Practice. [Online]. Available:
http://www.sciencedirect.com/science/article/pii/S0167739X15001788
[20] Srinivasa, K. G. and Hiriyannaiah, S., “Chapter five - comparative
study of different in-memory (no/new) sql databases,” in A Deep
Dive into NoSQL Databases: The Use Cases and Applications,
ser. Advances in Computers, Raj, P. and Deka, G. C., Eds.
Elsevier, 2018, vol. 109, pp. 133 – 156. [Online]. Available:
http://www.sciencedirect.com/science/article/pii/S0065245817300463
[21] Pokorný, J., “Database technologies in the world of big data,”
in Proceedings of the 16th International Conference on Computer
Systems and Technologies, ser. CompSysTech ’15, 2015, pp. 1–12.
[Online]. Available: http://doi.acm.org/10.1145/2812428.2812429
[22] Pavlo, A. and Aslett, M., “What’s really new with newsql?” SIGMOD
Rec., vol. 45, no. 2, pp. 45–55, Sep. 2016. [Online]. Available:
http://doi.acm.org/10.1145/3003665.3003674
[23] Kumar, R. and Charu, S., “Newsql databases: Scalable rdbms for oltp
needs to handle big data,” 03 2014.
[24] “cleardb overview,” https://www.cleardb.com/developers/platform/
overview, accessed: 2019-01-07.
[25] Golfarelli, M. and Rizzi, S., From Star Schemas to Big Data:
20$$+$$Years of Data Warehouse Research. Cham: Springer
International Publishing, 2018, pp. 93–107. [Online]. Available:
https://doi.org/10.1007/978-3-319-61893-7 6
[26] Sen, A. and Sinha, A. P., “A comparison of data warehousing
methodologies,” Commun. ACM, vol. 48, no. 3, pp. 79–84, Mar. 2005.
[Online]. Available: http://doi.acm.org/10.1145/1047671.1047673
[27] Barkhordari, M. and Niamanesh, M., “Chabok: a map-reduce
based method to solve data warehouse problems,” Journal of
Big Data, vol. 5, no. 1, p. 40, Oct 2018. [Online]. Available:
https://doi.org/10.1186/s40537-018-0144-5
[28] Chevalier, M., El Malki, M., Kopliku, A., Teste, O., and Tournier, R.,
“How can we implement a multidimensional data warehouse using
nosql?” in Enterprise Information Systems, Hammoudi, S., Maci-
aszek, L., Teniente, E., Camp, O., and Cordeiro, J., Eds. Cham:
Springer International Publishing, 2015, pp. 108–130.
[29] Yangui, R., Nabli, A., and Gargouri, F., “Automatic transformation
of data warehouse schema to nosql data base: Comparative study,”
Procedia Computer Science, vol. 96, pp. 255 – 264, 2016.
[Online]. Available: http://www.sciencedirect.com/science/article/pii/
S1877050916319391
[30] Biswajit Maity, N. C. D., “Challenges of implementing data
warehouse in mongodb environment,” Journal of Fundamental and
Applied Sciences, vol. 10, pp. 222–228, 2018. [Online]. Available:
https://www.ajol.info/index.php/jfas/article/view/171660
[31] Kovaevi, I. and Mekterovi, I., “Alternative business intelligence
engines,” in 2017 40th International Convention on Information
and Communication Technology, Electronics and Microelectronics
(MIPRO), May 2017, pp. 1385–1390.
[32] O’Neil, P., O’Neil, E., Chen, X., and Revilak, S., “The star schema
benchmark and augmented fact table indexing,” in Performance Eval-
uation and Benchmarking, Nambiar, R. and Poess, M., Eds. Berlin,
Heidelberg: Springer Berlin Heidelberg, 2009, pp. 237–252.
[33] El Malki, M., Kopliku, A., Sabir, E., and Teste, O., “Benchmarking big
data olap nosql databases,” in Ubiquitous Networking, Boudriga, N.,
Alouini, M.-S., Rekhis, S., Sabir, E., and Pollin, S., Eds. Cham:
Springer International Publishing, 2018, pp. 82–94.
[34] Zhu, J., Potti, N., Saurabh, S., and Patel, J. M., “Looking ahead
makes query plans robust: Making the initial case with in-memory
star schema data warehouse workloads,” Proc. VLDB Endow.,
vol. 10, no. 8, pp. 889–900, Apr. 2017. [Online]. Available:
https://doi.org/10.14778/3090163.3090167
[35] Makreshanski, D., Giannikis, G., Alonso, G., and Kossmann, D.,
“Many-query join: Efficient shared execution of relational joins on
modern hardware,” The VLDB Journal, vol. 27, no. 5, pp. 669–692,
Oct. 2018. [Online]. Available: https://doi-org.ezproxy.auckland.ac.
nz/10.1007/s00778-017-0475-4
[36] “Sap business warehouse7.5powered by sap hana,” https://www.sap.
com/documents/2016/03/445ef806-647c-0010-82c7-eda71af511fa.
html, accessed: 2019-01-15.
[37] “Enterprise bi in azure with sql data warehouse,” https:
//docs.microsoft.com/en-us/azure/architecture/reference-architectures/
data/enterprise-bi-sqldw, accessed: 2019-01-15.
[38] “Power bi - create and manage relationships in power
bi desktop,” https://docs.microsoft.com/en-us/power-bi/
desktop-create-and-manage-relationships, accessed: 2019-02-09.
[39] “Amazon redshift database developer guide api version 2012-12-
01,” https://docs.aws.amazon.com/redshift/latest/dg/redshift-dg.pdf,
accessed: 2019-01-15.
[40] “Online analytic processing with oracle database
12c,” https://docs.microsoft.com/en-us/power-bi/
desktop-create-and-manage-relationships, accessed: 2019-02-12.
[41] “Online analytic processing with oracle database 12c,” https:
//www.oracle.com/technetwork/database/options/advanced-analytics/
advanced-analytics-wp-12c-1896138.pdf, accessed: 2019-02-12.
[42] “Oracle in-memory overview,” https://www.oracle.com/technetwork/
database/in-memory/overview/twp-dbim-usage-2441076.html,
accessed: 2019-02-12.
[43] “Ibm db2 warehouse,” https://www.ibm.com/products/db2-warehouse,
accessed: 2019-02-12.
[44] “Zigzag join enablement for db2 star schema queries,”
https://www.ibm.com/developerworks/data/library/techarticle/
dm-1303zigzag/index.html, accessed: 2019-02-12.
[45] “atscale technical overview,” https://www.atscale.com/resource/
atscale-technical-overview, accessed: 2019-02-10.
[46] “atscale bi olap for hadoop,” https://www.atscale.com/resource/
atscale-bi-olap-for-hadoop-technical-doc, accessed: 2019-02-10.
[47] “Star schema enhancements in apache spark,” https://developer.ibm.
com/code/2018/04/16/star-schema-enhancements-in-apache-spark/,
accessed: 2019-02-10.
[48] “Sparklinedata overview,” https://github.com/SparklineData/
spark-druid-olap/wiki/Overview, accessed: 2019-02-10.
[49] “Stars and models: How to build and maintain star schemas using
sas data integration server in sas 9, nancy rausch, sas institute
inc., cary, nc,” https://support.sas.com/resources/papers/proceedings/
proceedings/sugi31/096-31.pdf, accessed: 2019-02-10.