This article was downloaded by: [141.214.17.

222]
On: 25 October 2014, At: 19:40
Publisher: Taylor & Francis
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,
37-41 Mortimer Street, London W1T 3JH, UK

Information Systems Management

Publication details, including instructions for authors and subscription information:
http://www.tandfonline.com/loi/uism20

Data Warehouse Administration and Management

a b c d
Alan Benander , Barbara Benander , Adam Fadlalla & Gregory James
a
An associate professor of Computer and Information Science at Cleveland State University.
b
An associate professor of Computer and Information Science at Cleveland State University.
c
An associate professor at Cleveland State University in the Computer and Information
Science Department.
d
Manager of Knowledge Discovery Services for National City Corp. and is pursuing his D.B.A.
degree at Cleveland State University.
Published online: 21 Dec 2006.

To cite this article: Alan Benander , Barbara Benander , Adam Fadlalla & Gregory James (2000) Data
Warehouse Administration and Management, Information Systems Management, 17:1, 71-80, DOI:
10.1201/1078/43190.17.1.20000101/31217.10

To link to this article: http://dx.doi.org/10.1201/1078/43190.17.1.20000101/31217.10

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained
in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no
representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the
Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and
are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and
should be independently verified with primary sources of information. Taylor and Francis shall not be liable for
any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever
or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of
the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematic
reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any
form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://
www.tandfonline.com/page/terms-and-conditions
 DATA
WEB-BASED
WAREHOUSING
LEARNING

DATA WAREHOUSE
ADMINISTRATION
AND MANAGEMENT
Alan Benander, Barbara Benander, Adam Fadlalla, and Gregory James
Downloaded by [141.214.17.222] at 19:40 25 October 2014

Data warehouses are huge repositories of legacy data. They are usually con-
figured as stars or snowflakes, and their application strength is their ability to
associate pieces of data in unique and multiple ways. Obviously, they are very
different from traditional relational databases, and as such they require admin-
istrators that possess a skill set different from that of traditional database
administrators. This article explores the skills a data warehouse administrator
must possess.

DATA WAREHOUSE IS A REPOSI- expertise that go beyond those required of a

ALAN BENANDER is an
associate professor of
Computer and Informa-
tion Science at Cleveland
State University.
BARBARA BENANDER
is an associate professor
of Computer and Infor-
mation Science at Cleve-
land State University.
ADAM FADLALLA is
an associate professor at
Cleveland State Univer-
sity in the Computer and
Information Science
Department.
GREGORY JAMES is
manager of Knowledge
Discovery Services for
National City Corp. and
is pursuing his D.B.A.
degree at Cleveland State
University.
© 2 0 0 0 C R C P R E S S L L C
A tory of integrated information, culled
from any number and variety of data
sources, including various databases and
legacy data sources. The size of a data ware-
house is usually massive and the data ware-
h o u s e t y p i c a l l y s t o re s a w i d e r a n g e o f
information that has been generated over long
periods of time. Data related to business sub-
jects such as products, markets, and customers
are all collected, integrated, and housed under
the data warehouse umbrella. When this vast
wealth of information is interfaced to decision
support tools that offer powerful data access
and analysis capabilities, the data warehouse
can be fully exploited by its users.
There are a number of data warehousing
tools and products currently on the market. All
of these products provide relatively easy access
to the data warehouse, both at the enterprise
and the data mart level. All of these tools also
provide useful facilities for administering and
managing a data warehouse. However, creation,
maintenance, and daily administration of data
warehouses are still formidable tasks that are far
from being fully automated.
The successful administration and manage-
ment of a data warehouse requires skills and
I N F O R M A T I O N S Y S T E M S
W I N T E R
traditional database administrator (DBA).
There is a need for the creation of a position
that is designated a “data warehouse adminis-
trator” (DWA). The tasks of a DWA encom-
pass those of a traditional DBA, but the DWA’s
job is considerably more complex because of
the nature of the data warehouse and its posi-
tion within an enterprise data architecture.
DATA WAREHOUSE DATABASES VS.
OPERATIONAL DATABASES
In order to better understand the distinction
between the roles of a DBA and a DWA, it is
imperative to understand the significant differ-
ences between an operational database and a
data warehouse. They differ not only in their
ultimate purposes, but also in the types and
amount of data stored in them and the meth-
ods by which they are populated, updated, and
accessed. Some distinguishing characteristics
are summarized in Exhibit 1.
There are three main types of data ware-
house: operational data stores (ODS), enterprise
data warehouses (EDW), and data marts (DM).
Operational data stores are the latest evolution
of customer information files (CIF). CIFs were
M A N A G E M E N T
2 0 0 0
 DATA WAREHOUSING
EXHIBIT 1 Operational Databases vs. Warehouse Databases
Characteristic Operational Databases
Users All users
Goal Recordkeeping (OLTP)
Update Online
Query level Atomic, detailed
Time horizon Mainly present
Data source Internal
Orientation Entity oriented (product,
account, customer …)
Data volumes Gigabytes
Process Transaction driven
Structure Relatively static
Downloaded by [141.214.17.222] at 19:40 25 October 2014
widely deployed in the past to maintain all com-
mon customer information in one central loca-
tion. Updates to this information are entered
into the CIF, and then all other systems that
share this data are synchronized to it. Name,
address, and phone number are good examples
of shared customer information. Today’s more
sophisticated ODS performs the same basic
function as the CIF but is also actively used by
customer support and sales personnel. It is the
nexus for all marketing and sales activities, and
it contains information pertaining to current,
prospective, and past customers. In addition to
the basic customer information, the ODS also
tracks all recent customer activity and market-
ing interactions with the customer. ODSs are
commonly deployed by corporations reorganiz-
ing away from product-line-oriented structures
to customer-oriented structures
Enterprise data warehouses are distinguished
from operational data stores by the scope and
quantity of the data they contain. They typically
contain product, accounting, and organizational
information as well as customer information.
Where an ODS may typically contain 12 to 18
months of data, an EDW will store three to five
years of data or more. Many EDWs are not
accessed directly by end users because of their
size and complexity. Their role is to store the
integrated, scrubbed, historical information that
is used as the standard data repository from
which multiple, special purpose data warehouses
(i.e., data marts) are populated. These secondary
data warehouses are then accessed by end users
with decision support and ad hoc query tools.
A data mart is a subject-oriented or depart-
ment-oriented data warehouse whose scope is
© 2 0 0 0 C R C P R E S S L L C I N F O R M A T I O N S Y S T E M S
W I N T E R
Warehouse Databases
Executives, analysts, customer service
representatives
Information analysis (OLAP, etc.),
decision support, database marketing
Batch
Aggregated, summarized, integrated
Historical + present + future
Internal + external
Category oriented (Product type,
account type, customer segment …)
Gigabytes/terabytes
Analysis driven
Dynamic
much smaller than that of an enterprise data
warehouse. Its purpose is to provide decision
support for a particular department within an
enterprise or about a particular subject within
an enterprise. Many companies decide to first
implement individual data marts before com-
mitting to a full-scale EDW. It has been esti-
mated that a data mart typically requires an
order of magnitude (1/10) less effort than that
of an EDW. In addition, it takes months, as
opposed to years, to build, and costs tens or
hundreds of thousands of dollars, versus the
millions of dollars needed for an EDW.7
All three types of data warehouses represent
data in fundamentally different ways than do
operational databases. In an operational data-
base that has been designed using traditional
relational database design principles, data is
represented as tables, whereas in a data ware-
house, there are many instances where data is
most naturally represented as cross-tabulations.
As an example, Exhibit 2 shows how sales data
might be represented using a relational data-
base design for an operational database. Exhibit
3 shows a two dimensional representation of
the same data. The two dimensions are Product
(P1, P2, P3, P4) and Region (E, C, W). Adding
a third dimension, such as Time (e.g., 1999,
2000), upgrades the two-dimensional represen-
tation to a cube, where a cell of the cube repre-
sents “sales of a specific product, in a specific
region, for a specific year.”
Differences between operational databases
and data warehouse databases also manifest
themselves sharply in their physical designs. An
important goal for an operational database is to
provide short update and query transactions.
M A N A G E M E N T
2 0 0 0
 DATA WAREHOUSING

EXHIBIT 2 A Sample Relational Sales Database

Product Region Sales

P1 E 20
P1 C 30
P1 W 10
P2 E 30
P2 C 25
P2 W 15
P3 E 60
P3 C 50
P3 W 40
P4 E 10
P4 C 40
P4 W 20
Downloaded by [141.214.17.222] at 19:40 25 October 2014

The physical structure of an operational data-
base is designed to ensure update efficiency
and data integrity over small sets of related
data. On the other hand, data warehouses are
typically used to answer analytic questions in a
user-friendly, ad hoc query environment. The
physical structure of a data warehouse is
designed to provide data loading and storage
efficiency and fast ad hoc query response times.
Special indexing techniques (for example, bit-
map indexes) and physically partitioned sche-
mas are commonly used in data warehouse
implementations.
Because of its enormous size, as well as its
inherent nature, a data warehouse requires very
close monitoring in order to maintain accept-
able efficiency and productivity. As more and
more users access the warehouse, the ware-
house grows both structurally and in size, caus-
ing potential system performance problems.
The data structure may need to change over
time as additional data is kept for analysis.
Some physical structures may grow dramati-
cally as the historical content increases.
Unusual business events such as mergers and
acquisitions and new product introductions
may require large system conversions or
enhancements to be implemented. All of these
changes require specialized tools and proce-
dures to facilitate version control, monitoring,
and system performance tuning.
Many data warehouses are designed using a
star schema model. A star schema includes a
central “fact” table surrounded by several
“dimension” tables. The fact table contains a
large number of rows that correspond to
observed business events or facts. The dimen-
sion tables contain classification and aggrega-
tion information about the central fact rows.
The dimension tables have a one-to-many rela-
tionship with rows in the central fact table. The
star schema design provides extremely fast
query response time, simplicity, and ease of
maintenance for read-only database structures.
However, star schemas are not well suited for
online update operations. Specialized data
warehousing tools have been built to explicitly
support star schema architectures. The more
recent versions of general-purpose relational
systems such as the IBM DB/2, Oracle, and
Microsoft’s SQL Server now support these
structures as well.

EXHIBIT 3 A Possible Data Warehouse Version of the

Database of Exhibit 2

E C W
P1 20 30 10
P2 30 25 15
P3 60 50 40
P4 10 40 20

© 2 0 0 0 C R C P R E S S L L C I N F O R M A T I O N S Y S T E M S M A N A G E M E N T
W I N T E R 2 0 0 0
 DATA WAREHOUSING
It is evident that a DBA who possesses
knowledge and skills pertinent to only a tradi-
tional operational database will not be able to
administer and manage a data warehouse. A
person knowledgeable of all the various com-
plexities of a data warehouse (i.e., a DWA), is
A DWA
needed.
must pay DATA WAREHOUSE ADMINISTRATION
special AND MANAGEMENT TASKS
attention to Traditional database administration methods
and techniques for operational database applica-
high-level
tions performed by the DBA have been devel-
business oped and refined over 20 years of industrywide
processes and use. Robust tools exist to aid in all aspects of the
job of a DBA. Data warehousing technology, on
to their effects
the other hand, has only recently moved beyond
upon the its initial stages, where most efforts are the first
database of their kind. Database management systems
Downloaded by [141.214.17.222] at 19:40 25 October 2014
have been enhanced significantly to enable the
structure.
creation of large-scale data warehouse databases,
and physical data storage systems have been
enhanced to handle the extremely large capaci-
ties as well. Specialized data warehouse adminis-
tration tools and techniques are also being
developed to handle the unique system require-
ments of data warehouses.
All of the usual operational database admin-
istration tasks apply to a data warehouse envi-
ronment. However, because of the differences
between an operational database and a data
warehouse previously mentioned, many of these
tasks are performed differently, focus on differ-
ent objectives, or are more complicated. For
example, whereas a DBA is generally concerned
with small, frequent, online updates to the data-
base, the DWA considers large, less frequent,
batch updates to the database. It is critical to
the success of a data warehouse that large, batch
updates are accomplished as quickly as possible
in order to make the refreshed data warehouse
available as soon as possible. In many situations,
the currency and availability of the data ware-
house provides a significant competitive advan-
tage. For example, a business’s ability to react
quickly to a sudden shift in consumer behavior
may result in significant customer retention or
in new customer acquisition. Frequent updates
to the data warehouse may be needed to sup-
port the decision-making cycle, which may be
monthly, weekly, or even daily.
Planning
A DWA must plan for a potentially enormous
growth rate of the data warehouse, anticipating
growth rates that are much higher than a DBA
© 2 0 0 0 C R C P R E S S L L C I N F O R M A T I O N S Y S T E M S
W I N T E R 2 0 0 0
would experience. In fact, a recent Gartner
Group management report stated the follow-
ing: “It is not unusual for organizations’ ware-
houses to experience a 100 percent growth-rate
per year. This is not the typical scenario that
DBAs take into account when sizing databases
for production. Compound this with the fact
that adding even incremental amounts of data
increases the usefulness of the overall ware-
house. Therefore, moderately small additions
may result in a multiplier effect for queries. In
the end, what one might experience is a drastic
growth in not only the size of the warehouse
but also the usage, and necessarily its ongoing
administration (security, tuning, etc.).”
Clearly such future expansion must be taken
into account by the DWA when planning a
data warehouse. He or she should consider
using system architectures to support such vol-
umes of data; these include SMP (Symmetric
Processing) and MPP (Massive Parallel Process-
ing) platforms, which introduce new levels of
design complexity (parallelism of database
operations and storage). DB2 Parallel, Informix
XPS, and Oracle Parallel offer versions that sup-
port parallel operations.8
Also as part of the planning stage, the DWA
must be involved with data-related issues that
pertain to data inconsistencies and data seman-
tics. Source data inconsistencies pose potential
serious problems to the integrity and function-
ality of the data warehouse. For example, even
though a particular data field is not essential for
all business units, it must be entered properly
by all business units in order for the integrated
views within the warehouse to be accurate. As
an example, suppose that manufacturing batch
numbers are not important to the sales depart-
ment, but are extremely important for manu-
facturing and customer service to be able to
track down all customers using a particular
batch of a product. If the sales department
does not enter the batch numbers, or enters
them inconsistently in their system, then track-
ing usage from sales back through the manufac-
turing process might be impossible.
DWAs also need to deal with issues regard-
ing data semantics. Data and its meaning
evolve over time. What was a preferred cus-
tomer last year may not be a preferred customer
this year by virtue of a change in business pol-
icy. Consequently, the DWA must pay special
attention to high-level business processes and
to their effects upon the database structure
(process-enforced integrity vs. structure-
enforced integrity). Processes that manage and
maintain data in all of the source systems need
M A N A G E M E N T
 DATA WAREHOUSING
to be examined when they change to ensure
that the meaning of the data has not also
changed. This could affect the fundamental
meaning of the data (as it is defined by its
source) or its integrated meaning (as it is
defined within the warehouse where many
D ata
more relationships are maintained).
Another difference in the challenges facing
warehouses the DWA as opposed to a DBA is in the area of
move beyond planning ahead for storage management.
Whereas a typical corporate database might
data consume several gigabytes of storage, a data
processing to warehouse may require several terabytes.
decision- Because of the warehouse’s enormous size, par-
allel processing is the norm when dealing with
support and a data warehouse. To achieve acceptable access
knowledge- rates and minimize data loss, the data in a data
based warehouse generally is distributed over multiple
disks, with concurrent access. Because of these
applications.
Downloaded by [141.214.17.222] at 19:40 25 October 2014
and other considerations, the traditional stor-
age management tasks of a DBA are signifi-
cantly expanded in a data warehouse
environment.
Another issue with which a DWA must be
concerned is the inclusion of potentially large
volumes of external data. The data in a data
warehouse is an information asset that may be
augmented, thereby appreciating in its overall
value. This is often accomplished by adding
external data acquired from outside organiza-
tions to enrich the data warehouse. Census
information, credit reports, purchasing habits,
demographics, and more are all available for
purchase. When external data is added to a
data warehouse, organizations can begin using
the augmented data related not only to their
existing customers, but also potential new cus-
tomers that fall within their marketing profiles.
This is an important capability for firms wish-
ing to expand their market share. Thus, to truly
leverage the power of a data warehouse, the
DWA must plan for the future inclusion of
valuable external data.
Design
The design goals for a data warehouse differ
from those of an operational database. The
major design goal for an operational database is
to meet business operational data require-
ments, while the major design goal for a data
warehouse is to meet business analysis informa-
tional requirements. Consequently, the DWA
cannot simply rely on conventional database
design techniques such as E-R diagrams, trans-
action processing constraints, database normal-
ization, and the like. A design approach for the
© 2 0 0 0 C R C P R E S S L L C I N F O R M A T I O N S Y S T E M S
W I N T E R 2 0 0 0
data warehouse uses the business subject–ori-
ented approach.
Data warehouses move beyond data process-
ing to decision-support and knowledge-based
applications. In this mission, the semantic
complexity of the underlying database structure
becomes even more complex. This additional
information requirement often takes the form
of more complex relationships among base
tables, which in turn require more database
keys and indexes. The need to index candidate
keys is associated with large tables of the kind
that pervade data warehouses. Indeed, multidi-
mensional databases, used to deploy data ware-
houses, often require more space to store their
indexes than they do for their main fact tables.
Some warehouse designers implement indexes
on columns with large domains as a substitute
for commonly executed select operations that
are known a priori to produce small result sets.
In such situations, the overhead of building and
maintaining the index is smaller than executing
the same “where” clause over and over.
The design of a data warehouse is intended
to provide integrated views of business objects
such as “Customer.” The working set of data
values that define “Customer” may not all
arrive at the warehouse at one time. Therefore,
integrity constraints that might be assumed by
the user of the warehouse might not be practi-
cally enforced during update. The DWA must
be sensitive to the temporal aspects of the data
warehouse. For example, with time, the mean-
ing of classifications can change. For example,
the classification “Middle Income” in 1979 vs.
“Middle income” in 1999 is probably different.
But an important issue is whether the 1979 or
the 1999 data is stored in historical records. If
the original data used to derive the classifica-
tions is no longer available, then there is no
ability to adjust it in the future.
Data warehouse designs must be robust to
deal with incompleteness and inconsistency,
unlike OLTP and operational database designs.
The potential for null values, has a significant
impact upon query strategies and adds complex-
ity to the warehouse architecture. Does one
impose restrictions on the warehouse and disal-
low nulls, potentially excluding a significant por-
tion of useful, albeit incomplete, data? Or does
one allow null values within the warehouse and
deal with the potential incompleteness on the
query side? As the number of data sources grows
(e.g., multiple operational databases that supply
data to the warehouse), the potential for “disin-
tegration” increases. This permits a situation in
which the total working set of attributes in the
M A N A G E M E N T
 DATA WAREHOUSING
integrated views within the warehouse are not
totally populated. Missing values may arise
through historical discrepancies (the data was
not collected during that time period), through
incomplete view integration (inconsistent can-
didate keys), or nonmandatory values (fields
M DDs give
that are important for one application may not
be important for another application).
flexible access Some argue that the RDBMS design
to large approach can fail to deliver adequate perfor-
mance as a result the massive size and com-
amounts of plexity of the data warehouse. For example,
data and do it typical queries against the data warehouse are
quickly by complex and ad hoc, often submitted by high-
level managers in the organization. Answering
“pre-digesting” highly complex business analysis queries may
data. require a large number of joins of huge tables.
This creates huge, temporary tables and is very
time-consuming. Expensive hardware solutions
Downloaded by [141.214.17.222] at 19:40 25 October 2014
do exist, including parallel database servers.
Furthermore, the complexity of the SQL state-
ments would likely be an impediment to the
nonsophisticated casual query writer. To solve
this problem, the DWA could create canned,
optimized queries. This solution, however, is
time-consuming for the DWA and does not
offer maximum flexibility, in terms of range of
available queries, to the users.
An alternative design model for a data ware-
house is the multidimensional database
(MDD). In the MDD model, data is stored in a
multidimensional array (hypercube) to allow
users to access, aggregate, analyze, and view
large amounts of data quickly. MDDs give flex-
ible access to large amounts of data and do it
quickly by “pre-digesting” data. Several prod-
ucts exist for creating and accessing MDDs.
These include eSSbase, Express, and LightShip
Server. MDDs usually keep the data warehouse
size to a minimum on the server through data
compression and provide client front-end tools.
Clearly, the DWA needs to be familiar with the
various new technologies for implementing a
data warehouse.
Implementation
Once the DWA has planned and designed the
data warehouse, he or she must choose an
implementation approach. It is the authors’
opinion that the DWA should use an incremen-
tal development approach, rather than an
enterprise approach, in creating a successful
data warehouse. The size and complexity of an
enterprise data warehouse makes an incremen-
tal approach the preferred strategy. For exam-
ple, one successful strategy is to approach each
© 2 0 0 0 C R C P R E S S L L C I N F O R M A T I O N S Y S T E M S
W I N T E R
business function (e.g., marketing, sales,
finance) that will use the data warehouse and
build the data model one function at a time.
With this approach, the DWA simply creates
data marts for each department. When com-
pleted, the enterprise data warehouse will com-
prise the integration of all the data marts.
Those who argue against the incremental
approach cite the inherent problems involved
in combining several data marts into a single
data warehouse. The integration of these data
marts, they maintain, is more difficult than ini-
tially creating a single enterprise data ware-
house. The authors contend, however, that the
complexity and cost involved with the enter-
prise approach is not worth the risk. With the
vast array of new technologies and modeling
approaches, it is a judicious approach for a
DWA to first become expert at creating a data
mart before attempting to create the enterprise
data warehouse.
Implementing a data warehouse requires
loading of data, which is a far more complex task
for a data warehouse, as compared with a con-
ventional database. The first major challenge for
the DWA is locating all data sources for the data
warehouse. As mentioned previously, a data
warehouse is created by using a variety of data
sources as input. These can include relational
databases, hierarchical databases, network data-
bases, conventional files, flat files, news wires,
HTML documents, and legacy systems. The
tasks of identifying and locating all sources are
the responsibility of the DWA. With frequent
corporate downsizing and the high turnover rate
in some businesses, it may prove quite a chal-
lenge for the DWA to identify and locate these
important inputs to the data warehouse.
After locating all of the data sources for the
data warehouse, the DWA must ascertain the
various formats in which the data to be input-
ted to the data warehouse is to be stored. The
heterogeneous nature of the data, originating
from multiple production and legacy systems,
as well as from external sources, creates another
challenge for the DWA. All of the data to be
stored in the data warehouse must be reformat-
ted to conform to the chosen data model of the
data warehouse. A number of management
tools are available to assist in this task, includ-
ing those from such vendors as Carleton, Plati-
num Technology, and Prism Solutions.
Data quality is a much greater challenge in a
data warehouse setting than in an operational
database setting. The authors’ experiences have
shown that nearly 20 percent of the data is
potentially inconsistent for one reason or
M A N A G E M E N T
2 0 0 0
 DATA WAREHOUSING
another. For example, it may happen that
address changes are not captured, records
belonging to the same customer are not merged
properly within the warehouse because of incon-
sistent candidate keys, or old errors in nonessen-
tial fields are replicated into the warehouse
A DWA
without notice. The data quality problem
becomes evident on the query side when incon-
must be much sistent results are returned. One particular exam-
ple occurred at a company that queried its data
more familiar
warehouse for duplicate Social Security num-
with the bers. The system returned one SSN several hun-
business rules dred times. Upon investigation, it was found
that a particular business office had entered the
and policies
SSN of their branch manager, instead of the cus-
than the DBA tomers’, for all special accounts because a favor-
counterpart. ite production report was sorted by customer
SSN rather than by manager!
There are many other data quality chal-
Downloaded by [141.214.17.222] at 19:40 25 October 2014
lenges inherent to heterogeneous data. For
example, different production systems may rep-
resent names differently, use different codes
that have the same meaning, and may store the
same attribute in different formats (e.g., many
formats exist for storing the date field).
Another issue that the DWA must address is
the possibility of changes in the information
source. Detection of changes in the information
source can vary in importance, depending on
the nature of the data warehouse. For example,
when it is not important for the warehouse to
be current and it is acceptable for the data ware-
house to be off-line occasionally, then ignoring
change detection may be acceptable. However,
if currency, efficiency, and continuous access are
required, then these changes must be propa-
gated up to the data warehouse. How does the
DWA detect changes in the input data sources
and propagate them up to the data warehouse?
It depends on the nature of the data source. For
example, if a source has active database capabil-
ities (such as triggers), then change notification
can be programmed automatically. If a data
source maintains a log that can be inspected,
then inspection of the log can reveal changes.
Detection of change in input sources and prop-
agation to the data warehouse is simply an addi-
tional function of the DWA, a task not normally
required of a traditional DBA.
Another traditional database task that is
expanded upon for the DWA is the creation of
various views that are appropriate for the vari-
ety of users. Whereas the DBA created opera-
tionally oriented views, the DWA must create
views for managers that are oriented toward
business decision making. The DWA must be
much more familiar with the business rules and
© 2 0 0 0 C R C P R E S S L L C I N F O R M A T I O N S Y S T E M S
W I N T E R 2 0 0 0
policies than the DBA counterpart. The DWA
will be in frequent communication with the
business managers and must understand clearly
their needs and requirements.
Many standard data warehouse queries spec-
ify aggregation values, and many of these aggre-
gations are requested over and over again. In
order to minimize the number of full-table
scans, the DWA should look for often-used
aggregates and store them as permanent values
in aggregate tables within the warehouse. These
tables are refreshed at the same time the ware-
house is updated, which requires more process-
ing during load but, over the long term,
eliminates redundant scans to produce the
same aggregate values even though the base
tables have not changed.
Maintenance
In traditional database administration, the
DBA is responsible for ensuring the security
and integrity of the database, defining user
views, and setting access permissions to the
database. The responsibilities of a DBA also
include implementing policies and procedures
related to database security. For a data ware-
house, security issues are even more critical. For
example, if a business failure occurred in some
aspect, many queries would need to be
answered, such as “What was the major con-
tributing factor?” Who should be allowed to
ask such queries and obtain the answers? The
very survival of the business could well depend
on proper security in the data warehouse, pro-
vided for by the DWA.
Some organizations are sensitive to internal
access. For example, at a large healthcare orga-
nization, various medical departments were
concerned about physicians from other depart-
ments viewing specialized test results, making
assumptions about the results’ meaning, and
coming to erroneous conclusions. There are
many situations where the data might imply
one thing but in reality means something quite
different. In these circumstances, knowledge
about the data and how it came into existence
is not captured within the warehouse. Special
security measures put in place by the DWA are
required to mitigate these risks.
The “knowledge about the data,” or meta-
data, is important information that must be
administered by the DWA. Metadata, which is
the syntactic, semantic, and pragmatic docu-
mentation of the data in the data warehouse, is
crucial for the success of a data warehouse. The
DWA must be familiar with the different
M A N A G E M E N T
 DATA WAREHOUSING
aspects of data: (1) syntactics, which defines
the structure of the database; (2) semantics,
which defines the meaning of the data; and (3)
pragmatics, which defines the source and life
cycle of the data (effective date, source system,
etc.). Business metadata defines relatively static
Equally
business objects and concepts such as branch
office code or product code. This data is typical
important to of commonly used “look-up” tables. Business
metadata, for example, would keep track of
the
how many systems maintain their own copies of
technological state codes and how many systems maintain
expertise of a copies of product codes.
Finally, backup and recovery of a database
DWA is the
that is the size of a data warehouse requires
possession of much more efficient techniques than a DBA
strong would use. Because of the enormous size of a
data warehouse and the resulting time that can
business
be consumed in the backup process, the DWA
acumen.
Downloaded by [141.214.17.222] at 19:40 25 October 2014
must carefully plan backup strategy during the
design phase of the data warehouse. For exam-
ple, the planned use of read-only tablespaces,
the use of a strategic partitioning scheme, and
the judicious choice of partition size is critical to
the success of efficient backup. In the complex
world of a data warehouse, it is especially crucial
to perform testing on backup strategies. As far as
recovery is concerned, the DWA must have a
recovery plan for each possible failure scenario.
The DWA must be familiar with the hardware
and software needed to implement backup and
recovery strategies. For example, there are a large
number of tape backup options on the market,
including a variety of tape media, standalone
tape drives, tape stackers, and tape silos. Of
course, given the size of the data warehouse, disk
backup is not as feasible as tape backup, but, if so
desired, disk technology can be used to accom-
plish disk-to-disk backup and mirror breaking.
Backup software packages include Ominiback II
(HP), ADSM (IBM), Alexandria (Sequent),
Epoch (Epoch Systems), and Networker
(Legato). It is possible to achieve different com-
binations of backup hardware and software that
can yield anywhere from 10 BG per hour to 500
GB per hour. 8 Again, due to the size of the data
warehouse, performance of the backup process
becomes an important issue. To minimize the
backup time, parallelism is used. Exhibit 4 sum-
marizes some of the differences between the
tasks of a DBA and those of a DWA.
The Organizational Role of a Data
Warehouse Administrator
As discussed on the foregoing text, the DWA
must be familiar with high-performance soft-
© 2 0 0 0 C R C P R E S S L L C I N F O R M A T I O N S Y S T E M S
W I N T E R 2 0 0 0
ware, hardware, and networking technologies.
Equally important to the technological exper-
tise of a DWA is the possession of strong busi-
ness acumen. Data warehouses are dynamic
resources used by business managers and deci-
sion makers to analyze complex business situa-
tions. The DWA must be familiar with the
relevant decision-making processes in order to
properly design and maintain the data ware-
house structure. The more specialized the deci-
sion support requirements, the more design
tradeoffs the DWA must consider. The DWA
should possess knowledge of the business orga-
nization itself as well as general business princi-
ples for which the data warehouse is being
developed. It is especially critical for a DWA to
balance the current requirements and capabili-
ties of the data warehouse while providing for
the rapid deployment of enhancements.
Finally, the DWA must possess excellent
communication skills. This is more critical for a
DWA than for a DBA, because the DWA will
be closely interacting with the managers in
using the data warehouse effectively. The DBA
often needs only to communicate with pro-
grammers and other technical personnel who
act as intermediaries in the specification and
design process. This is not usually the case for
the DWA, who must participate in the business
meetings directly in order to meet the fast
development life cycles of a typical data ware-
house project. If the DWA cannot communi-
cate in the language of nontechnical people,
the data warehouse project will suffer from mis-
understanding and delay.
FUTURE TRENDS
Data warehousing has given a competitive edge
to a number of enterprises that have, in recent
years, employed this technology. Whether the
data warehouse is an operational data store, a
data mart, or a full-blown enterprise data ware-
house, the administration and management
requirements of the enormous amount of data
in a data warehouse differ significantly from
those of a traditional DBA managing an opera-
tional database.
Because the characteristics (which are deter-
mined by the design goals) of a data warehouse
are in many ways very different than those of an
operational database, the data warehouse
administrator (DWA) must possess knowledge
and skills that his or her DBA counterpart need
not have. Differences in requirements show up
in all phases of the data warehouse develop-
ment: planning, designing, implementing, and
M A N A G E M E N T
 DATA WAREHOUSING

EXHIBIT 4 Tasks of DBA vs. DWA

Task Dimension DBA DWA

Planning Future growth in size Moderate Drastic
and usage
Data consistency and Transaction-level Aggregation-level
integration
Storage requirements Less issues More issues (e.g., disk arrays and
parallelism)
Design Goals Meet business operational Meet business analysis informational
data requirements requirements
Methodology Mainly ER diagrams Mainly dimension maps
Implementation Scope Application-oriented Enterprise-oriented
Approach Monolithic Incremental
Granularity Transaction-based updates Bulk updates
Query type Many reporting predefined Many analysis ad hoc queries
queries
Data level Detailed Summarized and aggregated
Downloaded by [141.214.17.222] at 19:40 25 October 2014

Maintenance Security View-based User-based

Security risk Security breaches relatively Security breaches relatively have far-
have contained impact reaching impact
Backup and recovery Backup less time-consuming Backup more time-consuming;
performance more critical

maintaining. Exhibit 4 summarizes some of
these differences.
The success of a data warehouse depends to
a large extent on the DWA, who must be very
knowledgeable about current data warehouse
hardware and software technology. In addition,
because of the nature and purpose of a data
warehouse, it is equally important for the suc-
cessful DWA to possess knowledge of the busi-
ness organization itself and general business
principles as well.
Emerging trends in data warehousing offer
competitive advantages to organizations poised
to take advantage of them. These exciting new
technologies also offer new challenges to the
data warehouse administrator, whose responsi-
bility it will be to leverage them for the benefit
of the organization.
One emerging area facilitated by the use of
data warehouse technology is Business Intelli-
gence. New forms of data are being integrated
with data warehouses to provide higher forms
of information to decision makers. Large blocks
of text, graphics, and even sound are being
added to traditional data. Many large telemar-
keting organizations have deployed systems for
their call centers that provide customer service
representatives with telemarketing scripts. The
systems are able to capture typed notes and
customer responses on a real-time basis. This
“nonstructured” data is stored in an integrated
© 2 0 0 0 C R C P R E S S L L C I N F O R M A T I O N S Y S T E M S
W I N T E R 2 0 0 0
fashion within the data warehouse environment
and is then available the next time contact is
made with the customer. It is also used to eval-
uate marketing strategies and tactics to
improve business performance. Newer genera-
tion data warehouses that include more varied
metadata and complex data types require spe-
cialized database features. New indexing struc-
tures and query capabilities are being provided
by DBMSs to support these nontraditional data
types. Querying a database for text, color, or
sound has become a reality during the past sev-
eral years.
In addition, Internet/intranet access allows
organizations to expose information within
their data warehouses to users outside their
immediate physical networks. Some organiza-
tions have opened up portions of their data
warehouses to customers. Federal Express, for
example, allows customers to query their parcel
tracking system over the Internet to obtain the
status of their shipment. A number of banks
and investment companies provide secured,
online access to customers’ accounts. Many
more organizations have enabled intranet
access within their network firewalls to essen-
tially broadcast data warehouse information to
employees regardless of where they work and
what kind of PC they may have. Intranet access
to data warehouses has also become critical to
field sales and support personnel to improve
M A N A G E M E N T
 DATA WAREHOUSING
their performance. Web-enabling data ware-
houses potentially places greater need for
online query response and 24x7 availability —
requirements not necessarily taken into
account when the data warehouse was origi-
nally designed.
The use of data warehousing will continue to
grow as more and more companies realize the
benefits afforded to them by this technology.
The role of the DWA will, to a large extent,
determine the success or failure of the data
warehouse within the organization. ▲
References
1. Anahory, S. and Murray, D., Date Warehousing in
the Real World, Addison-Wesley, Harlow, England
(1997), pp. 205–221.
2. Callaway, Erin, “License To Drive,” Inside
Technology Training, (July/August, 1998), pp.
Downloaded by [141.214.17.222] at 19:40 25 October 2014
18–21.
© 2 0 0 0 C R C P R E S S L L C I N F O R M A T I O N S Y S T E M S
W I N T E R 2 0 0 0
3. Gardner, Stephen R., “Building the Data
Warehouse,” Communications of the ACM, Vol.
41, No. 9, (Sept, 1998), pp. 52–60.
4. Gerdel, Thomas W., “LTV Steel computer
methods praised,” The Cleveland Plain Dealer
(July 7, 1996).
5. Radding, Alan, “Support Decision Makers with a
Data Warehouse,” Datamation, (March 15, 1995),
pp. 53–56.
6. Ricciuti Mike, “Multidimensional Analysis:
Winning the Competitive Game,” Datamation,
(Feb. 15, 1994), pp. 21–26.
7. van den Hoven, John, “Data Marts: Plan Big,
Build Small,” Information Systems Management,
(Winter, 1998), pp. 71–73.
8. Weldon, Jay-Louise, Warehouse Cornerstones,
Byte, (Jan., 1997), pp. 85–88.
M A N A G E M E N T