Difference Between Data Warehousing and Data Mining: Data Warehouse Architecture Three-Tier Data Warehouse Architecture

Data Warehouse Architecture
Three-Tier Data Warehouse Architecture

Generally a data warehouses adopts a three-tier architecture. Following are the three
tiers of the data warehouse architecture.
 Bottom Tier − The bottom tier of the architecture is the data warehouse database server. It
is the relational database system. We use the back end tools and utilities to feed data into
the bottom tier. These back end tools and utilities perform the Extract, Clean, Load, and
refresh functions.
 Middle Tier − In the middle tier, we have the OLAP Server that can be implemented in either
of the following ways.
o By Relational OLAP (ROLAP), which is an extended relational database
management system. The ROLAP maps the operations on multidimensional data to
standard relational operations.
o By Multidimensional OLAP (MOLAP) model, which directly implements the
multidimensional data and operations.
 Top-Tier − This tier is the front-end client layer. This layer holds the query tools and
reporting tools, analysis tools and data mining tools.
Difference Between Data Warehousing

and Data Mining
A Data Warehouse is an environment where essential data from multiple
sources is stored under a single schema. It is then used for reporting and
analysis. Data Warehouse is a relational database that is designed for query
and analysis rather than for transaction processing. It usually contains
historical data derived from transaction data. While a Data Warehouse is built
to support management functions.

Data Mining is used to extract useful information and patterns from data. The
data mining can be carried with any traditional database, but since a data
warehouse contains quality data, it is good to have data mining over the data
warehouse system. Data Mining supports knowledge discovery by finding
hidden patterns and associations, constructing analytical models, performing
classification and prediction.
Let us understand the Difference between Data Warehousing and Data Mining
in detailed
Key Features:
1. Data Warehouse:
The key features of a Data Warehouse are discussed below:
1. Subject Oriented: A data warehouse is subject-oriented as it provides
knowledge around a subject rather than the organization’s ongoing
operations. These subjects can be a product, customers, suppliers, sales,
revenue, etc. A data warehouse focuses on modeling and analysis of
data for decision making.

2. Integrated: A data warehouse is constructed by combining data from
heterogeneous sources such as relational databases, flat files, etc.
3. Time-Variant: The data present in the data warehouse provides
information with respect to a particular time period.
4. Non-volatile: Non-volatile means, data once entered into the
warehouse should not change.
Benefits of Data Warehouse:
1. Consistent and quality data
2. Cost reduction
3. More timely data access
4. Improved performance and productivity
Data Mining:
The key features of Data mining are discussed below:
1. Automatic discovery of patterns

2. Prediction of likely outcomes
3. Creation of actionable information
4. Focus on large data sets and databases
Benefits of data mining:
1. Direct marketing: The ability to predict who is most likely to be
interested in what products
2. Trend analysis: Understanding trends in the marketplace is a strategic
advantage because it helps reduce costs and timeliness to market.
3. Fraud detection: Data mining techniques can help discover which
insurance claims, cellular phone calls or credit card purchases are likely
to be fraudulent.
4. Forecasting in financial markets: Data mining techniques are extensively
used to help model financial markets.

The following diagram depicts the three-tier architecture of data warehouse −
Data Warehouse Models

From the perspective of data warehouse architecture, we have the following data
warehouse models −
 Virtual Warehouse
 Data mart
 Enterprise Warehouse
Virtual Warehouse
The view over an operational data warehouse is known as a virtual warehouse. It is
easy to build a virtual warehouse. Building a virtual warehouse requires excess
capacity on operational database servers.
Data Mart
Data mart contains a subset of organization-wide data. This subset of data is valuable
to specific groups of an organization.
In other words, we can claim that data marts contain data specific to a particular group.
For example, the marketing data mart may contain data related to items, customers,
and sales. Data marts are confined to subjects.
Points to remember about data marts −
 Window-based or Unix/Linux-based servers are used to implement data marts. They are
implemented on low-cost servers.
 The implementation data mart cycles is measured in short periods of time, i.e., in weeks
rather than months or years.
 The life cycle of a data mart may be complex in long run, if its planning and design are not
organization-wide.
 Data marts are small in size.
 Data marts are customized by department.
 The source of a data mart is departmentally structured data warehouse.
 Data mart are flexible.
Enterprise Warehouse
 An enterprise warehouse collects all the information and the subjects spanning an entire
organization
 It provides us enterprise-wide data integration.
 The data is integrated from operational systems and external information providers.
 This information can vary from a few gigabytes to hundreds of gigabytes, terabytes or
beyond.
Load Manager
This component performs the operations required to extract and load process.
The size and complexity of the load manager varies between specific solutions from
one data warehouse to other.
Load Manager Architecture

The load manager performs the following functions −
 Extract the data from source system.
 Fast Load the extracted data into temporary data store.
 Perform simple transformations into structure similar to the one in the data warehouse.
Extract Data from Source
The data is extracted from the operational databases or the external information
providers. Gateways is the application programs that are used to extract data. It is
supported by underlying DBMS and allows client program to generate SQL to be
executed at a server. Open Database Connection(ODBC), Java Database Connection
(JDBC), are examples of gateway.
Fast Load
 In order to minimize the total load window the data need to be loaded into the warehouse in
the fastest possible time.
 The transformations affects the speed of data processing.
 It is more effective to load the data into relational database prior to applying transformations
and checks.
 Gateway technology proves to be not suitable, since they tend not be performant when large
data volumes are involved.
Simple Transformations
While loading it may be required to perform simple transformations. After this has been
completed we are in position to do the complex checks. Suppose we are loading the
EPOS sales transaction we need to perform the following checks:
 Strip out all the columns that are not required within the warehouse.
 Convert all the values to required data types.

Warehouse Manager
A warehouse manager is responsible for the warehouse management process. It
consists of third-party system software, C programs, and shell scripts.
The size and complexity of warehouse managers varies between specific solutions.
Warehouse Manager Architecture

A warehouse manager includes the following −
 The controlling process
 Stored procedures or C with SQL
 Backup/Recovery tool
 SQL Scripts
Operations Performed by Warehouse Manager

 A warehouse manager analyzes the data to perform consistency and referential integrity
checks.
 Creates indexes, business views, partition views against the base data.
 Generates new aggregations and updates existing aggregations. Generates normalizations.
 Transforms and merges the source data into the published data warehouse.
 Backup the data in the data warehouse.
 Archives the data that has reached the end of its captured life.
Note − A warehouse Manager also analyzes query profiles to determine index and
aggregations are appropriate.
Query Manager
 Query manager is responsible for directing the queries to the suitable tables.
 By directing the queries to appropriate tables, the speed of querying and response
generation can be increased.
 Query manager is responsible for scheduling the execution of the queries posed by the user.
Query Manager Architecture

The following screenshot shows the architecture of a query manager. It includes the
following:
 Query redirection via C tool or RDBMS
 Stored procedures
 Query management tool
 Query scheduling via C tool or RDBMS
 Query scheduling via third-party software

Detailed Information
Detailed information is not kept online, rather it is aggregated to the next level of detail
and then archived to tape. The detailed information part of data warehouse keeps the
detailed information in the starflake schema. Detailed information is loaded into the
data warehouse to supplement the aggregated data.
The following diagram shows a pictorial impression of where detailed information is
stored and how it is used.
Note − If detailed information is held offline to minimize disk storage, we should make
sure that the data has been extracted, cleaned up, and transformed into starflake
schema before it is archived.

Difference Between Data Warehousing and Data Mining: Data Warehouse Architecture Three-Tier Data Warehouse Architecture

Uploaded by

Copyright:

Available Formats

Difference Between Data Warehousing and Data Mining: Data Warehouse Architecture Three-Tier Data Warehouse Architecture

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Difference Between Data Warehousing and Data Mining: Data Warehouse Architecture Three-Tier Data Warehouse Architecture

Uploaded by

Copyright:

Available Formats

Data Warehouse Architecture

Three-Tier Data Warehouse Architecture

Difference Between Data Warehousing

analysis. Data Warehouse is a relational database that is designed for query

and analysis rather than for transaction processing. It usually contains

to support management functions.

warehouse system. Data Mining supports knowledge discovery by finding

hidden patterns and associations, constructing analytical models, performing

classification and prediction.

The key features of a Data Warehouse are discussed below:

1. Subject Oriented: A data warehouse is subject-oriented as it provides

knowledge around a subject rather than the organization’s ongoing

operations. These subjects can be a product, customers, suppliers, sales,

revenue, etc. A data warehouse focuses on modeling and analysis of

data for decision making.

heterogeneous sources such as relational databases, flat files, etc.

3. Time-Variant: The data present in the data warehouse provides

information with respect to a particular time period.

4. Non-volatile: Non-volatile means, data once entered into the

warehouse should not change.

Benefits of Data Warehouse:

1. Consistent and quality data

3. More timely data access

4. Improved performance and productivity

The key features of Data mining are discussed below:

1. Automatic discovery of patterns

3. Creation of actionable information

4. Focus on large data sets and databases

Benefits of data mining:

1. Direct marketing: The ability to predict who is most likely to be

interested in what products

2. Trend analysis: Understanding trends in the marketplace is a strategic

advantage because it helps reduce costs and timeliness to market.

3. Fraud detection: Data mining techniques can help discover which

4. Forecasting in financial markets: Data mining techniques are extensively

used to help model financial markets.

Data Warehouse Models

Load Manager Architecture

 Convert all the values to required data types.

Warehouse Manager Architecture

 The controlling process

 Stored procedures or C with SQL

Operations Performed by Warehouse Manager

Query Manager Architecture

 Query redirection via C tool or RDBMS

 Query management tool

 Query scheduling via C tool or RDBMS

 Query scheduling via third-party software

You might also like