Data Mining

Data Mining
10.14.1 Purpose
Data mining is used to improve decision making by finding useful patterns and
insights from data.
10.14.2 Description
Data mining is an analytic process that examines large amounts of data from
different perspectives and summarizes the data in such a way that useful patterns
and relationships are discovered.
The results of data mining techniques are generally mathematical models or
equations that describe underlying patterns and relationships. These models can
be deployed for human decision making through visual dashboards and reports,
or for automated decision-making systems through business rule management
systems or in-database deployments.
Data mining can be utilized in either supervised or unsupervised investigations. In

a supervised investigation, users can pose a question and expect an answer that
can drive their decision making. An unsupervised investigation is a pure pattern
discovery exercise where patterns are allowed to emerge, and then considered for
applicability to business decisions.
Data mining is a general term that covers descriptive, diagnostic, and predictive
techniques:
• Descriptive: such as clustering make it easier to see the patterns in a set of
data, such as similarities between customers.
• Diagnostic: such as decision trees or segmentation can show why a
pattern exists, such as the characteristics of an organization's most
profitable customers.
• Predictive: such as regression or neural networks can show how likely
something is to be true in the future, such as predicting the probability that
a particular claim is fraudulent.
In all cases it is important to consider the goal of the data mining exercise and to
be prepared for considerable effort in securing the right type, volume, and quality
of data with which to work.
10.14.3 Elements
.1 Requirements Elicitation
The goal and scope of data mining is established either in terms of decision
requirements for an important identified business decision, or in terms of a
functional area where relevant data will be mined for domain-specific pattern
discovery. This top-down versus a bottom-up mining strategy allows analysts to
pick the correct set of data mining techniques.
Formal decision modelling techniques (see Decision Modelling (p. 265)) are used
to define requirements for top-down data mining exercises. For bottom-up
pattern discovery exercises it is useful if the discovered insight can be placed on
existing decision models, allowing rapid use and deployment of the insight.
Data mining exercises are productive when managed as an agile environment.

They assist rapid iteration, confirmation, and deployment while providing project
controls.
.2 Data Preparation: Analytical Dataset
Data mining tools work on an analytical dataset. This is generally formed by

merging records from multiple tables or sources into a single, wide dataset.
Repeating groups are typically collapsed into multiple sets of fields. The data may
be physically extracted into an actual file or it may be a virtual file that is left in the
database or data warehouse so it can be analyzed. Analytical datasets are split
into a set to be used for analysis, a completely independent set for confirming
that the model developed works on data not used to develop it, and a validation
set for final confirmation. Data volumes can be very large, sometimes resulting in
the need to work with samples or to work in-datastore so that the data does not
have to be moved around.
.3 Data Analysis
Once the data is available, it is analyzed. A wide variety of statistical measures are
typically applied and visualization tools used to see how data values are
distributed, what data is missing, and how various calculated characteristics
behave. This step is often the longest and most complex in a data mining effort
and is increasingly the focus of automation. Much of the power of a data mining
effort typically comes from identifying useful characteristics in the data. For
instance, a characteristic might be the number of times a customer has visited a
store in the last 80 days. Determining that the count over the last 80 days is more
useful than the count over the last 70 or 90 is key.
.4 Modelling Techniques
There are a wide variety of data mining techniques.
Some examples of data mining techniques are:

• classification and regression trees (CART), C5 and other decision tree
analysis techniques,
• linear and logistic regression,
• neural networks,
• support sector machines, and
• predictive (additive) scorecards.
The analytical dataset and the calculated characteristics are fed into these
algorithms which are either unsupervised (the user does not know what they are
looking for) or supervised (the user is trying to find or predict something specific).
Multiple techniques are often used to see which is most effective. Some data is
held out from the modelling and used to confirm that the result can be replicated
with data that was not used in the initial creation.
.5 Deployment
Once a model has been built, it must be deployed to be useful. Data mining
models can be deployed in a variety of ways, either to support a human decision
maker or to support automated decision-making systems. For human users, data
mining results may be presented using visual metaphors or as simple data fields.
Many data mining techniques identify potential business rules that can be
deployed using a business rules management system. Such executable business
rules can be fitted into a decision model along with expert rules as necessary.
Some data mining techniques—especially those described as predictive analytic
techniques—result in mathematical formulas. These can also be deployed as
executable business rules but can also be used to generate SQL or code for
deployment. An increasingly wide range of in-database deployment options allow
such models to be integrated into an organization's data infrastructure.
10.14.4 Usage Considerations
.1 Strengths
• Reveal hidden patterns and create useful insight during analysis—helping

determine what data might be useful to capture or how many people might be
impacted by specific suggestions.
• Can be integrated into a system design to increase the accuracy of the data.
• Can be used to eliminate or reduce human bias by using the data to determine
the facts.
.2 Limitations
• Applying some techniques without an understanding of how they work can

result in erroneous correlations and misapplied insight.
• Access to big data and to sophisticated data mining tool sets and software may
lead to accidental misuse.
• Many techniques and tools require specialist knowledge to work with.
• Some techniques use advanced math in the background and some
stakeholders may not have direct insights into the results. A perceived lack of
transparency can cause resistance from some stakeholders.
• Data mining results may be hard to deploy if the decision making they are
intended to influence is poorly understood.

Data Mining

Uploaded by

Copyright:

Available Formats

Data Mining

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Data Mining

Uploaded by

Copyright:

Available Formats

Data Mining

insights from data.

Data mining can be utilized in either supervised or unsupervised investigations. In

Data mining exercises are productive when managed as an agile environment.

.2 Data Preparation: Analytical Dataset

Data mining tools work on an analytical dataset. This is generally formed by

There are a wide variety of data mining techniques.

Some examples of data mining techniques are:

10.14.4 Usage Considerations

• Reveal hidden patterns and create useful insight during analysis—helping

• Applying some techniques without an understanding of how they work can

You might also like