Decision Tree Learning

In this paper we address the issue of privacy preserving data mining. Specifically, we consider a scenario in which two parties owning confidential databases wish to run a data mining algorithm on the union of their databases, without revealing any unnecessary information. Our work is motivated by the need both to protect privileged information and to enable its use for research or other purposes.

Naive Bayes is one of most effective classification algorithms. In many applications, however, a ranking of examples are more desirable than just classification. How to extend naive Bayes to improve its ranking performance is an interesting and useful question in practice. Weighted naive Bayes is an extension of naive Bayes, in which attributes have different weights. This paper investigates how to learn a weighted naive Bayes with accurate ranking from data, or more precisely, how to learn the weights of a weighted naive Bayes to produce accurate ranking. We explore various methods: the gain ratio method, the hill climbing method, and the Markov chain Monte Carlo method, the hill climbing method combined with the gain ratio method, and the Markov chain Monte Carlo method combined with the gain ratio method. Our experiments show that a weighted naive Bayes trained to produce accurate ranking outperforms naive Bayes.

Constraint satisfaction is becoming the paradigm of choice for solving many real-world problems. To date, most approaches to constraint satisfaction have focused on solving a problem using some form of backtrack search. Furthermore, the typical view is that a constraint satisfaction problem will be solved only once. However, in many real-world contexts, problems are solved repeatedly over time. Also such problems often exhibit some structure. This motivates the application of some form of learning to improve the performance of search from previously discovered solutions. In this paper we present an approach that uses knowledge about known solutions to a problem to improve search. The approach we present is based on a combination of decision tree learning and constraint satisfaction. We demonstrate that significant improvements, almost an order-of-magnitude, in search effort can be achieved using this hybrid approach over traditional search. We also show that the space complexity using this approach is almost negligible. This work is of interest in domains such as product configuration, and interactive constraint solving in general where the system takes the initiative by asking questions.

There is growing interest in scaling up the widely-used decision-tree learning algorithms to very large data sets. Although numerous diverse techniques have been proposed, a fast tree-growing algorithm without substantial decrease in accuracy and substantial increase in space complexity is essential. In this paper, we present a novel, fast decision-tree learning algorithm that is based on a conditional independence assumption. The new algorithm has a time complexity of O(m · n), where m is the size of the training data and n is the number of attributes. This is a significant asymptotic improvement over the time complexity O(m · n 2 ) of the standard decision-tree learning algorithm C4.5, with an additional space increase of only O(n). Experiments show that our algorithm performs competitively with C4.5 in accuracy on a large number of UCI benchmark data sets, and performs even better and significantly faster than C4.5 on a large number of text classification data sets. The time complexity of our algorithm is as low as naive Bayes'. Indeed, it is as fast as naive Bayes but outperforms naive Bayes in accuracy according to our experiments. Our algorithm is a core tree-growing algorithm that can be combined with other scaling-up techniques to achieve further speedup.

It has been observed that traditional decision trees produce poor probability estimates. In many applications, however, a probability estimation tree (PET) with accurate probability estimates is desirable. Some researchers ascribe the poor probability estimates of decision trees to the decision tree learning algorithms. To our observation, however, the representation also plays an important role. Indeed, the representation of decision trees is fully expressive theoretically, but it is often impractical to learn such a representation with accurate probability estimates from limited training data. In this paper, we extend decision trees to represent a joint distribution and conditional independence, called conditional independence trees (CITrees), which is a more suitable model for PETs. We propose a novel algorithm for learning CITrees, and our experiments show that the CITree algorithm outperforms C4.5 and naive Bayes significantly in classification accuracy.

The driving force behind the evolution of computing from automating automation [3] to automating learning [8] can be attributed to Machine Learning algorithms. By being able to generalize from examples, today's computers have the ability to perform autonomously and take critical decisions in almost any given task. In this report, we briefly discuss some of the foundations of what makes Machine Learning so powerful. Within the realm of supervised learning, we explore one of the most widely used classification technique known as Decision Tree Learning. We develop and analyze the ID3 algorithm, in particular we demonstrate how concepts such as Shannon's Entropy and Information Gain enables this form of learning to yield such powerful results. Furthermore, we introduce avenues through which the ID3 algorithm can be improved such as Gain ratio and the Random Forest Approach [4].

Accurate probability estimation generated by learning models is desirable in some practical applications, such as medical diagnosis. In this paper, we empirically study traditional decision-tree learning models and their variants in terms of probability estimation, measured by Conditional Log Likelihood (CLL). Furthermore, we also compare decision tree learning with other kinds of representative learning: naïve Bayes, Naïve Bayes Tree, Bayesian Network, K-Nearest Neighbors and Support Vector Machine with respect to probability estimation. From our experiments, we have several interesting observations. First, among various decision-tree learning models, C4.4 is the best in yielding precise probability estimation measured by CLL, although its performance is not good in terms of other evaluation criteria, such as accuracy and ranking. We provide an explanation for this and reveal the nature of CLL. Second, compared with other popular models, C4.4 achieves the best CLL. Finally, CLL does not dominate another wellestablished relevant measurement AUC (the Area Under the Curve of Receiver Operating Characteristics), which suggests that different decision-tree learning models should be used for different objectives. Our experiments are conducted on the basis of 36 UCI sample sets that cover a wide range of domains and data characteristics. We run all the models within a machine learning platform -Weka.

In data mining, large differences in prior class probabilities known as the class imbalance problem have been reported to hinder the performance of classifiers such as decision trees. Dealing with imbalanced and cost-sensitive data has been recognized as one of the 10 most challenging problems in data mining research. In decision trees learning, many measures are based on the concept of Shannon's entropy. A major characteristic of the entropies is that they take their maximal value when the distribution of the modalities of the class variable is uniform. To deal with the class imbalance problem, we proposed an off-centered entropy which takes its maximum value for a distribution fixed by the user. This distribution can be the a priori distribution of the class variable modalities or a distribution taking into account the costs of misclassification. Others authors have proposed an asymmetric entropy. In this paper we present the concepts of the three entropies and compare their effectiveness on 20 imbalanced data sets. All our experiments are founded on the C4.5 decision trees algorithm, in which only the function of entropy is modified. The results are promising and show the interest of off-centered entropies to deal with the problem of class imbalance.

Classification decision tree algorithms are used extensively for data mining in many domains such as retail target marketing, fraud detection, etc. Highly parallel algorithms for constructing classification decision trees are desirable for dealing with large data sets in reasonable amount of time. Algorithms for building classification decision trees have a natural concurrency, but are difficult to parallelize due to the inherent dynamic nature of the computation. In this paper, we present parallel formulations of classification decision tree learning algorithm based on induction. We describe two basic parallel formulations. One is based on Synchronous Tree Construction Approach and the other is based on Partitioned Tree Construction Approach. We discuss the advantages and disadvantages of using these methods and propose a hybrid method that employs the good features of these methods. We also provide the analysis of the cost of computation and communication of the proposed hybrid method. Moreover, experimental results on an IBM SP-2 demonstrate excellent speedups and scalability.

In this paper, we present "K-Means+ID3," a method to cascade k-Means clustering and the ID3 decision tree learning methods for classifying anomalous and normal activities in a computer network, an active electronic circuit, and a mechanical massbeam system. The k-Means clustering method first partitions the training instances into k clusters using Euclidean distance similarity. On each cluster, representing a density region of normal or anomaly instances, we build an ID3 decision tree. The decision tree on each cluster refines the decision boundaries by learning the subgroups within the cluster. To obtain a final decision on classification, the decisions of the k-Means and ID3 methods are combined using two rules: 1) the Nearest-neighbor rule and 2) the Nearestconsensus rule. We perform experiments on three data sets: 1) Network Anomaly Data (NAD), 2) Duffing Equation Data (DED), and 3) Mechanical System Data (MSD), which contain measurements from three distinct application domains of computer networks, an electronic circuit implementing a forced Duffing Equation, and a mechanical system, respectively. Results show that the detection accuracy of the K-Means+ID3 method is as high as 96.24 percent at a false-positive-rate of 0.03 percent on NAD; the total accuracy is as high as 80.01 percent on MSD and 79.9 percent on DED.

CRIME is one of the major problems encountered in any society and universities together with other higher institutions of learning are not exceptions. Thus, there is an urgent need for security agents and agencies to battle and eradicate crime. The Directorate of Students and Services Development (DSSD) are responsible for investigating and detecting criminals of any crime committed within the Redeemer’s University. DSSD faces major challenges when it comes to detecting the real perpetrators of several crimes. An improvement in their strategy can produce positive results and high success rates, which is the basic objective of this project. Several methods have been applied to solve similar problems in the literature but none was tailored to solving the problem in Redeemer’s University and other universities. This work therefore applied classification rule mining method to develop a system for detecting crimes in universities. Past data for both crimes and criminals were collected fr...

The required learning time and curse of dimensionality restrict the applicability of Reinforcement Learning(RL) on real robots. Difficulty in inclusion of initial knowledge and understanding the learned rules must be added to the mentioned problems. In this paper we address automatic state abstraction and creation of hierarchies in RL agent's mind, as two major approaches for reducing the number of learning trials, simplifying inclusion of prior knowledge, and making the learned rules more abstract and understandable. We formalize automatic state abstraction and hierarchy creation as an optimization problem and derive a new algorithm that adapts decision tree learning techniques to state abstraction. The proof of performance is supported by strong evidences from simulation results in nondeterministic environments. Simulation results show encouraging enhancements in the required number of learning trials, agent's performance, size of the learned trees, and computation time of the algorithm.

This paper describes a hybrid methodology that integrates genetic algorithms and decision tree learning in order to evolve useful subsets of discriminatory features for recognizing complex visual concepts. A genetic algorithm (GA) is used to search the space of all possible subsets of a large set of candidate discrimination features. Candidate feature subsets are evaluated by using C4.5, a decision-tree learning algorithm, to produce a decision tree based on the given features using a limited amount of training data. The classification performance of the resulting decision tree on unseen testing data is used as the fitness of the underlying feature subset. Experimental results are presented to show how increasing the amount of learning significantly improves feature set evolution for difficult visual recognition problems involving satellite and facial image data. In addition, we also report on the extent to which other more subtle aspects of the Baldwin effect are exhibited by the system.

This paper introduces a new methodology for discovering patterns in foodborne disease outbreaks using a data-driven approach. Specifically, our approach uses three data mining methods, namely attribute selection, decision tree learning, and association rule discovery, to extract previously unknown and meaningful patterns that connect specific types of foodborne diseases outbreaks with associated foods vehicles and consumption locations. We use this approach to study the four most common disease causing etiologies in the Center for Disease Control (CDC) database of foodborne disease outbreaks in the year 2006, namely Salmonella enteritidis, Salmonella typhimurium, Escherichia coli, and Norovirus. The analysis reveals numerous patterns of how each of these outbreaks types relates to specific foods and locations. The discovery of such patterns in foodborne disease outbreak data can be very useful is determination and implementation of suitable intervention techniques. In particular, if the associations between different food types and consumption locations are known then custom intervention techniques including specific training methods can be designed to train individuals in hygienic food handling, preparation, and consumption practices.

There had been an enormous increase in the crime in the recent past. Crimes are a common social problem affecting the quality of life and the economic growth of a society. With the increase of crimes, law enforcement agencies are continuing to demand advanced systems and new approaches to improve crime analytics and better protect their communities. Decision tree (J48) applied in the context of law enforcement and intelligence analysis holds the promise of alleviating such problem. Data mining is a way to extract knowledge out of usually large data sets; in other words it is an approach to discover hidden relationships among data by using artificial intelligence methods of which decision tree (J48) is inclusive. The wide range of machine learning applications has made it an important field of research. Criminology is one of the most important fields for applying data mining. Criminology is a process that aims to identify crime characteristics. This study considered the development o...

Diabetes Melitus (DM) merupakan penyakit kronis yang banyak diderita oleh penduduk Indonesia, penyakit ini disebabkan karena kadar glukosa dalam darah di atas nilai normal. Penyakit ini termasuk penyakit yang rumit dan mematikan, oleh karena itu dibutuhkan perawatan medis yang kontinu agar resiko terjadinya komplikasi bisa dihindari. Dari tahun ke tahun jumlah pengidap penyakit diabetes semakin bertambah. Merujuk dari sumber data Federasi Diabetes Internasional, pengidap penyakit diabetes sebanyak 10 juta jiwa pada tahun 2015 di Indonesia, ditahun 2040 diprediksi jumlah warga Indonesia yang terjangkit penyakit diabates mengalami meningkatan sebesar 16.2 juta jiwa penduduk Indonesia. Oleh sebab itu deteksi penyakit diabetes sangat diperlukan guna merendahkan komplikasi penyakit diabetes di waktu yang akan datang. Algortima C4.5 adalah salah satu metode yang bisa dimanfaatkan untuk meramal penyakit diabetes. C4.5 Decision Tree adalah algoritma yang bisa menghasilkan keputusan dengan c...

In this paper we present a novel, customizable IE paradigm that takes advantage of predicate-argument structures. We also introduce a new way of automatically identifying predicate argument structures, which is central to our IE paradigm. It is based on: (1) an extended set of features; and (2) inductive decision tree learning. The experimental results prove our claim that accurate predicate-argument structures enable high quality IE results.

We address the problem of designing a machine learning tool for the automatic diagnosis of Parkinson's disease that is capable of providing an explanation of its behavior in terms that are easy to understand by clinicians. For this purpose, we consider as machine learning tool the decision tree, because it provides the decision criteria in terms of both the features which are actually useful for the purpose among the available ones and how their values are used to reach the final decision, thus favouring its acceptance by clinicians. On the other side, we consider the random forest and the support vector machine, which are among the top performing machine learning tool that have been proposed in the literature, but whose decision criteria are hidden into their internal structures. We have evaluated the effectiveness of different approaches on a public dataset, and the results show that the system based on the decision tree achieves comparable or better results that state-of-the-art solutions, being the only one able to provide a plain description of the decision criteria it adopts in terms of the observed features and their values.

Abstract: In this work, decision tree learning algorithms and fuzzy inferencing systems are applied for galaxy morphology classification. In particular, the CART, the C4.5, the Random Forest and fuzzy logic algorithms are studied and reliable classifiers are developed to distinguish ...

Properly addressing the discretization process of continous valued features is an important problem during decision tree learning. This paper describes four multi-interval discretization methods for induction of decision trees used in dynamic fashion. We compare two known discretization methods to two new methods proposed in this paper based on a histogram based method and a neural net based method (LVQ). We compare them according to accuracy of the resulting decision tree and to compactness of the tree. For our comparison we use three data bases, IRIS domain, satellite domain and OHS domain (ovariel hyper stimulation).

In this work, decision tree learning algorithms and fuzzy inferencing systems are applied for galaxy morphology classification. In particular, the CART, the C4.5, the Random Forest and fuzzy logic algorithms are studied and reliable classifiers are developed to distinguish between spiral galaxies, elliptical galaxies or star/unknown galactic objects. Morphology information for the training and testing datasets is obtained from the

Artykuł ten jest czwartym z serii artykułów poświęconych przeglądowej prezentacji poszczególnych metod sztucznej inteligencji (AI) prezentowanych jako wyspy archipelagu, co zostało skomentowane i wyjaśnione w pierwszym artykule tego cyklu. Warto dodać, że wszystkie wcześniejsze artykuły z tej serii są także dostępne w serwisie Academia.edu. W prezentowanym tu artykule przedstawione są programy sztucznej inteligencji grające z ludźmi w gry strategiczne (warcaby, szachy, Go), dalej omówiona jest obszerna dziedzina komputerowego przetwarzania języka naturalnego, a na końcu omówione są uczące się drzewa decyzyjne.

Classification based on decision trees is one of the important problems in data mining and has applications in many fields. In recent years, database systems have become highly distributed, and distributed system paradigms, such as federated and peer-topeer databases, are being adopted. In this paper, we consider the problem of inducing decision trees in a large distributed network of genomic databases. Our work is motivated by the existence of distributed databases in healthcare and in bioinformatics, and by the emergence of systems which automatically analyze these databases, and by the expectancy that these databases will soon contain large amounts of highly dimensional genomic data. Current decision tree algorithms require high communication bandwidth when executed on such data, which are large-scale distributed systems. We present an algorithm that sharply reduces the communication overhead by sending just a fraction of the statistical data. A fraction which is nevertheless sufficient to derive the exact same decision tree learned by a sequential learner on all the data in the network. Extensive experiments using standard synthetic SNP data show that the algorithm utilizes the high dependency among attributes, typical to genomic data, to reduce communication overhead by up to 99 percent. Scalability tests show that the algorithm scales well with both the size of the data set, the dimensionality of the data, and the size of the distributed system.

This paper presents the results of an explorative study on predicting aspects of playing behavior for the major commercial title Tomb Raider: Underworld (TRU). Various supervised learning algorithms are trained on a large-scale set of in-game player behavior data, to predict when a player will stop playing the TRU game and, if the player completes the game, how long will it take to do so. Results reveal that linear regression models and other non-linear classification techniques perform well on the tasks and that decision tree learning induces small yet well-performing and informative trees. Moderate performance is achieved from the prediction models, which indicates the complexity of predicting player behavior based on a constrained set of gameplay metrics and the noise existent in the dataset examined, a generic problem in large-scale data collection from millions of remote clients.

This article presents a new classification algorithm, called CLEF, which induces a -machine by constructing its own features based on the training data. The features can be viewed as defining subsets of the instance space, and they allow CLEF to create useful non-linear functions over the input variables. The algorithm is guaranteed to find a classifier that separates the training instances, if such a separation is possible. We compare CLEF empirically to several other classification algorithms, including a well-known decision tree inducer, an artificial neural network inducer, and a support vector machine inducer. Our results show that the CLEF-induced -machines and support vector machines have similar accuracy on the suite tested, and that both are significantly more accurate than the other classifiers produced. We argue that the classifiers produced by CLEF are easy to interpret, and hence may be preferred over support vector machines in certain circumstances.

In this paper we present an approach to using decision tree learning as a basis for improving search performance for real-world constraint satisfaction problems. The problem used to evaluate the approach is a large real-world configuration benchmark from the automotive industry.

In this article we show that there is a strong connection between decision tree learning and local pattern mining. This connection allows us to solve the computationally hard problem of finding optimal decision trees in a wide range of applications by post-processing a set of patterns: we use local patterns to construct a global model. We exploit the connection between constraints in pattern mining and constraints in decision tree induction to develop a framework for categorizing decision tree mining constraints. This framework allows us to determine which model constraints can be pushed deeply into the pattern mining process, and allows us to improve the state-of-the-art of optimal decision tree induction.

There had been an enormous increase in the crime in the recent past. Crimes are a common social problem affecting the quality of life and the economic growth of a society. With the increase of crimes, law enforcement agencies are continuing to demand advanced systems and new approaches to improve crime analytics and better protect their communities. Decision tree (J48) applied in the context of law enforcement and intelligence analysis holds the promise of alleviating such problem. Data mining is a way to extract knowledge out of usually large data sets; in other words it is an approach to discover hidden relationships among data by using artificial intelligence methods of which decision tree (J48) is inclusive. The wide range of machine learning applications has made it an important field of research. Criminology is one of the most important fields for applying data mining. Criminology is a process that aims to identify crime characteristics. This study considered the development o...

Discretization is a critical component of data mining whereby continuous attributes of a data set are converted into discrete ones by creating intervals either before or during learning. There are many good reasons for preprocessing discretization, such as increased learning efficiency and classification accuracy, comprehensibility of data mining results, as well as the inherent limitation of a great majority of learning algorithms to handle only discrete data. Many preprocessing discretization techniques have been proposed to date, of which the Entropy-MDLP discretization has been accepted as by far the most effective in the context of both decision tree learning and rule induction algorithms. This paper presents a new discretization technique EDISC which utilizes the entropy-based principle but takes a class-tailored approach to discretization. The technique is applicable in general to any covering algorithm, including those that use the class-per-class rule induction methodology such as CN2 as well as those that use a seed example during the learning phase, such as the RULES family. Experimental evaluation has proved the efficiency and effectiveness of the technique as a preprocessing discretization procedure for CN2 as well as RULES-7, the latest algorithm among the RULES family of inductive learning algorithms.

We describe and evaluate an information-theoretic algorithm for datadriven induction of classification models based on a minimal subset of available features. The relationship between input (predictive) features and the target (classification) attribute is modeled by a tree-like structure termed an information network (IN). Unlike other decision-tree models, the information network uses the same input attribute across the nodes of a given layer (level). The input attributes are selected incrementally by the algorithm to maximize a global decrease in the conditional entropy of the target attribute. We are using the prepruning approach: when no attribute causes a statistically significant decrease in the entropy, the network construction is stopped. The algorithm is shown empirically to produce much more compact models than other methods of decision-tree learning, while preserving nearly the same level of classification accuracy.

Classification decision tree algorithms are used extensively for data mining in many domains such as retail target marketing, fraud detection, etc. Highly parallel algorithms for constructing classification decision trees are desirable for dealing with large data sets in reasonable amount of time. Algorithms for building classification decision trees have a natural concurrency, but are difficult to parallelize due to the inherent dynamic nature of the computation. In this paper, we present parallel formulations of classification decision tree learning algorithm based on induction. We describe two basic parallel formulations. One is based on Synchronous Tree Construction Approach and the other is based on Partitioned Tree Construction Approach. We discuss the advantages and disadvantages of using these methods and propose a hybrid method that employs the good features of these methods. We also provide the analysis of the cost of computation and communication of the proposed hybrid method. Moreover, experimental results on an IBM SP-2 demonstrate excellent speedups and scalability.

Accurate prediction of fault prone modules (a module is equivalent to a C function or a C+ + method) in software development process enables effective detection and identification of defects. Such prediction models are especially beneficial for large-scale systems, where verification experts need to focus their attention and resources to problem areas in the system under development. This paper presents a novel methodology for predicting fault prone modules, based on random forests. Random forests are an extension of decision tree learning. Instead of generating one decision tree, this methodology generates hundreds or even thousands of trees using subsets of the training data. Classification decision is obtained by voting. We applied random forests in five case studies based on NASA data sets. The prediction accuracy of the proposed methodology is generally higher than that achieved by logistic regression, discriminant analysis and the algorithms in two machine learning software packages, WEKA [I. H. Witten et al. (1999)] and See5. The difference in the performance of the proposed methodology over other methods is statistically significant. Further, the classification accuracy of random forests is more significant over other methods in larger data sets.

Accurate prediction of fault prone modules (a module is equivalent to a C function or a C+ + method) in software development process enables effective detection and identification of defects. Such prediction models are especially beneficial for large-scale systems, where verification experts need to focus their attention and resources to problem areas in the system under development. This paper presents a novel methodology for predicting fault prone modules, based on random forests. Random forests are an extension of decision tree learning. Instead of generating one decision tree, this methodology generates hundreds or even thousands of trees using subsets of the training data. Classification decision is obtained by voting. We applied random forests in five case studies based on NASA data sets. The prediction accuracy of the proposed methodology is generally higher than that achieved by logistic regression, discriminant analysis and the algorithms in two machine learning software packages, WEKA [I. H. Witten et al. (1999)] and See5. The difference in the performance of the proposed methodology over other methods is statistically significant. Further, the classification accuracy of random forests is more significant over other methods in larger data sets.

Most data mining algorithms assume static behavior of the incoming data. In the real world, the situation is different and most continuously collected data streams are generated by dynamic processes, which may change over time, in some cases even drastically. The change in the underlying concept, also known as concept drift, causes the data mining model generated from past examples to become less accurate and relevant for classifying the current data. Most online learning algorithms deal with concept drift by generating a new model every time a concept drift is detected. On one hand, this solution ensures accurate and relevant models at all times, thus implying an increase in the classification accuracy. On the other hand, this approach suffers from a major drawback, which is the high computational cost of generating new models. The problem is getting worse when a concept drift is detected more frequently and, hence, a compromise in terms of computational effort and accuracy is needed. This work describes a series of incremental algorithms that are shown empirically to produce more accurate classification models than the batch algorithms in the presence of a concept drift while being computationally cheaper than existing incremental methods. The proposed incremental algorithms are based on an advanced decision-tree learning methodology called "info-fuzzy network" (IFN), which is capable to induce compact and accurate classification models. The algorithms are evaluated on real-world streams of traffic and intrusion detection data.

In inductive databases, there is no conceptual difference between data and the models describing the data: both can be stored and queried using some query language. The approach that adheres most strictly to this philosophy is probably the one proposed by : in this approach, models are stored in relational tables and queried using standard SQL. The approach has been described in detail for association rule discovery. In this work, we study how decision tree induction can be integrated in this approach. We propose a representation format for decision trees similar to the format proposed earlier for association rules, and queryable using standard SQL; and we present a prototype system in which part of the needed functionality is implemented. In particular, we have developed an exhaustive tree learning algorithm able to answer a wide range of constrained queries.

This paper proposes an approach to sentence-level paraphrase identification by text canon-icalization. The source sentence pairs are first converted into surface text that approxi-mates canonical forms. A decision tree learn-ing module which employs simple lexical match- ...

The naive Bayesian classifier provides a simple and effective approach to classifier learning, but its attribute independence assumption is often violated in the real world. A number of approaches have sought to alleviate this problem. A Bayesian tree learning algorithm builds a decision tree, and generates a local naive Bayesian classifier at each leaf. The tests leading to a leaf can alleviate attribute inter-dependencies for the local naive Bayesian classifier. However, Bayesian tree learning still suffers from the small disjunct problem of tree learning. While inferred Bayesian trees demonstrate low average prediction error rates, there is reason to believe that error rates will be higher for those leaves with few training examples. This paper proposes the application of lazy learning techniques to Bayesian tree induction and presents the resulting lazy Bayesian rule learning algorithm, called Lbr. This algorithm can be justified by a variant of Bayes theorem which supports a weaker conditional attribute independence assumption than is required by naive Bayes. For each test example, it builds a most appropriate rule with a local naive Bayesian classifier as its consequent. It is demonstrated that the computational requirements of Lbr are reasonable in a wide cross-section of natural domains. Experiments with these domains show that, on average, this new algorithm obtains lower error rates significantly more often than the reverse in comparison to a naive Bayesian classifier, C4.5, a Bayesian tree learning algorithm, a constructive Bayesian classifier that eliminates attributes and constructs new attributes using Cartesian products of existing nominal attributes, and a lazy decision tree learning algorithm. It also outperforms, although the result is not statistically significant, a selective naive Bayesian classifier.

In today's world, there is an improved advance in hardware technology which increases the capability to store and record personal data about consumers and individuals. Data mining extracts knowledge to support a variety of areas as marketing, medical diagnosis, weather forecasting, national security etc successfully. Still there is a challenge to extract certain kinds of data without violating the data owners' privacy. As data mining becomes more enveloping, such privacy concerns are increasing. This gives birth to a new category of data mining method called privacy preserving data mining algorithm (PPDM). The aim of this algorithm is to protect the easily affected information in data from the large amount of data set. The privacy preservation of data set can be expressed in the form of decision tree. This paper proposes a privacy preservation based on data set complement algorithms which store the information of the real dataset. So that the private data can be safe from the unauthorized party, if some portion of the data can be lost, then we can recreate the original data set from the unrealized dataset and the perturbed data set.

Knowledge Discovery and Data Mining (KDD) is a multidisciplinary field of study that focuses on methodologies for extracting useful knowledge from data. During the latest Covid-19 pandemic, there was a significant uptick in online-based learning (e-learning) operations as every educational institution moved its operations to digital channels. To increase the quality of education in this new normal, it is necessary to determine the key factors in students’ performance. The main objective of this study is to exploit the regulating factors of education via digital platforms during the covid-19 pandemic by extracting knowledge and a set of rules by using the Decision Tree (j48) classifier.  In this study, we developed a conceptual framework using four datasets, each with a different set of attributes and instances, collected from “X-University” and Microsoft teams. ‘Final term’ and ‘Mid-term’ examinations acted as the root node for all four datasets. The findings of this study would ben...

While data miners can learn defect predictors from static code features, the performance improvement in such detectors is curiously static. One explanation for this ceiling effect is that static code features have limited information content. If so, then it should be useful to give our learners more knowledge about the problems they are processing. Data miners build predictors using a performance criteria P (e.g. accuracy), then assess them using another criteria Q (e.g. probability of detection). Typically, P = Q, so learners are blind to their purpose. Therefore, it is hardly surprising that they cannot find ways to better their performance. WHICH is a new data miner with P ≈ Q; i.e. a similar evaluation criteria is applied during training and testing. After matching P to the goals of a particular business application, we found that commonly used data miners perform no better than simple manual methods; and that WHICH outperforms other methods, coming close to a theoretical upper bound in performance (50 and 75 percentile of 70.9% and 80%, respectively). That is, with knowledge of the business application, it is possible to build new data miners that overcome ceiling effects.

This paper describes a method for classification of remote sensing images integrating the importance of texture with the eficiency of the artificial neural networks. The classification process consists of applying Gabor filters followed by neural network classification. This classification is based on a multi-layer perceptron neural network with backpropagation algorythm. Some results of experiments with synthetic and remote sensing images are presented and discussed.

Decision trees are probably the most popular and commonly used classification model. They are recursively built following a top-down approach (from general concepts to particular examples) by repeated splits of the training dataset. When this dataset contains numerical attributes, binary splits are usually performed by choosing the threshold value which minimizes the impurity measure used as splitting criterion (e.g. C4.5 gain ratio criterion or CART Gini's index). In this paper we propose the use of multi-way splits for continuous attributes in order to reduce the tree complexity without decreasing classification accuracy. This can be done by intertwining a hierarchical clustering algorithm with the usual greedy decision tree learning.