Abstract
In this paper, a new variant of Bagging named DepenBag is proposed. This algorithm obtains bootstrap samples at first. Then, it employs a causal discoverer to induce from each sample a dependency model expressed as a Directed Acyclic Graph (DAG). The attributes without connections to the class attribute in all the DAGs are then removed. Finally, a component learner is trained from each of the resulted samples to constitute the ensemble. Empirical study shows that DepenBag is effective in building ensembles of nearest neighbor classifiers.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Aha, D.W.: Lazy learning: special issue editorial. Artificial Intelligence Review 11, 7–10 (1997)
Alkoot, F.M., Kittler, J.: Moderating k-NN classifiers. Pattern Analysis & Applications 5, 326–332 (2002)
Blake, C., Keogh, E., Merz, C.J.: UCI repository of machine learning databases, Department of Information and Computer Science, University of California, Irvine, CA (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html
Breiman, L.: Bagging predictors. Machine Learning 24, 123–140 (1996)
Chickering, M.: Learning equivalence classes of bayesian networks structures. In: Proceedings of the 12th International Conference on Uncertainty in Artificial Intelligence, Portland, OR, pp. 150–157 (1996)
Cooper, G.F., Herskovits, E.: A bayesian method for the induction of probabilistic networks from data. Machine Learning 9, 309–347 (1992)
Dai, H., Li, G.: An improved approach for the discovery of causal models via MML. In: Proceedings of the 6th Pacific-Asia Conference on Knowledge Discovery and Data Mining, Taipei, Taiwan, pp. 304–315 (2002)
Dasarathy, B.V.: Nearest Neighbor Norms: NN Pattern Classification Techniques. IEEE Computer Society Press, Los Alamitos (1991)
Dietterich, T.G.: Ensemble learning. In: Arbib, M.A. (ed.) The Handbook of Brain Theory and Neural Networks, 2nd edn. MIT Press, Cambridge (2002)
Efron, B., Tibshirani, R.: An Introduction to the Bootstrap. Chapman & Hall, New York (1993)
Hall, L.O., Bowyer, K.W., Banfield, R.E., Bhadoria, D., Eschrich, S.: Comparing pure parallel ensemble creation techniques against bagging. In: Proceedings of the 3rd IEEE International Conference on Data Mining, Melbourne, FL, pp. 533–536 (2003)
Heckerman, D., Geiger, D., Chickering, D.M.: Learning bayesian networks: the combination of knowledge and statistical data. Machine Learning 20, 197–243 (1995)
Kononenko, I.: Estimating attributes: analysis and extensions of relief. In: Proceedings of the 7th European Conference on Machine Learning, Catania, Italy, pp. 171–182 (1994)
Krogh, A., Vedelsby, J.: Neural network ensembles, cross validation, and active learning. In: Tesauro, G., Touretzky, D.S., Leen, T.K. (eds.) Advances in Neural Information Processing Systems, vol. 7, pp. 231–238. MIT Press, Cambridge (1995)
Kuncheva, L.I.: Diversity in multiple classifier systems: special issue editorial. Information Fusion 6, 3–4 (2005)
Kuncheva, L.I., Whitaker, C.J.: Measures of diversity in classifier ensembles. Machine Learning 51, 181–207 (2003)
Latinne, P., Debeir, O., Decaestecker, C.: Mixing bagging and multiple feature subsets to improve classification accuracy of decision tree combination. In: Proceedings of the 10th Belgian-Dutch Conference on Machine Learning, Tilburg, The Netherlands (2000)
Liu, H., Setiono, R.: Chi2: feature selection and discretization of numeric attributes. In: Proceedings of the 7th IEEE International Conference on Tools with Artificial Intelligence, Washington, DC, pp. 388–391 (1995)
Murphy, K.: The bayes net toolbox for matlab. Computing Science and Statistics 33, 331–351 (2001)
Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Mateo (1988)
Robinson, R.W.: Counting unlabelled acyclic digraphs. In: Proceedings of the 5th Australian Conference on Combinatorial Mathematics, Melbourne, Australia, pp. 28–43 (1976)
Zhou, Z.-H., Yu, Y.: Adapt bagging to nearest neighbor classifiers. Journal of Computer Science and Technology 20, 48–54 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jiang, Y., Ling, JJ., Li, G., Dai, H., Zhou, ZH. (2005). Dependency Bagging. In: Ślęzak, D., Wang, G., Szczuka, M., Düntsch, I., Yao, Y. (eds) Rough Sets, Fuzzy Sets, Data Mining, and Granular Computing. RSFDGrC 2005. Lecture Notes in Computer Science(), vol 3641. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11548669_51
Download citation
DOI: https://doi.org/10.1007/11548669_51
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28653-0
Online ISBN: 978-3-540-31825-5
eBook Packages: Computer ScienceComputer Science (R0)