Article

On dataset complexity for case base maintenance

Authors:

Derek BridgeAuthors Info & Claims

ICCBR'11: Proceedings of the 19th international conference on Case-Based Reasoning Research and Development

Pages 47 - 61

https://doi.org/10.1007/978-3-642-23291-6_6

Published: 12 September 2011 Publication History

Abstract

We present what is, to the best of our knowledge, the first analysis that uses dataset complexity measures to evaluate case base editing algorithms. We select three different complexity measures and use them to evaluate eight case base editing algorithms. While we might expect the complexity of a case base to decrease, or stay the same, and the classification accuracy to increase, or stay the same, after maintenance, we find many counter-examples. In particular, we find that the RENN noise reduction algorithm may be over-simplifying class boundaries.

References

[1]

Brighton, H., Mellish, C.: On the consistency of information filters for lazy learning algorithms. In: Rauch, J., Zytkow, J. M. (eds.) PKDD 1999. LNCS (LNAI), vol. 1704, pp. 283-288. Springer, Heidelberg (1999)

Digital Library

[2]

Cummins, L.: Combining and Choosing Case Base Maintenance Algorithms. PhD thesis, Department of Computer Science, University College Cork, Ireland (forthcoming, 2011)

[3]

Cummins, L., Bridge, D.: Maintenance by a committee of experts: The MACE approach to case-base maintenance. In: McGinty, L., Wilson, D.C. (eds.) ICCBR 2009. LNCS, vol. 5650, pp. 120-134. Springer, Heidelberg (2009)

Digital Library

[4]

Delany, S. J.: The good, the bad and the incorrectly classified: Profiling cases for case-base editing. In: McGinty, L., Wilson, D.C. (eds.) ICCBR 2009. LNCS, vol. 5650, pp. 135-149. Springer, Heidelberg (2009)

Digital Library

[5]

Delany, S. J., Cunningham, P.: An analysis of case-based editing in a spam filtering system. In: Funk, P., González Calero, P. A. (eds.) ECCBR 2004. LNCS (LNAI), vol. 3155, pp. 128-141. Springer, Heidelberg (2004)

[6]

Doyle, D., Cunningham, P., Bridge, D., Rahman, Y.: Explanation oriented retrieval. In: Funk, P., González Calero, P. A. (eds.) ECCBR 2004. LNCS (LNAI), vol. 3155, pp. 157-168. Springer, Heidelberg (2004)

[7]

Fornells, A., Recio-García, J. A., Díaz-Agudo, B., Golobardes, E., Fornells, E.: Integration of a methodology for cluster-based retreval in jcolibri. In: McGinty, L., Wilson, D.C. (eds.) ICCBR 2009. LNCS, vol. 5650, pp. 418-433. Springer, Heidelberg (2009)

Digital Library

[8]

Frank, A., Asuncion, A.: UCI machine learning repository (2010)

[9]

Ho, T. K., Basu, M.: Measuring the complexity of classification problems. In: Procs. of the 15th Intl. Conference on Pattern Recognition, pp. 43-47 (2000)

[10]

Ho, T. K., Basu, M.: Complexity measures of supervised classification problems. IEEE Trans. on Pattern Analysis and Machine Intelligence 24(3), 289-300 (2002)

Digital Library

[11]

Maci`a, N., Bernadó-Mansilla, E., Orriols-Puig, A.: On the dimensions of data complexity through synthetic data sets. In: Procs. of the 11th Intl. Conference of the Catalan Association for Artificial Intelligence, pp. 244-252 (2008)

Digital Library

[12]

Massie, S., Craw, S., Wiratunga, N.: Complexity profiling for informed case-base editing. In: Roth-Berghofer, T. R., Göker, M. H., Güvenir, H. A. (eds.) ECCBR 2006. LNCS (LNAI), vol. 4106, pp. 325-339. Springer, Heidelberg (2006)

Digital Library

[13]

McKenna, E., Smyth, B.: Competence-guided case-base editing techniques. In: Blanzieri, E., Portinale, L. (eds.) EWCBR 2000. LNCS (LNAI), vol. 1898, pp. 186-197. Springer, Heidelberg (2000)

Digital Library

[14]

Orriols-Puig, A., Maci`a, N., Bernadó-Mansilla, E., Ho, T. K.: Documentation for the data complexity library in C++. Technical Report GRSI Report No. 2009001, Universitat Ramon Llull (2009)

[15]

Pranckeviciene, E., Ho, T. K., Somorjai, R.: Class separability in spaces reduced by feature selection. In: Procs. of the 18th Intl. Conference on Pattern Recognition, pp. 254-257 (2006)

Digital Library

[16]

Smyth, B., McKenna, E.: Modelling the competence of case-bases. In: Smyth, B., Cunningham, P. (eds.) EWCBR 1998. LNCS (LNAI), vol. 1488, pp. 208-220. Springer, Heidelberg (1998)

Digital Library

[17]

Tomek, I.: An experiment with the edited nearest-neighbor rule. IEEE Trans. on Systems, Man, and Cybernetics 6(6), 448-452 (1976)

Cited By

Barbon Junior SCeravolo PDamiani EMarques Tavares G(2020)Evaluating Trace Encoding Methods in Process MiningFrom Data to Models and Back10.1007/978-3-030-70650-0_11(174-189)Online publication date: 20-Oct-2020
https://dl.acm.org/doi/10.1007/978-3-030-70650-0_11
Nkisi-Orji IWiratunga NPalihawadana CRecio-García JCorsar D(2020)Clood CBR: Towards Microservices Oriented Case-Based ReasoningCase-Based Reasoning Research and Development10.1007/978-3-030-58342-2_9(129-143)Online publication date: 8-Jun-2020
https://dl.acm.org/doi/10.1007/978-3-030-58342-2_9
Khan MHayat HAwan I(2019)Hybrid case-base maintenance approach for modeling large scale case-based reasoning systemsHuman-centric Computing and Information Sciences10.1186/s13673-019-0171-z9:1(1-25)Online publication date: 1-Dec-2019
https://dl.acm.org/doi/10.1186/s13673-019-0171-z
Show More Cited By

Recommendations

Dataset Complexity and Gene Expression Based Cancer Classification
WILF '07: Proceedings of the 7th international workshop on Fuzzy Logic and Applications: Applications of Fuzzy Sets Theory

When applied to supervised classification problems, dataset complexity determines how difficult a given dataset to classify. Since complexity is a nontrivial issue, it is typically defined by a number of measures. In this paper, we explore complexity of ...
Dataset complexity in gene expression based cancer classification using ensembles of k-nearest neighbors

Objective: We explore the link between dataset complexity, determining how difficult a dataset is for classification, and classification performance defined by low-variance and low-biased bolstered resubstitution error made by k-nearest neighbor ...
Cyclomatic Complexity Density and Software Maintenance Productivity

A study of the relationship between the cyclomatic complexity metric (T. McCabe, 1976) and software maintenance productivity, given that a metric that measures complexity should prove to be a useful predictor of maintenance costs, is reported. The ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

ICCBR'11: Proceedings of the 19th international conference on Case-Based Reasoning Research and Development

September 2011

495 pages

ISBN:9783642232909

Sponsors

British Computer Society: BCS
Attensity: Attensity
Univ. of Greenwich: University of Greenwich

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 12 September 2011

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 29 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Barbon Junior SCeravolo PDamiani EMarques Tavares G(2020)Evaluating Trace Encoding Methods in Process MiningFrom Data to Models and Back10.1007/978-3-030-70650-0_11(174-189)Online publication date: 20-Oct-2020
https://dl.acm.org/doi/10.1007/978-3-030-70650-0_11
Nkisi-Orji IWiratunga NPalihawadana CRecio-García JCorsar D(2020)Clood CBR: Towards Microservices Oriented Case-Based ReasoningCase-Based Reasoning Research and Development10.1007/978-3-030-58342-2_9(129-143)Online publication date: 8-Jun-2020
https://dl.acm.org/doi/10.1007/978-3-030-58342-2_9
Khan MHayat HAwan I(2019)Hybrid case-base maintenance approach for modeling large scale case-based reasoning systemsHuman-centric Computing and Information Sciences10.1186/s13673-019-0171-z9:1(1-25)Online publication date: 1-Dec-2019
https://dl.acm.org/doi/10.1186/s13673-019-0171-z
Lorena AGarcia LLehmann JSouto MHo T(2019)How Complex Is Your Classification Problem?ACM Computing Surveys10.1145/334771152:5(1-34)Online publication date: 13-Sep-2019
https://dl.acm.org/doi/10.1145/3347711

View Options

View options

Figures

Tables

Media

View Table of Conten