CUPID - An Iterative Knowledge Discovery Framework

This paper describes the novel Knowledge Discovery system CUPID. Knowledge Discovery from Databases (KDD) is concerned with utilising techniques borrowed from fields such as machine learning (ML), statistics and databases to search for relationships and global patterns that may exist in large databases, but arehidden' among the vast amounts of data. The discovered knowledge can be helpful for building knowledge based systems and data analysis. The underlying principle behind CUPID is the use of a quantitative measure for theinterest' of a hypotheses. This measure provides a method of ranking competing hypotheses and thus allows the system to store the 'best' or 'most interesting' rules describing a database. CUPID is based on the ITRule algorithm of (Smyth & Goodman, 1992) and extends that algorithm with added functionality. CUPID provides four fundamental features. One, background knowledge in the form of attribute value generalisation hierarchies may be utilised. Two, prior domain knowledge which may be incorrect and incomplete may be provided by a domain expert. Three, knowledge may be re-used. Four, noise in the data set is handled in a well founded manner.