WEKA Analysis

Waikato

Environment

for

Knowledge

Weka is a popular suite of machine learning software written in Java, developed at theUniversity of Waikato, New Zealand. Weka is free software available under the GNU General Public License. The Weka (pronounced Way-Kuh) workbench[1] contains a collection of visualization tools and algorithms for data analysis andpredictive modeling, together with graphical user interfaces for easy access to this functionality. The original non-Java version of Weka was a TCL/TK front-end to (mostly third-party) modeling algorithms implemented in other programming languages, plus datapreprocessing utilities in C, and a Makefile-based system for running machine learning experiments. This original version was primarily designed as a tool for analyzing data from agricultural domains,[2][3] but the more recent fully Java-based version (Weka 3), for which development started in 1997, is now used in many different application areas, in particular for educational purposes and research. Advantages of Weka include: Free availability under the GNU General Public License Portability, since it is fully implemented in the Java programming language and thus runs on almost any modern computing platform A comprehensive collection of data preprocessing and modeling techniques Ease of use due to its graphical user interfaces Weka supports several standard data mining tasks, more specifically, data preprocessing, clustering, classification, regression, visualization, and feature selection. All of Weka's techniques are predicated on the assumption that the data is available as a single flat file or relation, where each data point is described by a fixed number of attributes (normally, numeric or nominal attributes, but some other attribute types are also supported). Weka provides access to SQL databases using Java Database Connectivity and can process the result returned by a database query. It is not capable of multi-relational data mining, but there is separate software for converting a collection of linked database tables into a single table that is suitable for processing using Weka.[4] Another important area that is currently not covered by the algorithms included in the Weka distribution is sequence modeling. Weka's main user interface is the Explorer, but essentially the same functionality can be accessed through the component-based Knowledge Flow interface and from the command line. There is also the Experimenter, which allows the systematic comparison of the predictive performance of Weka's machine learning algorithms on a collection of datasets.

The Explorer interface features several panels providing access to the main components of the workbench: The Preprocess panel has facilities for importing data from a database, a CSV file, etc., and for preprocessing this data using a so-called filtering algorithm. These filters can be used to transform the data (e.g., turning numeric attributes into discrete ones) and make it possible to delete instances and attributes according to specific criteria. The Classify panel enables the user to apply classification and regression algorithms (indiscriminately called classifiers in Weka) to the resulting dataset, to estimate the accuracy of the resulting predictive model, and to visualize erroneous predictions, ROC curves, etc., or the model itself (if the model is amenable to visualization like, e.g., a decision tree). The Associate panel provides access to association rule learners that attempt to identify all important interrelationships between attributes in the data. The Cluster panel gives access to the clustering techniques in Weka, e.g., the simple kmeans algorithm. There is also an implementation of the expectation maximization algorithm for learning a mixture of normal distributions The Select attributes panel provides algorithms for identifying the most predictive attributes in a dataset. The Visualize panel shows a scatter plot matrix, where individual scatter plots can be selected and enlarged, and analyzed further using various selection operators.

Open source ML algorithms

pre-processing classifiers clustering association rule

Installation
Download software from http://www.cs.waikato.ac.nz/ml/weka/ If you are interested in modifying/extending weka there is a developer version that includes the source code

Set the weka environment variable for java setenv WEKAHOME /usr/local/weka/weka-3-0-2 setenv CLASSPATH $WEKAHOME/weka.jar:$CLASSPATH

Routines are implemented as classes and logically arranged in packages Comes with an extensive GUI interface Weka routines can be used stand alone via the command line

 Eg. java weka.classifiers.j48.J48 -t $WEKAHOME/data/iris.arff

Data format
Uses flat text files to describe the data Can work with a wide variety of data files including its own .arff format and C4.5 file formats Data can be imported from a file in various formats: ARFF, CSV, C4.5, binary

Data can also be read from a URL or from an SQL database (using JDBC)

ARFF file
Attribute Relationship File Format (ARFF) is the text format file used by Weka to store data in a database. This kind of file is structured as follows ("weather" relational database): @relation weather @attribute outlook {sunny, overcast, rainy} @attribute temperature real @attribute humidity real @attribute windy {TRUE, FALSE} @attribute play {yes, no} The ARFF file contains two sections: the header and the data section. The first line of the header defines the relation name. Then there is the list of the attributes (@attribute...). Each attribute is associated with a unique name and a type. The latter describes the kind of data contained in the variable and what values it can have. The variables types are: numeric, nominal, string and date. The class attribute is by default the last one of the list. In the header section there can also be some comment lines, identified with a '%' at the beginning, which can describe the database content or give the reader information about the author. After that there is the data itself (@data), each line stores the attribute of a single entry separated by a comma.

Sample ARRF file @relation heart-disease-simplified @attribute age numeric @attribute sex { female, male} @attribute chest_pain_type { typ_angina, asympt, non_anginal, atyp_angina} @attribute cholesterol numeric

@attribute exercise_induced_angina { no, yes} @attribute class { present, not_present} @data 63,male,typ_angina,233,no,not_present 67,male,asympt,286,yes,present 67,male,asympt,229,yes,present 38,female,non_anginal,?,no,not_present Explorer: Preprocessing Pre-processing tools in WEKA are called filters WEKA contains filters for: Discretization, normalization, resampling, attribute selection, transforming, combining attributes, etc

Explorer: building classifiers Classifiers in WEKA are models for predicting nominal or numeric quantities Implemented learning schemes include: Decision trees and lists, instance-based classifiers, support vector machines, multi-layer perceptrons, logistic regression, Bayes nets,

Meta-classifiers include: Bagging, boosting, stacking, error-correcting output codes, locally weighted learning,

Explorer: Clustering Example showing simple K-means on the Iris dataset

Algorithm for decision tree learning

Algorithm decisionTree(D, A, T) 1 if D contains only training examples of the same class Cj C then 2 make T a leaf node labeled with class cj 3 elseif A = then 4 make T a leaf node labeled with cj, which is the most frequent class in D 5 else // D contains examples belonging to a mixture of classes. We select a single 6 // attribute to partition D into subsets so that each subset is purer 7 po =impurityEval-l (D); 8 for each attribute Ai {A1, A2, ., Ak} do 9 pi = impurityEval-2(Ai,D) 10 end 11 Select Ag {A1, A2, ,Ak} that gives the biggest impurity reduction, computed using P0 - Pi;

12 13 14 15 16 17 18 19 20 21 22 23 24

if P0 - Pg < threshold then //Ag does not significantly reduce impurity P0 make T a leaf node labeled with ci, the most frequent class in D, else // Ag is able to reduce impurity P0 Make T a decision node on Ag; Let the possible values of Ag be V1, V2, ... , Vm. Partition D into m disjoint subsets D1, D2 ,... , Dm based on the m values of Ag. for each Dj in { D1, D2,..., Dm } do if Dj then create a branch (edge) node Tj for vj as a child node of T; decisionTree(Dj, A-{Ag},Tj) // Ag is removed end end end end