Rule Extraction

Recent advances in functional brain imaging enable identification of active areas of a brain performing a certain function. Induction of logical formulas describing relations between brain areas and brain functions from functional brain images is a category of data mining. It is difficult, however, to apply conventional mining techniques to functional brain images due to several reasons, such as the difficulty of reducing images to symbolic data, possible existence of correlations between adjacent pixels in a image and the limited number of samples available from a single subject. Tsukimoto and Morita presented an algorithm for data mining from functional brain images and showed that the algorithm works well for artificial data. The algorithm consists of two steps. The first step is nonparametric regression. The second step is rule extraction from the linear formula obtained by the nonparametric regression. The authors have applied the algorithm to real f-MRI images. This paper repo...

... Concerning the research side, the issue of discrimination in credit, mortgage, insurance, labor market, education and other human activities has attracted much interest of re-searchers in economics and human sciences since late '50s, when a theory on the ... data miners want to ...

Artificial neural networks (ANNs) have been successfully applied to solve a variety of classification and function approximation problems. Although ANNs can generally predict better than decision trees for pattern classification problems, ANNs are often regarded as black boxes since ...

This paper presents a novel approach to knowledge extraction from large-scale datasets using a neural network when applied to the real-world problem of payment card fraud detection. Fraud is a serious and long term threat to a peaceful and democratic society. We present SOAR (Sparse Oracle-based Adaptive Rule) extraction, a practical approach to process large datasets and extract key generalizing

Advances in data mining have led to algorithms that produce accurate regression models for large and difficult to approximate data. Most of these use non-linear models to handle complex data-relationships in the input data. Problematic is their lack transparency, a requirement in many application domains where comprehensibility is of key importance. Rule-extraction algorithms have been proposed to solve this problem for classification by extracting comprehensible rulesets from the better performing, complex models. We present a new rule extraction algorithm for regression, based on active learning and the pedagogical approach to rule extraction. Empirical results show that improves on classical rule induction techniques, both in fidelity as well as accuracy.

Fuzzy neural networks (FNNs) provide a new approach for classification of multispectral data and to extract and optimize classification rules. Neural networks deal with issues on a numeric level, whereas fuzzy logic deals with them on a semantic or linguistic level. FNNs synthesize fuzzy logic and neural networks. Recently, there has been growing interest in the research community not only to understand how FNNs arrive at particular decisions but how to decode information stored in the form of connection strengths in the network. In this paper, we propose fuzzy neural network models for classification of pixels in multispectral images and to extract fuzzy classification rules. During the training phase, the connection strengths are updated. After training, classification rules are extracted by backtracking along the weighted paths through the FNN. The extracted rules are then optimized using a fuzzy associative memory (FAM) bank. The data mining system described above is useful in m...

Machine learning, a branch of artificial intelligence, is a scientific discipline that is
concerned with the design and development of algorithms that allow computers to
evolve behaviors based on empirical data, such as from sensor data or databases.
Artificial neural networks are composed of interconnecting artificial neurons
(programming constructs that mimic the properties of biological neurons). Artificial
neural networks may either be used to gain an understanding of biological neural
networks, or for solving artificial intelligence problems without necessarily creating a
model of a real biological system. In this paper, we made analysis with machine
learning algorithms and artificial neural networks classification from instances in data
set. Furthermore, machine learning algorithms and artificial neural networks with
constituted rules.

Neural network has many advantages such as non-linear module prediction,
automatic learning from training data, high adaptation et al. It also is successfully
applied to many applications in different domains, like control system, civil
engineering, weather prediction, financial analysis and recognition of image and
speech et al. However, due to the results made by the neural networks are difficult to
explain the decision process of neural networks is supposed as a black box. This is
because the learned knowledge is stored in the weights and structure of the network
and the decision process of neural networks is a numerical value operation. But the
explanation of decision making is important to some applications such like credit
approval application and medical diagnosing software. Oppositely, expert system
stores the knowledge in the knowledge base with symbolic logic rule form and
inferences with this logic rules. Hence, the decision making process of expert system
is close to human thought. By this, users also can verify the conclusion made by the
expert system, and then they are more likely to trust the results. But it is not easy to
build the knowledge base of the expert system. The acquiring knowledge process is
needed a frequently and lengthily interview between a knowledge engineer and
domain experts, and the process may takes a lone time. Moreover, many domain
experts could not explain their domain knowledge explicitly.
Therefore, rule extraction algorithm is becoming more and more important in
explaining the knowledge from the neural networks. Rule extraction algorithm is a
method to get rules from a neural network and these rules can show the learned
knowledge within the neural network. With this kind of algorithm, we can build a
hybrid intelligence system. The system can use neural network module to learn the
knowledge from training data automatically and then transforms the knowledge to
symbolic rules. Next, it uses these rules to construct the knowledge base of expert
system module. Therefore, the hybrid intelligence system can have the advantages of
both neural network and expert system.
In this thesis, a rule extraction algorithm is analyzed and designed to extract rules
from neural networks. The algorithm is a kind of decompositional rule extraction
IV
algorithm, and it means the algorithm extract rules from each neuron independently.
The notion of the algorithm is designing a quick equation to find the maximum and
minimum values of a set of inputs of a neuron, and then it can observe if all of these
inputs in the set will make the neuron activated (or all not). Next, the input set will be
decomposed into smaller subsets until all the inputs in these subsets will make the
neuron activated (or all not). Because the decomposition process is like a tree, the
algorithm is named as “Bound Decomposition Tree Algorithm” (BDT).
The algorithm is simple and efficient and it also can reduce the extracted rules.
The reduce option of the algorithm can make simple extracted rules, reduce the
number of these rules and improve the efficiency.
In the thesis, we use a linear equation and three familiar data used in machine
learning (MONK’s, the voting data, and mushroom data) to test the algorithm and
compare with other two algorithm, MofN and Garcez. Using the algorithm, we also
build a hybrid intelligence system which combines advantages of neural network and
expert system. In the last years, the hybrid intelligence system was applied into some
applications, such as “abnormal web access”, “the expert system of coronary artery
disease in Chinese and western medicine”, “employee recruitment with Biodata” and
“advance process control for PECVD”. These applications will also been discussed in
the thesis.

Link prediction is a task that in graph-based data models, as well as, in complex networks not only to predict edges that will appear in a near future but also to find missing edges. NELL is a never ending language learner system that has the ability to continuously learn to extract structured information from unstructured text (fetched from web pages) and map this information to a continuously growing knowledge base. NELL's knowledge base can be seen as a complex network, allowing us to apply graph mining techniques to extract new knowledge and enhance the system performance. In this paper we present Prophet, a link prediction component that can be connected to NELL allowing the it to infer new rules and misplaced connections among nodes, thus, helping the never-ending system to learn more and better each day. We also show that Prophet can extract new knowledge that cannot be obtained using traditional first order rule extraction procedures.

Classification is one of the data mining problems receiving enormous attention
in the database community. Although artificial neural networks (ANNs) have been
successfully applied in a wide range of machine learning applications, they are however
often regarded as black boxes, i.e., their predictions cannot be explained. To enhance the
explanation of ANNs, a novel algorithm to extract symbolic rules from ANNs has been
proposed in this paper. ANN methods have not been effectively utilized for data mining
tasks because how the classifications were made is not explicitly stated as symbolic rules
that are suitable for verification or interpretation by human experts. With the proposed
approach, concise symbolic rules with high accuracy, that are easily explainable, can be
extracted from the trained ANNs. Extracted rules are comparable with other methods in
terms of number of rules, average number of conditions for a rule, and the accuracy. The
effectiveness of the proposed approach is clearly demonstrated by the experimental results
on a set of benchmark data mining classification problems.

We derive a linear neural network model of the chemotaxis control circuit in the nematode Caenorhabditis elegans and demonstrate that this model is capable of producing nematodelike chemotaxis. By expanding the analytic solution for the network output in time-derivatives of the network input, we extract simple computational rules that reveal how the model network controls chemotaxis. Based on these rules we find that optimized linear networks typically control chemotaxis by computing the first time-derivative of the chemical concentration and modulating the body turning rate in response to this derivative. We argue that this is consistent with behavioral studies and a plausible mechanism for at least one component of chemotaxis in real nematodes.

The purpose of this project is to present a direct
method for rules extraction and the use of Neural
Network for classification. A direct method with
simplification (DMS) to extract rules from data
using a hash table is also developed. Apart from the
rules, DMS can also produce the corresponding
decision tree for each subset with predefined class
value. The merge of the paths of all the call values
can produce the decision tree of the data. Therefore,
DMS works as both direct and indirect method.
The advantages of DMS are the ability to produce
rules with a maximum number of conditions and
prevention of subtree overlapping. Moreover, an
incremental Backpropagation Neural Network
(IBNN) is created using the instances of data
grouping according to their predefined class.
Comparisons show that IBNN outperforms DMS.
Simulation results are provided.

Artificial neural network (ANN) may achieve successful classification value, but obtained results sometimes can be unintelligible and may not be interpreted. In literature, different rule extraction methods from trained ANN are applied to overcome this problem. This study presented a new method of extraction correct and intelligible rules from Trained ANN using variable neighbourhood search (VNS) metaheuristic method. Since the VNS method is independent from the ANN, it does not modify the results of ANN. Real time Wisconsin Breast Cancer (WBC) data was examined for determining the feasibility of the suggested method. The VNS method was used to obtain the best fitness values belong to input attributes, Id, which was maximized the fitness function Sr of output node r. The suggested method was the computational evaluated on investigated data sets and obtained results from the suggested approach indicated that had a potential to produce accurately and cored rules. The proposed method o...

Business rules have attained a major role in the development of software systems for businesses. They influence the business behavior based on the decisions enforced upon a wide range of aspects. As the business requirements are subjected to frequent amendments, new business rules have to be evolved to reinstate the previously formed ones. Business analysts count on the assistance from the IS developers for this accomplishment. As business rules are set and owned by the business, provisions must be enforced for business rule management by the business analysts directly. In order to get a clear picture of what the current policies and terms in the business are, business users utilize the document as a valuable tool. Unfortunately, documents are not updated as per the modifications in the source code. Moreover, as the software becomes larger, documents become increasingly large and hence difficult to understand and maintain. Thus they turn out to be a non-useful resource in such conditions. With a motive to resolve the above issues, several tools for extracting the rules from the business program code have been developed. In this paper, a robust architecture for an extraction engine to isolate rules from the base source code has been proposed. This suggested model involves slicing of code segments which results in easy identification of domain variables which in turn would result in extraction of business rules, validating them and exchanging them with the newly formulated business policies. This is a new research dimension which paves waves for efficient and quick business rule management strategy.