Frequent Itemset Mining

Salah satu metode data mining yang cukup popular untuk mendapatkan hubungan antar sebuah item dengan item tertentu adalah metode aturan asosiasi dengan menggunakan algoritma APriori, metode ini cukup tepat untuk menghasilkan pola aturan hubungan antar jenis item yang terjual berdasarkan data penjualan. Nilai support pada frequent item dan confidence pada rules yang didapat bisa memberikan wawasan yang dapat ditindaklanjuti oleh pengelola minimarket, koperasi dan lain sebagainya. Pengkategorian jenis produk pada minimarket cukup banyak sementara jumlah keseluruhan transaksi dalam satu tahun juga sangat banyak, sedangkan jumlah jenis item terjual dalam suatu transaksi sangat sedikit, dengan demikian nilai threshold tidak bisa terlampau tinggi. Pada penelitian ini metode aturan asosiasi dilakukan per event atau periode tertentu yang berkaitan dengan hari besar umat Islam, didapatkan rule tertinggi yaitu makanan ringan => sembako dengan confidence sebesar 46% dan support sebesar 16% yang terjadi pada bulan Ramadhan.

We present a depth-first algorithm, PatriciaMine, that discovers all frequent itemsets in a dataset, for a given support threshold. The algorithm is main-memory based and employs a Patricia trie to represent the dataset, which is space efficient for both dense and sparse datasets, whereas alternative representations were adopted by previous algorithms for these two cases. A number of optimizations have been introduced in the implementation of the algorithm. The paper reports several experimental results on real and artificial datasets, which assess the effectiveness of the implementation and show the better performance attained by PatriciaMine with respect to other prominent algorithms.

Frequent pattern mining is one of the most important task for discovering useful meaningful patterns from large collection of data.Mining of association rules from frequent pattern from massive collection of data is of interest for many industries which can provide guidance in decision making processes such as cross marketing, market basket analysis, promotion assortment etc. The techniques of discovering association rule from data have traditionally focused on identifying relationship between items predicting some aspect of human behavior, usually buying behavior. In this paper ,the study includes three classical frequent pattern mining methods that are Apriori, Eclat, FP growth and discusses some issues related with these algorithms.

During the recent years, a number of efficient and scalable frequent itemset mining algorithms for big data analytics have been proposed by many researchers. Initially, MapReduce-based frequent itemset mining algorithms on Hadoop cluster were proposed. Although, Hadoop has been developed as a cluster computing system for handling and processing big data, but the performance of Hadoop does not meet the expectation for the iterative algorithms of data mining, due to its high I/O, and writing and then reading intermediate results in the disk. Consequently, Spark has been developed as another cluster computing infrastructure which is much faster than Hadoop due to its in-memory computation. It is highly suitable for iterative algorithms and supports batch, interactive, iterative, and stream processing of data. Many frequent itemset mining algorithms have been re-designed on the Spark, and most of them are Apriori-based. All these Spark-based Apriori algorithms use Hash Tree as the underlying data structure. This paper investigates the efficiency of various data structures for the Spark-based Apriori. Although, the data structure perspective has been investigated previously, but for MapReduce-based Apriori, and it must be re-investigated in the distributed computing environment of Spark. The considered underlying data structures are Hash Tree, Trie, and Hash Table Trie. The experimental results on the benchmark datasets show that the performance of Spark-based Apriori with Trie and Hash Table Trie are almost similar but both perform many times better than Hash Tree in the distributed computing environment of Spark.

The efficiency of frequent itemset mining algorithms is determined mainly by three factors: the way candidates are generated, the data structure that is used and the implementation details. Most papers focus on the first factor, some describe the underlying data structures, but implementation details are almost always neglected. In this paper we show that the effect of implementation can be more important than the selection of the algorithm. Ideas that seem to be quite promising, may turn out to be ineffective if we ...

Mining association rules in relational databases is a significant computational task with lots of applications. A fundamental ingredient of this task is the discovery of sets of attributes (itemsets) whose frequency in the data exceeds some threshold value. In previous work [9] we have introduced an approach to this problem which begins by carrying out an efficient partial computation of the necessary totals, storing these interim results in a set-enumeration tree. This work demonstrated that making use of this structure can significantly reduce the ...

Data mining place viral aspect in many of the applications like market –basket analysis, fraud detection etc. In data mining association rule mining and frequent pattern mining, both are key feature of market-basket analysis. In a given large amount of transnational database where each record consists of items purchased by customer at store. One of the basic market basket analysis algorithm is an Apriori, which generate all candidates item-set frequent pattern. In this research paper we describe the improved candidate 1-itemsets generation and candidate 2-itemsets generation from traditional technique. This algorithm utilizes the dynamic programming approach to facilitate fast candidate itemset generation and searching. We have compared results with previous approach that optimize the database scans and eliminate duplicate candidate itemset generation. This technique helps research scholar.

—The paper presents a parallel implementation of a Dynamic Itemset Counting (DIC) algorithm for many-core systems, where DIC is a variation of the classical Apriori algorithm. We propose a bit-based internal layout for transactions and itemsets with the assumption that such a representation of the transaction database fits in main memory. This technique reduces the memory space for storing the transaction database and also simplifies support counting and candidate itemsets generation via logical bitwise operations. Implementation uses OpenMP technology and thread-level parallelism. Experimental evaluation on the platforms of Intel Xeon CPU and Intel Xeon Phi coprocessor with large synthetic database showed good performance and scalability of the proposed algorithm.

Anomaly detection is vital for automated data analysis, with specific applications spanning almost every domain. In this paper, we propose a hybrid supervised learning of anomaly detection using frequent itemset mining and random forest with an ensemble probabilistic voting method, which outperforms the alternative supervised learning methods through the commonly used measures for anomaly detection: accuracy, true positive rate (i.e. recall) and false positive rate. To justify our claim, a benchmark dataset is used to evaluate the efficiency of the proposed approach, where the results illustrate its benefits. K EYWORDS

Frequent item-set mining is a data analysis method which is used to find the relationship between the different items in the given database. Plenty of research work and progress has been made over the decades due to its wider applications. Recently, BitTableFI and Index-BitTableFI approaches have been applied for mining frequent item-sets and results are significant.  They use Bit Table as the base data structure and exploits the bit table both horizontally and vertically. However still needs simple and efficient approach for mining frequent itemsets from the given dataset. This paper introduces the Compact BitTable approach for mining frequent itemsets (CBT-fi)  which clusters(groups) the similar transaction into one and forms a compact bit-table structure which reduces the memory consumption as well as frequency of checking the itemsets in the redundant transaction. Finally we present result, which shows the proposed algorithm has better than the existing algorithms.

The development for data mining technology in healthcare is growing today as knowledge and data mining are a must for the medical sector. Healthcare organizations generate and gather large quantities of daily information. Use of IT allows for the automation of data mining and information that help to provide some interesting patterns which remove manual tasks and simple data extraction from electronic records, a process of electronic data transfer which secures medical records, saves lives and cuts the cost of medical care and enables early detection of infectious diseases. In this research paper an improved Apriori algorithm names enhanced parallel and distributed apriori (EPDA) is presented for the health care industry, based on the scalable environment known as Hadoop MapReduce. The main aim of the work proposed is to reduce the huge demands for resources and to reduce overhead communication when frequent data are extracted, through split-frequent data generated locally and the early removal of unusual data. The paper shows test results, whereby the EPDA performs in terms of the time and number of rules generated with a database of healthcare and different minimum support values.

Abstract: Data Mining is the process of extracting interesting and previously unknown patterns and correlations form huge data stored in databases. Association rule mining-a descriptive mining technique of data mining, is the process of discovering items or literals ...

In the phase of association rule mining, frequent itemset mining is an important move. Conventional methods to mine frequent itemsets in the big data epoch pose major obstacles as there is insufficient processing capacity and memory space. This work presents a novel frequent itemset mining technique that utilizes both matrix-based pruning method and Spark. The presented Spark based distributed (SBD) frequent itemset mining technique can efficiently reduce the number of campaign itemsets and also boost the performance of iterative computing. The experimental analysis is carried out using typical benchmark datasets to compare the presented SBD algorithm with the previous algorithm such as parallel FP-growth and distributed frequent itemset mining. The numerical results obtained from the experimental analysis indicate that SBD algorithm has better scalability and efficiency. Furthermore, a case study is accomplished to verify the efficacy of the presented SBD frequent itemset mining technique