A high utility itemset mining algorithm based on subsume index

Abstract

High utility itemset mining addresses the limitations of frequent itemset mining by introducing measures of interestingness that reflect the significance of an itemset beyond its frequency of occurrence. Among such algorithms, level-wise candidate generation-and-test approaches suffer from the drawbacks of having an immense candidate pool and requiring several database scans. Meanwhile, methods based on pattern growth tend to consume large amounts of memory to store conditional trees. We propose an efficient algorithm, called Index High Utility Itemsets Mine (IHUI-Mine), for application to high utility itemsets. The subsume index, which has been employed to mine frequent itemsets, is extended in IHUI-Mine to the discovery of high utility itemsets. In addition to the enumeration and search strategies inherited from the subsume index, we introduce a new property to specifically accelerate the computation of transaction-weighted utilization for high utility itemsets. Furthermore, given that bitmaps are used for database representation, the real utility of candidates can be verified from the recorded transactions rather than by resorting to the entire database. The computational complexity of IHUI-Mine is analyzed, and tests conducted on publicly available synthetic and real datasets further demonstrate that the proposed algorithm outperforms existing state-of-the-art algorithms.

Appendices

Appendix 1: Proof of Property 4.1

For $\forall X\subseteq subsume(i)$, we have the following two cases.

(1) The case that $X=\emptyset$. Thus, $i\cup X= i$, so we have $TWU(i)=TWU(i\cup X)$.

(2) The case that $X\ne \emptyset$. Suppose $X= b_1b_2\ldots b_k$, where $b_j\in {{\mathbf {I}}}(1\le j\le k)$. Since $X\subseteq subsume(i)$ for $\forall b_j\in X$, we have $b_j\in subsume(i)$. According to Definition 4.1, $g(i)\subseteq g(b_j)$. Thus, $g(i\cup X)=g(i)\cap g(X)=g(i)\cap g(b_1)\cap g(b_2)\cap \ldots \cap g(b_k)=g(i)$, and we have $TWU(i)=TWU(i\cup X)$.

Appendix 2: Proof of Property 4.2

We prove the property by contradiction. Suppose there exists $j\notin subsume(i)$, such that $i\cup j$ is an HTWUI. Since the size of the set returned by the function g(X) (described in Sect. 4.1) decreases as the size of the itemset X increases, and $i\subseteq i\cup j$, we have $g(i\cup j)\subseteq g(i)$, i.e., $TWU(i\cup j)\le TWU(i)$. Assume $TWU(i\cup j)=TWU(i)$, this means $g(i)=g(i\cup j)=g(i)\cap g(j)$, which leads to $g(i)\subseteq g(j)$. According to Definition 4.1, we can easily find that $j\in subsume(i)$, which contradicts the hypothesis. Thus, $TWU(i\cup j)<TWU(i) = min\_util$.

Appendix 3: Proof of Property 4.3

For the left side of equation:

$$\begin{aligned} {\textit{TWU}}(X\cup i) =\sum _{t\in (g(X\cup i))}{} \textit{TU}(t)=\sum _{t\in (g(X)\cap g(i))}{} \textit{TU}(t) \end{aligned}$$

For the right side of equation:

$$\begin{aligned}&{\textit{TWU}}(X)-\sum _{t\in (g(X)-g(i))}{} \textit{TU}(t)\\&\quad =\sum _{t\in g(X)}{} \textit{TU}(t)-\sum _{t\in (g(X)-g(i))}{} \textit{TU}(t)\\&\quad =\sum _{t\in (g(X)-(g(X)-g(i)))}{} \textit{TU}(t) =\sum _{t\in (g(X)\cap g(i))}{} \textit{TU}(t) \end{aligned}$$

So we have

$$\begin{aligned} {\textit{TWU}}(X\cup i)= {\textit{TWU}}(X)-\sum _{t\in (g(X)-g(i))}{} \textit{TU}(t). \end{aligned}$$