Accelerating the discovery of unsupervised-shapelets

Abstract

Over the past decade, time series clustering has become an increasingly important research topic in data mining community. Most existing methods for time series clustering rely on distances calculated from the entire raw data using the Euclidean distance or Dynamic Time Warping distance as the distance measure. However, the presence of significant noise, dropouts, or extraneous data can greatly limit the accuracy of clustering in this domain. Moreover, for most real world problems, we cannot expect objects from the same class to be equal in length. As a consequence, most work on time series clustering only considers the clustering of individual time series “behaviors,” e.g., individual heart beats or individual gait cycles, and contrives the time series in some way to make them all equal in length. However, automatically formatting the data in such a way is often a harder problem than the clustering itself. In this work, we show that by using only some local patterns and deliberately ignoring the rest of the data, we can mitigate the above problems and cluster time series of different lengths, e.g., cluster one heartbeat with multiple heartbeats. To achieve this, we exploit and extend a recently introduced concept in time series data mining called shapelets. Unlike existing work, our work demonstrates the unintuitive fact that shapelets can be learned from unlabeled time series. We show, with extensive empirical evaluation in diverse domains, that our method is more accurate than existing methods. Moreover, in addition to accurate clustering results, we show that our work also has the potential to give insight into the domains to which it is applied. While a brute-force algorithm to discover shapelets in an unsupervised way could be untenably slow, we introduce two novel optimization procedures to significantly speed up the unsupervised-shapelet discovery process and allow it to be cast as an anytime algorithm.

Appendix: Proof of Theorem 1

Theorem 1

$$\begin{aligned} \sqrt{{\sum }_{i=1}^{n-1} \left( {A_i -A_{i+1} } \right) ^{2}}-\sqrt{{\sum }_{i=1}^{n-1} \left( {B_i -B_{i+1} } \right) ^{2}}\le 2\sqrt{{\sum }_{i=1}^n \left( {A_i -B_i } \right) ^{2}} \end{aligned}$$

(10)

We will prove (10).

Proof

Without loss of generality, assume

$$\begin{aligned} \left( {\hbox {A}_1 -\hbox {B}_1 } \right) ^{2}\le \left( {\hbox {A}_\mathrm{n} -\hbox {B}_\mathrm{n} } \right) ^{2} \end{aligned}$$

(11)

(If not, reverse both $A$ and $B$. That is, replace $A$ with ($A_{n}, A_{n-1},{\ldots }, A_{1}$), and likewise for $B$. Note that (10) holds for $A$ and $B$ iff it holds for $A$ and $B$ reversed.)

Note that the inequality (10) is equivalent to

$$\begin{aligned} \hbox {d}\left( {\hbox {A},\hbox {A}^{\prime }} \right) -\hbox {d}(\hbox {B},\hbox {B}^{\prime })\le 2\hbox {d}(\hbox {A},\hbox {B}) \end{aligned}$$

(12)

where $d(X, Y)$ is Euclidean distance, i.e., $d\left( {X,Y} \right) =\sqrt{{\sum }_i^n (X_i -Y_i )^{2}}$, and

$$\begin{aligned} A^{{\prime }}= & {} \left( {A_1 ,A_1 ,A_2 ,A_3 ,\ldots A_{n-1} } \right) ,\\ B^{{\prime }}= & {} \left( {B_1 ,B_1 ,B_2 ,B_3 ,\ldots B_{n-1} } \right) . \end{aligned}$$

So, to prove (10), it suffices to prove (12). Here is a proof of (10).

The triangle inequality holds for Euclidean distance. Applying it twice shows

$$\begin{aligned} \hbox {d}\left( {\hbox {A},\hbox {A}^{\prime }} \right) \le \hbox {d}\left( {\hbox {A},\hbox {B}} \right) +\hbox {d}\left( {\hbox {B},\hbox {A}^{\prime }} \right) \le \hbox {d}\left( {\hbox {A},\hbox {B}} \right) +\hbox {d}\left( {\hbox {B},\hbox {B}^{{\prime }}} \right) +\hbox {d}(\hbox {B}^{ {\prime }},\hbox {A}^{\prime }). \end{aligned}$$

Rearranging gives

$$\begin{aligned} \hbox {d}\left( {\hbox {A},\hbox {A}^{\prime }} \right) -\hbox {d}\left( {\hbox {B},\hbox {B}^{\prime }} \right) \le \hbox {d}\left( {\hbox {A},\hbox {B}} \right) +\hbox {d}(\hbox {A}^{\prime },\hbox {B}^{\prime }) \end{aligned}$$

(13)

By inspection, $d(A^{{\prime }},B^{\prime })^{2}=d(A,B)^{2}-\left( {A_n -B_n } \right) ^{2}+(A_1 -B_1 )^{2}$, so

$$\begin{aligned} \hbox {d}\left( {\hbox {A}^{\prime },\hbox {B}^{\prime }} \right) =\sqrt{\hbox {d}(\hbox {A},\hbox {B})^{2}-\left( {\hbox {A}_\mathrm{n} -\hbox {B}_\mathrm{n} } \right) ^{2}+\left( {\hbox {A}_1 -\hbox {B}_1 } \right) ^{2}} \end{aligned}$$

(14)

Together with (11), this gives

$$\begin{aligned} \hbox {d}\left( {\hbox {A}^{\prime },\hbox {B}^{\prime }} \right) \le \sqrt{\hbox {d}\left( {\hbox {A},\hbox {B}} \right) ^{2}}=\hbox {d}(\hbox {A},\hbox {B}) \end{aligned}$$

(15)

Substituting this into (13) gives,

$$\begin{aligned} \hbox {d}\left( {\hbox {A},\hbox {A}^{\prime }} \right) -\hbox {d}\left( {\hbox {B},\hbox {B}^{\prime }} \right) \le \hbox {d}\left( {\hbox {A},\hbox {B}} \right) +\hbox {d}\left( {\hbox {A},\hbox {B}} \right) =2\hbox {d}(\hbox {A},\hbox {B}) \end{aligned}$$

(16)

proving (12) which proves (10). $\square$