IoT-Based Small Scale Anomaly Detection Using Dixon’s Q Test for e-Health Data

Abstract

Anomaly detection in the smart application domain can significantly improve the quality of data processing, especially when the size of a dataset is too small. Internet of Things (IoT) enables the development of numerous applications where sensor-data-aware anomalies can affect the decision making of the underlying system. In this paper, we propose a scheme: IoTDixon, which works on the Dixon’s Q test to identify point anomalies from a simulated normally distributed dataset. The proposed technique involves Q statistics, Kolmogorov–Smirnov test, and partitioning of a given dataset into a specific data packet. The proposed techniques use Q-test to detect point anomalies. We find that value 76.37 is statistically significant where $P=0.012<\alpha =0.05$, thus rejecting the null hypothesis for a test data packet. In other data packets, no such significance is observed; thus, no outlier is statistically detected. The proposed approach of IoTDixon can help to improve small-scale point anomaly detection for a small-size dataset as shown in the conducted experiments.

1. Introduction

IoT has brought enormous opportunities to allow the developments of a multitude of smart applications, namely health monitoring, smart city, smart transportation, and smart industry. Sensors are used in IoT-based ecosystems to generate data streams in regular intervals to provide real-time monitoring support to the owner of the given IoT system [1]. Such sensors may sometimes become faulty or generate erroneous data which must be detected at an early stage; otherwise, it can create serious troubles for decision making in the following stage of the applications [2,3].

The problem becomes very difficult when the size of the dataset is too small. This can happen due to an abrupt change of one of the data point values compared to the rest [4,5,6]. Thus, the detection of outliers in a small-size dataset is a trivial task. This situation may be exaggerated when applied for an IoT-based system which is resource constrained in nature.

Performing high-end data analytics in a resource-limited IoT device is not always feasible. With existing deep learning and machine learning algorithms, one can find outliers from a dataset [7,8,9]. However, due to lack of hardware resources such as processor and memory, an IoT device may face severe difficulty [10]. Further, depicting a point anomaly from a very small-size dataset makes the whole process questionable [11,12].

In this paper, we propose the IoTDixon scheme to detect point anomalies from very small-size dataset for the IoT-based environment. The IoTDixon uses both Dixon’s Q test and Kolmogorov–Smirnov test statistics to help find the anomaly. We consider a small-size dataset with 42 samples, which is assumed to be normally distributed. Such a dataset is further subdivided into six equal data packets, each with seven samples. We then perform the normality test of these data packets. Once this test is satisfied, the data packets are fed to Dixon’s method for finding a point anomaly. Finally, we obtain an equal number of anomalies as of the data packets which are further investigated against the P values. If the P value is less than a given confidence level $\alpha$, we infer it as statistically significant and declare the point as an outlier.

The key contributions of this work can be presented as follows:

• To propose IoTDixon scheme to detect point anomalies from small-size dataset;
• To integrate Dixon’s Q test as a key statistic for detection of outlier points from small-size data packets;
• To integrate Kolmogorov–Smirnov test statistic as the normality checker.

Novelty of the work: Our work is the first ever study that uses Dixon’s Q test and Kolmogorov–Smirnov test together to find small-size anomalies in IoT-based simulated scenarios. The study provides a scheme to divide a small dataset into further smaller data packets of constant size. We prefer to use the insertion sort to arrange the data points in ascending order for each of the data packets due to its faster response time. The presented IoTDixon algorithm has linear time complexity, thus making it appropriate for IoT-based devices.

The rest of the paper is presented as follows: Section 2 presents the formulation and derivation of Dixon’s Q test. Section 3 presents the IoTDixon methodology. Section 4 provides results. Section 5 concludes the paper.

2. Dixon’s Q Test

Dixon’s Q test can be used to detect an anomaly from a dataset as follows [13,14,15]: We assume that a dataset contains n samples each denoted by $x_i$. Such samples must be arranged in ascending order as follows: $x_1\le x_2\le x_3\le x_4\cdots \le x_n$. We can define the statistic as Equation (1).

$$r_{j,i-1}=\frac{(x_n-x_{n-j})}{(x_n-x_i)}$$

(1)

The j on r is denoted as the number of anomalies which the data analyst suspects at the higher end of the given dataset. The i represents the number of anomalies that are suspected to be deposited at the lower end of the dataset.

The r values define six ways to perform different analytics based on the cumulative and density distribution functions, such as $r_{10},r_{11},r_{12},r_{20},r_{21},r_{22}$ as shown below Equations (2)–(7) for $n\le 30$ under the one-tail distribution. The r values are specified for following range of samples in the dataset $r_{10}:3\ge n\le 7,r_{11}:8\ge n\le 10,$$r_{21}:11\ge n\le 13,r_{22}:14\ge n\le 30$. However, it is slightly changed for a two-sided scenario as follows $r_{10}:3\ge n\le 10,r_{11}:8\ge n\le 10,$ and $r_{21}:11\ge n\le 13$.

$$r_{10}=\frac{(x_2-x_1)}{(x_n-x_1)}OR\frac{(x_n-x_{n-1})}{(x_n-x_1)}$$

(2)

$$r_{11}=\frac{(x_2-x_1)}{(x_{n-1}-x_1)}OR\frac{(x_n-x_{n-1})}{(x_n-x_2)}$$

(3)

$$r_{12}=\frac{(x_2-x_1)}{(x_{n-2}-x_1)}OR\frac{(x_n-x_{n-1})}{(x_n-x_3)}$$

(4)

$$r_{20}=\frac{(x_3-x_1)}{(x_n-x_1)}OR\frac{(x_n-x_{n-2})}{(x_n-x_1)}$$

(5)

$$r_{21}=\frac{(x_3-x_1)}{(x_{n-1}-x_1)}OR\frac{(x_n-x_{n-2})}{(x_n-x_2)}$$

(6)

$$r_{22}=\frac{(x_3-x_1)}{(x_{n-2}-x_1)}OR\frac{(x_n-x_{n-2})}{(x_n-x_3)}$$

(7)

2.1. Probability Density of r

The Dixon’s ratio r follows Equation (1). However, joint probability density for $x_i,x_n,$ and $x_{n-j}$ can be obtained from Equation (8). We can use a combinatorial normalization factor along with the density functions multiplied by the integration of possible values over the three variables excluding three points which are being used in the calculation. We can express the formulation based on the three observations such as $i-1$, $n-j-i-1$, and $j-1$ which are below $x_j$ and within the range from $x_i$ to $x_{n-j}$ and from $x_{n-j}$ to $x_n$, respectively. L and M are two variables, as shown in Equations (9) and (10).

$$P(x_i,x_{n-j},x_n)=\frac{n!}{(i-1)!(n-j-i-1)!(j-1)!}\ast L\ast M$$

(8)

where,

$$L={[{\int }_{-\infty }^{x_i}\varphi \left(t\right)dt]}^{i-1}{[{\int }_{x_i}^{n-j}\varphi \left(t\right)dt]}^{n-j-i-1}$$

(9)

and

$$M={[{\int }_{x_{n-j}}^{x_n}\varphi \left(t\right)dt]}^{j-1}\varphi \left(x_i\right)\varphi \left(x_{n-j}\right)\varphi \left(x_n\right)$$

(10)

We can use $\varphi \left(t\right)={\left(2\pi \right)}^{-\frac{1}{2}}exp[-\frac{x^2}{2}]$ as the density function of the given standard normal distribution. Equation (1) is obtained when $j=i=1$, and we normally use r instead of $t_{10}$ to avoid the ambiguity.

2.2. Jacobian Probability Density of r

The three variables $x_i,x_{n-j}$, and $x_n$ can be expressed as $\{x,v,r\}$, where v denotes the Jacobian transformation. Now, the variables can be rearranged as follows: $x=x_n$, $v=x_n-x_i$,a dn $r=\frac{(x_n-x_{n-j})}{v}$. Now, to find the probability density of r on the Jacobian, we can integrate $-\infty <x<\infty$ and $0\le v<\infty$. Equation (11) shows the Jacobian evolved probability distribution of r. $\widehat{L}$ and $\widehat{M}$ present the variables as given in Equations (12) and (13), respectively.

$$P\left(r\right)=\frac{n!}{(i-1)!(n-j-i-1)!(j-1)!}\ast \widehat{L}\ast \widehat{M}$$

(11)

where,

$$\widehat{L}={\int }_{-\infty }^{\infty }{\int }_0^{\infty }{[{\int }_{-\infty }^{x-v}\varphi \left(t\right)dt]}^{i-1}{[{\int }_{x-v}^{x-rv}\varphi \left(t\right)dt]}^{n-j-i-1}$$

(12)

and

$$\widehat{M}={[{\int }_{x-rv}^x\varphi \left(t\right)dt]}^{j-1}\varphi (x-v)\varphi (x-rv)\varphi \left(x\right)vdvdx$$

(13)

2.2.1. Derivation of $r_{10}$

We can derive all the r with various i and j. We can find $r_{10}$ when $j=i=1$ as shown in Equation (14) and part calculations in Equations (15) and (16).

$$P\left(r_{10}\right)=\frac{n!}{(n-3)!}\ast \widehat{L}\ast \widehat{M}$$

(14)

where,

$$\widehat{L}={\int }_{-\infty }^{\infty }{\int }_0^{\infty }{[{\int }_{x-v}^{x-rv}\varphi \left(t\right)dt]}^{n-3}$$

(15)

and

$$\widehat{M}=\varphi (x-v)\varphi (x-rv)\varphi \left(x\right)vdvdx$$

(16)

2.2.2. Derivation of $r_{11}$

We can find $r_{11}$ when $j=1$ and $i=2$ as shown in Equation (17) and part calculations in Equations (18) and (19)

$$P\left(r_{11}\right)=\frac{n!}{(n-4)!}\ast \widehat{L}\ast \widehat{M}$$

(17)

where,

$$\widehat{L}={\int }_{-\infty }^{\infty }{\int }_0^{\infty }[{\int }_{-\infty }^{x-v}\varphi \left(t\right)dt]{[{\int }_{x-v}^{x-rv}\varphi \left(t\right)dt]}^{n-4}$$

(18)

and

$$\widehat{M}=\varphi (x-v)\varphi (x-rv)\varphi \left(x\right)vdvdx$$

(19)

2.2.3. Derivation of $r_{12}$

We can find $r_{12}$ when $j=1$ and $i=3$ as shown in Equation (20) and part calculations in Equations (21) and (22).

$$P\left(r_{12}\right)=\frac{n!}{2!(n-5)!}\ast \widehat{L}\ast \widehat{M}$$

(20)

where,

$$\widehat{L}={\int }_{-\infty }^{\infty }{\int }_0^{\infty }{[{\int }_{-\infty }^{x-v}\varphi \left(t\right)dt]}^2{[{\int }_{x-v}^{x-rv}\varphi \left(t\right)dt]}^{n-5}$$

(21)

and

$$\widehat{M}=\varphi (x-v)\varphi (x-rv)\varphi \left(x\right)vdvdx$$

(22)

2.2.4. Derivation of $r_{20}$

We can find $r_{20}$ when $j=2$ and $i=1$ as shown in Equation (23) and part calculations in Equations (24) and (25).

$$P\left(r_{20}\right)=\frac{n!}{(n-4)!}\ast \widehat{L}\ast \widehat{M}$$

(23)

where,

$$\widehat{L}={\int }_{-\infty }^{\infty }{\int }_0^{\infty }{[{\int }_{x-v}^{x-rv}\varphi \left(t\right)dt]}^{n-4}$$

(24)

and

$$\widehat{M}=[{\int }_{x-rv}^x\varphi \left(t\right)dt]\varphi (x-v)\varphi (x-rv)\varphi \left(x\right)vdvdx$$

(25)

2.2.5. Derivation of $r_{21}$

We can find $r_{21}$ when $j=2$ and $i=2$ as shown in Equation (26) and part calculations in Equations (27) and (28).

$$P\left(r_{21}\right)=\frac{n!}{(n-5)!}\ast \widehat{L}\ast \widehat{M}$$

(26)

where,

$$\widehat{L}={\int }_{-\infty }^{\infty }{\int }_0^{\infty }[{\int }_{-\infty }^{x-v}\varphi \left(t\right)dt]{[{\int }_{x-v}^{x-rv}\varphi \left(t\right)dt]}^{n-5}$$

(27)

and

$$\widehat{M}=[{\int }_{x-rv}^x\varphi \left(t\right)dt]\varphi (x-v)\varphi (x-rv)\varphi \left(x\right)vdvdx$$

(28)

2.2.6. Derivation of $r_{22}$

We can find $r_{22}$ when $j=2$ and $i=3$ as shown in Equation (29) and part calculations in Equations (30) and (31).

$$P\left(r_{22}\right)=\frac{n!}{2!(n-6)!}\ast \widehat{L}\ast \widehat{M}$$

(29)

where,

$$\widehat{L}={\int }_{-\infty }^{\infty }{\int }_0^{\infty }{[{\int }_{-\infty }^{x-v}\varphi \left(t\right)dt]}^2{[{\int }_{x-v}^{x-rv}\varphi \left(t\right)dt]}^{n-6}$$

(30)

and

$$\widehat{M}=[{\int }_{x-rv}^x\varphi \left(t\right)dt]\varphi (x-v)\varphi (x-rv)\varphi \left(x\right)vdvdx$$

(31)

2.3. Cumulative Distribution of R

We can rewrite Equations (8)–(10) in terms of cumulative normal distribution $\Phi \left(x\right)$ as shown in Equation (32)

$$P\left(r\right)=\frac{n!}{(i-1)!(n-j-i-1)!(j-1)!}\ast \overline{L}\ast \overline{M}$$

(32)

where,

$$\overline{L}={\int }_{-\infty }^{\infty }{\int }_0^{\infty }{[\Phi (x-v)]}^{i-1}$$

(33)

and

$$\overline{M}={[\Phi (x-rv)-\Phi (x-v)]}^{n-j-i-1}{[\Phi \left(x\right)-\Phi (x-rv)]}^{j-1}\varphi (x-v)\varphi (x-rv)\varphi \left(x\right)vdvdx$$

(34)

The cumulative distribution function $CDF\left(R\right)$ can be expressed as Equation (35), where $0\le r\le 1$ and $CDF\left(0\right)=0$ and $CDF\left(1\right)=1$. With a given probability $\alpha$, we can find the roots as the critical values from Equation (36) with monotonic increment from 0 to 1.

$$CDF\left(R\right)={\int }_0^{R}P\left(r\right)dr$$

(35)

$$CDF\left(R\right)=(1-\alpha )$$

(36)

2.4. Probability Density of r

Equation (32) can be rewritten on three $\varphi$ terms with $x^2,v^2$ as shown in Equation (37) and part variables as shown in Equations (38) and (39). Herein, N represents the normalization factor with constant term ${[2\pi ]}^{-\frac{3}{2}}$, and $J(x,r,v)$ depicts the terms with $\Phi$.

$$P\left(r\right)=\frac{n!}{(i-1)!(n-j-1-1)!(j-1)!}{[2\pi ]}^{-\frac{3}{2}}{\int }_{-\infty }^{\infty }{e}^{-\frac{3x^2}{2}}{\int }_0^{\infty }{e}^{-\frac{(1+r^2)v^2}{2}}\ast C\ast D$$

(37)

$P\left(r\right)=N{\int }_{-\infty }\infty e^{-\frac{3x^2}{2}}{\int }_0^{\infty }{e}^{-\frac{(1+r^2)v^2}{2}}J(x,v,r)e^{xv(1+r)}vdvdx$, where

$$C={[\Phi (x-v)]}^{i-1}{[\Phi (x-rv)-\Phi (x-v)]}^{n-j-i-1}{[\Phi \left(x\right)-\Phi (x-rv)]}^{j-1}$$

(38)

$$D=e^{xv(1+r)}vdvdx$$

(39)

We can modify variable $t^2=\frac{(1+r^2)v^2}{2}$ and $u^2=\frac{3x^2}{2}$ to change the integration into the Gauss–Hermite quadrants as shown in Equation (40), where $x\left(u\right)=u\sqrt{\frac{2}{3}}$ and $v(t,r)=t\sqrt{\frac{2}{(1+r^2)}}$.

$$P\left(r\right)=N\sqrt{\frac{2}{3}}\sqrt{\frac{2}{(1+r^2)}}{\int }_{-\infty }^{\infty }{e}^{-u^2}{\int }_0^{\infty }{e}^{-t^2}J(x\left(u\right),v(t,r),r)e^{\frac{2ut(1+r)}{\sqrt{3(1+r^2)}}}dtdu$$

(40)

Thus, the quadrature rules can be formulated as follows: Equations (41) and (42), where $w_l$, $t_l$ represent weights and abscissas of the $n_{hh}$ point belonging to half-range Hermite quadrature.

$${\int }_0^{\infty }{e}^{-t^2}f\left(t\right)dt\approx \sum _{l=1}^{n_{hh}}{w}_{l}f\left(t_l\right)$$

(41)

$${\int }_{-\infty }^{\infty }{e}^{-t^2}g\left(u\right)du\approx \sum _{k=1}^{n_{fh}}{w}_{k}g\left(u_k\right)$$

(42)

Now, the $CDF\left(R\right)$ can be computed as Equation (43), where $w_m,y_m$ refer to the Gauss–Legendre weights and abscissas on the given range of $[-1,1]$ with $y=\frac{2r}{R}-1$ as the variable transformation on the range $[0,R]$ to $[-1,1]$.

$$CDF\left(R\right)\approx \frac{R}{2}\sum _{m=1}^{n_{g}l}{w}_{m}P\left(\frac{R_{y_m}}{2}\right)$$

(43)

2.5. Range Test

The data analyst must use the Dixon’s Q test once on any dataset. The Q test is performed by Equation (44). The gap denotes an absolute difference $|x-y|$, where $x,y$ are real numbers. The metric properties on such absolute difference should hold the following inequalities, (i) $|x-y|\ge 0$, (ii) $|x-y|=0$ when $x=y$, (iii) $|x-y|=|y-x|$, and (iv) $|x-z|\le |x-y|+|y-z|$.

$$Q=\frac{Gap}{Range}$$

(44)

The Q is then tested against the $Q_{critical}$, i.e., table-wise reference value based on a given confidence interval and provided number of observations. A rejection is provided when $Q>Q_{critical}$; otherwise, an acceptance is made.

3. System Design

The IoTDixon flow chart is shown in Figure 1. The flow chart shows the process behind the proposed methodology where a small test data stream can be fed for anomaly detection. Initially, the small test dataset was divided into m data packets each with seven samples. We then performed each of the data packets for a test of normal distribution by using the Kolmogorov–Smirnov test. Upon notification as the normal distribution, the respective data packet was then processed for the Dixon’s test. Finally, all the anomalies from each of the data packets were collected by the data analysts for further investigations.

3.1. IoTDixon Algorithm

We present the IoTDixon algorithm for detection of single anomaly from a packet of samples taken from an small test data stream $X=\{x_1,x_2,x_3,\cdots ,x_n\}$. The IoTDixon technique works as follows: We assume that IoT-based health sensor data are being streamed on a regular interval to an edge device. We assume that small test data stream is divided into $m=\lceil \frac{len\left(X\right)}{\eta }\rceil$ number of packets, where $\eta$ is a given as the amount of samples in each data packet. We select the appropriate r statistic based on the sample size $\eta$. By default, we use $r_{10}$ as the Dixon statistic where a sample of data packet must lie within the range of $[3,7]$. However, one can change the range and r statistic according to the need to adjust with the packet size. The partitioning task required $\mathcal{O}\left(m\right)$ that depends on the $\eta$. We then perform the insertion sort on the $j^{th}$ generated data packet $x_{\eta j}$. In this study, we select $\eta =7$; however, as mentioned earlier, it can be changed to other values, though keeping the minimum limit to three samples. We select insertion sort due to its faster response time for small datasets typically less than 10. Thus, we expect the sorting time to be around $\mathcal{O}\left(1\right)$, i.e., negligible. Then, we perform the normality checking procedure by using the Kolmogorov–Smirnov test in $\mathcal{O}\left(\eta \right)$ time. Once the given data packet is statistically proven to be normally distributed, then the data packet is forwarded for the two-sided Dixon’s Q test. Finally, the outlier $o_{\eta j}$ from the sample data packet is returned. Thus, we can find the maximum m number of outliers as the total number of data packets is m. It is the duty of the data analyst to further investigate the obtained outliers to find the most effective anomaly. The overall time complexity of the proposed algorithm can be obtained in a linear time $\mathcal{O}(m+\eta )$. The IoTDixon Algorithm is shown in Algorithm 1.

Algorithm 1: IoTDixon Algorithm

Input: IoT-based small test data stream X where $1\le i\le n$ and select appropriate $r_{j,i-1}$ statistic for Dixon’s Q test depending on sample packet size $\eta$ Output: Possible anomalies Make packet $x_{\eta j}$ of each $\eta$ amount of small test data samples while(m=$\lceil \frac{len\left(X\right)}{\eta }\rceil >0$)do

3.2. Kolmogorov–Smirnov Algorithm

The Kolmogorov–Smirnov (KS) test is used to evaluate whether a random sample selected from a dataset is drawn from a fixed normal distribution function $F\left(x\right)$, i.e., one-sample test [16,17]. It can also be used to evaluate whether two datasets belong to the same fixed distribution, i.e., two-sample test. The KS test requires no a-priori knowledge about the distribution of samples under consideration. In this study, we use a one-sample KS test on the data packet $x_{\eta j}$. We find $D_{max}^{+}=\sqrt{\eta }max\{\frac{t}{\eta }-F\left(x_t\right)\},\forall t,1\le t\le \eta$, $D_{max}^{-}=\sqrt{\eta }max\{F\left(x_t\right)-\frac{t-1}{\eta }\},\forall t,1\le t\le \eta$, and $D_{max}=max\{D_{max}^{+},D_{max}^{-}\}$, where $D_{max}^{+}$ represents the maximum positive), $D_{max}^{-}$ refers to the maximum negative, and $D_{max}$ denotes the maximum absolute. The null hypothesis $H_0$ is expressed as the data packet $x_{\eta j}$ follows the normal distribution. The alternative hypothesis $H_1$ is that the data do not follow the normal distribution. The $H_0$ can be rejected when the $D_{max}>D_{c\alpha }$ at the confidence level $\alpha =0.05$. This infers that the data packet $x_{\eta j}$ is not normally distributed. Otherwise, we fail to reject the $H_0$ and infer that $x_{\eta j}$ is normally distributed. The Kolmogorov–Smirnov Algorithm is shown in Algorithm 2.

3.3. Dixon’s Q Algorithm

We present the Dixon’s Q algorithm where we provide the normally distributed small test data packet $x_{\eta j}$ which is ranked or ordered. A potential outlier sample $x_t$ can be tested as follows: $Q=r_{10}=\frac{\left|\right(x_t-x_{t+1}\left)\right|}{\left|\right(x_{max}-x_{min}\left)\right|}$. If $Q_{<}{Q}_{c\alpha }$, we fail to reject the null hypothesis $H_0$, which implies that the sample $x_t$ is not an outlier. Otherwise, we infer to accept the null hypothesis $H_0$, which implies that the sample $x_t$ is an outlier. In both cases, the null hypothesis $H_0$ can be stated as follows: there is no significant difference between the suspected data and the rest of $x_{\eta j}$; thus, it is not an outlier. Dixon’s Q two–sided algorithm is shown in Algorithm 3.

Algorithm 2: Kolmogorov–Smirnov Algorithm

Input: IoT data packet $x_{\eta j}$ Output: Kolmogorov–Smirnov estimate $D_{max}$ $D_{max}^{+}=\sqrt{\eta }max\{\frac{t}{\eta }-F\left(x_t\right)\},\forall t,1\le t\le \eta$ $D_{max}^{-}=\sqrt{\eta }max\{F\left(x_t\right)-\frac{t-1}{\eta }\},\forall t,1\le t\le \eta$ $D_{max}=max\{D_{max}^{+},D_{max}^{-}\}$ if ($D_{max}>D_{c\alpha =0.05}$) then

Algorithm 3: Dixon’s Q two-sided Algorithm

Input: Normally distributed small test data packet $x_{\eta j}$ Output: A point outlier Calculate Q statistic $Q=r_{10}=\frac{\left|\right(x_t-x_{t+1}\left)\right|}{\left|\right(x_{max}-x_{min}\left)\right|}$ if ($Q_{>}{Q}_{c\alpha }$) then

3.4. IoTDixon Dataset

We performed the study under the R distribution framework where we used the dixonTest package for performing the Q test. We created a dataset that has 42 samples with a mean of 72 and standard deviation of 2 to simulate the pulse rate per minute of a human being. The dataset is then partitioned into six equally sized subset of data packets having $\eta =7$ named $x1,x2,x3,x4,x5,x6$. Such partitioned subsets are then used for checking anomalies per packet level.

4. Results

We obtained the IoTDixon normal distribution and evaluate them against (i) histogram with density curve (top left), (ii) plot of data points (top right), (iii) box plot (bottom left), and (iv) QQ plot (bottom right) for each of the six data packets. Figure 2 shows the packet wise evaluation of normality for following (a) x1 dataset, (b) x2 dataset, (c) x3 dataset, (d) x4 dataset, (e) x5 dataset, and (f) x6 dataset. We also present the overall normality evaluation for the whole dataset comprising 42 samples as in Figure 3.

We perform KS-test on each of the data packets. The cumulative distribution plots for each of the six data packets are shown in Figure 4, where x1, x2, x3, x4, x5, and x6 datasets are considered separately. All the six data packets are considered as normally distributed for proceeding the Dixon’s Q test.

We perform the Dixon test on each of the 6 packets: x1, x2, x3, x4, x5, and x6. We find the Q statistic value for each of them as shown in

Table 1. IoTDixon aware anomaly detection.

	Q	P	POS	Anomaly
x1	0.477	0.1343	5	75.69
x2	0.300	0.533	5	67.56
x3	0.378	0.311	2	68.86
x4	0.365	0.344	6	75.58
x5	0.090	1	7	73.85
x6	0.665	0.012 ***	3	76.37 ***

. The probability of the Dixon test is shown as P. The position of selected anomaly from each data packet is mentioned under the POS column, and the corresponding value of anomaly point is shown under the anomaly column.

We find that value 76.37 is statistically significant where $P=0.012<\alpha =0.05$, thus rejecting the null hypothesis for x6 data packet. In other data packets, no such significance is observed; thus, no outlier is statistically detected.

The proposed work is performed for the first time to showcase the use of the Dixon’s Q-test for detecting point anomalies from a small-scale dataset. In this study, we used a health dataset as a case study to validate the applicability of the proposed methodology. This approach can be deployed at a resource-constrained IoT-edge device connected to a health sensor as a proof of concept for the purpose of detection of point anomalies from a small-scale dataset. IoT devices are less processing capacity aware; thus, such systems should be fed with a lightweight scheme with a minimal amount dataset at their vicinity. Doing so can certainly improve the existing scenario of large-scale dataset-aware anomaly detection schemes toward minimalistic processing consumption deployments. Thus, the proposed technique can support at the edge real-life implications where small numbers of samples are collected and mitigated for localized anomaly detection. This can further minimize the overhead of a high amount anomaly eradication procedure in the later phase of application.

5. Conclusions

This paper presents a novel IoTDixon methodology that can work on small-size data packets obtained from the given IoT dataset. As the Q test only provides a single anomaly from a small data packet, it can be useful for sensor data gathering wherein a few repetitions of averaging are performed. The proposed techniques uses Q-test to detect point anomalies. We find that value 76.37 is statistically significant where $P=0.012<\alpha =0.05$, thus rejecting the null hypothesis for a test data packet. The IoTDixon algorithm can be useful in real-life applications with a small fragment of data analytics scenario, for example gathering small number of health data by an IoT sensor and identifying any anomaly present therein. Thus, anomaly detection can be imposed at the IoT-edge devices to detect possible point anomalies from a small set of data instead of searching them from a very large dataset that can be difficult in terms of power and processing capacity utilization by the resource-constrained IoT devices.