Image Anomalies: A Review and Synthesis of Detection Methods

Abstract

We review the broad variety of methods that have been proposed for anomaly detection in images. Most methods found in the literature have in mind a particular application. Yet we focus on a classification of the methods based on the structural assumption they make on the “normal” image, assumed to obey a “background model.” Five different structural assumptions emerge for the background model. Our analysis leads us to reformulate the best representative algorithms in each class by attaching to them an a-contrario detection that controls the number of false positives and thus deriving a uniform detection scheme for all. By combining the most general structural assumptions expressing the background’s normality with the proposed generic statistical detection tool, we end up proposing several generic algorithms that seem to generalize or reconcile most methods. We compare the six best representatives of our proposed classes of algorithms on anomalous images taken from classic papers on the subject, and on a synthetic database. Our conclusion hints that it is possible to perform automatic anomaly detection on a single image.

Appendix: Dual Formulation of Sparsity Models

Sparsity-based variational methods lack the direct interpretation enjoyed by other methods as to the proper definition of an anomaly. By reviewing the first simplest method of this kind proposed in [9], we shall see that its dual interpretation points to the detection of the worst anomaly. Let D a dictionary representing “normal” image patches. For a given patch p, the normal patch corresponding to p is $\hat{p}=D\hat{x}$ where

$$\begin{aligned} \hat{x} = \mathop {\mathrm{arg\,min}}\limits _x \left\{ \frac{1}{2}\Vert p-Dx\Vert _2^2 + \lambda \Vert x\Vert _1 \right\} . \end{aligned}$$

One can derive the following dual optimization problem: Let $z = p-Dx$,

$$\begin{aligned} \min _x \left\{ \frac{1}{2}\Vert z\Vert _2^2 + \lambda \Vert x\Vert _1 \right\} \text { s.t } z=p-Dx. \end{aligned}$$

The Lagrangian is in this case

$$\begin{aligned} \mathcal {L}(x,z,\eta )&= \frac{1}{2}\Vert z\Vert _2^2 + \lambda \Vert x\Vert _1 + \eta ^T(p - Dx - z)\\&= \eta ^Tp + \left( \frac{1}{2}\Vert z\Vert _2^2 - \eta ^Tz\right) + (\lambda \Vert x\Vert _1 - \eta ^TDx). \end{aligned}$$

The dual problem is then

$$\begin{aligned} \mathcal {G}(\eta )&= \inf _{x,z} \mathcal {L}(x,z,\eta )\\&= \eta ^T p + \inf _{z}\left( \frac{1}{2}\Vert z\Vert _2^2 - \eta ^Tz\right) + \inf _{x}(\lambda \Vert x\Vert _1 - \eta ^TDx). \end{aligned}$$

Consider first $\inf _{z}\left( \frac{1}{2}\Vert z\Vert _2^2 - \eta ^Tz\right)$: This part is differentiable in z so that

$$\begin{aligned} \partial _z \left( \frac{1}{2}\Vert z\Vert _2^2 - \eta ^Tz\right) = z - \eta ; \end{aligned}$$

therefore, the inf is achieved for $z=\eta$. The inf is in this case

$$\begin{aligned} \inf _{z}\left( \frac{1}{2}\Vert z\Vert _2^2 - \eta ^Tz\right) = -\frac{1}{2}\Vert \eta \Vert _2^2 \end{aligned}$$

As for $\inf _{x}(\lambda \Vert x\Vert _1 - \eta ^TDx)$: This part is not differentiable (because not smooth); nevertheless, the subgradient exists. Let v such that $\Vert x\Vert _1 = v^Tx$ (for all i $v_i \in {-1, 1}$). The subgradient of $\Vert .\Vert _1$ gives v.

$$\begin{aligned} \partial _x \left( \lambda \Vert x\Vert _1 - \eta ^TDx\right)&= \partial _x \left( \lambda v^Tx - \eta ^TDx\right) \\&= \lambda v - D^T\eta \end{aligned}$$

A necessary condition to attain the infimum is then $0 \in \{\lambda v - D^T\eta \}$. This leads to $v = \frac{D^T\eta }{\lambda }$ with the condition that $\Vert D^T\eta \Vert _\infty \le \lambda$ (because $\Vert v\Vert _\infty \le 1$) which can be injected into the previous equation which gives

$$\begin{aligned} \inf _{x}(\lambda \Vert x\Vert _1 - \eta ^TDx)&= \inf _{x}(\lambda v^Tx - \eta ^TDx)\\&= \lambda \left( \frac{D^T\eta }{\lambda }\right) ^T x - \eta ^T Dx\\&= \eta ^T Dx - \eta ^T Dx\\&= 0 \end{aligned}$$

Finally,

$$\begin{aligned} \mathcal {G}(\eta ) = \eta ^Tp - \frac{1}{2}\Vert \eta \Vert _2^2. \end{aligned}$$

Therefore, the dual problem is

$$\begin{aligned} \sup _\eta \left\{ \eta ^Tp - \frac{1}{2}\Vert \eta \Vert _2^2\right\} \text { s.t. } \Vert D^t\eta \Vert _\infty \le \lambda \end{aligned}$$

which is equivalent to

$$\begin{aligned} \sup _\eta \left\{ -\frac{1}{2}\Vert p - \eta \Vert _2^2\right\} \text { s.t. } \Vert D^t\eta \Vert _\infty \le \lambda . \end{aligned}$$

It can be reformulated in a penalized version as

$$\begin{aligned} \hat{\eta } = \mathop {\mathrm{arg\,min}}\limits _\eta \left\{ \frac{1}{2}\Vert p - \eta \Vert _2^2 + \lambda ' \Vert D^T\eta \Vert _\infty \right\} . \end{aligned}$$

(15)

While $D\hat{x}$ represents the “normal” part of the patch p, $\hat{\eta }$ represents the anomaly. Indeed, the condition $\Vert D^T\eta \Vert _\infty \le \lambda$ imposes to $\eta$ to be far from the patches represented by D. Moreover, for a solution $\eta ^*$ of the dual to exist (and so that the duality gap does not exist) it requires that $\eta ^* = p - Dx^*$, i.e., $p = Dx^* + \eta ^*$ which confirms the previous observation. Notice that the solution of (15) exists by an obvious compactness argument and is unique by the strict convexity of the dual functional.