Feature Extraction in FLIR Imagery Using the Continuous Wavelet Transform (CWT) Romain Murenzi,1 Lance Kaplan,1;2 Fernando Mujica,1;3 and Sandor Der 4 murenzi@hubble.cau.edu, lmkaplan@hubble.cau.edu, fmujica@eedsp.gatech.edu, sder@arl.mil Center for Theoretical Studies of Physical Systems 1;2 Department of Physics 1 and Engineering 2 , Clark Atlanta University Atlanta, GA 30314 Phone: (404) 880-8655 Center for Signal and Image Processing 3 School of Electrical Engineering, Georgia Institute of Technology Atlanta, GA 30332 Phone: (404) 894-2969 Army Research Laboratory 4 ARL-SE-SE, MD 20783 Phone: (301) 394-0807 Abstract This paper investigates the use of the Continuous Wavelet Transform (CWT) for detection of targets in cluttered environments. We rst de ne the multiple 2D CWT energy densities in the feature space. We then use the spatial energy density as a computational core for our target-detection algorithm. Results from FLIR images are presented at the end of the paper. We used both the Mexican Hat wavelet and target-adapted wavelets in the simulations. Introduction Automatic target detection and recognition (ATD/R) consists of computer processing to detect and recognize target signatures embedded in a cluttered environment. Typical targets include planes and tanks; clutter includes grass, trees, topographical features, and atmospheric phenomena (e.g., clouds and smoke). In general, the problem can be modeled using the following equation: s(~x) = n(~x) + X T (~x ? ~x ); N l=1 l ol (1) where n(~x) represents an additive noise (clutter + measurement noise), Tl (~x) are targets to be detected and recognized, and s(~x) represents the accessible measured signal. Automatic or assisted target detection and identication for FLIR imagery requires the ability to extract the essential features of an object from cluttered environments under the condition that the range to the target is unknown. Multiscale techniques (e.g., the multidimensional continuous wavelet transform (MCWT)) are highly desirable, because they can extract and normalize both the unknown scale and orientation of the target. The typical features to be extracted from the image of a target in a cluttered environment are the position of the target, the spatial extent (size) of the target (i.e., the scale), and the shape of the target, including its orientation and symmetry. To extract such target features, it is preferable not to work in the image space, but rather to map the image into a feature space of position, scale, and orientation | the MCWT performs this mapping exactly, as discussed in detail in [1, 2]. The 2D CWT is a decomposition [3, 4] of an image on a set of dilated, rotated, and translated versions of a single function called the mother wavelet. The scale dependence allows sensitivity to variations in sensor resolution as well as determination of target size. Rotation dependence leads to robust behavior, under varying target orientations. Integration over all the parameters of the squaredmodulus of the CWT gives the total energy in the signal, which is Parseval's energy conservation condition. Therefore, integration on a subset of the parameters gives an energy density in the remaining variables. A total of 14 energy densities (4 1D densities, 6 2D densities, and 4 3D densities) exist. Those densities have what we refer to as CWT features. In the next section; we de ne 2D CWT energy densities. We then describe an algorithm for target detection in cluttered environments, using the position energy density, and present results of the algorithm on FLIR data from the TRIM2 database, using the anisotropic Mexican hat wavelet, as well as targetadapted wavelets. Continuous Wavelet Transform and Energy Densities in Feature Space The 2D CWT is a linear transformation from 2~ L2 (IR2 ; d2~x) to L2 (IR+   [0; 2[IR2 ; da a3 d d b), Z 1  ( 1 r (~x ? ~b))s(~x) ; (2) a a  IR2 where s is the input image, S is the CWT, is the mother wavelet, a is the dilation parameter,  is the rotation parameter (with r , the corresponding 2D rotation matrix), ~b = [bx ; by ]0 is the translation vector, the asterisk denotes complex conjugation, and the hat denotes the Fourier transform. Considering the energy density js(~x)j2 , js^(~k)j2 , we have the following energy-conservation theorem: S (a; ; ~b) = Z IR2 d2 ~x d2~xjs(~x)j2 = = Z d2~kjs^(~k)j2 (3) IR ZZZ 1 d2~b djS (a; ; ~b)j2 : 2 a3 This theorem leads to the interpretation of jS (a; ; ~b)j2 as an energy density of the signal, s, in position, scale, and orientation parameters. It is clear that any partial integration of the CWT energy density in any subset of the parameter space gives an energy density in the remaining variables. We restrict the analysis to 2D energy densities, giving four 1D dimensional densities, six 2D densities, and four 3D densities. We have thus the following 2D CWT energy densities:  Space (range and aspect) energy density E34 (bx ; by ) = Z 1da Z 2 djS(a; ; b ; b )j2 (4)  0 a3 0 x y  Scale-angle energy density Z +1db Z +1db jS(a; ; b ; b )j2 (5) E12 (a; ) = x ?1 y ?1 x y  Anisotropy angle and aspect energy density (bx; by ) = j~bj(cos( ); sin( )) Z 1daZ +1dj~bjj~bjjS(a; ; j~bj; )j2 (6) E24 (; ) = 0 a3 0  Scale-range energy density E13 (a; j~bj) = Z 2 dZ 2 d jS(a; ; j~bj; )j2  0  0 (7)  Anisotropy angle-range energy density E23 (; j~bj) = Z 1da Z 2d jS(a; ; j~bj; )j2  0 a3 0 (8)  Scale-aspect energy density E14 (a; ) = Z 1dj~bjj~bj Z 2djS(a; ; j~bj; )j2: (9)  0 0 We are currently being investigating these energy densities in order to build a detector based on CWT 2D features of a given target type under various conditions. These features will then be used as inputs to a convolutional neural network (CNN) algorithm [5]. These densities will also be used to design an ATR algorithm for detection, classi cation, and recognition of targets in FLIR and SAR imagery. Detection Algorithm Using the Position (Range-Aspect) Energy Density Consider the FLIR images shown in gures 1 and 2: they contain a certain number of targets embedded in a cluttered environment. To enhance these images, we compute a 2D CWT energy density over position, while suppressing clutter at the same time. First, we compute the CWT in the position representation at all relevant scales a = aj and angles  = j , that is, S (aj ; j ; bx ; by ). For detection, we take the image obtained for each xed a = aj and  = j , threshold it, and add all the images together. Thresholding is performed in an adaptive way, becoming more severe for smaller a. The e ect of this adaptation is to suppress the clutter information, while preserving the target information. Note that other nonlinear transformations (e.g., enhancement, morphological operators) may also be applied. After wavelet image integration, the target information is  ?? s(~ x) ?? -? @ @@ @@ R S (a1; 1; ~b)j2 j .. . S (an ; n; ~b)j2 j - 6Threshold-  +f  @@ @@R - 6Threshold-. ?? ? - 6Threshold ? - - E3;4(~b) Figure 3: Detection algorithm. sulting composite image. These centroids correspond to the positions of potential targets; one controls the false-alarm rate by adjusting the thresholds to eliminate ambiguous targets. 50 100 150 200 Application to Detection of Target in FLIR Images 250 300 We apply the CWT-based algorithm to the two images shown in gures 1 and 2 taken from the TRIM2 database. We then compute the composite images as described above. Figures 4 and 5 show the results for the anisotropic Mexican Hat wavelet (with anisotropy parameter  = Figure 1: A FLIR image consisting of targets (tanks M1) 5). Figures 6 and 7 show the results using a mother and clutter, from the TRIM2 database. wavelet adapted to the targets. We performed all calculations at scales a = 1=8; 1=4; 1=2; 1 and orientations  = 0; 4 ; 2 . 350 400 450 500 100 200 300 400 500 50 100 150 50 200 100 250 150 300 200 350 250 400 300 450 350 400 500 100 200 300 400 500 450 500 Figure 2: A FLIR image consisting of targets (helicopters) and clutter, from the TRIM2 database. 100 200 300 400 500 Figure 4: Results for gure 1, using an anisotropic Mexican Hat wavelet with anisotropy parameter  = 5:0 . reinforced and becomes visually enhanced. Figure 3 shows a block diagram of the proposed target detec- As the gures show, the CWT-based detection altion algorithm. gorithm makes the targets stand out from their backTo complete the steps of the detection algorithm, grounds with more de nition than the original imwe compute the centroids ~b = ~bi ; i = 1; :::; L in the re- ages. This suggests that the CWT may be used 50 50 100 100 150 150 200 200 250 250 300 300 350 350 400 400 450 450 500 500 100 200 300 400 500 100 200 300 400 500 Figure 5: Results for gure 2, using an anisotropic Mexi- Figure 7: Results for gure 2, using a target-adapted can Hat wavelet with anisotropy parameter  = 5:0 . wavelet. References 50 100 150 200 250 300 350 400 450 500 100 200 300 400 500 Figure 6: Results for gure 1, using a target-adapted wavelet. for target detection, as described in earlier, and that CWT-based features might be appropriate as input to a convolutional neural network [5]. Conclusions In this paper, we presented a family of wavelet features based on the CWT, and we examined the e ectiveness of the CWT in separating targets from background in synthetic FLIR imagery. Target signatures extracted from a training set can be formed into the CWT feature space, allowing targets to be detected through the nonlinear matching algorithm described in [5]. Our future work will include automated learning of wavelet features, using arti cial neural nets on a large set of real FLIR and SAR images. Acknowledgments The authors wish to thank Drs. Marvin Cohen and Mark J. T. Smith for their valuable assistance in our research. View publication stats [1] Jean-Pierre Antoine, K. Bouyoucef, P. Vandergheyst and R. Murenzi.,"Target Detection and Target Recognition Using 2 Dimensional Continuous Isotropic and Anisotropic Waveletes," in Automatic Object Recognition V, SPIE-1995, Vol. 2485, 20-31, (1995). [2] Sang-II Park, Mark J.T. Smith and R. Murenzi, \Multidimensional Wavelets for Target Detection and Recognition," in SPIE International Symposium on Aerospace/Defense Sensing and Controls, Orlando, Florida, April 8-12, (1996). [3] J-P. Antoine and R. Murenzi, \Two Dimensional Directional Wavelets and the Scale-Angle Representation," Signal Processing 52, 259-281, (1996). [4] J-P. Antoine, R. Murenzi and P. Vandergheynst, \Two Dimensional Directional Wavelets in Image Processing," International Journal for Imaging Systems and Technology, Vol.7, 152-165, (1996). [5] Syed A. Rizvi and Vincent Mirelli, \Automatic Target Recognition Using a Multi-layer Convolution Neural Network," preprint (1996).