Classification of remote sensing imagery with high spatial resolution

Invited Paper Classification of remote sensing imagery with high spatial resolution Mathieu Fauvela,b , Jon Aevar Palmasona , Jon Atli Benediktssona , Jocelyn Chanussotb , and Johannes R. Sveinssona a Department of Electrical and Computer Engineering, University of Iceland, Hjardarhaga 2-6, 107 Reykjavik, Iceland; b Signals and Images Laboratory – LIS, Grenoble, LIS / ENSIEG – Domaine Universitaire – BP 46 – 38402 Saint-Martin-d’Hères Cedex, France. ABSTRACT Classification of high resolution remote sensing data from urban areas is investigated. The main challenge in classification of high resolution remote sensing image data is to involve local spatial information in the classification process. Here, a method based on mathematical morphology is used in order to preprocess the image data using spatial operators. The approach is based on building a morphological profile by a composition of geodesic opening and closing operations of different sizes. In the paper, the classification is performed on two data sets from urban areas; one panchromatic and one hyperspectral. These data sets have different characteristcs and need different treatments by the morphological approach. The approach can directly be applied on the panchromatic data. However, some feature extraction needs to be done on the hyperspectral data before the approach can be applied. Both principal and independent components are considered here for such feature extraction. A neural network approach is used for the classification of the morphological profiles and its performance in terms of accuracies is compared to the classification of a fuzzy possibilistic approach in the case of the panchromatic data and the conventional maximum likelhood method based on the Gaussian assumption in the case of the case of hyperspectral data. Also, different types of feature extraction methods are considered in the classification process. Keywords: Mathematical Morphology, Urban Areas, Classification, High Resolution Remote Sensing Data, Hyperspectral Data, Panchromatic Data. 1. INTRODUCTION The classification of high resolution urban remote sensing imagery is a challenging research problem. Here, we consider the classification of such data by both considering the classification of panchromatic imagery (single data channel) and hyperspectral images (multiple data channels). Panchromatic images are characterized by a very high spatial resolution. The high spatial resolution allows to identify small structures in a dense urban area. However, the analyze of the scene by considering only the value of the pixel will produce very poor classification compared to the fine resolution; for example it will not be able to distinguish between a pixel belonging to the roof either of a small house or of a large building if both the roofs have the same reflectance. To solve this problem, some local spatial information is needed. An interesting approach to provide such information is based on the theory of Mathematical Morphology, which provides tools to analyze spatial relationship between pixels. Recently, advance mathematical morphology has been successfully applied to geoscience and remote sensing1, 2 and has been proved its usefulness is remotely sensed images analysis. Hyperspectral urban data not only contain lot of spectral information, covered throughout the data channels, but also spatial information, covered by each individual band. Therefore, a single image, drawn from the data set, does not involve much spectral information, as well as spectral properties of the data set cannot bring Further author information: (Send correspondence to J.A. Benediktsson) J.A. Benediktsson: E-mail: benedikt@hi.is, Telephone: +354 525 4670 Image and Signal Processing for Remote Sensing XI, edited by Lorenzo Bruzzone Proceedings of SPIE Vol. 5982 (SPIE, Bellingham, WA, 2005) 0277-786X/05/$15 · doi: 10.1117/12.637224 Proc. of SPIE Vol. 5982 598201-1 forth spatial information. Consequently, joint spectral/spatial classifier is needed for classification of urban hyperspectal data. Benediktsson et al.8 have proposed the use of extended morphological profiles for hyperspectral urban data, i.e., to build morphological profiles based on more than one image (such as in the panchromatic case) and use several principal components for that purpose. Here, both Principal Component Analysis (PCA) and Independent Component Analysis (ICA) are used to create base images for morphological profiles. The paper is organized as follows. In Section 2, basic definitions of mathematical morphology are reviewed, followed by the discussion of a composition of morphological operators that are used to define the morphological profile which characterizes the structures present in an the image. Experimental results on an IKONOS panchromatic image are given. In Section 3, the extended morphological profile for hyperspectral data is discussed along with a review of PCA and ICA for feature extraction. Classification results are given for DAIS hyperspectral data. Finally, in Section 4, conclusions are drawn. 2. MATHEMATICAL MORPHOLOGY Mathematical Morphology (MM) is a theory aiming to analyze spatial relationship between pixels. MM was introduced by Matheron and Serra in the 1960s to study porous media. Nowadays, several morphological operators are available for extracting structural information in spatial data.3 In the following subsection, some basic notions of MM are reviewed. Then, concepts of the Morphological Profile and of the Derivative of the Morphological Profile are detailed. 2.1. Theoretical Notions In image analysis, data are represented in discrete space Z n , and an image f is a mapping of a subset Df of Z n , f : Df ⊂ Z n → {0, . . . , fmax } (1) where fmax is the maximum value of the image. With MM, objects of interest are viewed as a subsets of the image. Then, several sets of known size and shape (such as disk, square or line) can be used to characterize their morphology. These sets are called Structuring Elements (SEs). An SE always has an origin, which generally is its symmetric center. The origin allows the positioning of the SE at a given pixel x of f , i.e., the origin coincides with x. For binary images (i.e., fmax = 1), MM are mainly based on set operators such as the union, intersection, complementation and translation: SE is positioning on each pixel x and a set operators is applied between the set which x belongs to and SE. For grey level images, intersection ∩ of two sets becomes the infimum ∧ and the union ∪ becomes supremum ∨. For two images f and g and a given pixel x: (f ∧g)(x) = min[f (x), g(x)] and (f ∨g)(x) = max[f (x), g(x)]. We now give the definitions of the two fundamental morphological operators, erosion and dilation. More developments on MM can be found in the literature.3, 4 Definition 2.1 (Erosion). The erosion ǫB (f ) of an image f by a structuring element B is defined as
f−b , (2) ǫB (f ) = b∈B where fb is the translation by vector b of f , i.e., fb (x) = f (x − b). The eroded value at a given pixel x is the minimum value of the image in the window defined by the SE when its origin is at x. The eroded value shows where the SE fits the objects in the input image. Definition 2.2 (Dilation). The dilation δB (f ) of an image f by a structuring element B is defined as  f−b . (3) δB (f ) = b∈B The dilated value at a given pixel x is the maximum value of the image in the window defined by the SE when its origin is at x. The dilated value shows where the SE hits the objects in the input image. Proc. of SPIE Vol. 5982 598201-2 Erosion and dilation are dual transformations with respect to the complementation: ǫB (f ) = [δB ([f ]c )]c , (4) where [ ]c is the complementation operator: [f ]c (x) = fmax − f (x). This property shows the dual effect of erosion and dilation. When erosion expands dark objects, dilation shrinks them (and vice-versa for bright objects). Moreover, bright structures that cannot contain the SE are removed by erosion (similar effects are seen for dark objects using dilation). Hence, both erosion and dilation are non-invertible transformation. Figure 1 shows examples of erosion and dilation. These two operators are the basic tools of MM. The operators that will be discussed next, opening and closing, are a combination of erosion and dilation. Definition 2.3 (Opening). The opening γB (f ) of an image f by an SE B is defined as the erosion of f by B followed by the dilation with the SE B ∗ : γB (f ) = δB [ǫB (f )]. (5) The idea to dilate the eroded image is to recover most structures of the original image, i.e., structures that were not removed by the erosion. Definition 2.4 (Closing). The closing φB (f ) of an image f by an SE B is defined as the dilation of f by B followed by the erosion with the SE B: φB (f ) = ǫB [δB (f )]. (6) Figure 2 shows result of closing and opening of an IKONOS image by a 5x5 square SE. It can be seen that structures of size less than the SE are totally removed. Eventhough opening and closing are powerful operators, their major drawback is that they are not connected filters. It can be seen in Figure 2, at the bottom, that the two small houses have merged into one after the closing operation. This structure can be seen as a building now. To avoid thisproblem, geodesic morphology and reconstruction can be used. Reconstruction filters are connected filters and they have been proven not to introduce discontinuities.2 Reconstruction filters are based on geodesic morphology. (1) Definition 2.5 (Geodesic dilation). The geodesic dilation δg (f ) of size 1 consists in dilating a marker f with respect to a mask g, δg(1) (f ) = δ (1) (f ) ∧ g. (7) The geodesic dilation of size n is obtained by performing n successive geodesic dilations of size 1: Definition 2.6 (geodesic erosion). The geodesic erosion is the dual transformation of the geodesic dilation, (1) (f ) ∨ g. ǫ(1) g (f ) = ǫ (8) Definition 2.7 (Reconstruction). The reconstruction by dilation (erosion) of a marker f with respect to (n+1) a mask g consists of repeating a geodesic dilation (erosion) of size one until stability is achieved, i.e., δg (f ) = (n) (n+1) (n) δg (f ) (i.e., ǫg (f ) = ǫg (f )): Recg (f ) = δg(n) (f ), (9) ǫ(n) g (f ). (10) Rec∗g (f ) = With Definition 2.7, it is possible to define connected transformation that satisfy the following assertion: If the structure of the image cannot contain the SE then it is totally removed, else it is totally preserved. These operators are called opening/closing by reconstruction. The true definition is with the transposed SE B̌. Transposition of B corresponds to its symmetric set with respect to its origin. For simplicity, we only consider SE whose origin is also the symmetric center, so B̌ = B. ∗ Proc. of SPIE Vol. 5982 598201-3 Definition 2.8 (Opening-Closing by reconstruction). The opening by reconstruction of an image f is defined as the reconstruction by dilation of f from the erosion of size n of f . Closing by reconstruction is defined by duality: (n) γR = Recf (ǫ(n) (f )), (11) (n) φR (12) = Rec∗f (δ (n) (f )). Figure 2 shows results of opening and closing by reconstruction. It can be clearly seen that these transformations introduce less noise than the classical opening-closing. Shapes are preserved and the structures still present after transformation are of a size greater than or equal to the SE. Therefore, the use of opening and closing by reconstruction allows us to characterize morphological characteristics of the structures present in an image. In addition, to determine the size or shape of all the objects present in an image, it is necessary to use a range of different SE size. This concept is called Granulometry.3 Definition 2.9 (Granulometry). A granulometry Φλ is defined by a transformation having a size parameter λ and satisfying the three following axioms: • Anti-extensivity: The transformed image is less than or equal to the original image. • Increasingness: The ordering relation between image is preserved. • Absorption: The composition of two transformations Φ of different size λ and ν will give always the result of transformation with the biggest size: Φλ Φν = Φν Φλ = Φmax(λ,ν) . (13) Granulometries are typically used for the analysis of the size distribution of structures in images. Classical granulometry by opening is built by successive opening operation of an increasing size. By doing so, an image is progressively simplified. Using connected operators, like opening by reconstruction, no shape noise is introduced. Anti-Granulometry is defined with the same axioms as granulometry and by replacing anti-extensivity axiom by extensivity. The granulometry concept could be used to create a feature vector from a single image. The next subsection describes the concepts of the Morphological Profile and of the Derivative of the Morphological Profile. 2.2. Morphological Profile and Derivative of the Morphological Profile Here, we will review the concepts of the Morphological Profile and of the Derivative of the Morphological Profile, both of which are based on granulometry and anti-granulometry The opening profile (OP) at the pixel x of the image I is defined as an n-dimensional vector: (i) OPi (x) = γR (x), ∀i ∈ [0, n]. Erosion Original image (14) Dilation Figure 1. Erosion and dilation of a grey level image by a 5x5 square SE. Proc. of SPIE Vol. 5982 598201-4 ..-. Opening Closing Opening by Reconstruction Closing by Reconstruction Figure 2. Opening, closing, opening by reconstruction, and closing by reconstruction of a grey level image by a 5x5 square SE. Also, the closing profile (CP) at the pixel x of the image I is defined as a n-dimensional vector: (i) CPi (x) = φR (x), ∀i ∈ [0, n]. (15) Clearly we have CP0 (x) = OP0 (x) = I(x) and n is the total number of opening or closing. The OP made with opening by reconstruction satisfy the three axioms of granulometry. It is the same for CP with antigranulometry. Therefore OP (CP) could be defined as a granulometry (anti-granulometry) made with opening (closing) by reconstruction. By collating the OP and the CP, the Morphological Profile (MP) of the image I is defined as 2n + 1-dimensional a vector: M P (x) = {CPn (x), CPn−1 (x), . . . , CP1 (x), I(x), OP1 (x), . . . , OPn−1 (x), OPn (x)} , (16) where ⎧ ⎨ = CPn−i = I(x) M Pi (x) ⎩ = OPi if 0 ≤ i < n, if i = n, if n < i ≤ 2n. (17) Finally, the Derivative of the Morphological Profile (DMP) is defined as 2n-dimensional vector equal to the discrete derivative of MP: DM Pi (x) = M Pi−1 − M Pi . (18) Informations provided by DMP are both spatial and radiometric. For a given pixel, if the DMP is balanced (around a center point), that should signify that a pixel belonging to a structure that is small compared to the SE was used to build the DMP. On the other hand, an unbalanced DMP (to the left or to the right) should indicate that the pixel belongs to a large structure. Then, the unbalance of the profile, indicates that the pixel belongs to a darker (left side) or a brighter (right side) structure than the surrounding pixels. Finally, the amplitude of the DMP gives an information about the local contrast of the structure. Figures 3 and 4 presents the MP and the DMP obtained for the IKONOS image. In this case, the SE used were discs with increasing radius (3, 6 and 9 pixels long radius). From this example, it is clear that the DMP should help in discrimination. However, different approaches can be used. In the next section we will discuss the classification of the DMP using neural networks and fuzzy logic. 2.3. Classification of Panchromatic Imagery As said previously, DMP provides information to discriminate classes. Figure 5 gives “spectral responses” examples of DMPs for three different classes. For classification, we assume that each class has a typical DMP. Based on this assumption, two classifications methods were considered. They are based on two interpretation of the DMP. For the first one,5 we consider the DMP as a multispectral image, then the classification is performed with classical pattern recognition algorithms. For the second approach,6 the DMP is a fuzzy measurement of Proc. of SPIE Vol. 5982 598201-5 the characteristic size and contrast of each structure. Both approach will be briefly presented and tested in the next subsections. For both approaches, the tested image is an IKONOS panchromatic image from Reykjavik, Iceland. This image is a 975 × 639 pixel with 1-m spatial resolution. The test area is in the center of Reykjavik. It comprises residential, commercial and open areas. Six classes of interest are defined: large buildings (1), small buildings (2), residential lawns (3), streets (4), open areas (5) and shadows (6). The number of training and test samples for each information class is listed in Table 1. A 17-dimensional MP was created (8 openings and 8 closings) using a disc SE. Therefore, a 16-dimensional DMP was used as input data. 2.4. DMP Viewed as a Multispectral Image Here, each DM Pi (x) is considered as a channel of a multispectral image. This way, classification methods applied to multispectral images can be applied. Due to the possibly high dimensionality of the DMP, feature extraction and feature selection methods could also be used. For this experiment, a conjugate gradient neural network was used for classification. The number of hidden neurons was selected to be twice the number of input features. Two different feature extraction methods15 (Discriminant Analysis Feature Extraction (DAFE) and Decision Boundary Feature Extraction (DBFE)) were also applied on the DMP. DAFE is a method which is intended to enhance class separability. DBFE is a method which is intended to extract discriminant informations from the decision boundary. The Neural Network (NN) was trained with a reference map and 25% of the labelled samples were used for training. The other samples were use for testing. Table 2 shows the achieved accuracies for the different approaches. In the table, AVE refers to average classification accuracy, i.e., the mean accuracy for the individual classes and OA denotes overall classification accuracy, i.e., the classification accuracy for all pixels in the training or test sets. It is evidence that NN performed better using the DMP as compared using the single panchromatic band. However, using the DBFE gives quantitative results which are a little worse than without feature extraction. DAFE had singularity problems and performed not so well. More details on these data and experiments are given in Benediktsson et. al.5 Table 1. DAIS University area: Information classes and samples. No. 1 2 3 4 5 6 Class Name Large Buildings Small Buildings Residential Lawns Streets Open Areas Shadows Total Samples Train Test 7729 30916 8539 34155 8788 35147 9800 39202 10967 43867 6451 25806 52274 209093 2.5. DMP Viewed as a Possibility Distribution In this approach, the classification is based on a fuzzy interpretation of the DMP and on a possibilistic definition for the classes. The DMP is interpreted as a fuzzy measure of the size of a structure. The contrast of the local structure is provided by the maximum value of the DMP. From Figure 5 it is clear that buildings are of a bigger size and contrast than roads or shadows. For classification, pre-defined DMP Π(n) for each class n is needed. They can be build with simple statements such as: “A large building is a bright object with a large size and a high contrast” or “A shadow is a dark object with a high contrast but a unknown size”. Dark and bright can be defined as on the left side of the DMP and on the right side of the DMP while fuzzy definition of size (large ...) Figure 3. MP made with 3 closing-opening by reconstruction. SE were discs with increasing radius (3, 6 and 9 pixels long radius) Proc. of SPIE Vol. 5982 598201-6 Figure 4. DMP derives from MP on Figure 3. The profile was normalized for visual inspection. Table 2. Test Accuracies in percentage for original IKONOS Image, the entire DMP, the DAFE, the DBFE and the FPM. Data Original gray value 1 Features ❤❤❤❤ Method ❤❤❤ ❤❤❤ Class No 1 2 3 4 5 6 Ave OA Entire DMP 16 DAFE 10 DBFE 10 Entire DMP 16 NN NN NN NN FPM 63.6 7.3 0.0 2.8 90.9 95.5 43.3 41.6 73.2 60.6 40.2 61.6 52.7 92.5 63.8 62.5 54.4 45.3 45.9 61.6 38.2 91.8 56.2 54.0 70.7 57.4 46.5 68.7 43.3 89.5 62.9 60.6 47.6 67.8 58.8 9.8 52.2 83.3 53.3 52.1 depends on the DMP size. For definition facilities, the pre-defined DMP has trapezoidal shape. Then, for each pixel x a degree of membership Cn (x) is defined for each class n as  Π (n) i DM Pi (x) ×   i (19) Cn (x) = DM P (x) × i i i Πi (n) where Πi (n) is a possibility distribution. Then another membership degree αn (x) is defined by comparing the local contrast with the corresponding model (low or high, depends on the considered class, for example high for building class). The final decision for the pixel x is taken by selecting the class n as following: nselected (x) = argmaxn {αn (x) × Cn (x)}. (20) Table 2 shows the achieved accuracies for the Fuzzy Possibilistic Model approach. Here, results are evaluated with all the labelled samples and that can explained why visual results are not consistent with qualitative results. More details on these experiments are given in Ref.6 It should be underlined that numerical and visual results should be considered in a relative ways. Some preand post-filtering could increase these accuracies. 3. EXTENDED MORPHOLOGICAL PROFILE The morphological profile approach was applied to panchromatic data in the previous section. The method has been extended for hyperspectral data applications. A characteristic image or images needs to be extracted from the data. It was suggested to use the first principal component (PC) of the hyperspectral data for such 1 1 1 0.9 0.9 0.9 0.8 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.4 0.4 0.4 0.3 0.3 0.3 0.2 0.2 0.1 0.1 0 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 Shadow Class 6 0 0.2 0.1 1 1.5 2 2.5 3 3.5 4 4.5 5 Road Class 5.5 6 0 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 Building class Figure 5. Examples of typical DMPs ”“spectral responses” obtained for the MP in 4 Proc. of SPIE Vol. 5982 598201-7 a purpose.9 Although that approach seems reasonable because PCA is optimal for data representation in the mean square sense, it should not be forgotten that with only one PC, the hyperspectral data are reduced from potentially several hundred data channels into one single data channel. Some important information may be contained in the other PCs. Therefore, we apply an extension to the approach in10 and build an extended morphological profile from several different PCs. This extension was proposed in Benediktsson et. al.8 For example, we could decide to use the PCs that account to certain percentage of the total variation in the image, e.g., 95% or 99%. If two PCs fill up the variation threshold, morphological profiles are constructed for each of the PCs. An example of the extended morphological profile is shown in Figure 6. Each profile is now represented by multi dimensional feature vector, which now are stacked into single vector to be used for classification. In general mathematical notation, the extended morphological profile is represented by M Pext (x) = {M PPC 1 (x), M PPC 2 (x), . . . , M PPC n (x)} , (21) where M PPC i (x), i = 1, ..., n, are the morphological profiles constructed from principal components according to (14). Obviously, the computations will be more intensive for this approach. On the other hand, better information should be extracted from the hyperspectral data than for the simple approach proposed in.9 Also, some redundancies should be observed for the extended morphological profile. Principal component analysis (PCA) and Independent component analysis (ICA) as feature extraction method are discussed in the next two subsections. 3.1. PCA Feature Extraction The aim of PCA is to transform the data into lower dimensional subspace which is optimal in the sense of sum-square error. PCA decorrelates the original data set and makes the transformed features uncorrelated to each other. The eigenvectors, ei , i = 1, 2, ..., d, of the covariance matrix with corresponding eigenvalues λi form orthogonal basis for the new feature space, which original data set is projected onto. Eigenvalues measure the contribution of each eigenvector to the original feature space and the eigenvectors are sorted in a decreasing order of eigenvalues. Usually, only the first k eigenvectors are used but the remaining d−k dimensions are skipped. The projection onto lower dimensional subspace contains noise. The value of k may be determined in order to have the transformed data include a particular percentage of the original variance, e.g., 95% or 99% of the original variance. The transformation matrix Ax of size d × k includes the k eigenvectors and projection is given by y = ATx (x − µx ). (22) where x is the feature vector and µx is the mean. After the transformation, the correlation between different features has been removed and the covariance matrix of y, Σy , is a diagonal matrix with entires λi . Profile from PC 1 Closings Original Openings Profile from PC 2 Closings Openings Original '-- *--- .4:_ •--c' 4-rU- -V Figure 6. Extended morphological profile of two images. Each of the original profile has 2 opening and 2 closings. Circular structuring element with radius increment 4 was used (r = 4, 8). Proc. of SPIE Vol. 5982 598201-8 3.2. ICA Feature Extraction ICA was introduced in 1980s as effective method for Blind Source Separation (BSS), i.e., to separate data into underlaying information component. The method has also been used for the purpose of feature extraction. To demonstrate a BSS problem, let us consider sensed signals, xi (t), i = 1, 2, 3, as linear mixtures of the sources, si (t), i = 1, 2, 3: x1 (t) = a11 s1 (t) + a12 s2 (t) + a13 s3 (t), x2 (t) = a21 s1 (t) + a22 s2 (t) + a23 s3 (t), x3 (t) = a31 s1 (t) + a32 s2 (t) + a33 s3 (t). (23) It is of interest to find the unknown sources, si (t), without prior knowledge about the mixing coefficients, aij . The set of equations can be written in vector form as, x = As, (24) where A is referred to as the mixing matrix with size n × m. The vector x represents n sensed signals and m assumed sources are in s. ICA can recover up to m = n independent sources but in practice, m < n should be expected. By definition, the sources are assumed to be statistically independent, which is a stronger requirement than being uncorrelated.12 Then, the probability density can be expressed as, p(s) = m  p(si ), (25) i=1 where p(si ) are the probability density functions for the individual source signals. During the unmixing process, we seek m × n transformation matrix W such as, u = W x. (26) where u contains the unmixed signals - the sources. Bell and Sejnowski published their approach of blind signal deconvolution based on ICA by minimizing the mutual information13  p(u) I(u1 , ..., um ) = E log m (27) i=1 p(ui ) m H(ui ) − H(u), = (28) i=1 where H(ui ) = −E{log(p(ui ))} is the entropy for random variable ui and H(u) = −E{log(p(u))} is the joint entropy for u = [u1 u2 · · · um ]T . At it’s minimum, (27) becomes zero. The unmixing matrix W is optimized by a natural gradient algorithm14 where learning rate can control the convergence speed. Varshney and Arora propose two ICA feature extraction algorithms in.14 The algorithm applied here starts by data set decorrelation using the whitening transform or PCA. Independent components are extracted from the m most important principal components with the accumulative variance of 99%. The n − m rest of the PCs are not used. Algorithm breaks when mutual information has reached it’s minimum, according to estimation by (28). 3.3. Experimental Results for DAIS Data Experiments were done on Digital Airborne Imaging Spectrometer (DAIS) 7915 data. The DAIS 7915 imaging spectrometer was designed by DLR and has 79 channels. Seventy two of the spectral channels are in the visible light and near infrared regions of the spectrum, i.e., correspond to the wavelengths 0.4–2.4 µm. Seven thermal Proc. of SPIE Vol. 5982 598201-9 infrared bands were not used. Some channels were skipped due to noise. Therefore, a total of 62 bands were used. The flight altitude was chosen as the lowest available for the airplane, which resulted in a spatial resolution of 2.6m per pixel. The test site is around the Engineering School at the University of Pavia the image is 243 by 243 pixels. There were eight information classes defined for University area data set: asphalt, tree, meadow, gravel, bitumen, soil, parking lot and roof. The number of training and test samples for each information class is listed in Table 3. Table 3. DAIS University area: Information classes and samples. No. 1 2 3 4 5 6 7 8 Class Name Asphalt Tree Meadow Gravel Bitumen Soil Parking lot Metallic roof Total Samples Train Test 137 129 131 135 136 137 98 117 110 96 133 80 130 135 92 90 967 919 3.3.1. Statistical Classification To get baseline results for the University area data set, the experiments were started by using the Gaussian Maximum Likelihood (ML) classifier. Feature Extraction, i.e., DAFE, DBFE and Non-Parametric Weighted Feature Extraction (NWFE),15 was also used in the experiments in order to reduce the data sets. For DBFE and NWFE, the reduction was according to the 99% variance criterion but for the DAFE a 100% criterion was used.15 Results for the ML classification of the raw and reduced data sets are listed in Table 4 Table 4. DAIS University area: Classification accuracies (%) obtained from maximum likelihood classification of raw data, with and without Feature Extraction. Data FE Features Class 1 2 3 4 5 6 7 8 Ave OA Raw bands 62 Train Test 100.0 69.8 100.0 82.2 100.0 75.2 100.0 21.4 100.0 28.1 100.0 75.0 100.0 59.3 100.0 74.4 100.0 60.7 100.0 61.3 Raw bands DAFE 7 Train Test 81.0 59.7 90.8 84.4 89.7 93.4 92.9 66.7 92.7 47.9 92.5 80.0 80.0 70.4 100.0 73.3 90.0 72.0 89.3 72.7 Raw bands DBFE 23 Train Test 97.8 63.6 99.2 77.8 98.5 86.9 100.0 36.8 100.0 44.8 100.0 82.5 100.0 70.4 100.0 80.0 99.4 64.9 99.4 68.0 Raw bands NWFE 19 Train Test 90.5 61.2 93.1 83.0 97.1 97.1 99.0 62.4 98.2 40.6 98.5 67.5 93.8 74.1 100.0 81.1 96.3 70.9 96.0 72.1 Using the 62 raw data bands gave the lowest overall test accuracies, i.e., 61.3% of the test samples were assigned to the correct class. The highest test accuracy was obtained after a reduction by the DAFE method (72.7% overall accuracy). For the DAFE, seven features were used based on an 100% accumulative variance in this eight class problem. For the DBFE and NWFE, the feature reduction was according to the 99% variance and original data set transformed into 23 and 19 features, respectively. 3.3.2. Principal Components The three most important PCs correspond to 99% of the total data set variance. Principal components one through four are displayed in Figure 7 and classification accuracies, using these these band as input features are given in table 5. PCs four and above are noisy and make negligible contribution to the total data set variance. The best accuracies for the principal components was experienced using the three most important PCs where 46.6% of test samples samples were assigned to correct class, respectively. Proc. of SPIE Vol. 5982 598201-10 Figure 7. DAIS University area, most important principal components, 1st (left) through 4th (right). Table 5. DAIS University area: Classification accuracies (%) obtained from Neural Network classification of most important Principal Components. Data set Features Class 1 2 3 4 5 6 7 8 Ave OA PC 1 Train 71.5 74.0 4.4 12.2 0.0 68.4 49.2 96.7 47.1 47.3 1 Test 24.0 87.4 2.2 11.1 0.0 27.5 40.7 52.2 30.7 31.4 PCs 1, 2 2 Train Test 81.8 58.1 80.9 84.4 69.1 48.9 84.7 67.5 90.9 40.6 43.6 16.3 0.0 0.0 0.0 0.0 56.4 39.5 57.2 42.1 PCs 1, 3 Train 83.2 86.3 52.9 85.7 82.7 94.0 0.0 0.0 60.6 61.9 2, 3 Test 55.0 86.7 83.9 56.4 38.5 52.5 0.0 0.0 46.6 48.7 3.3.3. Independent Components The PCs of University area data set, normalized to unit variance, are transformed according to the independent component analysis procedure. The ICs are displayed in Figure 8 and classification accuracies, using ICs as input to the neural network, are given in Table 6. For the three ICs, 61.9% of the test samples were correctly classified. This was a much improved overall accuracy on what was obtained using the three PCs in Table 5. 3.3.4. Morphological Profiles of Principal Components Simple and extended morphological profiles were constructed from each of the principal components by adding four openings and four closing to the original image. A disk-shaped structuring element with radius increment of 2 pixels was used. The profiles have nine, 18 and 27 input features and the classification accuracies are shown in Table 7. In all three instances, accuracies were improved compared to classification of PCs, shown in Table 5. The highest accuracy was obtained for the 3-PCs morphological profile where the classifier was correct for 79.0% of the test samples. et Figure 8. DAIS University area, independent components, 1st (left) through 3rd (right). Proc. of SPIE Vol. 5982 598201-11 Table 6. DAIS University area: Classification accuracies (%) obtained from Neural Network classification of Independent Components. Data set Features Class 1 2 3 4 5 6 7 8 Ave OA IC 1 1 Train Test 70.8 76.7 35.9 37.0 0.0 0.0 6.1 8.5 0.0 0.0 67.7 68.8 45.4 37.0 72.8 21.1 37.3 31.2 37.8 30.8 IC 2 1 Train Test 77.4 66.7 0.0 0.0 58.8 43.1 14.3 9.4 22.7 0.0 90.2 57.5 56.2 42.2 66.3 40.0 48.2 32.4 49.5 32.1 IC 3 1 Train Test 75.9 79.1 44.3 15.6 55.9 72.3 73.5 63.2 0.0 0.0 58.6 15.0 0.0 0.0 95.7 68.9 50.5 39.3 49.2 40.3 ICs 1, 3 Train 61.3 84.7 80.9 63.3 91.8 91.0 77.7 100.0 81.3 80.9 2, 3 Test 46.5 88.9 94.2 39.3 37.5 48.8 69.6 50.0 59.3 61.9 Table 7. DAIS University area: Classification accuracies (%) obtained from neural network classification of morphological profiles of most important Principal Components. Data set # op/cl SE step Features Class 1 2 3 4 5 6 7 8 Ave OA PC 4 2 9 Train 94.2 0.0 57.4 0.0 92.7 0.0 93.8 0.0 42.3 44.6 1 Test 76.0 0.0 35.8 0.0 74.0 0.0 89.6 0.0 34.4 36.9 PCs 1, 2 4 2 18 Train Test 94.9 72.1 100.0 74.1 98.5 66.4 89.8 63.2 100.0 67.7 100.0 55.0 97.7 69.6 98.9 56.7 97.5 65.6 97.6 66.6 PCs 1, 4 2 27 Train 90.5 92.4 99.3 79.6 100.0 99.2 100.0 100.0 95.1 95.3 2, 3 Test 79.8 80.7 99.3 59.0 71.9 88.8 91.9 50.0 77.7 79.0 3.3.5. Morphological Profiles of Independent Components Morphological profiles of three independent components in Figure 8 were constructed and used as input to the neural network classifier. Classification accuracies are shown in Table 8. Table 8. DAIS University area: Classification accuracies (%) obtained from neural network classification of morphological profiles of Independent Components. Data set # op/cl SE step Features Class 1 2 3 4 5 6 7 8 Ave OA IC 1 4 2 9 Train Test 69.3 45.7 50.4 35.6 45.6 0.7 0.0 0.0 97.3 59.4 64.7 60.0 80.0 65.2 94.6 36.7 62.7 37.9 62.8 36.3 IC 2 4 2 9 Train Test 86.9 62.8 67.2 48.9 77.9 76.6 29.6 30.8 97.3 68.8 93.2 75.0 60.8 39.3 92.4 46.7 75.7 56.1 76.2 55.4 IC 3 4 2 9 Train Test 0.0 0.0 74.0 54.1 89.7 75.9 0.0 0.0 0.0 0.0 0.0 6.3 97.7 89.6 100.0 81.1 45.2 38.4 45.3 40.9 ICs 1, 2, 3 4 2 27 Train Test 0.0 0.0 98.5 74.8 100.0 100.0 100.0 87.2 100.0 86.5 0.0 0.0 100.0 90.4 100.0 61.1 74.8 62.5 71.9 65.3 By the construction of an extended morphological profile of three ICs, classification accuracies were improved little from the experiments listed in Table 6. 3.3.6. Morphological Profiles of PCs with Feature Extraction The three feature extraction methods were applied to the morphological profile of two and three original PCs, respectively. The feature reduction for the DAFE was based on a 100% variance, resulting in seven features. Proc. of SPIE Vol. 5982 598201-12 Fewer features were used in case of reduction by the DBFE and NWFE methods. For those approaches, the features were reduced based on 87 to 92% total variance. The classification accuracies are given in Table 9. Table 9. DAIS University area: Classification accuracies (%) obtained from neural network classification of morphological profiles of most important Principal Components with Feature Extraction. Data set PCs 1, 2 PCs 1, 2, 3 PCs 1, 2 PCs 1, 2, 3 PCs 1, 2 PCs 1, 2, 3 # op/cl 4 4 4 4 4 4 SE step 2 2 2 2 2 2 FE DAFE DAFE DBFE DBFE NWFE NWFE Features 7 7 6 6 7 6 Class Train Test Train Test Train Test Train Test Train Test Train Test 1 63.5 50.4 45.3 16.3 62.8 61.2 0.0 0.0 86.1 79.8 89.8 78.3 2 83.2 60.7 80.2 51.9 77.1 63.0 77.9 63.7 87.0 82.2 98.5 88.1 3 84.6 30.7 83.1 76.6 80.9 35.8 79.4 85.4 83.8 34.3 100.0 89.8 4 83.7 64.1 74.5 53.8 0.0 0.0 45.9 41.0 78.6 68.4 80.6 62.4 5 100.0 72.9 98.2 71.9 99.1 63.5 97.3 67.7 100.0 66.7 99.1 49.0 6 88.7 72.5 85.0 57.5 92.5 32.5 97.7 73.8 100.0 71.3 97.7 66.3 7 87.7 71.9 80.8 80.7 70.0 60.7 95.4 75.6 90.0 80.0 90.0 76.3 8 98.9 58.9 75.0 35.6 92.4 46.7 98.9 51.1 100.0 52.2 100.0 48.9 Ave 86.3 60.3 77.7 55.5 71.8 45.4 74.1 57.3 90.7 66.9 94.5 69.9 OA 85.4 59.0 77.4 56.0 72.9 46.1 73.1 56.9 90.5 67.1 94.6 72.1 Classification accuracies were not improved over the results in Table 7 in any of the experiments listed in Table 9 except when the 2-PCs morphological profile was reduced by the NWFE method. 3.3.7. Morphological Profiles of ICs with Feature Extraction Finally, the same feature extraction methods were applied to extended morphological profile of three independent components and the accuracies are given in Table 10. Table 10. DAIS University area: Classification accuracies (%) obtained from neural network classification of morphological profiles of Independent Components with Feature Extraction. Data set # op/cl SE step FE Features Class 1 2 3 4 5 6 7 8 Ave OA PCs 1, 2, 3 4 2 DAFE 6 Train Test 0.0 0.0 79.4 71.9 89.7 47.4 96.9 71.8 0.0 0.0 0.0 0.0 89.2 74.8 89.1 50.0 55.5 39.5 53.7 42.7 PCs 1, 2, 3 4 2 DBFE 5 Train Test 80.3 62.8 31.3 40.7 73.5 70.8 64.3 69.2 92.7 60.4 0.0 0.0 90.8 84.4 97.8 77.8 66.3 58.3 64.5 60.5 PCs 1, 2, 3 4 2 NWFE 5 Train Test 85.4 56.6 90.8 85.9 63.2 86.1 0.0 0.0 99.1 70.8 96.2 57.5 93.1 91.9 100.0 67.8 78.5 64.6 79.8 65.9 When compared to the results in Table 8, the test accuracies were not improved using the DAFE and DBFE methods. However, a small improvement in terms of overall accuracies was obtained for reduction by NWFE and 65.9% test samples were assigned to the correct class according to Table 10. 3.3.8. Summary of DAIS University Area Experiments Classification accuracies obtained by the neural network classifier are compared to statistical classification accuracies. Bar chart is displayed in Figure 9. Reduction of the the extended morphological profiles always lead to less accuracies for the DAFE and DBFE methods. The reduction by the NWFE increased overall accuracies for two morphological profiles out of three, along with the case when the NWFE was applied to the original raw data, prior to ML-classification. Proc. of SPIE Vol. 5982 598201-13 L. -a—————. aoot'raoiss Classification Figure 9. DAIS University area, comparison of feature extraction methods. 4. CONCLUSIONS Classification has been performed on high resolution imagery from urban areas using morphological profiles. Two different data sets were used in analysis, i.e., a panchromatic data set and hyperspectral data. In the panchromatic case, acceptable accuracies were achieved, regardless of the classification methods used. However, the obtained accuracies are still far below 100%, meaning that some improvements are still required. In particular, we defined the DMP with only one shape of SE. It could be useful to explore different shape of SE (disk and line of a different orientation) for the same image in order to increase the accuracies. Recently in Ref,7 we fused NN results and FPM results to improve the classification. The obtained results motivate us to continue in that direction. For the hyperspectral data, the best overall accuracies of test data were obtained by using extended morphological profiles based on principal components. However, the classification of extended morphological profiles was in most cases not seen to have a significant advantage, in term of classification accuracies, over statistical classification of the original raw data set. On the other hand, we have noted in our experiments that morphological approaches improve the visual interpretation on classes, which represent structure shaped objects, such as roads, houses and their shadows. Furthermore, the classification maps obtained by the neural network classification of extended morphological profiles seem to be less noisy than when maximum likelihood classification is applied on the raw data. ACKNOWLEDGMENTS This research was partially supported by the Icelandic Science Fund, the Research Fund of the University of Iceland, and the Jules Verne Program of the French and Icelandic governments (PAI EGIDE). The authors would like to thank Professor Paolo Gamba of the University of Pavia, Italy, for providing the reference data for the DAIS data. REFERENCES 1. P. Soille and M. Pesaresi, “Advances in mathematical morphology applied to geoscience and remote sensing,” IEEE Transactions on Geoscience and Remote Sensing 40, pp. 2042–2055, september 2002. 2. M. Pesaresi and J. A. Benediktsson, “A new approach for the morphological segmentation of high-resolution satellite imagery,” IEEE Transactions on Geoscience and Remote Sensing 39, pp. 309–320, february 2001. 3. P. Soille, Morphological Image Analysis, Principles and Applications- 2nd edition, Springer, 2003. 4. J. Serra, Image Analysis and Mathematical Morphology, Volume 2: Theoretical Advances, U.K. Academic, 1988. 5. J. A. Benediktsson, M. Pesaresi, and K. Arnason, “Classification and feature extraction for remote sensing images from urban areas based on morphological transformations,” IEEE Transactions on Geoscience and Remote Sensing 41, pp. 1940–1949, September 2003. Proc. of SPIE Vol. 5982 598201-14 6. J. Chanussot, J. A. Benediktsson, and M. Fauvel, “Classification of remote sensing images from urban areas using a fuzzy possibilistic model,” IEEE Transactions on Geoscience and Remote Sensing Letters , accepted for publication. 7. M. Fauvel, J. Chanussot, and J. A. Benediktsson, “Fusion of methods for the classification of remote sensing images from urban areas,” in IEEE Geoscience and Remote Sensing Synposium, 2005. 8. J. A. Benediktsson, J. A. Palmason and J. R. Sveinsson, “Classification of hyperspectral data from urban areas based on extended morphological profiles,” IEEE Transactions on Geoscience and Remote Sensing, vol. 43, no. 3, pp. 480–491, 2005. 9. J. A. Palmason, J. A. Benediktsson and K. Arnason, “Morphological Transformations and Feature Extraction for Urban Data with High Spectral and Spatial Resolution,” Proeedings of IGARSS 2003, Toulouse, France, CD Rom, IEEE Publications 2003. 10. F. Dell’Acqua, P. Gamba, A. Ferrari, J. A. Palmason, J. A. Benediktsson and K. Arnason, “Exploiting Spectral and Spatial Information in Hyperspectral UrbanData with High Resolution” IEEE Geoscience and Remote Sensing Letters, vol. 1, no. 3, pp. 322–326, 2004. 11. K. Fukunaga, Introduction to Statistical Pattern Recognition, 2nd ed, Academic Press, New York, 1990. 12. A. Hyvrinen, J. Karhunen and E. Oja, Independent copmonent analysis, John Wiley and Sons, New York, 2001 13. A. J. Bell and T. J. Sejnowski, Blind separation and blind deconvolution: an information-theoretic approach IEEE ICASSP-95, vol. 5, pp. 3415–3418, 1995. 14. P.K. Varshney and M.K. Arora, Advanced Image Processing Techniques for Remotely Sensed Hyperspectral Data, Springer Verlag, Berlin, 2003. 15. D. A. Landgrebe, Signal Theory Methods in Multispectral Remote Sensing, John Wiley and Sons, Hoboken, New Jersey, 2003. Proc. of SPIE Vol. 5982 598201-15