Many facial-analysis approaches rely on robust and accurate automatic facial landmarking to corre... more Many facial-analysis approaches rely on robust and accurate automatic facial landmarking to correctly function. In this paper, we describe a statistical method for automatic facial-landmark localization. Our landmarking relies on a parsimonious mixture model of Gabor wavelet features, computed in coarse-to-fine fashion and complemented with a shape prior. We assess the accuracy and the robustness of the proposed approach in extensive cross-database conditions conducted on four face data sets (Face Recognition Grand Challenge, Cohn-Kanade, Bosphorus, and BioID). Our method has 99.33% accuracy on the Bosphorus database and 97.62% accuracy on the BioID database on the average, which improves the state of the art. We show that the method is not significantly affected by low-resolution images, small rotations, facial expressions, and natural occlusions such as beard and mustache. We further test the goodness of the landmarks in a facial expression recognition application and report landmarking-induced improvement over baseline on two separate databases for video-based expression recognition (Cohn-Kanade and BU-4DFE).
Our group within the University of Amsterdam participated in the large-scale visual concept detec... more Our group within the University of Amsterdam participated in the large-scale visual concept detection task of ImageCLEF 2009. Our experiments focus on increasing the robustness of the individual concept detectors based on the bag-of-words approach, and less on the hierarchical nature of the concept set used. To increase the robustness of individual concept detectors, our experiments emphasize in particular the role of visual sampling, the value of color invariant features, the influence of codebook construction, and the effectiveness of kernel-based learning parameters. The participation in ImageCLEF 2009 has been successful, resulting in the top ranking for the large-scale visual concept detection task in terms of both EER and AUC. For 40 out of 53 individual concepts, we obtain the best performance of all submissions to this task. For the hierarchical evaluation, which considers the whole hierarchy of concepts instead of single detectors, using the concept likelihoods estimated by our detectors directly works better than scaling these likelihoods based on the class priors.
Color constancy algorithms are often evaluated by using a distance measure that is based on mathe... more Color constancy algorithms are often evaluated by using a distance measure that is based on mathematical principles, such as the angular error. However, it is unknown whether these distance measures correlate to human vision. Therefore, the main goal of our paper is to analyze the correlation between several performance measures and the quality, obtained by using psychophysical experiments, of the output images generated by various color constancy algorithms. Subsequent issues that are addressed are the distribution of performance measures, suggesting additional and alternative information that can be provided to summarize the performance over a large set of images, and the perceptual significance of obtained improvements, i.e., the improvement that should be obtained before the difference becomes noticeable to a human observer.
Our aim is to analyze and evaluate di erent color models to be used for the purpose of 3-D object... more Our aim is to analyze and evaluate di erent color models to be used for the purpose of 3-D object recognition by color-metric histogram matching according to the following criteria: invariance to the geometry of the object and illumination circumstances, high discriminative power, and noise robustness.
International Journal of Computer Vision, Apr 22, 2010
Color is a powerful visual cue in many computer vision applications such as image segmentation an... more Color is a powerful visual cue in many computer vision applications such as image segmentation and object recognition. However, most of the existing color models depend on the imaging conditions that negatively affect the performance of the task at hand. Often, a reflection model (e.g., Lambertian or dichromatic reflectance) is used to derive color invariant models. However, this approach may be too restricted to model real-world scenes in which different reflectance mechanisms can hold simultaneously. Therefore, in this paper, we aim to derive color invariance by learning from color models to obtain diversified color invariant ensembles. First, a photometrical orthogonal and non-redundant color model set is computed composed of both color variants and invariants. Then, the proposed method combines these color models to arrive at a diversified color ensemble yielding a proper balance between invariance (repeatability) and discriminative power (distinctiveness). To achieve this, our fusion method uses a multiview approach to minimize the estimation error. In this way, the proposed method is robust to data uncertainty and produces properly diversified color invariant ensembles. Further, the proposed method is extended to deal with temporal data by predicting the evolution of observations over time.
Proceedings Cvpr Ieee Computer Society Conference on Computer Vision and Pattern Recognition Ieee Computer Society Conference on Computer Vision and Pattern Recognition, Jun 20, 2009
Edge-based color constancy makes use of image derivatives to estimate the illuminant. However, di... more Edge-based color constancy makes use of image derivatives to estimate the illuminant. However, different edge types exist in real-world images such as shadow, geometry, material and highlight edges. These different edge types may have a distinctive inf luence on the performance of the illuminant estimation. Therefore, in this paper, an extensive analysis is provided of different edge types on the performance of edge-based color constancy methods. First, an edge-based taxonomy is presented classifying edge types based on their ref lectance properties (e.g. material, shadow-geometry and highlights). Then, a performance evaluation of edge-based color constancy is provided using these different edge types. From this performance evaluation, it is derived that certain edge types are more valuable than material edges for the estimation of the illuminant. To this end, the weighted Grey-Edge algorithm is proposed in which certain valuable edge types are more emphasized for the estimation of the illuminant.
The user has requested enhancement of the downloaded file. All in-text references underlined in b... more The user has requested enhancement of the downloaded file. All in-text references underlined in blue are added to the original docume and are linked to publications on ResearchGate, letting you access and read them immediately.
In this paper, our goal is to analyze and evaluate various color features to be used for the purp... more In this paper, our goal is to analyze and evaluate various color features to be used for the purpose of image retrieval by color-metric histogram matching independent of varying imaging conditions.
Color constancy aims to compute object colors despite differences in the color of the light sourc... more Color constancy aims to compute object colors despite differences in the color of the light source. Gamut-based approaches are very promising methods to achieve color constancy. In this paper, the gamut mapping approach is extended to incorporate higher-order statistics (derivatives) to estimate the illuminant.
Journal of the Optical Society of America a Optics Image Science and Vision, Oct 1, 2009
Color constancy algorithms are often evaluated by using a distance measure that is based on mathe... more Color constancy algorithms are often evaluated by using a distance measure that is based on mathematical principles, such as the angular error. However, it is unknown whether these distance measures correlate to human vision. Therefore, the main goal of our paper is to analyze the correlation between several performance measures and the quality, obtained by using psychophysical experiments, of the output images generated by various color constancy algorithms. Subsequent issues that are addressed are the distribution of performance measures, suggesting additional and alternative information that can be provided to summarize the performance over a large set of images, and the perceptual significance of obtained improvements, i.e., the improvement that should be obtained before the difference becomes noticeable to a human observer.
Proceedings 2001 International Conference on Image Processing, Feb 1, 2001
In this paper mode filtering of color images is explored. An existing framework based on local hi... more In this paper mode filtering of color images is explored. An existing framework based on local histograms is extended to multi-channel images. Within this framework three color mode operations are proposed; 1. global mode operation for edge sharpening, noise reduction and small object removal, 2. constrained mode operation for white noise filtering while preserving detail, and 3. uncertain data mode filtering to incorporate prior knowledge about the certainty of the measurements into the mode computation. Results obtained for a variety of images indicate the feasibility of color mode filtering.
In this paper, indexing is used as a common framework to represent, index and retrieve images on ... more In this paper, indexing is used as a common framework to represent, index and retrieve images on the basis of color and shape invariants. To evaluate the use of color and shape invariants for the purpose of image retrieval, experiments have been conducted on a database consisting of 500 images of multicolored objects. Images in the database show a considerable amount of noise, specularities, occlusion and fragmentation resulting in a good representation of views from everyday life as it appears in home video and consumer photography in general. The experimental results show that image retrieval based on both color and shape invariants provides excellent retrieval accuracy. Image retrieval based on shape invariants yields poor discriminative power and worst computational performance whereas color based invariants image retrieval provides high discrimination power and best computational performance. Furthermore, the experimental results reveal that identifying multicolored objects entirely on the basis of color invariants is to a large degree robust to partial occlusion and a change in viewing position.
Proceedings of the 4th International Conference on Database and Expert Systems Applications, 1993
In this paper, a method is discussed to store and retrieve images e ciently from an image databas... more In this paper, a method is discussed to store and retrieve images e ciently from an image database on the basis of the data structure called E() representation. The E() representation is a spatial knowledge representation preserving the spatial information between objects embedded in symbolic images as an iconic index for the purpose of e cient image retrieval. The image retrieval method is invariant under, at least, the a ne transformation (i.e. translation, rotation and scale) and is able to deal with substantial object occlusion. A metric is de ned to express similarity between symbolic images. Initial experiments carried out for two applications show that the image retrieval method is very e cient and robust to similarity retrieval in image databases. Together with the inherent high parallelism, it makes the method a promising image retrieval method. keywords: image database, image indexing, similarity retrieval, spatial relations, E() representation, metric, spatial query language.
Many facial-analysis approaches rely on robust and accurate automatic facial landmarking to corre... more Many facial-analysis approaches rely on robust and accurate automatic facial landmarking to correctly function. In this paper, we describe a statistical method for automatic facial-landmark localization. Our landmarking relies on a parsimonious mixture model of Gabor wavelet features, computed in coarse-to-fine fashion and complemented with a shape prior. We assess the accuracy and the robustness of the proposed approach in extensive cross-database conditions conducted on four face data sets (Face Recognition Grand Challenge, Cohn-Kanade, Bosphorus, and BioID). Our method has 99.33% accuracy on the Bosphorus database and 97.62% accuracy on the BioID database on the average, which improves the state of the art. We show that the method is not significantly affected by low-resolution images, small rotations, facial expressions, and natural occlusions such as beard and mustache. We further test the goodness of the landmarks in a facial expression recognition application and report landmarking-induced improvement over baseline on two separate databases for video-based expression recognition (Cohn-Kanade and BU-4DFE).
Our group within the University of Amsterdam participated in the large-scale visual concept detec... more Our group within the University of Amsterdam participated in the large-scale visual concept detection task of ImageCLEF 2009. Our experiments focus on increasing the robustness of the individual concept detectors based on the bag-of-words approach, and less on the hierarchical nature of the concept set used. To increase the robustness of individual concept detectors, our experiments emphasize in particular the role of visual sampling, the value of color invariant features, the influence of codebook construction, and the effectiveness of kernel-based learning parameters. The participation in ImageCLEF 2009 has been successful, resulting in the top ranking for the large-scale visual concept detection task in terms of both EER and AUC. For 40 out of 53 individual concepts, we obtain the best performance of all submissions to this task. For the hierarchical evaluation, which considers the whole hierarchy of concepts instead of single detectors, using the concept likelihoods estimated by our detectors directly works better than scaling these likelihoods based on the class priors.
Color constancy algorithms are often evaluated by using a distance measure that is based on mathe... more Color constancy algorithms are often evaluated by using a distance measure that is based on mathematical principles, such as the angular error. However, it is unknown whether these distance measures correlate to human vision. Therefore, the main goal of our paper is to analyze the correlation between several performance measures and the quality, obtained by using psychophysical experiments, of the output images generated by various color constancy algorithms. Subsequent issues that are addressed are the distribution of performance measures, suggesting additional and alternative information that can be provided to summarize the performance over a large set of images, and the perceptual significance of obtained improvements, i.e., the improvement that should be obtained before the difference becomes noticeable to a human observer.
Our aim is to analyze and evaluate di erent color models to be used for the purpose of 3-D object... more Our aim is to analyze and evaluate di erent color models to be used for the purpose of 3-D object recognition by color-metric histogram matching according to the following criteria: invariance to the geometry of the object and illumination circumstances, high discriminative power, and noise robustness.
International Journal of Computer Vision, Apr 22, 2010
Color is a powerful visual cue in many computer vision applications such as image segmentation an... more Color is a powerful visual cue in many computer vision applications such as image segmentation and object recognition. However, most of the existing color models depend on the imaging conditions that negatively affect the performance of the task at hand. Often, a reflection model (e.g., Lambertian or dichromatic reflectance) is used to derive color invariant models. However, this approach may be too restricted to model real-world scenes in which different reflectance mechanisms can hold simultaneously. Therefore, in this paper, we aim to derive color invariance by learning from color models to obtain diversified color invariant ensembles. First, a photometrical orthogonal and non-redundant color model set is computed composed of both color variants and invariants. Then, the proposed method combines these color models to arrive at a diversified color ensemble yielding a proper balance between invariance (repeatability) and discriminative power (distinctiveness). To achieve this, our fusion method uses a multiview approach to minimize the estimation error. In this way, the proposed method is robust to data uncertainty and produces properly diversified color invariant ensembles. Further, the proposed method is extended to deal with temporal data by predicting the evolution of observations over time.
Proceedings Cvpr Ieee Computer Society Conference on Computer Vision and Pattern Recognition Ieee Computer Society Conference on Computer Vision and Pattern Recognition, Jun 20, 2009
Edge-based color constancy makes use of image derivatives to estimate the illuminant. However, di... more Edge-based color constancy makes use of image derivatives to estimate the illuminant. However, different edge types exist in real-world images such as shadow, geometry, material and highlight edges. These different edge types may have a distinctive inf luence on the performance of the illuminant estimation. Therefore, in this paper, an extensive analysis is provided of different edge types on the performance of edge-based color constancy methods. First, an edge-based taxonomy is presented classifying edge types based on their ref lectance properties (e.g. material, shadow-geometry and highlights). Then, a performance evaluation of edge-based color constancy is provided using these different edge types. From this performance evaluation, it is derived that certain edge types are more valuable than material edges for the estimation of the illuminant. To this end, the weighted Grey-Edge algorithm is proposed in which certain valuable edge types are more emphasized for the estimation of the illuminant.
The user has requested enhancement of the downloaded file. All in-text references underlined in b... more The user has requested enhancement of the downloaded file. All in-text references underlined in blue are added to the original docume and are linked to publications on ResearchGate, letting you access and read them immediately.
In this paper, our goal is to analyze and evaluate various color features to be used for the purp... more In this paper, our goal is to analyze and evaluate various color features to be used for the purpose of image retrieval by color-metric histogram matching independent of varying imaging conditions.
Color constancy aims to compute object colors despite differences in the color of the light sourc... more Color constancy aims to compute object colors despite differences in the color of the light source. Gamut-based approaches are very promising methods to achieve color constancy. In this paper, the gamut mapping approach is extended to incorporate higher-order statistics (derivatives) to estimate the illuminant.
Journal of the Optical Society of America a Optics Image Science and Vision, Oct 1, 2009
Color constancy algorithms are often evaluated by using a distance measure that is based on mathe... more Color constancy algorithms are often evaluated by using a distance measure that is based on mathematical principles, such as the angular error. However, it is unknown whether these distance measures correlate to human vision. Therefore, the main goal of our paper is to analyze the correlation between several performance measures and the quality, obtained by using psychophysical experiments, of the output images generated by various color constancy algorithms. Subsequent issues that are addressed are the distribution of performance measures, suggesting additional and alternative information that can be provided to summarize the performance over a large set of images, and the perceptual significance of obtained improvements, i.e., the improvement that should be obtained before the difference becomes noticeable to a human observer.
Proceedings 2001 International Conference on Image Processing, Feb 1, 2001
In this paper mode filtering of color images is explored. An existing framework based on local hi... more In this paper mode filtering of color images is explored. An existing framework based on local histograms is extended to multi-channel images. Within this framework three color mode operations are proposed; 1. global mode operation for edge sharpening, noise reduction and small object removal, 2. constrained mode operation for white noise filtering while preserving detail, and 3. uncertain data mode filtering to incorporate prior knowledge about the certainty of the measurements into the mode computation. Results obtained for a variety of images indicate the feasibility of color mode filtering.
In this paper, indexing is used as a common framework to represent, index and retrieve images on ... more In this paper, indexing is used as a common framework to represent, index and retrieve images on the basis of color and shape invariants. To evaluate the use of color and shape invariants for the purpose of image retrieval, experiments have been conducted on a database consisting of 500 images of multicolored objects. Images in the database show a considerable amount of noise, specularities, occlusion and fragmentation resulting in a good representation of views from everyday life as it appears in home video and consumer photography in general. The experimental results show that image retrieval based on both color and shape invariants provides excellent retrieval accuracy. Image retrieval based on shape invariants yields poor discriminative power and worst computational performance whereas color based invariants image retrieval provides high discrimination power and best computational performance. Furthermore, the experimental results reveal that identifying multicolored objects entirely on the basis of color invariants is to a large degree robust to partial occlusion and a change in viewing position.
Proceedings of the 4th International Conference on Database and Expert Systems Applications, 1993
In this paper, a method is discussed to store and retrieve images e ciently from an image databas... more In this paper, a method is discussed to store and retrieve images e ciently from an image database on the basis of the data structure called E() representation. The E() representation is a spatial knowledge representation preserving the spatial information between objects embedded in symbolic images as an iconic index for the purpose of e cient image retrieval. The image retrieval method is invariant under, at least, the a ne transformation (i.e. translation, rotation and scale) and is able to deal with substantial object occlusion. A metric is de ned to express similarity between symbolic images. Initial experiments carried out for two applications show that the image retrieval method is very e cient and robust to similarity retrieval in image databases. Together with the inherent high parallelism, it makes the method a promising image retrieval method. keywords: image database, image indexing, similarity retrieval, spatial relations, E() representation, metric, spatial query language.
Uploads
Papers by Theo Gevers