Alexander Toet received his PhD in physics from the University of Utrecht in the Netherlands in 1987, where he worked on visual spatial localization (hyperacuity) and image processing. His is currently a senior research scientist at TNO (Soesterberg, The Netherlands) where he investigates multimodal image fusion, image quality, computational models of human visual search and detection, and the quantification of visual target distinctness. He also investigates crossmodal perceptual interactions between the visual, auditory, olfactory and tactile senses, with the aim to deploy these interactions to tune the perceived affective quality of both virtual and real environments. He is currently affiliated with the Faculty of Social and Behavioral Sciences of the University of Utrecht where he researches the perception of multimodal imagery and the emotional bias induced by affective visual backgrounds. He has published over 100 papers in refereed journals, 115 papers in refereed conference proceedings, was co-editor of a book on the mathematical description of shape in images, was Guest Editor of Special Issue of Optical Engineering on “Advances in Target Acquisition Modeling II”, and is currently Area Editor of the ISIF Journal of Advances in Information Fusion. He organized and directed 3 international workshops on Search and Target Acquisition, Combinatorial algorithms for Military Applications, and on the mathematical description of shape in images. He is also Fellow of The International Society for Optical Engineering (SPIE) and Senior Member of The Institute of Electrical & Electronics Engineers (IEEE).
We have tested whether natural dynamic backgrounds affect perceived facial dominance. Facial eval... more We have tested whether natural dynamic backgrounds affect perceived facial dominance. Facial evaluation is based on just two fundamental dimensions of facial appearance: valence and dominance. Perceived facial valence has been shown to be biased towards background valence. However, it is currently unknown, if the perception of facial dominance is also context dependent. In this study, participants rated the perceived dominance of neutral faces superimposed on everyday dynamic backgrounds that were either classified as low (weak) or high (strong) in dominance. Neutral faces were perceived as significantly more dominant when seen on a strong dynamic background than on either a weak or neutral background. Thus, background dominance enhances perceived facial dominance. Since dynamic textures are ubiquitous this finding is relevant for the design and experience of both our daily environment and multimedia content.
<div class="eventBoxedText"> Post-Meeting Proceedings Manuscript Due Date: 8 Marc... more <div class="eventBoxedText"> Post-Meeting Proceedings Manuscript Due Date: 8 March 2010 </div> <br/><br/>This conference is focused upon "Situation Awareness (SA)" for vehicle navigation under poor.
The fusion and enhancement of multiband nighttime imagery for surveillance and navigation has bee... more The fusion and enhancement of multiband nighttime imagery for surveillance and navigation has been the subject of extensive research for over two decades. Despite the ongoing efforts in this area there is still only a small number of static multiband test images available for the development and evaluation of new image fusion and enhancement methods. Moreover, dynamic multiband imagery is also currently lacking. To fill this gap we present the TRICLOBS dynamic multi-band image data set containing sixteen registered visual (0.4–0.7μm), near-infrared (NIR, 0.7–1.0μm) and long-wave infrared (LWIR, 8–14μm) motion sequences. They represent different military and civilian surveillance scenarios registered in three different scenes. Scenes include (military and civilian) people that are stationary, walking or running, or carrying various objects. Vehicles, foliage, and buildings or other man-made structures are also included in the scenes. This data set is primarily intended for the development and evaluation of image fusion, enhancement and color mapping algorithms for short-range surveillance applications. The imagery was collected during several field trials with our newly developed TRICLOBS (TRI-band Color Low-light OBServation) all-day all-weather surveillance system. This system registers a scene in the Visual, NIR and LWIR part of the electromagnetic spectrum using three optically aligned sensors (two digital image intensifiers and an uncooled long-wave infrared microbolometer). The three sensor signals are mapped to three individual RGB color channels, digitized, and stored as uncompressed RGB (false) color frames. The TRICLOBS data set enables the development and evaluation of (both static and dynamic) image fusion, enhancement and color mapping algorithms. To allow the development of realistic color remapping procedures, the data set also contains color photographs of each of the three scenes. The color statistics derived from these photographs can be used to define color mappings that give the multi-band imagery a realistic color appearance.
Cognitive biases and heuristics are pervasive simplifications and distortions in judgement and re... more Cognitive biases and heuristics are pervasive simplifications and distortions in judgement and reasoning that systematically affect human decision making. Knowledge in this area may enable us to foresee and reduce detrimental effects of biases or to influence others more effectively. We therefore performed a literature study to assess the influence of personal characteristics (cognitive abilities, expertise, personality, cultural background) on the occurrence of cognitive biases. We found that each of the aforementioned factors can affect cognitive biases, though not much is known about the effects of culture. Also, factors that appear to reduce a cognitive bias may in fact mitigate (suppress or override) its behavioral effect rather than preventing the bias from occurring at all. The general picture that arises is that bias susceptibility and the occurrence of biases depend on thinking style (heuristic versus deliberate), where thinking style is associated with an individual " s personal characteristics. In general, biases are reduced when a deliberate (analytical) thinking style is applied. However, whether a specific (heuristic or deliberate) thinking style actually reduces or enhances a given type of bias also depends on the context.
We propose a multi-scale image fusion scheme based on guided filtering. Guided filtering can effe... more We propose a multi-scale image fusion scheme based on guided filtering. Guided filtering can effectively reduce noise while preserving detail boundaries. When applied in an iterative mode, guided filtering selectively eliminates small scale details while restoring larger scale edges. The proposed multi-scale image fusion scheme achieves spatial consistency by using guided filtering both at the decomposition and at the recombination stage of the multi-scale fusion process. First, size-selective iterative guided filtering is applied to decompose the source images into approximation and residual layers at multiple spatial scales. Then, frequency-tuned filtering is used to compute saliency maps at successive spatial scales. Next, at each spatial scale binary weighting maps are obtained as the pixelwise maximum of corresponding source saliency maps. Guided filtering of the binary weighting maps with their corresponding source images as guidance images serves to reduce noise and to restore spatial consistency. The final fused image is obtained as the weighted recombination of the individual residual layers and the mean of the approximation layers at the coarsest spatial scale. Application to multiband visual (intensified) and thermal infrared imagery demonstrates that the proposed method obtains state-of-the-art performance for the fusion of multispectral nightvision images. The method has a simple implementation and is computationally efficient. INTRODUCTION The increasing deployment and availability of co-registered multimodal imagery from different types of sensors has spurred the development of image fusion techniques. The information provided by different sensors registering the same scene can either be (partially) redundant or complementary and may be corrupted with noise. Effective combinations of complementary and partially redundant multispectral imagery can therefore visualize information that is not directly evident from the individual input images. For instance, in nighttime (low-light) outdoor surveillance applications, intensified visual (II) or near-infrared (NIR) imagery often provides a detailed but noisy representation of a scene. While different types of noise may result from several processes associated with the underlying sensor physics, additive noise is typically the predominant noise component encountered in II and NIR imagery (Petrovic & Xydeas, 2003). Additive noise can be modelled as a random signal that is simply added to the original signal. As a result, additive noise may obscure or distort relevant image details. In addition, targets of interest like persons or cars How to cite this article Toet (2016), Iterative guided image fusion. PeerJ Comput. Sci. 2:e80; DOI 10.7717/peerj-cs.80
We measured the effects of sound and visual dynamic elements on user experience of a serious game... more We measured the effects of sound and visual dynamic elements on user experience of a serious game, with special interest in engagement and arousal. Engagement was measured through questionnaires and arousal through the SAM and electromyography (EMG). We adopted the EMG of the corrugator (frown muscle) and the zygomatic muscle (smile muscle) as indicators for arousal and valence. We hypothesized that sound and dynamic elements would increase engagement, while user characteristics would determine the amount of arousal. We find that the addition of dynamic elements to the game increases the user experience. Sound increases the ease of navigation, but does not determine user experience, probably due to the dominance of other game elements and user characteristics. The subjective evaluation is inconsistent with the physiological data on arousal. Hence, further research is required to elucidate the relation between arousal and engagement.
Abstract: This report results from a contract tasking TNO Human Factors Institute as follows: The... more Abstract: This report results from a contract tasking TNO Human Factors Institute as follows: The contractor will analyze low-light visible, uncooled long-wave infrared, and laser radar imagery showing vehicles and men deployed in military relevant nighttime scenarios, both in the open and in the trees, during nighttime conditions. The contractor will perform assess the relevant features in multiple image modalities. An optimal image fusion scheme will then be developed which combines the relevant features in a single stereo image.
We have tested a prototype dual-band NVG system consisting of two NVGs fitted with filters that s... more We have tested a prototype dual-band NVG system consisting of two NVGs fitted with filters that split the NVG sensitive range into a short (visual) and a long wavelength (NIR) band. The Color-the-night technique (see Hogervorst & Toet, SPIE D&S'08) was used to fuse the images of the two sensors. We designed a color scheme especially optimized for the detection of camouflaged targets. The added value of this system was evaluated in an experiment in which observers had to detect targets (green and blue tubes).
The increasing availability and deployment of imaging sensors operating in multiple spectral band... more The increasing availability and deployment of imaging sensors operating in multiple spectral bands has led to a requirement for methods that combine the signals from these sensors in an effective and ergonomic way for presentation to the human operator. Effective combinations of complementary and partially redundant multispectral imagery can provide information that is not directly evident from the individual input images.
Abstract. It is well known that navigation through VEs may have a negative effect on the physical... more Abstract. It is well known that navigation through VEs may have a negative effect on the physical well-being of observers, by inducing cybersickness. In this study we investigated if cybersickness interacts with the appraisal of a simulated environment. Using the Half-Life 2 level editor we created a VE representing fictitious rural and urban areas. This environment was displayed by means of back projection onto a semi-transparent (frosted) screen, and was viewed from 1 m distance.
Night vision goggles have been in use for many years and limitations in their use have been well ... more Night vision goggles have been in use for many years and limitations in their use have been well studied through training research and flight experience. However, advances in technology have led to improvements in NVG display capabilities and in some cases helmet mounted display (HMD) technology has begun replacing NVG systems.
Iodine-based compound semiconductors may allow one to build a portable gamma-ray spectrometer wit... more Iodine-based compound semiconductors may allow one to build a portable gamma-ray spectrometer with improved efficiency and energy resolution compared to many other portable spectrometer devices. Iodine-based semiconductors have a wide band gap that allows these detectors to operate without any cooling mechanism.
NASA's Synthetic Vision Systems (SVS) project is developing technologies with practical applicati... more NASA's Synthetic Vision Systems (SVS) project is developing technologies with practical applications to eliminate low visibility conditions as a causal factor to civil aircraft accidents while replicating the operational benefits of clear day flight operations, regardless of the actual outside visibility condition.
The objective of this research was to determine if there was an improvement in human observer per... more The objective of this research was to determine if there was an improvement in human observer performance, identifying potential weapons or threat objects, when imagery is presented in three dimensions instead of two dimensions. Also it was desired to quantify this potential improvement in performance by evaluating the change in N50 cycle criteria, for this task and target set.
ABSTRACT We present the design and first test results of the TRICLOBS (TRI-band Color Low-light O... more ABSTRACT We present the design and first test results of the TRICLOBS (TRI-band Color Low-light OBServation) system The TRICLOBS is an all-day all-weather surveillance and navigation tool. Its sensor suite consists of two digital image intensifiers (Photonis ICU's) and an uncooled longwave infrared microbolometer (XenICS Gobi 384). The night vision sensor suite registers the visual (400-700 nm), the near-infrared (700-1000 nm) and the longwave infrared (8-14 µm) bands of the electromagnetic spectrum.
Abstract: TNO has completed the construction of the TRICLOBS 3-band night vision system. This new... more Abstract: TNO has completed the construction of the TRICLOBS 3-band night vision system. This newly developed system consists of two digital image intensifiers (Photonis ICUs) and an uncooled longwave infrared microbolometer (XenICS Gobi 384). This night vision sensor suite registers the visual (400-700 nm), the near-infrared (700-1000 nm) and the longwave infrared bands of the electromagnetic spectrum.
Abstract–Modern electro-optical imaging sensor suites are designed to expand the conditions under... more Abstract–Modern electro-optical imaging sensor suites are designed to expand the conditions under which humans can operate. The sensors that constitute a suite typically provide non-literal images (ie images that do not resemble a photograph taken of the same scene). Examples are long-and short-wave infrared, laser radar, and synthetic aperture radar systems. The different sensors typically produce complementary information.
We have tested whether natural dynamic backgrounds affect perceived facial dominance. Facial eval... more We have tested whether natural dynamic backgrounds affect perceived facial dominance. Facial evaluation is based on just two fundamental dimensions of facial appearance: valence and dominance. Perceived facial valence has been shown to be biased towards background valence. However, it is currently unknown, if the perception of facial dominance is also context dependent. In this study, participants rated the perceived dominance of neutral faces superimposed on everyday dynamic backgrounds that were either classified as low (weak) or high (strong) in dominance. Neutral faces were perceived as significantly more dominant when seen on a strong dynamic background than on either a weak or neutral background. Thus, background dominance enhances perceived facial dominance. Since dynamic textures are ubiquitous this finding is relevant for the design and experience of both our daily environment and multimedia content.
<div class="eventBoxedText"> Post-Meeting Proceedings Manuscript Due Date: 8 Marc... more <div class="eventBoxedText"> Post-Meeting Proceedings Manuscript Due Date: 8 March 2010 </div> <br/><br/>This conference is focused upon "Situation Awareness (SA)" for vehicle navigation under poor.
The fusion and enhancement of multiband nighttime imagery for surveillance and navigation has bee... more The fusion and enhancement of multiband nighttime imagery for surveillance and navigation has been the subject of extensive research for over two decades. Despite the ongoing efforts in this area there is still only a small number of static multiband test images available for the development and evaluation of new image fusion and enhancement methods. Moreover, dynamic multiband imagery is also currently lacking. To fill this gap we present the TRICLOBS dynamic multi-band image data set containing sixteen registered visual (0.4–0.7μm), near-infrared (NIR, 0.7–1.0μm) and long-wave infrared (LWIR, 8–14μm) motion sequences. They represent different military and civilian surveillance scenarios registered in three different scenes. Scenes include (military and civilian) people that are stationary, walking or running, or carrying various objects. Vehicles, foliage, and buildings or other man-made structures are also included in the scenes. This data set is primarily intended for the development and evaluation of image fusion, enhancement and color mapping algorithms for short-range surveillance applications. The imagery was collected during several field trials with our newly developed TRICLOBS (TRI-band Color Low-light OBServation) all-day all-weather surveillance system. This system registers a scene in the Visual, NIR and LWIR part of the electromagnetic spectrum using three optically aligned sensors (two digital image intensifiers and an uncooled long-wave infrared microbolometer). The three sensor signals are mapped to three individual RGB color channels, digitized, and stored as uncompressed RGB (false) color frames. The TRICLOBS data set enables the development and evaluation of (both static and dynamic) image fusion, enhancement and color mapping algorithms. To allow the development of realistic color remapping procedures, the data set also contains color photographs of each of the three scenes. The color statistics derived from these photographs can be used to define color mappings that give the multi-band imagery a realistic color appearance.
Cognitive biases and heuristics are pervasive simplifications and distortions in judgement and re... more Cognitive biases and heuristics are pervasive simplifications and distortions in judgement and reasoning that systematically affect human decision making. Knowledge in this area may enable us to foresee and reduce detrimental effects of biases or to influence others more effectively. We therefore performed a literature study to assess the influence of personal characteristics (cognitive abilities, expertise, personality, cultural background) on the occurrence of cognitive biases. We found that each of the aforementioned factors can affect cognitive biases, though not much is known about the effects of culture. Also, factors that appear to reduce a cognitive bias may in fact mitigate (suppress or override) its behavioral effect rather than preventing the bias from occurring at all. The general picture that arises is that bias susceptibility and the occurrence of biases depend on thinking style (heuristic versus deliberate), where thinking style is associated with an individual " s personal characteristics. In general, biases are reduced when a deliberate (analytical) thinking style is applied. However, whether a specific (heuristic or deliberate) thinking style actually reduces or enhances a given type of bias also depends on the context.
We propose a multi-scale image fusion scheme based on guided filtering. Guided filtering can effe... more We propose a multi-scale image fusion scheme based on guided filtering. Guided filtering can effectively reduce noise while preserving detail boundaries. When applied in an iterative mode, guided filtering selectively eliminates small scale details while restoring larger scale edges. The proposed multi-scale image fusion scheme achieves spatial consistency by using guided filtering both at the decomposition and at the recombination stage of the multi-scale fusion process. First, size-selective iterative guided filtering is applied to decompose the source images into approximation and residual layers at multiple spatial scales. Then, frequency-tuned filtering is used to compute saliency maps at successive spatial scales. Next, at each spatial scale binary weighting maps are obtained as the pixelwise maximum of corresponding source saliency maps. Guided filtering of the binary weighting maps with their corresponding source images as guidance images serves to reduce noise and to restore spatial consistency. The final fused image is obtained as the weighted recombination of the individual residual layers and the mean of the approximation layers at the coarsest spatial scale. Application to multiband visual (intensified) and thermal infrared imagery demonstrates that the proposed method obtains state-of-the-art performance for the fusion of multispectral nightvision images. The method has a simple implementation and is computationally efficient. INTRODUCTION The increasing deployment and availability of co-registered multimodal imagery from different types of sensors has spurred the development of image fusion techniques. The information provided by different sensors registering the same scene can either be (partially) redundant or complementary and may be corrupted with noise. Effective combinations of complementary and partially redundant multispectral imagery can therefore visualize information that is not directly evident from the individual input images. For instance, in nighttime (low-light) outdoor surveillance applications, intensified visual (II) or near-infrared (NIR) imagery often provides a detailed but noisy representation of a scene. While different types of noise may result from several processes associated with the underlying sensor physics, additive noise is typically the predominant noise component encountered in II and NIR imagery (Petrovic & Xydeas, 2003). Additive noise can be modelled as a random signal that is simply added to the original signal. As a result, additive noise may obscure or distort relevant image details. In addition, targets of interest like persons or cars How to cite this article Toet (2016), Iterative guided image fusion. PeerJ Comput. Sci. 2:e80; DOI 10.7717/peerj-cs.80
We measured the effects of sound and visual dynamic elements on user experience of a serious game... more We measured the effects of sound and visual dynamic elements on user experience of a serious game, with special interest in engagement and arousal. Engagement was measured through questionnaires and arousal through the SAM and electromyography (EMG). We adopted the EMG of the corrugator (frown muscle) and the zygomatic muscle (smile muscle) as indicators for arousal and valence. We hypothesized that sound and dynamic elements would increase engagement, while user characteristics would determine the amount of arousal. We find that the addition of dynamic elements to the game increases the user experience. Sound increases the ease of navigation, but does not determine user experience, probably due to the dominance of other game elements and user characteristics. The subjective evaluation is inconsistent with the physiological data on arousal. Hence, further research is required to elucidate the relation between arousal and engagement.
Abstract: This report results from a contract tasking TNO Human Factors Institute as follows: The... more Abstract: This report results from a contract tasking TNO Human Factors Institute as follows: The contractor will analyze low-light visible, uncooled long-wave infrared, and laser radar imagery showing vehicles and men deployed in military relevant nighttime scenarios, both in the open and in the trees, during nighttime conditions. The contractor will perform assess the relevant features in multiple image modalities. An optimal image fusion scheme will then be developed which combines the relevant features in a single stereo image.
We have tested a prototype dual-band NVG system consisting of two NVGs fitted with filters that s... more We have tested a prototype dual-band NVG system consisting of two NVGs fitted with filters that split the NVG sensitive range into a short (visual) and a long wavelength (NIR) band. The Color-the-night technique (see Hogervorst & Toet, SPIE D&S'08) was used to fuse the images of the two sensors. We designed a color scheme especially optimized for the detection of camouflaged targets. The added value of this system was evaluated in an experiment in which observers had to detect targets (green and blue tubes).
The increasing availability and deployment of imaging sensors operating in multiple spectral band... more The increasing availability and deployment of imaging sensors operating in multiple spectral bands has led to a requirement for methods that combine the signals from these sensors in an effective and ergonomic way for presentation to the human operator. Effective combinations of complementary and partially redundant multispectral imagery can provide information that is not directly evident from the individual input images.
Abstract. It is well known that navigation through VEs may have a negative effect on the physical... more Abstract. It is well known that navigation through VEs may have a negative effect on the physical well-being of observers, by inducing cybersickness. In this study we investigated if cybersickness interacts with the appraisal of a simulated environment. Using the Half-Life 2 level editor we created a VE representing fictitious rural and urban areas. This environment was displayed by means of back projection onto a semi-transparent (frosted) screen, and was viewed from 1 m distance.
Night vision goggles have been in use for many years and limitations in their use have been well ... more Night vision goggles have been in use for many years and limitations in their use have been well studied through training research and flight experience. However, advances in technology have led to improvements in NVG display capabilities and in some cases helmet mounted display (HMD) technology has begun replacing NVG systems.
Iodine-based compound semiconductors may allow one to build a portable gamma-ray spectrometer wit... more Iodine-based compound semiconductors may allow one to build a portable gamma-ray spectrometer with improved efficiency and energy resolution compared to many other portable spectrometer devices. Iodine-based semiconductors have a wide band gap that allows these detectors to operate without any cooling mechanism.
NASA's Synthetic Vision Systems (SVS) project is developing technologies with practical applicati... more NASA's Synthetic Vision Systems (SVS) project is developing technologies with practical applications to eliminate low visibility conditions as a causal factor to civil aircraft accidents while replicating the operational benefits of clear day flight operations, regardless of the actual outside visibility condition.
The objective of this research was to determine if there was an improvement in human observer per... more The objective of this research was to determine if there was an improvement in human observer performance, identifying potential weapons or threat objects, when imagery is presented in three dimensions instead of two dimensions. Also it was desired to quantify this potential improvement in performance by evaluating the change in N50 cycle criteria, for this task and target set.
ABSTRACT We present the design and first test results of the TRICLOBS (TRI-band Color Low-light O... more ABSTRACT We present the design and first test results of the TRICLOBS (TRI-band Color Low-light OBServation) system The TRICLOBS is an all-day all-weather surveillance and navigation tool. Its sensor suite consists of two digital image intensifiers (Photonis ICU's) and an uncooled longwave infrared microbolometer (XenICS Gobi 384). The night vision sensor suite registers the visual (400-700 nm), the near-infrared (700-1000 nm) and the longwave infrared (8-14 µm) bands of the electromagnetic spectrum.
Abstract: TNO has completed the construction of the TRICLOBS 3-band night vision system. This new... more Abstract: TNO has completed the construction of the TRICLOBS 3-band night vision system. This newly developed system consists of two digital image intensifiers (Photonis ICUs) and an uncooled longwave infrared microbolometer (XenICS Gobi 384). This night vision sensor suite registers the visual (400-700 nm), the near-infrared (700-1000 nm) and the longwave infrared bands of the electromagnetic spectrum.
Abstract–Modern electro-optical imaging sensor suites are designed to expand the conditions under... more Abstract–Modern electro-optical imaging sensor suites are designed to expand the conditions under which humans can operate. The sensors that constitute a suite typically provide non-literal images (ie images that do not resemble a photograph taken of the same scene). Examples are long-and short-wave infrared, laser radar, and synthetic aperture radar systems. The different sensors typically produce complementary information.
Uploads
Papers by Alexander Toet