Chris Benton

Multi-sensor information fusion aims at extracting and combining useful information from different sensors. This paper addresses the problem of estimating and visualising motion information from Multi-sensor information fusion aims at extracting and combining useful information from different sensors. This paper addresses the problem of estimating and visualising motion information from a pair of visible and infrared cameras, using an optical flow technique. Videos from cameras sensitive to visible light are rich in texture and colour information such that a moving target can readily be positioned. On the other hand, videos from infrared cameras provide extra information which cannot be detected in the visible-light spectrum. In this paper we introduce a stochastic rule for combining optical flow computed from two (or more) sources. We also propose a novel motion-contingent selection method for the fusion of the co-registered visible and infrared video sources.

We present a reliable real-time optical flow estimation framework which can be used in surveillance applications and video analysis. In normal imaging environments, reliability can be achieved by combining an extended optical flow constraint with a smoothing procedure and a masking procedure. In noisy environments, total least squares is adopted to ensure accuracy. The proposed system is able to recover up to 31 frames of dense optical flow per second using a Xeon 3.06GHz workstation, which makes it very useful in a range of surveillance systems that are based on standard PC hardware.

The current paper presents a novel adaptive multiscale scheme to estimate optical flow from image sequences. The scheme models estimation uncertainties which are used to reduce the influence of unreliable intermediate estimates on accuracy. The experimental results show that the proposed method provides more accurate estimates for both small and large motions than a standard multiscale scheme in which an increment is added to an intermediate estimate regardless of estimation certainty.

Nonlinear processing can be used to recover the motion of contrast modulations of binary noise patterns. A nonlinear stage has also been proposed to explain the perception of forward motion in motion sequences which typically elicit reversed-phi. We examined perceived direction of motion for stimuli in which these reversed motion sequences were used to modulate the contrast of binary noise patterns. A percept of forward motion could be elicted by both luminance-defined and contrast-defined stimuli. The perceived direction of motion seen in the contrast-defined stimuli showed a profound carrier dependency. The replacement of a static carrier by a dynamic carrier can reverse the perceived direction of motion. Forward motion was never seen with dynamic carriers. For luminance- and contrast-defined patterns the reversed motion percept increasingly dominated, with increases in the spatial frequency and temporal frequency of the modulation. Differences in the patterns of responses to the two stimuli over spatial and temporal frequency were abolished by the addition of noise to the luminance-defined stimulus. These data suggest the possibility that a single mechanism may mediate the perception of luminance- and contrast-defined motion.

Speed discrimination thresholds were measured for first- and second-order Gaussian bars and edges as a function of speed and the spatial scale of the modulation signal. Discrimination thresholds were generally higher for the second-order patterns when compared with modulations of luminance. There were no systematic effects of variations in the width of the bars and edges. The results are discussed in relation to mechanisms for the explicit recovery of contrast modulations and the influence of the form of the carrier signal on visual performance in second-order motion tasks.

Current computational models of motion processing in the primate motion pathway do not cope well with image sequences in which a moving pattern is superimposed upon a static texture. The use of non-linear operations and the need for contrast normalization in motion models mean that the separation of the influences of moving and static patterns on the motion computation is not trivial. Therefore, the response to the superposition of static and moving patterns provides an important means of testing various computational strategies. Here we describe a computational model of motion processing in the visual cortex, one of the advantages of which is that it is highly resistant to interference from static patterns.

In this study, we show that negative polarity noise patterns appear to have a higher contrast than positive polarity noise patterns with identical expected Fourier amplitude spectra. This demonstrates a failure of contrast constancy over changes in pattern polarity. An examination of local contrast measures shows that negative polarity noise has a wider distribution of local contrast values than positive polarity noise. We propose that the difference in apparent contrast between the two patterns may be based upon spatial non-linearities in the combination of local contrast measures.

Velocity matching using the method of Constant Stimuli shows that perceived velocity varies with contrast [Thompson, P. (1982). Perceived rate of movement depends upon contrast. Vision Research, 22, 377–380]. Random contrast jitter would therefore be expected to increase the slopes of psychometric functions, and thus the velocity discrimination threshold. However, McKee, S., Silverman, G., and Nakayama, K. [(1986) Precise velocity discrimination despite random variation in temporal frequency. Vision Research, 26, 609–620] found no effect of contrast jitter on thresholds, using the method of single stimuli. To determine whether this apparent discrepancy is due to the difference in methodology, or to the different ranges of temporal frequencies used in the two studies, we used the method of single stimuli to measure psychometric functions at three different velocities (0.5, 2.0 and 4.0°/s). We found that contrast jitter increased thresholds at low but not at high velocities. Separate analysis of the psychometric functions at each contrast level showed that increases in contrast increased perceived velocity at low standard speeds (0.5°/s) but not at high. We conclude that the effect of contrast on perceived speed is real, and not a methodological artefact, but that it is found only at low temporal frequencies.

When a static textured background is covered and uncovered by a moving bar of the same mean luminance we can clearly see the motion of the bar. Texture-defined motion provides an example of a naturally occurring second-order motion. Second-order motion sequences defeat standard spatio-temporal energy models of motion perception. It has been proposed that second-order stimuli are analysed by separate systems, operating in parallel with luminance-defined motion processing, which incorporate identifiable pre-processing stages that make second-order patterns visible to standard techniques. However, the proposal of multiple paths to motion analysis remains controversial. Here we describe the behaviour of a model that recovers both luminance-defined and an important class of texture-defined motion. The model also accounts for the induced motion that is seen in some texture-defined motion sequences. We measured the perceived direction and speed of both the contrast envelope and induced motion in the case of a contrast modulation of static noise textures. Significantly, the model predicts the perceived speed of the induced motion seen at second-order texture boundaries. The induced motion investigated here appears distinct from classical induced effects resulting from motion contrast or the movement of a reference frame.

Despite detailed psychophysical, neurophysiological and electrophysiological investigation, the number and nature of independent and parallel motion processing mechanisms in the visual cortex remains controversial. Here we use computational modelling to evaluate evidence from two psychophysical studies collectively thought to demonstrate the existence of three separate and independent motion processing channels. We show that the pattern of psychophysical results can largely be accounted for by a single mechanism. The results demonstrate that a low-level luminance based approach can potentially provide a wider account of human motion processing than generally thought possible.

It has been widely accepted that standard low-level computational approaches to motion processing cannot extract texture-defined motion without applying some pre-processing nonlinearity. This has motivated accounts of motion perception in which luminance- and texture-defined motion are processed by separate mechanisms. Here, we introduce a novel method of image description where motion sequences may be described in terms of their local spatial and temporal gradients. This allows us to assess the local velocity information available to standard low-level motion mechanisms. Our analysis of several texture-motion stimuli shows that the information indicating correct texture-motion velocity and/or direction is present in the raw luminance measures. This raises the possibility that luminance-motion and texture-motion may be processed by the same cortical mechanisms. Our analysis offers a way of looking at texture-motion processing that is, to our knowledge, new and original.

A gradient-based image analysis technique is applied to a class of non-Fourier stimuli. To create the stimuli, n translating sine waves with identical spatial and temporal frequencies, but separated by 2π/n radians, are spatially randomly sampled to produce a Pn stimulus. For n⩾2, the stimuli are non-Fourier. Local image gradients are represented in the form of a gradient plot, a histogram which shows the frequency of ranges of temporal gradient/spatial gradient pairs occurring. It is shown that the gradient plots contain features, oriented in gradient space, which indicate correct non-Fourier velocity. As n increases, so too does the complexity of the gradient plots, a finding which may account for the concomitant decrease in perceived coherent motion [Vision Res 37 (1997) 1459]. This paper demonstrates that the gradient plot and associated velocity plots are a useful way of assessing gradient-based motion information. Compared to the traditional Fourier based approach, gradient-based analysis can lead to different judgement of the motion information available to standard models of low-level motion processing.

A theory of early motion processing in the human and primate visual system is presented which is based on the idea that spatio-temporal retinal image data is represented in primary visual cortex by a truncated 3D Taylor expansion that we refer to as a jet vector. This representation allows all the concepts of differential geometry to be applied to the analysis of visual information processing. We show in particular how the generalised Stokes theorem can be used to move from the calculation of derivatives of image brightness at a point to the calculation of image brightness differences on the boundary of a volume in space-time and how this can be generalised to apply to integrals of products of derivatives. We also provide novel interpretations of the roles of direction selective, bi-directional and pan-directional cells and of type I and type II cells in V5/MT.

Two low-level motion models are applied to a second-order stimulus, a translating contrast modulation of static binary noise. Both models have been used to demonstrate equivalence between energy and gradient algorithms and can be split into a motion-opponent stage followed by a contrast-normalised stage. Analysis of results shows no directional bias at the motion-opponent stage but a strong bias, indicating the correct direction of second-order motion, at the contrast-normalised stage. This demonstrates that the intrinsically non-linear process of contrast-normalisation may play a part in the detection of second-order motion.

We produced morph sequences between identities at a variety of viewpoints, ranging from the three quarter leftward facing view, to the three quarter rightward facing view. We measured the strength of identity adaptation as a function of changing test viewpoint whilst keep the adaptation viewpoint constant, and as a function of adaptation viewpoint whilst keeping test viewpoint constant. Our results show a substantial decrease in adaptation as the angle between adaptation and test viewpoint increases. These findings persisted when we introduced controls for low-level retinotopic adaptation, leading us to conclude that our results show strong evidence for viewpoint dependence in the high-level encoding of facial identity. Our findings support models in which identity is encoded, to a large degree, by viewpoint dependent non-retinotopic neural mechanisms. Functional imaging studies suggest the fusiform gyrus as the most likely location for this mechanism.

We tested the hypothesis that the right cerebral hemisphere contributes to the enhanced body image distortions seen in women when compared to men. Using classical psychophysics, 60 right-handed healthy participants (30 women) were briefly presented with size-distorted pictures of themselves, another person (an experimenter), and a non-corporal, familiar object (a coke bottle) to the central, right, and left visual field. Participants had to decide whether the presented stimulus was fatter or thinner than the real body/object, and thus compare the presented picture with the stored representation of the stimulus from memory. From these data we extracted the amount of image distortion at which participants judged the various stimuli to be veridical. We found that right visual field presentations (initial left hemisphere processing) revealed a general "fatter" bias, which was more evident for bodies than for objects. Crucially, a "fatter" bias with own body presentations in the left visual field (initial right hemisphere processing) was only found for women. Our findings suggest that right visual field presentation results in a general size overestimation, and that this overestimation is more pronounced for bodies than for objects. Moreover, the particular "fatter" bias after own body presentations to the left visual field in women supports the notion of a specific role of the right hemisphere in sex-specific body image distortion.

Prolonged viewing of a face can result in a change of our perception of subsequent faces. This process of adaptation is believed to be functional and to reflect optimization-driven changes in the neural encoding. Because it is believed to target the neural systems underlying face processing, the measurement of face aftereffects is seen as a powerful behavioral technique that can provide deep insights into our facial encoding. Face identity aftereffects have typically been measured by assessing the way in which adaptation changes the perception of images from a test sequence, the latter commonly derived from morphing between two base images. The current study asks to what extent such face aftereffects are driven by the test sequence used to measure them. Using subjects trained to respond either to identity of expression, we examined the effects of identity and expression adaptation on test stimuli that varied in both identity and expression. We found that face adaptation produced measured aftereffects that were congruent with the adaptation stimulus; the composition of the test sequences did not affect the measured direction of the face aftereffects. Our results support the view that face adaptation studies can meaningfully tap into the intrinsically multidimensional nature of our representation of facial identity.

How does an animal conceal itself from visual detection by other animals? This review paper seeks to identify general principles that may apply in this broad area. It considers mechanisms of visual encoding, of grouping and object encoding, and of search. In most cases, the evidence base comes from studies of humans or species whose vision approximates to that of humans. The effort is hampered by a relatively sparse literature on visual function in natural environments and with complex foraging tasks. However, some general constraints emerge as being potentially powerful principles in understanding concealment—a ‘constraint’ here means a set of simplifying assumptions. Strategies that disrupt the unambiguous encoding of discontinuities of intensity (edges), and of other key visual attributes, such as motion, are key here. Similar strategies may also defeat grouping and object-encoding mechanisms. Finally, the paper considers how we may understand the processes of search for complex targets in complex scenes. The aim is to provide a number of pointers towards issues, which may be of assistance in understanding camouflage and concealment, particularly with reference to how visual systems can detect the shape of complex, concealed objects.

Recent studies have shown that reaction times to expressions of anger with averted gaze and fear with direct gaze appear slower than those to direct anger and averted fear. Such findings have been explained by appealing to the notion of gaze/expression congruence with aversion (avoidance) associated with fear, whereas directness (approach) is associated with anger. The current study examined reactions to briefly presented direct and averted faces displaying expressions of fear and anger. Participants were shown four blocked series of faces; each block contained an equal mix of two facial expressions (neutral plus either fear or anger) presented at one viewpoint (either full face or three quarter leftward facing). Participants were instructed to make rapid responses classifying the expressions as either neutral or expressive. Initial analysis of reaction time distributions showed differences in distribution shape with reactions to averted anger and direct fear showing greater skew than those to direct anger and averted fear. Computational modelling, using a diffusion model of decision making and reaction time, showed a difference in the rate of information accrual with more rapid rates of accrual when viewpoint and expression were congruent. This analysis supports the notion of signal congruence as a mechanism through which gaze and viewpoint affect our responses to facial expressions.

Interacting with a dynamic environment calls for close coordination between the timing and direction of motor behaviors. Accurate motor behavior requires the system to predict where the target for action will be, both when action planning is complete and when the action is executed. In the current study, we investigate the time course of velocity information accrual in the period leading up to a saccade toward a moving object. In two experiments, observers were asked to generate saccades to one of two moving targets. Experiment 1 looks at the accuracy of saccades to targets that have trial-by-trial variations in velocity. We show that the pattern of errors in saccade landing position is best explained by proposing that trial-by-trial target velocity is taken into account in saccade planning. In Experiment 2, target velocity stepped up or down after a variable interval after the movement cue. The extent to which the movement endpoint reflects pre- or post-step velocity can be used to identify the temporal velocity integration window; we show that the system takes a temporally blurred snapshot of target velocity centered ∼200 ms before saccade onset. This estimate is used to generate a dynamically updated prediction of the target's likely future location.

A growing number of studies in vision research employ analyses of how perturbations in visual stimuli influence behavior on single trials. Recently, we have developed a method along such lines to assess the time course over which object velocity information is extracted on a trial-by-trial basis in order to produce an accurate intercepting saccade to a moving target. Here, we present a simplified version of this methodology, and use it to investigate how changes in stimulus contrast affect the temporal velocity integration window used when generating saccades to moving targets. Observers generated saccades to one of two moving targets which were presented at high (80%) or low (7.5%) contrast. In 50% of trials, target velocity stepped up or down after a variable interval after the saccadic go signal. The extent to which the saccade endpoint can be accounted for as a weighted combination of the pre- or post-step velocities allows for identification of the temporal velocity integration window. Our results show that the temporal integration window takes longer to peak in the low when compared to high contrast condition. By enabling the assessment of how information such as changes in velocity can be used in the programming of a saccadic eye movement on single trials, this study describes and tests a novel methodology with which to look at the internal processing mechanisms that transform sensory visual inputs into oculomotor outputs.

Evidence suggests that underlying the human system processing facial expressions are two types of representation of expression: one dependent on identity and the other independent of identity. We recently presented findings indicating that identity-dependent representations are encoded using a prototype-referenced scheme, in a manner notably similar to that proposed for facial identity. Could it be that identity-independent representations are encoded this way too? We investigated this by adapting participant to anti-expressions and asking them to categorize the expression aftereffect in a prototype probe that was either the same (congruent) or different (incongruent) identity to that of the adapter. To distinguish between encoding schemes, we measured how aftereffect magnitude changed in response to variations in the strength of adapters. The increase in aftereffect magnitude with adapter strength characteristic of prototype-referenced encoding was observed in both congruent and, crucially, incongruent conditions. We conclude that identity-independent representations of expression are indeed encoded using a prototype-referenced scheme. The striking similarity between the encoding of facial identity and both representations of expression raises the possibility that prototype-referenced encoding might be a common scheme for encoding the many types of information in faces needed to enable our complex social interactions.

We used visual search to explore whether the preattentive mechanisms that enable rapid detection of facial expressions are driven by visual information from the displacement of features in expressions, or other factors such as affect. We measured search slopes for luminance and contrast equated images of facial expressions and anti-expressions of six emotions (anger, fear, disgust, surprise, happiness, and sadness). Anti-expressions have an equivalent magnitude of facial feature displacements to their corresponding expressions, but different affective content. There was a strong correlation between these search slopes and the magnitude of feature displacements in expressions and anti-expressions, indicating feature displacement had an effect on search performance. There were significant differences between search slopes for expressions and anti-expressions of happiness, sadness, anger, and surprise, which could not be explained in terms of feature differences, suggesting preattentive mechanisms were sensitive to other factors. A categorization task confirmed that the affective content of expressions and anti-expressions of each of these emotions were different, suggesting signals of affect might well have been influencing attention and search performance. Our results support a picture in which preattentive mechanisms may be driven by factors at a number of levels, including affect and the magnitude of feature displacement. We note that indirect effects of feature displacement, such as changes in local contrast, may well affect preattentive processing. These are most likely to be nonlinearly related to feature displacement and are, we argue, an important consideration for any study using images of expression to explore how affect guides attention. We also note that indirect effects of feature displacement (for example, changes in local contrast) may well affect preattentive processing. We argue that such effects are an important consideration for any study using images of expression to explore how affect guides attention.

Our visual representation of facial expression is examined in this study: is this representation built from edge information, or does it incorporate surface-based information? To answer this question, photographic negation of grey-scale images is used. Negation preserves edge information whilst disrupting the surface-based information. In two experiments visual aftereffects produced by prolonged viewing of images of facial expressions were measured. This adaptation-based technique allows a behavioural assessment of the characteristics encoded by the neural systems underlying our representation of facial expression. The experiments show that photographic negation of the adapting images results in a profound decrease of expression aftereffect. Our visual representation of facial expression therefore appears to not just be built from edge information, but to also incorporate surface information. The latter allows an appreciation of the 3-D structure of the expressing face that, it is argued, may underpin the subtlety and range of our non-verbal facial communication.

Adaptation is a powerful experimental technique that has recently provided insights into how people encode representations of facial identity. Here, we used this approach to explore the visual representation of facial expressions of emotion. Participants were adapted to anti-expressions of six facial expressions. The participants were then shown an average face and asked to classify the face’s expression using one of six basic emotion descriptors. Participants chose the emotion matching the anti-expression they were adapted to significantly more often than they chose any other emotion (e.g., if they were adapted to antifear, they classified the emotion on the average face as fear). The strength of this aftereffect of adaptation decreased as the strength of the anti-expression adapter decreased. These findings provide evidence that visual representations of facial expressions of emotion are coded with reference to a prototype within a multidimensional framework.

Edges are fundamental properties of our environment and the objects we interact with. There is a lack of research on the haptic perception of edges, especially the sharpness of an edge. Skinner et al. [2013 PLoS ONE, 8(9): e73283] found that haptic discriminability of sharpness was clearly superior when using a relatively unrestrained, free exploration strategy compared with a static single touch strategy. In the free exploration condition two distinct movement patterns were frequently used by participants: a proximal-distal movement of the fingerpad across the test edge and a medial-lateral movement of the fingerpad along the test edge. Here, using the same stimuli and two-alternative forced-choice method of constant stimuli as Skinner et al. (2013), we demonstrate that a proximal-distal movement results in substantially lower sharpness discrimination thresholds than a medial-lateral movement. The underlying neurophysiology and implications for the design of haptic displays are considered.