Abstract
Inferences drawn from functional magnetic resonance imaging (fMRI) studies are dependent on the statistical criteria used to define different brain regions as “active” or “inactive” under the experimental manipulation. In fMRI studies of multisensory integration, additional criteria are used to classify a subset of the active brain regions as “multisensory.” Because there is no general agreement in the literature on the optimal criteria for performing this classification, we investigated the effects of seven different multisensory stat-istical criteria on a single test dataset collected as human subjects performed auditory, visual, and auditory-visual object recognition. Activation maps created using the different criteria differed dramatically. The classification of the superior temporal sulcus (STS) was used as a performance measure, because a large body of converging evidence demonstrates that the STS is important for auditory-visual integration. A commonly proposed criterion, “supra-additivity” or “super-additivity”, which requires the multisensory response to be larger than the summed unisensory responses, did not classify STS as multisensory. Alternative criteria, such as requiring the multisensory response to be larger than the maximum or the mean of the unisensory responses, successfully classified STS as multisensory. This practical demonstration strengthens theoretical arguments that the super-additivity is not an appropriate criterion for all studies of multisensory integration. Moreover, the importance of examining evoked fMRI responses, whole brain activation maps, maps from multiple individual subjects, and mixed-effect group maps are discussed in the context of selecting statistical criteria.
Similar content being viewed by others
References
Alais, D. and Burr, D. (2004) The ventriloquist effect results from near-optimal bimodal integration. Curr. Biol. 14, 257–262.
Amedi, A., Jacobson, G., Hendler, T., Malach, R., and Zohary, E. (2002) Convergence of visual and tactile shape processing in the human lateral occipital complex. Cereb. Cortex 12, 1202–1212.
Amedi, A., Malach, R., Hendler, T., Peled, S., and Zohary, E. (2001) Visuo-haptic object-related activation in the ventral visual pathway. Nat. Neurosci. 4: 324–330.
Argall, B. D., Saad, Z. S., and Beauchamp, M. S. A simplified method for intersubject overaging on the cartical surface using SUMA. Human Brain Mapping, in press.
Beauchamp, M. S. (2005) See me, hear me, touch me: multisensory integration in lateral occipital-temparal cortex. Curr, opin. Neurobiol. 15, 145–153.
Beauchamp, M. S., Argall, B. D., Bodurka, J., Duyn, J. H., and Martin, A. (2004a) Unraveling multisensory integration: patchy organization within human STS multisensory cortex. Nat. Neurosci. 7, 1190–1192.
Beauchamp, M. S., Cox, R. W., and DeYoe, E. A. (1997) Graded effects of spatial and featural attention on human area MT and associated motion processing areas. J. Neurophysiol. 77, 516–520.
Beauchamp, M. S., Haxby, J. V., Jennings, J. E., and DeYoe, E. A. (1999) An fMRI version of the Farnsworth-Munsell 100-Hue test reveals multiple color-selective areas in human ventral occipitotemporal cortex. Cereb. Cortex 9, 257–263.
Beauchamp, M. S., Lee, K. E., Argall, B. D., and Martin, A. (2004b) Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 41, 809–823.
Beauchamp, M. S., Lee, K. E., Haxby, J. V., and Martin, A. (2002) Parallel visual motion processing streams for manipulable objects and human movements. Neuron 34, 149–159.
Beauchamp, M. S., Lee, K. E., Haxby, J. V., and Martin, A. (2003) fMRI responses to video and point-light displays of moving humans and manipulable objects. J. Cogn. Neurosci. 15, 991–1001.
Brainard, D. H. (1997) The psychophysics toolbox. Spat. Vis. 10, 433–436.
Bremmer, F., Schlack, A., Shah, N. J., et al. (2001) Polymodal motion processing in posterior parietal and premotor cortex: a human fMRI study strongly implies equivalencies between humans and monkeys. Neuron 29, 287–296.
Calvert, G. A. (2001). Crossmodal processing in the human brain: insights from functional neuroimaging studies. Cereb. Cortex 11, 1110–1123.
Calvert, G. A., Hansen, P. C., Iversen, S. D., and Brammer, M. J. (2001) Detection of audio-visual integration sites in humans by application of electrophysiological criteria to the BOLD effect. NeuroImage 14, 427–438.
Cox, R. W. (1996) AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Comput. Biomed. Res. 29, 162–173.
Dale, A. M. (1999) Optimal experimental design for event-related fMRI. Hum. Brain Mapp. 8, 109–114.
Falchier, A., Clavagnier, S., Barone, P., and Kennedy, H. (2002) Anatomical evidence of multimodal integration in primate striate cortex. J. Neurosci. 22, 5749–5759.
Fischl, B., Sereno, M. I., and Dale, A. M. (1999) Cortical surface-based analysis. II: inflation, flattening, and a surface-based coordinate system. NeuroImage 9, 195–207.
Foxe, J. J., Wylie, G. R., Martinez, A., et al. (2002) Auditory-somatosensory multisensory processing in auditory association cortex: an fMRI study. J. Neurophysiol. 88, 540–543.
Friston, K. J., Holmes, A. P., Price, C. J., Buchel, C., and Worsley, K. J. (1999). Multisubject fMRI studies and conjunction analyses. NeuroImage 10, 385–396.
Haxby, J. V., Ungerleider, L. G., Clark, V. P., Schouten, J. L., Hoffman, E. A., and Martin, A. (1999) The effect of face inversion on activity in human neural systems for face and object perception. Neuron 22, 189–199.
Laurienti, P. J., Burdette, J. H., Wallace, M. T., Yen, Y. F., Field, A. S., and Stein, B. E. (2002) Deactivation of sensory-specific cortex by cross-modal stimuli. J. Cogn. Neurosci. 14, 420–429.
Laurienti, P. J., Perrault Jr., T. J., Stanford, T. R., Wallace, M. T., and Stein, B. E. (2005) On the use of superadditivity as a metric for characterizing multisensory integration in functional neuroimaging studies. Exp. Brain Res, in press.
McCullagh, P. & Nelder, J. A. (1983). Generalized Linear Models, Vol. 37 of Monographs on Statistics and Applied Probability, 1st edition, Chapman and Hall, London.
Mitchell, J. P., Heatherton, T. F., and Macrae, C. N. (2002) Distinct neural systems subserve person and object knowledge. Proc. Natl. Acad. Sci. USA 99, 15238–15243.
Nichols, T., Brett, M., Andersson, J., Wager, T., and Poline, J. B. (2005) Valid conjunction inference with the minimum statistic. NeuroImage 25, 653–660.
Pelli, D. G. (1997) The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spat. Vis. 10, 437–442.
Petit, L. and Beauchamp, M. S. (2003) Neural basis of visually guided head movements studied with fMRI. J. Neurophysiol. 89, 2516–2527.
Raichle, M. E., MacLeod, A. M., Snyder, A. Z., Powers, W. J., Gusnard, D. A., and Shulman, G. L. (2001) A default mode of brain function. Proc. Natl. Acad. Sci. USA 98, 676–682.
Saad, Z. S., Ropella, K. M., DeYoe, E. A., and Bandettini, P. A. (2003) The spatial extent of the BOLD response. NeuroImage 19, 132–144.
Schroeder, C. E., Smiley, J., Fu, K. G., McGinnis, T., O’Connell, M. N., and Hackett, T. A. (2003) Anatomical mechanisms and functional implications of multisensory convergence in early cortical processing. Int. J. Psychophysiol. 50, 5–17.
Talairach, J. and Tournoux, P. (1988) Co-planar stereotaxic atlas of the human brain, Thieme Medical Publishers, New York.
Van Atteveldt, N., Formisano, E., Goebel, R., and Blomert, L. (2004) Integration of letters and speech sounds in the human brain. Neuron 43, 271–282.
Worsley, K. J., and Friston, K. J. (1995) Analysis of fMRI time-series revisited — again. NeuroImage 2, 173–181.
Wright, T. M., Pelphrey, K. A., Allison, T., McKeown, M. J., and McCarthy, G. (2003) Polysensory interactions along lateral temporal regions evoked by audiovisual speech. Cereb. Cortex 13, 1034–1043.
Xiong, J. H., Gao, J. H., Lancaster, J. L., and Fox, P. T. (1995) Clustered pixels analysis for functional MRI activation studies of the human brain. Hum. Brain Mapp. 3, 287–301.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Beauchamp, M.S. Statistical criteria in fMRI studies of multisensory integration. Neuroinform 3, 93–113 (2005). https://doi.org/10.1385/NI:3:2:093
Issue Date:
DOI: https://doi.org/10.1385/NI:3:2:093