Mining EEG-FMRI Using Independent Component Analysis

International Journal of Psychophysiology 73 (2009) 53–61 Contents lists available at ScienceDirect International Journal of Psychophysiology j o u r n a l h o m e p a g e : w w w. e l s ev i e r. c o m / l o c a t e / i j p s yc h o Mining EEG–fMRI using independent component analysis Tom Eichele a,⁎, Vince D. Calhoun b,c, Stefan Debener d a Department of Biological and Medical Psychology, University of Bergen, 5009 Bergen, Norway The Mind Research Network, Albuquerque, New Mexico, United States c Department of ECE, University of New Mexico, Albuquerque, New Mexico, United States d Biomagnetic Center, Dept. of Neurology, University Hospital Jena, Erlanger Allee 101, D-07747 Jena, Germany b a r t i c l e i n f o Article history: Received 8 September 2008 Received in revised form 26 November 2008 Accepted 23 December 2008 Available online 15 February 2009 Keywords: ICA PCA EEG ERP fMRI Single trial analysis Group analysis a b s t r a c t Independent component analysis (ICA) is a multivariate approach that has become increasingly popular for analyzing brain imaging data. In contrast to the widely used general linear model (GLM) that requires the user to parameterize the brain's response to stimuli, ICA allows the researcher to explore the factors that constitute the data and alleviates the need for explicit spatial and temporal priors about the responses. In this paper, we introduce ICA for hemodynamic (fMRI) and electrophysiological (EEG) data processing, and one of the possible extensions to the population level that is available for both data types. We then selectively review some work employing ICA for the decomposition of EEG and fMRI data to facilitate the integration of the two modalities to provide an overview of what is available and for which purposes ICA has been used. An optimized method for symmetric EEG-fMRI decomposition is proposed and the outstanding challenges in multimodal integration are discussed. © 2009 Elsevier B.V. All rights reserved. 1. Introduction Since its inception in 1992 (Frahm et al., 1992; Kwong et al., 1992; Ogawa et al., 1992) functional magnetic resonance imaging has become a major tool to study human brain function (Amaro and Barker, 2006; Bandettini et al., 2000; Huettel et al., 2004). This was propelled by the non-invasiveness, excellent spatial resolution, flexible experiment design and repeatability. The signal in fMRI is based on changes in magnetic susceptibility of the blood during brain activation, it does not directly reflect neuro-electric activity in the brain. The signal is based upon the complex coupling of neuronal activity, metabolic activity and blood flow parameters in the brain (Heeger and Ress, 2002; Logothetis and Wandell, 2004; Raichle and Mintun, 2006). The hemodynamic response is delayed and smoothed relative to the neuronal activity, and it is usually not possible to reconstruct the neuro-electric process from the hemodynamic process. Nevertheless, the hemodynamic signal remains a very informative surrogate for neuronal activity. Moreover, what fMRI alone lacks in temporal resolution and direct relationships with neuro-electric activity could be compensated by the combination with concurrent EEG measurements (Debener et al., 2006; Eichele et al., 2005). A wide variety of studies have been performed to delineate the neuronal correlates of cognitive functions (for a review see e.g. Cabeza and Nyberg, 2000), use activity patterns to predict perception and behavior, identify aberrant regional activation in clinical populations, and many more. Inferences made from fMRI data rely most often on pre-specified temporal models, i.e. the correlation between a predicted time course that models the activation to stimuli/responses and the measured data. However, in many studies the temporal dynamics of event related or intrinsic regional activations are difficult to predict due to the lack of a well-understood brain-activation model. In contrast, independent component analysis (ICA) is an exploratory, data-driven tool that can reveal inter-subject and inter-event differences in the temporal dynamics of the fMRI signal without a prior model. ICA is increasingly utilized as a tool for evaluating the hidden spatio-temporal structure contained within electrophysiological and hemodynamic brain imaging data. The strength of ICA is its ability to reveal function-relevant dynamics for which a temporal model cannot be specified a priori (Calhoun and Adali, 2006; Eichele et al., 2008b), or is not available, such as in resting state data (Damoiseaux et al., 2006; Fox and Raichle, 2007). Below, we will introduce an ICA generative model and the extension of ICA to group data. We will review how the method has been used in fMRI, EEG and concurrent EEG-fMRI research, and identify some of the outstanding challenges for future work. 2. Independent component analysis ⁎ Corresponding author. Tel.: +47 45224919; fax: +47 555 89872. E-mail address: tom.eichele@psybp.uib.no (T. Eichele). 0167-8760/$ – see front matter © 2009 Elsevier B.V. All rights reserved. doi:10.1016/j.ijpsycho.2008.12.018 ICA is a multivariate statistical method used to uncover hidden sources from multiple data channels (e.g., electrodes, microphones, 54 T. Eichele et al. / International Journal of Psychophysiology 73 (2009) 53–61 images) such that these sources are maximally independent. Typically, it assumes a generative model where observations are assumed to be linear mixtures of statistically independent sources. Unlike principal component analysis (PCA), which decorrelates the data, ICA includes higher-order statistics to achieve independence. An intuitive example of ICA can be given by a scatter-plot of two independent signals s1 and s2. Fig. 1a (left, middle) show the projections for PCA and ICA, respectively, for a linear mixture of s1 and s2 and Fig. 1a shows a plot of the two independent signals (s1 and s2) in a scatter-plot. PCA finds the orthogonal vectors a1,a2, but cannot identify the independent vectors. In contrast, ICA is able to find the independent vectors of the linearly mixed signals (s1,s2), and is thus able to restore the original sources. A typical ICA model assumes that the recorded signal consists of statistically independent and non-Gaussian sources that are linearly mixed. Consider an observed M–dimensional random vector denoted by x = [x1,x2,⋯,xM]T: x = As ð1Þ where s = [s1,s2,⋯,sN ]T is an N-dimensional vector whose elements are the random variables that refer to the independent sources and AM × N is an unknown mixing matrix. Typically M ≥ N, so that A is usually of full rank. The goal of ICA is to estimate an unmixing matrix WN × M such that y given by y = Wx ð2Þ is a good approximation to the ‘true’ sources s. Two commonly used ICA algorithms for EEG and fMRI data derived within these formulations are Infomax (Bell and Sejnowski, 1995) and FastICA (Hyvärinen et al., 2001). Both Infomax and FastICA typically work with a fixed nonlinearity or one that is selected from a small set, e.g., two in the case of extended Infomax (Lee et al., 1999). These algorithms work well for symmetric distributions and are less accurate for skewed distributions and for sources close to Gaussian. An intuitive introduction into ICA is provided by Stone (Stone, 2004), and a detailed mathematical description by Hyvärinen (Hyvärinen et al., 2001). 3. ICA of EEG data The initial application of single-subject temporal ICA to EEG data was introduced by Makeig using multichannel event related potentials (Makeig et al., 1997). Since then, ICA of EEG signals has become popular for a wide user community, facilitated through the opensource toolbox EEGLAB (www.sccn.eeglab.edu). The use of ICA for multi-channel EEG recordings has been reviewed (Onton et al., 2006), and a conceptual framework for using ICA for the study of eventrelated brain dynamics has been proposed by Makeig and colleagues (Makeig et al., 2004b). The popularity of ICA is in large part due to two key features. First, it is a powerful way to remove artefacts from EEG data (Jung et al., 2000a,b), and second, it helps to disentangle otherwise mixed brain signals (Makeig et al., 2002). One of the most common artefacts typically identified by ICA is shown in Fig. 2. Eye blinks can often be easily identified by their characteristic topography and time course. Also very common in EEG recordings are ICs reflecting lateral eye movements, and muscle activity (EMG). ICA from high-density EEG recordings typically reveals a number of these components. Beyond artifacts, various examples have been published demonstrating the potential of ICA for the separation of event-related brain activity patterns (Debener et al., 2005a,b; Delorme et al., 2007; Makeig et al., 2002; Onton et al., 2005). These studies have combined ICA with single-trial EEG analysis, thereby allowing exploration of brain dynamics beyond the evoked fraction of the signal that is preserved in the ERP. Typically, the inspection of non-artefact ICs suggests that some can be specifically linked to stimulus processing and explain condition-specific variance, which affords functional inferences about event-related component activity (for a review, see Onton and Makeig, 2006). 4. ICA of fMRI data Following its first application to fMRI (McKeown et al., 1998), ICA has been successfully utilized in a number of fMRI studies, especially in those that have proven challenging to analyze with the standard regression-type approaches (Calhoun and Adali, 2006; McKeown et al., 2003). Spatial ICA of fMRI finds systematically non-overlapping, temporally coherent brain regions without constraining the shape of the temporal response. Note that ICA can be used to discover either spatially or temporally independent components (Stone, 2004). Due to the data structure, in which the number of observed volume elements (voxels) by far exceeds the number of observed time points, most applications to fMRI use the former approach and seek components that are maximally independent in space. The aim of ICA is then to factor a two dimensional data matrix (voxels-by-timepoints) into a product of a set of time courses and a set of independent spatial patterns. The choice of spatial or temporal independence has been somewhat controversial, although these are just two possible modeling assumptions. McKeown et al. (1998) argued that the sparse distributed nature of the spatial pattern for typical cognitive activation paradigms would work well with spatial ICA (sICA). Furthermore, since the proto-typical image artifacts are also sparse and localized, e.g., vascular pulsation, CSF flow signals in the ventricles, or breathing induced motion (signal localized to strong tissue contrast near discontinuities), the Infomax algorithm with a sparse prior is very well suited for spatial analysis, and has also been used for temporal ICA as have decorrelation-based algorithms (Calhoun et al., 2001; Petersen et al., 2000). Stone et al. (1999) proposed a method which attempts to maximize both spatial and temporal independence. An interesting Fig. 1. Scatterplot of two independent, mixed signals illustrates the need for higher order statistics as an alternative to orthogonal projection as a means to faithfully un-mix the data. T. Eichele et al. / International Journal of Psychophysiology 73 (2009) 53–61 55 Fig. 2. Illustration of artefact removal from EEG data by means of ICA. A Section of selected channels from a multi-channel EEG recording is shown, with ongoing EEG oscillations in the alpha range evident at occipital electrodes and two eye blinks at fronto-polar channels. B Un-mixing of the EEG data into a set of independent components. Each component can be described on the basis of a spatial pattern (map) and a time course (activation). C Back-projection of all but components 1 and 2 reveals artefact-corrected EEG data. combination of spatial and temporal ICA was presented by Seifritz et al. (2002). They used an initial sICA to reduce the spatial dimensionality of the data by locating a region of interest in which they then subsequently performed temporal ICA to study in more detail the structure of the response in the human auditory cortex. 5. Group ICA models for EEG and fMRI Unlike univariate methods such as the general linear model (GLM), ICA does not naturally generalize to a method suitable for drawing inferences about observations from multiple subjects. For example, when using the GLM, the investigator specifies a fixed set of regressors, and so drawing inferences about group data comes naturally, since all individuals in the group share the same regressors. In ICA, by contrast, different individuals in the group will have different time courses and component maps. Even if components are very similar across subjects they require a subsequent clustering effort to be grouped together. Accordingly, it is not immediately clear how to draw inferences about group data by using single-subject ICA. To overcome this problem, several multi-subject ICA approaches have been proposed (Beckmann and Smith, 2005; Calhoun et al., 2001; Esposito et al., 2005; Guo and Pagnoni, 2008; Lukic et al., 2002; Schmithorst and Holland, 2004; Svensen et al., 2002). The various approaches differ in terms of how the data are preprocessed and organized prior to the ICA analysis and what types of outputs are generated. One possibility is to perform single-subject ICA and then attempt to combine the output into a group post hoc by using a clustering metric or simple correlation of the components (Calhoun et al., 2001; Esposito et al., 2005). This has the advantage of allowing for unique spatial and temporal features, but has the disadvantage that the components are not necessarily unmixed in the same way for each subject. Other approaches involve ICA computed on condensed group data. The advantage of these approaches is that they perform one single ICA on the group data, in which subject-specific and condition-specific contributions can be identified. Note that a temporal concatenation approach allows for unique time courses for each subject, but assumes common group maps. A spatial concatenation approach on the other hand allows for unique maps but assumes common time courses. It appears that temporal concatenation works better for fMRI data (Guo and Pagnoni, 2008; Schmithorst and Holland, 2004). Temporal concatenation is implemented in MELODIC (http://www.fmrib.ox.ac. uk/fsl/) and in GIFT (http://icatb.sourceforge.net/). GIFT additionally implements a back-projection step which returns subject specific maps and timecourses for further analysis (Calhoun et al., 2001; Eichele et al., 2008a). A detailed comparison of several group ICA approaches including temporal concatenation and tensor ICA is provided in recent papers (Guo and Pagnoni, 2008; Schmithorst and Holland, 2004). While group spatial ICA is more prominent in fMRI, it has been more typical in EEG research to employ single-subject temporal ICA and cluster individual component sets along features of interest for inferences about groups of subjects (Debener et al., 2005a,b; Onton 56 T. Eichele et al. / International Journal of Psychophysiology 73 (2009) 53–61 et al., 2005, 2006). In the case of group temporal ICA on EEG time domain data, the spatial concatenation, aggregation and data reduction with PCA precedes component estimation, enabling the identification of components that contribute to event-related potentials (Eichele et al., 2008a), and is implemented in EEGIFT (http://icatb.sourceforge.net/). A similar group ICA model in combination with partial least squares was proposed recently by Kovacevic (Kovacevic and McIntosh, 2007). The drawback however is that processes which are not well time-locked across subjects, such as ongoing EEG rhythms, cannot be satisfyingly reconstructed in this model. The accuracy of component detection and back-reconstruction with such a group ICA model depends on the degree of intra- and inter-individual latency jitter of event related EEG processes (see also Moosmann et al., 2008). We focus on the group ICA approach implemented in the GIFT software, which is freely available and has been designed for analysis of EEG and fMRI data. GIFT uses multiple data reduction steps following data concatenation, and enables a statistical comparison of individual maps and time courses. GIFT essentially estimates a mixing matrix which has partitions that are unique to each subject. Once the mixing matrix is estimated, the component maps for each subject can be computed by projecting the single subject data onto the inverse of the partition of the unmixing matrix that corresponds to that subject. 1 Mathematically, if we let Xi = F− Yi be the L × V reduced data matrix i from subject i, where Yi is the K × V data matrix containing the 1 preprocessed fMRI or EEG data, F− is the L × K reducing matrix i determined by the PCA decomposition, V is the number of fMRI voxels or EEG timepoints, K is the number of fMRI time points or EEG channels and L is the size of the dimension following reduction. The reduced data from all subjects is concatenated into a matrix and reduced using PCA to N dimensions the number of components to be estimated. The N × V reduced, concatenated matrix for the M subjects is X=G 2 − 14 3 F−1 1 Y1 5: v F−1 Y M M ð3Þ where G− 1 is an N × M reducing matrix (also determined by a PCA decomposition) and is multiplied on the right by the LM × V concatenated data matrix for the M subjects. Following ICA estimation, we can write X =  Ŝ, where  is the N × N mixing matrix and Ŝ is the N × V component fMRI map or EEG timecourse. Substituting this expression for X into Eq. (3) and multiplying both sides by G, we have 2 G Ŝ = 4 3 F−1 1 Y1 5: v −1 FM YM ð4Þ Partitioning the matrix G by subject provides the following expression 3 2 −1 3 G1 F1 Y1 4 v 5 Ŝ = 4 5: v GM F−1 Y M M 2 ð5Þ We then write the equation for subject i by working only with the elements in partition i of the above matrices such that −1 Gi  Ŝi = Fi Yi : ð6Þ The matrix Ŝi in Eq. (6) contains the single subject maps for subject i and is calculated from the following equation  −1 −1 Fi Yi : Ŝi = Gi  ð7Þ We now multiply both sides of Eq. (6) by Fi and write Yi ≈Fi Gi  Ŝi ; ð8Þ which provides the ICA decomposition of the data from subject i, contained in the matrix Yi. The N × V matrix Sî contains the N source maps in fMRI and timecourses in the EEG modality and the K × N matrix FiGi  is the single subject mixing matrix and contains the fMRI component time course or the EEG component topography for each of the N components Fig. 3. 6. Application of ICA to concurrent EEG-fMRI recordings It has become popular to collect multiple types of imaging and other (e.g. genetic) data from the same participants, often in settings where relatively large groups are sampled. Each imaging method informs on a limited domain and typically provides both common and unique information about the problem in question. Approaches for combining or fusing data in brain imaging can be conceptualized as having a place on an analytic spectrum with meta-analysis (highly distilled data) to examine convergent evidence at one end and highly detailed large-scale computational modeling at the other end (Husain et al., 2002). In between are methods that attempt to perform a direct data fusion (Horwitz and Poeppel, 2002). One promising data fusion approach is to first process each image type and extract features from different modalities. These features are then examined for relationships among the data types at the group level. This approach allows us to take advantage of the ‘cross’-information among data types and when performing multimodal fusion provides a natural link among different data types (Ardnt, 1996; Savopol and Armenakis, 2002). Here we focus selectively on ‘integration by prediction’, where typically some feature from the EEG (e.g. alpha power, P300 amplitude) is convolved with a canonical hemodynamic response function and used as a predictor of hemodynamic activity in a GLM. Integration-byprediction is based on the assumption that the hemodynamic response is linearly related to local changes in neuronal activity, in particular local field potentials (Heeger and Ress, 2002; Lauritzen and Gold, 2003; Logothetis et al., 2001). Since large-scale synchronous field potentials are the basis for the scalp EEG (Nunez, 1995), such integration can be achieved by investigating correlations between BOLD and scalp EEG. This can be done either continuously over time, as in the study of background rhythms and epileptic discharges in the EEG (for a review see Laufs et al., 2008), or in the context of inducing variation in some cognitive operation (Debener et al., 2006, 2005; Eichele et al., 2005). ICA has been used in this context on many levels: for artifact reduction during preprocessing, for feature extraction from the EEG, decomposition of fMRI, parallel decomposition of EEG and fMRI, and joint ICA of multimodal data. 6.1. EEG artifact reduction Besides using ICA for feature extraction and making inferences, it has been successfully employed during pre-processing for denoising of EEG data acquired in the scanner. Temporal ICA is very useful for artifact reduction in EEG data (Jung et al., 2000a,b), and it is similarly useful in pre-processing of simultaneously acquired EEG–fMRI data. Principally, the same artifacts affect the in-scanner EEG recording as outside the scanner. In addition, in-scanner EEG recordings suffer from further artifacts, most prominently the pulse artifact. A number of authors have used ICA for removing the pulse artifact, given that template-based rejection algorithms (Allen et al., 2000, 1998) do not always sufficiently control for this type of artifact. ICA has been shown to be helpful in reducing the pulse artifact only at lower field strengths (Debener et al., 2008, 2007; Mantini et al., 2007a,b). Since this artifact seems to violate the spatial stationarity assumption inherent in temporal ICA, other existing tools such as optimal basis sets (OBS) T. Eichele et al. / International Journal of Psychophysiology 73 (2009) 53–61 57 Fig. 3. Prototypical independent components from fMRI data. The figure shows the activation maps of nine independent components from an event related fMRI study (Eichele et al., 2008b) rendered onto the MNI template at representative transverse slices. The maps are shown in neurological convention (left hemisphere is on the left). Activations are plotted in red, deactivations in blue. To the left of each map, the hemodynamic response functions within the respective ICs as estimated via deconvolution from 1–20 s after stimulus onset, in arbitrary (range-scaled) amplitude units are displayed. The group average from the 13 participants is plotted as a solid line, error bars indicate ±1 S.E.M., dotted lines represent all individual estimates. The empirical HRFs were used to estimate single trial amplitudes in the fMRI data. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.) might be more advantageous (Niazy et al., 2005). It is our experience that ICA returns better unmixing results when used subsequent to pulse artifact correction with OBS or template-based rejection algorithms (Debener et al., 2007). 6.2. EEG feature extraction Another powerful option is to use ICA to identify and select a brain signal of interest, such as the power modulation of the posterior alpha rhythm (Feige et al., 2005), frontal theta (Scheeringa et al., 2008), or single trial variability in the error-related negativity (ERN, Debener et al., 2005). ICA here provides an unmixed and denoised source waveform, and the assumption is that this improves the correlation between EEG and fMRI signals. This approach assumes that unmixed data allows a better matching of specific EEG timecourses to their underlying spatial sources in the fMRI. An intuitive example is activity in the 8–12 Hz range, and a common ICA observation often ignored in the literature is that this frequency range is occupied by at least posterior alpha and central mu sources. These rhythms are regionally and functionally separable, and it is easily conceivable that sampling alpha activity from single or grouped posterior electrodes in which all these activities mix provides a less clear measure of any of these subprocesses than decomposition and selection of one feature (Feige et al., 2005). Similar considerations apply to other rhythms, and to event related activity. For example, the isolation of an ERN-like topography and time course separates the variability associated with response-locked error-related processing from stimulus-locked and background activity (Debener et al., 2005). In other circumstances where it is difficult to define a feature of interest it seems more appropriate to employ ICA for artifact reduction only and choose the back-projected, mixed EEG rather than the sources for correlation analysis with the fMRI (Eichele et al., 2005). The results from both method choices may provide comparable results (Bagshaw and Warbrick, 2007). However, one caveat is that temporal independence in the EEG does not necessarily equate with different regional generators of the activity in the fMRI, such that a variety of scenarios are conceivable in which unmixing of the EEG could reduce sensitivity of the EEG–fMRI correlation. Also, although the mixing problem is acknowledged and addressed in the EEG, this is not extended in these publications to the treatment of fMRI data, where it exists as well. 6.3. fMRI decomposition While the work cited above used ICA to decompose EEG data for a better integration with the fMRI BOLD signal, more recently alternative approaches have been used. Here, ICA is applied to address 58 T. Eichele et al. / International Journal of Psychophysiology 73 (2009) 53–61 the spatio-temporal mixing problem that is inherent in fMRI data as well, while the EEG signals were not subjected to ICA. For instance, Mantini et al. (2009, 2007b) decomposed resting state and event related fMRI data with fastICA and subsequent component clustering across subjects and extracted distributed regional networks. In the resting state data (Mantini et al., 2007b), the BOLD signals of these networks were correlated with power fluctuations in the delta, theta, alpha, beta, and gamma bands from concurrently recorded EEG. Each network was characterized by a signature that involved variable contributions from the different bands. The interesting idea of this work is the perspective that wide-band EEG activity principally describes large-scale brain networks more comprehensively than narrow-band activity. In the event related data (Mantini et al., 2009), the amplitude fluctuation of the P300 revealed correlations with attentional networks. The approach illustrates a means of how to meaningfully reduce the multiple testing problems in fMRI without defining regions of interest a priori. However, again, only one modality was unmixed. In this case the EEG remained mixed. can use the convolved trial-by-trial modulation in EEG components to predict the fMRI, or (vice versa) employ deconvolution and single trial estimation on the fMRI component timecourses, and use these as predictors for the trial-by-trial amplitude modulation in the EEG. Parallel group ICA provides a means to disentangle and visualize large scale networks both in their spatial and temporal form, given coherent neuronal sources jointly express scalp electrophysiologic and hemodynamic features (Calhoun and Adali, 2006; Debener et al., 2006; Makeig et al., 2004a; McKeown et al., 2003; Onton et al., 2006). 6.5. Joint ICA for multimodal data Joint ICA (jICA) is another approach, which enables the joint decomposition of multi-modal data that have been collected from the same sample of subjects. Here, we primarily consider a set of extracted features from each subject's data, and these data form the multiple observations. Given two (or more) sets of group data, XF and XG we concatenate the two datasets side-by-side to form XJ and write the likelihood as 6.4. Parallel EEG-fMRI decomposition The two approaches described above each solve a part of the mixing problem and make way for refined spatio-temporal mapping. However, the choice to unmix only one modality but not the other is somewhat inconsistent with the reasoning that leads researchers to use ICA in the first place. We now lay out how parallel unmixing could be used for EEG-fMRI integration. The motivation for implementing parallel EEG-fMRI decomposition is based on the data structure and the presumed physiology of signal generation in EEG and fMRI. First, consider that a concurrent experiment generates a multidimensional dataset consisting of about 104 to 105 volume elements by 102 to 103 timepoints sampled in the fMRI, by 102 to 103 timepoints by 102 to 103 trials by 32–128 scalp channels sampled in the EEG, by a number of participants that constitute a group study, providing a rich and complex source of information. The utility of ICA lies in the visualization and exploratory assessment of such data. Second, we can assume that processing of simple stimuli and tasks produces widespread event-related responses that are spatially and temporally mixed across the brain (Baudena et al., 1995; Halgren et al., 1995a,b; Halgren and Marinkovic, 1995; Makeig et al., 2004a; Onton et al., 2006). The scalp EEG samples a spatially degraded version of brain activity, where the potential is a mixture of independent timecourses from large-scale synchronous field potentials. Similarly, the neurovascular transformation of the electrophysiological activity into hemodynamics provides temporally degraded and spatially mixed signals across the fMRI volume (Calhoun and Adali, 2006; McKeown et al., 2003). The spatial and temporal mixing in both modalities creates situations in which prediction of fMRI activity by EEG features in a mass–univariate voxel-by-voxel framework may be difficult since neither the predictor, nor the response variables are any likely to represent single and coupled sources of variability. Thus, one improvement to achieve a more symmetric treatment of the data is to unmix both modalities in parallel. We have developed such an analysis framework for multisubject data that employs Infomax ICA (Bell and Sejnowski, 1995) to recover a set of statistically independent maps from the fMRI, and independent time-courses from the EEG separately (Eichele et al., 2008a). We utilized the approach that is described above, i.e. create aggregate data containing observations from all subjects, estimate a single set of ICs and then back-reconstruct these in the individual data (Calhoun et al., 2001; Schmithorst and Holland, 2004). The analysis is done on the single trial level rather than averaged data (cf. Calhoun et al., 2006), and it does not assume a joint mixing matrix for both modalities (cf. Calhoun et al., 2006; cf. Moosmann et al., 2008). The components are matched across modalities by correlating their trial-to-trial modulation, where we LðWÞ = N V Y Y n=1 v=1   pJ;n uJ;v ; ð9Þ where uJ = WxJ. Here, we use the notation in terms of random variables such that each entry in the vectors uJ and xJ correspond to a random variable, which is replaced by the observation for each sample n = 1,…,N as rows of matrices UJ and XJ. When posed as a maximum likelihood problem, we estimate a joint unmixing matrix W such that the likelihood L(W) is maximized. Let the two datasets XF and XG have dimensionality N × V1 and N × V2, then we have LðWÞ = N Y n=1 V1 Y v=1 ! V2  Y   pF;n uF;v pG;n uG;v ; ð10Þ v=1 Depending on the data types in question, the above formula can be made more or less flexible. This formulation characterizes the basic jICA approach and assumes that the sources associated with the two data types modulate the same way across N subjects. Note that the assumption of the same linear covariation for both modalities is fairly strong. However, it has the advantage of providing a parsimonious way to link multiple data types and has been demonstrated in a variety of cases with meaningful results, for example for fusing ERPs with fMRI contrast images (Calhoun, Adali et al., 2006). The extension of joint ICA for concurrent single trial data is described in a simulation study by Moosmann and colleagues (Moosmann et al., 2008). Although robust results can be acquired with this methodology, broader validity of the application to real single trial data would require an iterative procedure with estimation and removal of the hemodynamic transfer function prior to fusion (see below). 7. Challenges The methodological and conceptual development in the field of multimodal integration is ongoing, and ICA plays a prominent role in this effort. With respect to EEG-fMRI integration we will highlight two issues that are currently being addressed. First, we discuss the utility of hemodynamic deconvolution and single trial estimation in the fMRI, and second, the need for more flexible modeling of the EEG-fMRI coupling. 7.1. Deconvolution An outstanding problem that was not addressed in the work described above is the assumptions made about the hemodynamic T. Eichele et al. / International Journal of Psychophysiology 73 (2009) 53–61 response function (HRF). Most previous concurrent EEG-fMRI studies assumed a generic transfer HRF to the transient neuronal response evoked by discrete sensory stimulation (e.g. derived from Boynton et al., 1996) to model the hemodynamic activation, whereby the parameters of the HRF such as shape and latency of the peak and undershoot are implicitly assumed to be fixed across the entire brain and across subjects. However, quite some variability of the HRF exists across regions and subjects (Aguirre et al., 1998; Glover, 1999; Handwerker et al., 2004). Inclusion of temporal and dispersion derivative terms of the HRF into the model can alleviate such differences on the first level. However, it ignores the potential for amplitude bias induced by model mismatches due to variable hemodynamic delays and is typically not helpful for 2nd level random effects analyses (Calhoun et al., 2004). Therefore, in order to achieve a more sensitive and less biased analysis it is more desirable to estimate subject- and region-specific HRFs. Another consideration is that the eliciting conditions that generate a response which conforms to the canonical HRF are not necessarily valid for intrinsic/resting state activity and also more complex cognitive operations. An illustrative example of estimating the HRF from EEG-data has been published by de Munck et al. (2008, 2007). Here, HRFs were directly estimated from the alpha power modulation in concurrent EEG–fMRI data, and indeed show systematic latency and shape differences from the canonical HRF. The regional activation with significant HRFs was much more widespread than what 59 was previously reported, which can be taken as an indication that the sparser activation patterns in earlier studies (Goldman et al., 2002; Moosmann et al., 2003) could have resulted from poor sensitivity. Apart from estimating the HRF from the EEG, event related HRFs can be estimated from independent component timecourses through deconvolution (Eichele et al., 2008b). While typically noise-tolerant models are needed for voxel-by-voxel deconvolution, ICA forms large regions of interest with quasi-denoised time courses such that deconvolution can be done simply by forming a convolution matrix of the stimulus onsets with an assumed duration of the HRF. FMRI single trial response amplitudes can then be recovered by fitting a design matrix (X) containing separate predictors for the onset times of each trial convolved with the estimated HRF onto the IC timecourse, estimating the scaling coefficients (β) in the multiple linear regression model y = β·X + ε using least squares. In this approach, the HRF is estimated and then effectively removed from the hemodynamic data, leaving just the amplitude modulation across trials, which has the same resolution as the EEG trial-by-trial dynamics (see Fig. 4). 7.2. Integration There are a variety of ways how one can conceive the coupling between electrophysiology and hemodynamic signals. Linear regression Fig. 4. EEG–fMRI integration with deconvolution. The spatial ICA of the fMRI data results in individual maps and timecourses. The single trial HRF amplitude modulation estimated from the IC timecourses is used for prediction of EEG activity. In order to recover the amplitude modulation (AM), the pseudoinverse of a convolution matrix generated from the stimulus timing and an assumed HRF length is multiplied with the IC timecourse, yielding individually and regionally specific HRFs. These HRFs are then convolved separately with each stimulus onset, yielding a design matrix (X) with predictors for each trial 1..n. The regression of the design onto the IC timecourse (y) yields the single trial amplitude modulation for this IC (β1.. βn). In the EEG, a group temporal ICA provides independent source timecourses, whose trial-by-trial modulation are extracted and correlated with the fMRI activity. 60 T. Eichele et al. / International Journal of Psychophysiology 73 (2009) 53–61 between fMRI and EEG is usually used in combination with ICA to investigate links between modalities (e.g., Debener et al., 2006). In searching for such one-to-one mappings it is assumed that a particular EEG feature is related to a particular fMRI network. Although this is justifiable since both the fMRI maps and EEG topographies are different from each other and stationary, it appears oversimplified since in principle several fMRI networks can affect several EEG signals. One explanation might be that one component is a generator, i.e. it has an open field configuration, is reasonably large and close to the cortex surface to generate a scalp potential, while other components transiently and remotely modulate this source, while being electrophysiologically silent on the scalp, since these otherwise should induce topographic variability in the EEG. This explanation already implies that the fMRI networks are coupled with each other, which is reflected in their functional connectivity (Eichele et al., 2008b; Jafri et al., 2008), and effective connectivity (Stevens et al., 2007). Another related explanation is that the fMRI networks might be considered spatially independent nodes in regionally distributed, functionally coherent source networks, such that many nodes can contribute directly or indirectly to the scalp potentials in a given paradigm/task. Here, the EEG components, although temporally independent on the scalp, would not represent a single source but still a weighted average of multiple spatially independent regional sources. Both these explanations are physiologically plausible (Baudena et al., 1995; Halgren et al., 1995a,b; Halgren and Marinkovic,1995), and although it is typically not the aim of integrationby-prediction to directly solve the inverse problem, it is helpful to consider in which way the topographies and timecourses relate to each other. If we assume that many fMRI networks affect a particular EEG feature one way of addressing this is by employing multiple regression with a design that contains all fMRI trial-by-trial modulations for prediction of EEG activity. The problem of collinearity between regressors can be addressed by orthogonalization (Andrade et al., 1999), stepwise selection, or a decomposition of the fMRI component timecourses into set of unrelated factors by means of e.g. PCA or ICA (similar to the procedure in Seifritz et al., 2002). However, such a treatment also means that the interpretation of the results becomes less straightforward. Other options include canonical correlation analysis (CCA) in order to treat the problem. Here, however, 2nd level inference is less well defined. Also, a joint (temporal) ICA of the sIC and tIC trial-by-trial modulations appears feasible (Calhoun et al., 2006; Moosmann et al., 2008), but has as yet not been explored in detail. A conceptually more advanced approach that, however, needs detailed specifications would be to adapt dynamic causal models that explain the EEG/ERP responses as changes in the effective connectivity between independent sources (Friston, 2005; Friston et al., 2003; Garrido et al., 2007). 8. Conclusion ICA is a powerful data driven approach that has been successfully used to analyze EEG, fMRI, and simultaneous EEG-fMRI data. It has been shown repeatedly that the integrations of both modalities can be achieved on the (statistical) source level, as provided by ICA. The overview provided here demonstrates the utility and diversity of the various existing ICA-based approaches for the analysis of brain imaging data. Two key challenges in the field, the estimation of the HRF and a more flexible way of data integration, were identified. Multimodal integration by means of ICA is still under development. However, we expect ICA to contribute to more refined ways of achieving comprehensive joint mapping of electrophysiology and hemodynamic signals. Acknowledgment This work was supported by a grant from the L. Meltzer University Fund (801616) to TE, and by the National Institutes of Health, under Grants 1 R01 EB 000840, 1 R01 EB 005846, and 1 R01 EB 006841 to VDC. References Aguirre, G.K., Zarahn, E., D'Esposito, M., 1998. The variability of human, BOLD hemodynamic responses. Neuroimage 8, 360–369. Allen, P.J., Polizzi, G., Krakow, K., Fish, D.R., Lemieux, L., 1998. Identification of EEG events in the MR scanner: the problem of pulse artifact and a method for its subtraction. Neuroimage 8, 229–239. Allen, P.J., Josephs, O., Turner, R., 2000. A method for removing imaging artifact from continuous EEG recorded during functional MRI. Neuroimage 12, 230–239. Amaro Jr., E., Barker, G.J., 2006. Study design in fMRI: basic principles. Brain Cogn. 60, 220–232. Andrade, A., Paradis, A.L., Rouquette, S., Poline, J.B., 1999. Ambiguous results in functional neuroimaging data analysis due to covariate correlation. Neuroimage 10, 483–486. Ardnt, C., 1996. Information Gained by Data Fusion. SPIE Proc., vol. 2784. Bagshaw, A.P., Warbrick, T., 2007. Single trial variability of EEG and fMRI responses to visual stimuli. Neuroimage 38, 280–292. Bandettini, P.A., Birn, R.M., Donahue, K.M., 2000. Functional MRI. Background, methodology, limits and implementation. In: Cacioppo, J.T., Tassinary, L.G., Berntson, G.G. (Eds.), Handbook of Psychophysiology. Cambridge University Press, Cambridge, UK. Baudena, P., Halgren, E., Heit, G., Clarke, J.M., 1995. Intracerebral potentials to rare target and distractor auditory and visual stimuli. III. Frontal cortex. Electroencephalogr. Clin. Neurophysiol. 94, 251–264. Beckmann, C.F., Smith, S.M., 2005. Tensorial extensions of independent component analysis for multisubject FMRI analysis. Neuroimage 25, 294–311. Bell, A.J., Sejnowski, T.J., 1995. An information-maximization approach to blind separation and blind deconvolution. Neural Comput. 7, 1129–1159. Boynton, G.M., Engel, S.A., Glover, G.H., Heeger, D.J., 1996. Linear systems analysis of functional magnetic resonance imaging in human V1. J. Neurosci. 16, 4207–4221. Cabeza, R., Nyberg, L., 2000. Imaging cognition II: An empirical review of 275 PET and fMRI studies. J. Cogn. Neurosci. 12, 1–47. Calhoun, V., Adali, T., 2006. Unmixing fMRI with independent component analysis. IEEE Eng. Med. Biol. Magazine 25, 79–90. Calhoun, V.D., Adali, T., McGinty, V., Pekar, J.J., Watson, T., Pearlson, G.D., 2001. fMRI activation in a visual-perception task: network of areas detected using the general linear model and independent component analysis. NeuroImage 14, 1080–1088. Calhoun, V.D., Adali, T., Pearlson, G.D., Kiehl, K.A., 2006. Neuronal chronometry of target detection: fusion of hemodynamic and event-related potential data. Neuroimage 30, 544–553. Calhoun, V.D., Adali, T., Pearlson, G.D., Pekar, J.J., 2001. A method for making group inferences from functional MRI data using independent component analysis. Hum. Brain Mapp. 14, 140–151. Calhoun, V.D., Stevens, M.C., Pearlson, G.D., Kiehl, K.A., 2004. fMRI analysis with the general linear model: removal of latency-induced amplitude bias by incorporation of hemodynamic derivative terms. Neuroimage 22, 252–257. Damoiseaux, J.S., Rombouts, S.A., Barkhof, F., Scheltens, P., Stam, C.J., Smith, S.M., Beckmann, C.F., 2006. Consistent resting-state networks across healthy subjects. Proc. Natl. Acad. Sci. U. S. A. 103, 13848–13853. de Munck, J.C., Goncalves, S.I., Huijboom, L., Kuijer, J.P., Pouwels, P.J., Heethaar, R.M., Lopes da Silva, F.H., 2007. The hemodynamic response of the alpha rhythm: an EEG/ fMRI study. Neuroimage 35, 1142–1151. de Munck, J.C., Goncalves, S.I., Faes, T.J., Kuijer, J.P., Pouwels, P.J., Heethaar, R.M., Lopes da Silva, F.H., 2008. A study of the brain's resting state based on alpha band power, heart rate and fMRI. Neuroimage 42 (1), 112–121. Debener, S., Makeig, S., Delorme, A., Engel, A.K., 2005a. What is novel in the novelty oddball paradigm? Functional significance of the novelty P3 event-related potential as revealed by independent component analysis. Brain Res. Cogn. Brain Res. 22, 309–321. Debener, S., Ullsperger, M., Siegel, M., Fiehler, K., von Cramon, D.Y., Engel, A.K., 2005b. Trialby-trial coupling of concurrent electroencephalogram and functional magnetic resonance imaging identifies the dynamics of performance monitoring. J. Neurosci. 25, 11730–11737. Debener, S., Ullsperger, M., Siegel, M., Engel, A.K., 2006. Single-trial EEG–fMRI reveals the dynamics of cognitive function. Trends Cogn. Sci. 10, 558–563. Debener, S., Strobel, A., Sorger, B., Peters, J., Kranczioch, C., Engel, A.K., Goebel, R., 2007. Improved quality of auditory event-related potentials recorded simultaneously with 3-T fMRI: removal of the ballistocardiogram artefact. Neuroimage 34, 587–597. Debener, S., Mullinger, K.J., Niazy, R.K., Bowtell, R.W., 2008. Properties of the ballistocardiogram artefact as revealed by EEG recordings at 1.5, 3 and 7 T static magnetic field strength. Int. J. Psychophysiol. 67, 189–199. Delorme, A., Westerfield, M., Makeig, S., 2007. Medial prefrontal theta bursts precede rapid motor responses during visual selective attention. J. Neurosci. 27, 11949–11959. Eichele, T., Calhoun, V.D., Moosmann, M., Specht, K., Jongsma, M.L., Quiroga, R.Q., Nordby, H., Hugdahl, K., 2008a. Unmixing concurrent EEG–fMRI with parallel independent component analysis. Int. J. Psychophysiol. 67, 222–234. Eichele, T., Debener, S., Calhoun, V.D., Specht, K., Engel, A.K., Hugdahl, K., von Cramon, D.Y., Ullsperger, M., 2008b. Prediction of human errors by maladaptive changes in eventrelated brain networks. Proc. Natl. Acad. Sci. U. S. A. 105, 6173–6178. Eichele, T., Specht, K., Moosmann, M., Jongsma, M.L., Quiroga, R.Q., Nordby, H., Hugdahl, K., 2005. Assessing the spatiotemporal evolution of neuronal activation with single-trial event-related potentials and functional MRI. Proc. Natl. Acad. Sci. U. S. A. 102, 17798–17803. T. Eichele et al. / International Journal of Psychophysiology 73 (2009) 53–61 Esposito, F., Scarabino, T., Hyvarinen, A., Himberg, J., Formisano, E., Comani, S., Tedeschi, G., Goebel, R., Seifritz, E., Di Salle, F., 2005. Independent component analysis of fMRI group studies by self-organizing clustering. Neuroimage 25, 193–205. Feige, B., Scheffler, K., Esposito, F., Di Salle, F., Hennig, J., Seifritz, E., 2005. Cortical and subcortical correlates of electroencephalographic alpha rhythm modulation. J. Neurophysiol. 93, 2864–2872. Fox, M.D., Raichle, M.E., 2007. Spontaneous fluctuations in brain activity observed with functional magnetic resonance imaging. Nat. Rev. Neurosci. 8, 700–711. Frahm, J., Bruhn, H., Merboldt, K.D., Hanicke, W., 1992. Dynamic MR imaging of human brain oxygenation during rest and photic stimulation. J. Magn. Reson. Imaging 2, 501–505. Friston, K.J., 2005. A theory of cortical responses. Philos. Trans. R. Soc. Lond. B Biol. Sci. 360, 815–836. Friston, K.J., Harrison, L., Penny, W., 2003. Dynamic causal modelling. Neuroimage 19, 1273–1302. Garrido, M.I., Kilner, J.M., Kiebel, S.J., Friston, K.J., 2007. Evoked brain responses are generated by feedback loops. Proc. Natl. Acad. Sci. U. S. A. 104, 20961–20966. Glover, G.H., 1999. Deconvolution of impulse response in event-related BOLD fMRI. Neuroimage 9, 416–429. Goldman, R.I., Stern, J.M., Engel Jr., J., Cohen, M.S., 2002. Simultaneous EEG and fMRI of the alpha rhythm. Neuroreport 13, 2487–2492. Guo, Y., Pagnoni, G., 2008. A unified framework for group independent component analysis for multi-subject fMRI data. Neuroimage 42, 1078–1093. Halgren, E., Marinkovic, K., 1995. General principles for the physiology of cognition as suggested by intracranial ERPs. In: Ogura, C., Koga, Y., Shimokochi, M. (Eds.), Recent Advances in Event-Related Brain Potential Research. Elsevier, Amsterdam, pp. 1072–1084. Halgren, E., Baudena, P., Clarke, J.M., Heit, G., Liegeois, C., Chauvel, P., Musolino, A., 1995a. Intracerebral potentials to rare target and distractor auditory and visual stimuli. I. Superior temporal plane and parietal lobe. Electroencephalogr. Clin. Neurophysiol. 94, 191–220. Halgren, E., Baudena, P., Clarke, J.M., Heit, G., Marinkovic, K., Devaux, B., Vignal, J.P., Biraben, A., 1995b. Intracerebral potentials to rare target and distractor auditory and visual stimuli. II. Medial, lateral and posterior temporal lobe. Electroencephalogr. Clin. Neurophysiol. 94, 229–250. Handwerker, D.A., Ollinger, J.M., D'Esposito, M., 2004. Variation of BOLD hemodynamic responses across subjects and brain regions and their effects on statistical analyses. Neuroimage 21, 1639–1651. Heeger, D.J., Ress, D., 2002. What does fMRI tell us about neuronal activity? Nat. Rev. Neurosci. 3, 142–151. Horwitz, B., Poeppel, D., 2002. How can EEG/MEG and fMRI/PET data be combined? Hum. Brain Mapp. 17, 1–3. Huettel, S.A., Song, A.W., McCarthy, G., 2004. Functional Magnetic Resonance Imaging. Sinauer, Sunderland, MA. Husain, F.T., Nandipati, G., Braun, A.R., Cohen, L.G., Tagamets, M.A., Horwitz, B., 2002. Simulating transcranial magnetic stimulation during PET with a large-scale neural network model of the prefrontal cortex and the visual system. NeuroImage 15, 58–73. Hyvärinen, A., Karhunen, J., Oja, E., 2001. Independent Component Analysis. John Wiley & Sons, New York. Jafri, M.J., Pearlson, G.D., Stevens, M., Calhoun, V.D., 2008. A method for functional network connectivity among spatially independent resting-state components in schizophrenia. Neuroimage 39, 1666–1681. Jung, T.P., Makeig, S., Humphries, C., Lee, T.W., McKeown, M.J., Iragui, V., Sejnowski, T.J., 2000a. Removing electroencephalographic artifacts by blind source separation. Psychophysiology 37, 163–178. Jung, T.P., Makeig, S., Westerfield, M., Townsend, J., Courchesne, E., Sejnowski, T.J., 2000b. Removal of eye activity artifacts from visual event-related potentials in normal and clinical subjects. Clin. Neurophysiol. 111, 1745–1758. Kovacevic, N., McIntosh, A.R., 2007. Groupwise independent component decomposition of EEG data and partial least square analysis. Neuroimage 35, 1103–1112. Kwong, K.K., Belliveau, J.W., Chesler, D.A., Goldberg, I.E., Weisskoff, R.M., Poncelet, B.P., Kennedy, D.N., Hoppel, B.E., Cohen, M.S., Turner, R., et al., 1992. Dynamic magnetic resonance imaging of human brain activity during primary sensory stimulation. Proc. Natl. Acad. Sci. U. S. A. 89, 5675–5679. Laufs, H., Daunizeau, J., Carmichael, D.W., Kleinschmidt, A., 2008. Recent advances in recording electrophysiological data simultaneously with magnetic resonance imaging. Neuroimage 40, 515–528. Lauritzen, M., Gold, L., 2003. Brain function and neurophysiological correlates of signals used in functional neuroimaging. J. Neurosci. 23, 3972–3980. Lee, T., Girolami, M., Sejnowski, T., 1999. Independent component analysis using an extended infomax algorithm for mixed subgaussian and supergaussian sources. Neural Comput. 11, 417–441. Logothetis, N.K., Wandell, B.A., 2004. Interpreting the BOLD signal. Annu. Rev. Physiol. 66, 735–769. Logothetis, N.K., Pauls, J., Augath, M., Trinath, T., Oeltermann, A., 2001. Neurophysiological investigation of the basis of the fMRI signal. Nature 412, 150–157. 61 Lukic, A.S., Wernick, M.N., Hansen, L.K., Strother, S.C., 2002. An ICA algorithm for analyzing multiple data sets. Int.Conf.on Image Processing (ICIP). Makeig, S., Jung, T.P., Bell, A.J., Ghahremani, D., Sejnowski, T.J., 1997. Blind separation of auditory event-related brain responses into independent components. Proc. Natl. Acad. Sci. U. S. A. 94, 10979–10984. Makeig, S., Westerfield, M., Jung, T.P., Enghoff, S., Townsend, J., Courchesne, E., Sejnowski, T.J., 2002. Dynamic brain sources of visual evoked responses. Science 295, 690–694. Makeig, S., Debener, S., Onton, J., Delorme, A., 2004a. Mining event-related brain dynamics. Trends Cogn. Sci. 8, 204–210. Makeig, S., Debener, S., Onton, J., Delorme, A., 2004b. Mining event-related brain dynamics. Trends Cogn. Sci. 8, 204–210. Mantini, D., Perrucci, M.G., Cugini, S., Ferretti, A., Romani, G.L., Del Gratta, C., 2007a. Complete artifact removal for EEG recorded during continuous fMRI using independent component analysis. Neuroimage 34, 598–607. Mantini, D., Perrucci, M.G., Del Gratta, C., Romani, G.L., Corbetta, M., 2007b. Electrophysiological signatures of resting state networks in the human brain. Proc. Natl. Acad. Sci. U. S. A. 104, 13170–13175. Mantini, D., Corbetta, M., Perrucci, M.G., Romani, G.L., Del Gratta, C., 2009. Large-scale brain networks account for sustained and transient activity during target detection. Neuroimage 44, 265–274. McKeown, M.J., Makeig, S., Brown, G.G., Jung, T.P., Kindermann, S.S., Bell, A.J., Sejnowski, T.J., 1998. Analysis of fMRI data by blind separation into independent spatial components. Hum. Brain Mapp. 6, 160–188. McKeown, M.J., Hansen, L.K., Sejnowski, T.J., 2003. Independent component analysis of functional MRI: what is signal and what is noise? Curr. Opin. Neurobiol. 13, 620–629. Moosmann, M., Ritter, P., Krastel, I., Brink, A., Thees, S., Blankenburg, F., Taskin, B., Obrig, H., Villringer, A., 2003. Correlates of alpha rhythm in functional magnetic resonance imaging and near infrared spectroscopy. Neuroimage 20, 145–158. Moosmann, M., Eichele, T., Nordby, H., Hugdahl, K., Calhoun, V.D., 2008. Joint independent component analysis for simultaneous EEG–fMRI: principle and simulation. Int. J. Psychophysiol. 67, 212–221. Niazy, R.K., Beckmann, C.F., Iannetti, G.D., Brady, J.M., Smith, S.M., 2005. Removal of FMRI environment artifacts from EEG data using optimal basis sets. Neuroimage 28, 720–737. Nunez, P.L., 1995. Neocortical Dynamics and Human EEG Rhythms. Oxford University Press, New York. Ogawa, S., Tank, D.W., Menon, R.S., Ellermann, J.M., Kim, S.G., Merkle, H., Ugurbil, K., 1992. Intrinsic signal changes accompanying sensory stimulation: functional brain mapping with magnetic resonance imaging. Proc. Natl. Acad. Sci. U. S. A. 89, 5951–5955. Onton, J., Makeig, S., 2006. In: Neuper, C., Pfurtscheller, G. (Eds.), Information-based Modeling Of Event-Related Brain Dynamics. Progress in Brain Research, vol. 159. Elsevier, Amsterdam, pp. 99–120. Onton, J., Delorme, A., Makeig, S., 2005. Frontal midline EEG dynamics during working memory. Neuroimage 27, 341–356. Onton, J., Westerfield, M., Townsend, J., Makeig, S., 2006. Imaging human EEG dynamics using independent component analysis. Neurosci. Biobehav. Rev. 30, 808–822. Petersen, K., Hansen, L., Kolenda, T., Rostrup, E., Strother, S., 2000. On the independent components of functional neuroimages. Int.Conf.on ICA and BSS, pp. 615–620. Raichle, M.E., Mintun, M.A., 2006. Brain work and brain imaging. Annu. Rev. Neurosci. 29, 449–476. Savopol, F., Armenakis, C., 2002. Mergine of heterogeneous data for emergency mapping: data integration or data fusion? Proc.ISPRS. Scheeringa, R., Bastiaansen, M.C., Petersson, K.M., Oostenveld, R., Norris, D.G., Hagoort, P., 2008. Frontal theta EEG activity correlates negatively with the default mode network in resting state. Int. J. Psychophysiol. 67, 242–251. Schmithorst, V.J., Holland, S.K., 2004. Comparison of three methods for generating group statistical inferences from independent component analysis of functional magnetic resonance imaging data. J. Magn. Reson. Imaging 19, 365–368. Seifritz, E., Esposito, F., Hennel, F., Mustovic, H., Neuhoff, J.G., Bilecen, D., Tedeschi, G., Scheffler, K., Di Salle, F., 2002. Spatiotemporal pattern of neural processing in the human auditory cortex. Science 297, 1706–1708. Stevens, M.C., Kiehl, K.A., Pearlson, G.D., Calhoun, V.D., 2007. Functional neural networks underlying response inhibition in adolescents and adults. Behav. Brain Res. 181, 12–22. Stone, J.V., 2004. Independent Component Analysis: A Tutorial Introduction. MIT press, Cambridge, MA. Stone, J.V., Porrill, J., Buchel, C., Friston, K., 1999. Spatial, temporal, and spatiotemporal independent component analysis of fMRI data. Proc.Leeds Statistical Research Workshop. Svensen, M., Kruggel, F., Benali, H., 2002. ICA of fMRI Group Study Data. NeuroImage 16, 551–563.