Augmented Cognition, Universal Access and Social Intelligence in the Information Society Ray Adams and Satinder Gill CIRCUA, Collaborative International Research Centre for Universal Access, School of Computing Science, Middlesex University, Ravensfield House, The Burroughs, Hendon, London NW4 4BT, United Kingdom r.g.adams@mdx.ac.uk,s.gill@mdx.ac.uk Abstract. The two concepts of universal access and augmented cognition have both contributed significantly to providing the intended users of modern information and communication technology with the necessary resources to achieve enhanced interaction and performance. The two concepts share a number of important features including; the improvement of user performance, the use of concepts from cognitive psychology, a consideration of user modelling, a user sensitive approach, support for customisation, personalisation, adaptation and adaptive systems. They differentially emphasise; short term and long term demands, ambient intelligence, ubiquitous computing, people with disabilities, the Information Society and social skills. Since the present research programme (CIRCUA) is focussed upon the design and evaluation of universally accessible systems within a vocational context, the concepts of universal access and augmented are both very relevant, though both need to draw more upon the concept of social intelligence if they to tackle key issues of the Information Society. 1 Introduction The concept of universal access is presented in the following words [18] “In order for these new technologies to be truly effective, they must provide communication modes and interaction modalities across different languages and cultures, and should accommodate the diversity of requirements of the user population at large, including disabled and elderly people, thus making the Information Society universally accessible, to the benefit of mankind”. The concept of augmented cognition is summarised as follows. “Augmented Cognition seeks to extend a user’s abilities via computational technologies that are explicitly designed to address and accommodate bottlenecks, limitations, and biased in cognition, such as limitations in attention, memory, learning, comprehension, visualization abilities, and decision making.” [19]. Both universal access and augmented cognition draw upon currently developing theories of human cognition that underpin both generic and specific cognitive user models. The notion of cognitive overload is relevant to both universal access and augmented cognition, as shown below, by means of user modelling approaches. Conversely, user modelling is potentially a significant contributor to a better D.D. Schmorrow, L.M. Reeves (Eds.): Augmented Cognition, HCII 2007, LNAI 4565, pp. 231–240, 2007. © Springer-Verlag Berlin Heidelberg 2007 232 R. Adams and S. Gill understanding of both universal access and cognitive overload and can benefit from an involvement in both. However, though universal access explicitly states the need for a thorough consideration of the social aspects of the Inclusive Information Society, augmented cognition tends to focus more on the individual at work, though possessing the potential that is now being applied to social factors that mitigate or exacerbate overload [20]. Cognitive overload offers many possible ways to resolve problems of the Information Society. If that is so, then socially relevant concepts might support the application of augmented cognition to the solution of social or vocational problems within the context of universal access. The present two goals are: (a) the creation of accessible systems that support vocational and social effectiveness for people with special requirements and (b) to build the joint contribution of cognitive and social factors into UA design methodologies for socially relevant systems. 2 The Concept of Universal Access How will the concept of universal access support the achievement of these two goals? To create accessible systems to support vocational and social effectiveness for people with special requirements requires a number of contributory factors. The Universal Access framework provides (i) the motivation to achieve inclusivity in society, (ii) methods to model the key parameters of intended users, technology platforms, contexts of use and social contexts, (iii) evaluation methods for accessible, interactive systems, (iv) methodologies to design and build such systems and (v) the potential to include cognitive and social factors into UA design methodologies for socially relevant systems [1] [2] [3]. 3 The Concept of Augmented Cognition How will the concept of augmented cognition support the achievement of these two goals? Augmented cognition took the traditional concept of cognitive overload and revitalised it with the newer notion of augmented cognition, using sensors and gauges to identify and measure the cognitive states of an individual so that any overloads could be mitigated by the provision of computational technologies. Clearly, this is highly analogous to the case where individuals face accessibility problems and limitations that can be overcome, by the provision of appropriately designed, adaptive technology. Augmented cognition provides (i) a rational basis for the identification of different types of cognitive overload, (ii) ways to measure cognitive overloads, (iii) methods to identify and provide suitable methods of augmentation (Adams, 2006) and emerging ways to consider team and social factors that interact with overload problems [20]. If so, then there are clear synergies between these two user-centred concepts i.e. augmented cognition and universal access. Augmented Cognition, Universal Access and Social Intelligence 233 4 The Concept of Social Intelligence The third and essential concept to introduce here is social intelligence. How can social intelligence contribute to the above two goals? How can social intelligence help us to develop the concepts of universal access and augmented cognition? The essence of social intelligence can be seen as knowledge in co-action [13]. These researchers define social intelligence as “the ability of actors and agents to manage their relationships with each other.” These relationships would engage us with the different agents (the people, tools, artefacts and technologies in the environment). 5 The Improvement of User Performance If the above considerations are correct, then universal access, augmented cognition and social intelligence can all contribute substantially to the performance of interactive systems in synergistic but distinct ways. But how do they interact in this context? Universal Access aims to provide the intended users with systems that are accessible to them in terms of their abilities and disabilities, their tasks, their hardware and software and context of use. Such systems should contain the capability for customisation, personalisation, adaptability (i.e. change before use) and adaptivity (i.e. change during use). Augmented Cognition provides systems that can actively adapt (i.e. change during use) on the basis of the detection and measurement of different types of cognitive overload. To do so, such augmentation technologies must have the capabilities to be adaptive and to learn about the users and their requirements proactively, providing projective guidance and support [20]. Since, performance enhancements can be seen as drawing upon both accessibility and cognitive overload, it is worth asking about the relationship between them. One way is to expand the notion of accessibility, as considered next, to envelop cognitive overload and social intelligence. 6 Accessibility and Cognitive Overload Accessibility problems can occur in different ways. Here are five complementary ways to conceive of accessibility, including cognitive accessibility that is very closely related to cognitive overload and thus to augmented cognition. There is the first level of accessibility (physical access). Consider that individuals or groups who are denied access face substantial disadvantage and are effectively on the wrong side of a massive digital divide in our societies, separating the haves from the have-nots. Denial of access to information resources, whether deliberate or accidental, occurs at a cost and occurs for a variety of reasons due, in part, to the multiple levels of accessibility that are required to function properly. The second type of accessibility problem involves lack of access from local systems to the necessary resources and communications (connectivity access). Digital poverty may be associated with a lack of resources. Geographical or organizational location may be related to lack of connectivity or excessively high costs. The third type of accessibility problem involves the design of the interface, an aspect of design that has been given more prominence by universal access. For 234 R. Adams and S. Gill people with disabilities (long term or short term), older adults and those with atypical perceptual skills, the present rash of interactive interfaces and Graphical User Interfaces (GUIs) often presents insurmountable accessibility problems. The fourth type of accessibility problem (cognitive access) involves cognitive processes like navigation and comprehension. It is the nearest version of accessibility to cognitive overload. People will inevitably vary in levels of education, culture, IST experience etc. Such variations may render the navigability of a system problematic or the contents of such a system inaccessible. In addition, problems occur with attention, memory, learning, visualization abilities, and decision making.” [19] will impair cognitive accessibility through cognitive overload. Thus, it is suggested that cognitive accessibility and cognitive overload are closely related, if not synonymous, terms. Fifth, and perhaps most importantly, we sometimes act as if IST were an end in itself. It is not. If an IST system does not allow individuals to access their goals, that system will be little used. It will have been a commercial or social failure. On a positive note, we live in an age of considerable pressures that may encourage inclusive design based, inter alia, on; (a) the political will to create an inclusive society, (b) the substantial skills-shortages faced by modern societies and (c) the need of IST vendors to achieve better market share in order to be successful. I will leave it to the reader to decide which are the most important of these factors and whether the three factors will always work together. However, as social creatures, access to objectives is best seen as a social activity, within the context of the inclusive information society. As the following table shows, accessibility types can also be related to the well developed internet layered model [16], which provides an important cross-validation of the above, new five-level approach to accessibility. Table 1. Accessibility and the Internet layered model Comparing: Accessibility types Internet layered model 1 Hardware access Physical 2 connectivity access data link 3 interface access Network 4 cognitive access Transport 7 goal / social access Application 7 The Importance of the Social Context When considering human performance, social context is very important for many reasons, particularly as it can be seen to influence user behaviour when using Information Society Technologies (IST). For example, it has been found that social context and emotional tone (task-oriented vs. socio-emotional) influenced the use of emoticons in short internet chats [9]. Social intelligence is a major consideration in our everyday lives, at work, education or leisure. It is often undervalued by researchers who look at cognitive aspects of users [3] [12]. Social intelligence is a Augmented Cognition, Universal Access and Social Intelligence 235 central aspect of the folk psychology of today’s citizens and cannot be ignored when working within the context of the Information Society ([14] The concept of social intelligence has at least two components i.e. the person’ skills to handle social settings and also their knowledge of the social world and it implications for them. It is clear that cognitive user models can incorporate them within their existing architectures. Simplex Two [1] [4] [5] [15] can incorporate “to understand and manage other people” as an attribute of the executive function. “To engage in adaptive social interactions” can be incorporated as a function of the complex output module. The long term memory and mental models modules can be used to cover “the individual's fund of knowledge about the social world”. On this basis, there is no need to postulate a distinct social intelligence module at the moment, though social intelligence may reflect a distinct domain of knowledge and skills within currently postulated cognitive modules. 8 Social Context, Skills and Social Intelligence Social intelligence is a major aspect of the triarchic theory of human intelligence [22]. On this view, intelligence comprises analytical, creative, and practical abilities [22] [23] [24]. There are also three performance components of social intelligence i.e. (i) problem solving in social settings, (ii) executive functions to make plans and execute them and (iii) the ability to learn to develop these component skills. All forms of intelligence are predicted to be sensitive to the context of measurement, so performance in a social task (like negotiation) would not correlate perfectly with performance on an intellectual task (like syllogistic reasoning), because of the differential influence of context. Social intelligence can also be subdivided into two types, knowing what to say (declarative knowledge) and knowing how to do actions (procedural knowledge) [14]. The concept of social intelligence is of the most relevance to inclusion and accessibility within the Information Society. The status of social intelligence is not completely clear. If it were a completely distinct faculty of the human brain then, when confounding factors (like overall speed of response) are controlled, any correlation between social and other forms of intelligence should be vanishingly small. However, if social intelligence is seen as based on a subset of knowledge and skills within common systems and processes like long term memory, then correlations may be low but never zero. It is clearly difficult to distinguish empirically between these two views of social intelligence, since that would require the detection of true zero correlations. However, there is no current evidence for the complete independence of social intelligence. A recent substantial review paper [14] applied the dichotomy between fluid and crystallized intelligence to social intelligence. Thus, crystallized intelligence covers a person’s knowledge of their social context. Fluid intelligence is skill based, i.e. a person’s capability for social problem solving. Exploratory factor analyses found that crystallized social intelligence fluid social intelligence but was well correlated with academic intelligence. However, none of the correlations were zero. Parsimony would suggest that there is basis to postulate a distinct social intelligence module at the moment, though social intelligence may be based on distinct knowledge and skills within a more generic cognitive architecture, as shown in figure one. R. Adams and S. Gill Fig. 1. Simplex two 236 Augmented Cognition, Universal Access and Social Intelligence 237 9 Social Intelligence and Disability If social intelligences cannot be dissociated completely from other forms of intelligence or cognition, on the basis between correlations between intelligence tests, then a consideration of cognitive disabilities, itself very relevant to universal access, could provide additional evidence. If social intelligence depends selectively on different regions of the brain, then certain brain injuries will be associated with the impairment of aspects of social intelligence but not other cognitive facilities. Some researchers [10] [11] have proposed that there are at least seven different types of intelligence, each with its own brain system. If so, then there could be evidence that specific brain injuries are associated with different intelligences. If social intelligence can be dissociated from other cognitive facilities, then some people with specific head injuries would retain their social intelligence but present with deficits in other functions and conversely, those with different head injuries could display the reverse pattern of impaired social intelligence with other cognitive functions intact. In psychoneurological case studies, injuries to the prefrontal lobes of the cerebral cortex can selectively impair personal and social intelligence, leaving other abilities intact [1] [4]. In contrast, injuries to the occipital and parietal lobes can be associated with the impairment of cognitive functions whilst social abilities remain unscathed. Similar evidence can be found when considering clinical conditions like Down syndrome and Alzheimer's disease. They are often associated with damaged cognitive sequelae but little or no impairment of social intelligence. By contrast, Pick's disease can spare some cognitive abilities while severely hitting a person's social skills. Autism spectrum disorder (ASD) and Asperger’s syndrome are classically associated with a deficit in social intelligence and lack of empathy but not necessarily with learning difficulties, the so-called “theory of mind” explanation [17] [7] [8]. There is some suggestion that they do not find traditional ways to learn social skills to be conducive and so acquire them badly [8]. Infantile autisms, like Kanner's syndrome and Williams' syndrome, are associated with significantly impair of empathy for other people in social settings. Whilst the diagnosis of mental retardation includes deficits in both social and academic intelligence, they are considered to be only lowly correlated [1] [4]. Furthermore, some researchers [26] have concluded that there are three different brain systems that are associated with social intelligence (i): a cortical subsystem which relies on long-term memory to make complex social judgments; (ii) a frontal-dominant subsystem to organize and create social behaviours; and (iii) a limbic-dominant subsystem to support emotional responses. If so, different brain lesions will lead to different types of impairment of social intelligence. From this consideration of social intelligence and disability, it is evident that all the evidence suggests that if social intelligence is correlated with other cognitive functions, any correlations are small but important and not statistically insignificant. If so, social intelligence is an important, semi-autonomous component of human cognition and worthy of consideration along with universal access and augmented cognition based solutions for accessibility problems and cognitive overload when using socially significant systems 238 R. Adams and S. Gill 10 Social Intelligence and Cognitive User Modelling There are two relevant groups who will use the notion of social intelligence within the context of universal access. First, there are the researchers who seek to understand the psychology of the intended users, for cognitive user modelling. Second, there are the designers and developers who work on unified user interface design [21]. For the first group, it is recommended that social intelligence can be treated as sub-domains of knowledge and skills within an overall cognitive architecture of the user. For the second group, as suggested below, social intelligence may require a more special treatment if it is not undervalued accidentally in the design process. The theoretical perspective is considered elsewhere [6] and concludes that most, if not all the presently offered components of human cognition can subsume many aspects of social intelligence. 11 Socially Intelligent Interactive Design From the perspective of the Universal Access interactive system design, social skills must be acknowledged, understood and included by the designers and eventually be supported by the system itself. This is particularly important to the development of socially based and socially aware systems, for example, the creation of an accessible negotiation system based on remote, visual conferencing. In that case, we propose that designers and developers should consider social intelligence as a separate factors in addition to the nine cognitive components already provided by Simplex Two [3]. 12 Discussion In this paper we have shown how both universal access and augmented cognition potentially have substantial areas of synergy within the context of the emergence of the Inclusive Information Society [21]. We have also defined a new, five level expansion of the concept of accessibility and introduced the concept of cognitive accessibility to link accessibility and cognitive overload. Furthermore, we have found that social intelligence and social skills are an important development area for both universal access and augmented cognition. Additionally, social intelligence, at the moment, can be subsumed under the current architecture of Simplex [3]. Finally all three concepts of universal access, augmented cognition and social intelligence provide powerful toolboxes for us to meet our objectives, namely (a) the creation of accessible systems that support vocational and social effectiveness for people with special requirements and (b) to build the joint contribution of cognitive and social factors into UA design methodologies for socially relevant systems. References 1. Adams, R.: Universal access through client-centred cognitive assessment personality profiling. In: Stary, C., Stephanidis, C. (eds.) User-Centered Interaction Paradigms for Universal Access in the Information Society. LNCS, vol. 3196, pp. 3–15. Springer, Heidelberg (2004) Augmented Cognition, Universal Access and Social Intelligence 239 2. Adams, R.G.: Applying advanced concepts of cognitive overload and augmentation in practice: the future of overload. In: Schmorrow, D.D., Stanney, K.M., Reeves, L.M. (eds.) Foundations of Augmented Cognition, 2nd edn. Arlington Virginia: Strategic Analysis (2006) 3. Adams, R.: Decision and Stress: Cognition and e-Accessibility in the Information Workplace. Universal Access in the Information Society (in press) (2007) 4. Adams, R., Langdon, P.: Principles and concepts for information and communication technology design. Journal of Visual Impairment and Blindness 97, 602–611 (2003) 5. Adams, R., Langdon, P.: Assessment, Insight and Awarenes. In: User Centred Design that Includes Users With Special Needs. In: Keates, S., Clarkson, P.J., Langdon, P. and Robinson, P (eds.) London: Springer (2004) 6. Adams, R., Gill, S.P.: User modelling and social intelligence. In: HCII 2007 proceedings (In press) (2007) 7. Baron-Cohen, S.: Mindblindness: An essay on autism and theory of mind. MIT Press, Cambridge (1995) 8. Baron-Cohen, S., Golan, O., Chapman, E., Grander, Y.: Transported to a world of emotion. Psychologist 20, 76–77 (2007) 9. Derks, D., Bos, A.E.R., von Grumbkow, J.: Emoticons and social interaction on the Internet: the importance of social context. Computers in Human Behavior 23, 842–849 (2007) 10. Gardner, H.: Frames of mind: The theory of multiple intelligences. New York: Basic Books (1983) 11. Gardner, H.: Multiple intelligences: The theory in practice. New York: Basic Books (1993) 12. Gill, S.: Designing of inclusivity. HCII proceedings (In press) (2007) 13. Gill, S.P., Borchers, J.: Knowledge in co-action: social intelligence in collaborative design activity. AI & Society 17, 322–339 (2003) 14. Kihlstrom, J.F., Cantor, N.: Social intelligence. In: Sternberg, R.J. (ed.) Handbook of intelligence, 2nd edn. pp. 359–379. Cambridge University Press, Cambridge (2000) 15. Langdon, P., Adams, R., Clarkson, P.J.: Universal Access to Assistive Technology through Client-Centred Cognitive Assessment. In: Carbonell, N., Stephanidis, C. (eds.) Universal Access. Theoretical Perspectives, Practice, and Experience. LNCS, vol. 2615, pp. 555– 567. Springer, Heidelberg (2003) 16. Moseley, R.: Developing web applications. Wiley, West Sussex UK (2007) 17. Premack, D., Woodruff, G.: Does the chimpanzee have a theory of mind? Behavioral & Brain Sciences 1, 515–526 (1978) 18. Salvendy, G.: Foreword. In: Stephanidis, C. (ed.) User interfaces for all: concepts, methods and tools, Lawrence Erlbaum Associates, Mahwah (2001) 19. Schmorrow, D.D., Stanney, K.M., Reeves, L.M.: Foundations of Augmented Cognition, 2nd edn. Arlington Virginia: Strategic Analysis (2006) 20. Smith, T.J., Henning, R.A., Adams, R.G.: Social cybernetics of augmented cognition – a control systems analysis. In: Schmorrow, D.D., Stanney, K.M., Reeves, L.M. Foundations of Augmented Cognition, 2nd edn. Arlington Virginia: Strategic Analysis (2006) 21. Stephanidis, C.: User interfaces for all: new perspectives into human-computer interaction. In: Stephanidis, C. (ed.) User interfaces for all: concepts, methods and tools, Lawrence Erlbaum Associates, Mahwah (2001) 22. Sternberg, R.J.: Toward a triarchic theory of human intelligence. Behavioral & Brain Sciences. Behavioral & Brain Sciences 7, 269–315 (1984) 240 R. Adams and S. Gill 23. Sternberg, R.J.: Beyond IQ: A triarchic theory of human intelligence. Cambridge University Press, New York (1985) 24. Sternberg, R.J.: The triarchic mind: A new theory of intelligence. New York: Viking (1988) 25. Sternberg, R.J. (ed.): Handbook of intelligence, 2nd edn., pp. 359–379. Cambridge University Press, Cambridge 26. Taylor, E.H., Cadet, J.L.: Social intelligence, a neurological system? Psychological Reports 64, 423–444 (1989)