- I am a Computer Scientist specializing in Human-Computer Interaction, Computer Vision, and Embodied Interaction. My r... moreI am a Computer Scientist specializing in Human-Computer Interaction, Computer Vision, and Embodied Interaction. My research areas span Technology in Learning, Embodied Technologies, and Interaction for the Blind. I am Director of the Texas Aedit
Research Interests:
Research Interests:
Research Interests:
This article studies fine motor strategies for precise spatial manipulation in close-to-body interactions. Our innate ability for precise work is the result of the confluence of visuo-tactile perception, proprioception, and bi-manual... more
This article studies fine motor strategies for precise spatial manipulation in close-to-body interactions. Our innate ability for precise work is the result of the confluence of visuo-tactile perception, proprioception, and bi-manual motor control. Contrary to this, most mixed-reality (MR) systems are designed for interactions at arms length. To develop guidelines for precise manipulations in MR systems, there is a need for a systematic study of motor strategies including physical indexing, bi-manual coordination, and the relationship between visual and tactile feedback. To address this need, we present a series of experiments using three variations of a tablet-based MR interface using a close-range motion capture system and motion-tracked shape proxies. We investigate an elaborate version of the classic peg-and-hole task that our results strongly suggests the critical need for high precision tracking to enable precise manipulation.
Research Interests:
Metin Akay, Dartmouth, USA Costas Balas, University of Crete, Greece Arvind Bansal, Kent State University, USA Lisa Bates, Mayo Foundation, USA George Bebis, University of Nevada at Reno, USA Gary Benson, Mount Sinai, USA O. Bukhres, PUI,... more
Metin Akay, Dartmouth, USA Costas Balas, University of Crete, Greece Arvind Bansal, Kent State University, USA Lisa Bates, Mayo Foundation, USA George Bebis, University of Nevada at Reno, USA Gary Benson, Mount Sinai, USA O. Bukhres, PUI, USA Sergio Cerutti, ...
Slate-type devices allow Individuals with Blindness or Severe Visual Impairment (IBSVI) to read in place with the touch of their fingertip by audio-rendering the words they touch. Such technologies are helpful for spatial cognition while... more
Slate-type devices allow Individuals with Blindness or Severe Visual Impairment (IBSVI) to read in place with the touch of their fingertip by audio-rendering the words they touch. Such technologies are helpful for spatial cognition while reading. However, users have to move their fingers slowly, or they may lose their place on screen. Also, IBSVI may wander between lines without realizing they did. We addressed these two interaction problems by introducing a dynamic speech-touch interaction model and an intelligent reading support system. With this model, the speed of the speech will dynamically change with the user’s finger speed. The proposed model is composed of (1) an Audio Dynamics Model and (2) an Off-line Speech Synthesis Technique. The intelligent reading support system predicts the direction of reading, corrects the reading word if the user drifts, and notifies the user using a sonic gutter to help him/her from straying off the reading line. We tested the new audio dynamics...
Research Interests:
The following full text is a publisher's version. For additional information about this publication click this link.
Research Interests:
Research Interests:
Research Interests:
Research Interests:
Research Interests:
Research Interests: Computer Science, Human Computer Interaction, Wireless Sensor Networks, Virtual Reality, Human Gait Analysis, and 12 moreMagnetic sensors, Virtual Environment, Gait Analysis, Sensor Fusion, Mobile Device, Acceleration, Data Handling, Wireless Sensor Network, Acceleration Measurement, Data loss handing, Human Walking Detection Algorithm, and Walking in place technique
This review discusses the reported studies investigating the use of bioprinting to develop functional organ-on-chip systems from a manufacturing perspective. These organ-on-chip systems model the liver, kidney, heart, lung, gut, bone,... more
This review discusses the reported studies investigating the use of bioprinting to develop functional organ-on-chip systems from a manufacturing perspective. These organ-on-chip systems model the liver, kidney, heart, lung, gut, bone, vessel, and tumors to demonstrate the viability of bioprinted organ-on-chip systems for disease modeling and drug screening. In addition, the paper highlights the challenges involved in using bioprinting techniques for organ-on-chip system fabrications and suggests future research directions. Based on the reviewed studies, it is concluded that bioprinting can be applied for the automated and assembly-free fabrication of organ-on chip systems. These bioprinted organ-on-chip systems can help in the modeling of several different diseases and can thereby expedite drug discovery by providing an efficient platform for drug screening in the preclinical phase of drug development processes.
Research Interests:
Research Interests:
The following full text is a publisher's version. For additional information about this publication click this link.
Research Interests:
Research Interests:
Research Interests:
Research Interests:
Research Interests:
Research Interests: Computer Science, Human Computer Interaction, Wireless Sensor Networks, Virtual Reality, Human Gait Analysis, and 12 moreMagnetic sensors, Virtual Environment, Gait Analysis, Sensor Fusion, Mobile Device, Acceleration, Data Handling, Wireless Sensor Network, Acceleration Measurement, Data loss handing, Human Walking Detection Algorithm, and Walking in place technique
Research Interests:
Medicinal and aromatic plants represent a consistent part of the natural biodiversity endowment of many countries around the world. These species provide an important contribution to health, local economies, cultural integrity and... more
Medicinal and aromatic plants represent a consistent part of the natural biodiversity endowment of many countries around the world. These species provide an important contribution to health, local economies, cultural integrity and ultimately the well-being of people, particularly the rural poor and fragile social group, e.g., the elderly, children and women. Their role has been increasingly acknowledged over the last
Research Interests:
Research Interests:
Research Interests:
Research Interests:
Page 1. xxviii Additional Reviewers Manoj Aggarwal Joni Alon Arnon Amir Vassilis Athitsos Selim Balcisoy Kobus Barnard Carlos Barron John Barron Adrien Bartoli Ronen Basri John Bastian Paul Beardsley George Bebis Moshe Ben-Ezra Alex Berg... more
Page 1. xxviii Additional Reviewers Manoj Aggarwal Joni Alon Arnon Amir Vassilis Athitsos Selim Balcisoy Kobus Barnard Carlos Barron John Barron Adrien Bartoli Ronen Basri John Bastian Paul Beardsley George Bebis Moshe Ben-Ezra Alex Berg Margrit Betke David Beymer Horst B0ischof Alesandro Bissacco Giuseppe Boccignone Luca Bogoni Ruud Bolle Leon Bottou Gabriel Brostow Tomas Brodsky Owen Carmichael Yaron Caspi Thanarat Chalidabhongse Olivier Chapelle Xilin Chen Yang Cheng German Cheung Alessandro Chiuso ...
Research Interests:
Research Interests:
We present an agent-based framework for human hand and head tracking. The benefits of this architecture are that it simplifies system organization, facilitates flexible extension, and supports hierarchical abstraction. The framework... more
We present an agent-based framework for human hand and head tracking. The benefits of this architecture are that it simplifies system organization, facilitates flexible extension, and supports hierarchical abstraction. The framework employs low-level processing agents that computes motionvector clusters, skin-color patches and edge. Mid-level agents fuse the features computed into hand and head candidates. Finally, high-level agents select among these candidates to yield consistent interpretations of hand and head groupings. Hence more domain-specific knowledge is encoded into the agents as we ascend the agent hierarchy. We demonstrate the effectiveness of our system by defining a set of gestures (move, zoom, and rotate) that were recognized and tracked.
Human multimodal communicative behaviors form a tightly integrated whole. By matching up gestural features with a carefully time-tagged transcription of the speech, we can observe how gesture features and discourse unit transitions... more
Human multimodal communicative behaviors form a tightly integrated whole. By matching up gestural features with a carefully time-tagged transcription of the speech, we can observe how gesture features and discourse unit transitions cohere. Pointing is a key gestural component. Although pointing appears to be simply a matter of aiming the hand at some target, it is in fact a complex process of several components including an origo (zero point of reference) and a deictic field. In our experiments where subjects make action plans around a terrain map, such origos become key organizational loci around which the discourse may be built. Our vision-based approach extracts ‘origo histograms’ from stereo video describing the locus of motion of a speaker’s dominant hand. A N N ‘fuzzy correlation’ of these histograms yields a correlation space in which similar origos are clustered. By locating the cluster transitions we can locate topical shifts in the discourse. We show results by comparing t...
Research Interests:
The Maker phenomenon revolves around the technology-enabled hands-on production of artifacts, and has been consistently proposed as a vehicle for STEM education and to inculcate a 'Maker mindset' in children. While Making is... more
The Maker phenomenon revolves around the technology-enabled hands-on production of artifacts, and has been consistently proposed as a vehicle for STEM education and to inculcate a 'Maker mindset' in children. While Making is physical and activities for children typically take place in physical venues like Maker camps and workshops, much information about Making resides online. To understand how virtual online resources may support inherently physical Making activity, we investigate the landscape of Maker websites for children through a perception-based study and a content analysis study. Our findings showed that Maker websites for children are of 3 types: associated with a fixed space, a transient space, or without any physical presence. These provide different support structures for learning and Maker mindset development. Further research is needed to extend the experience of children in Making activities beyond the limitations of physical visits to Makerspaces.1
Research Interests:
Our work is concerned with how embodied communication involving speech and gestures may be mediated through mobile tele-robotics and augmented reality to support hands-on distance mentoring. Following work in the psycholinguistics of... more
Our work is concerned with how embodied communication involving speech and gestures may be mediated through mobile tele-robotics and augmented reality to support hands-on distance mentoring. Following work in the psycholinguistics of embodied communication (e.g., meaning is expressed through gesture, gaze, and speech), a four design-implement-test-deploy-evaluate study was undertaken. We investigated whether and how powerful multimodal language to support explanation and mentoring may be mediated over distance through the designs.
Research Interests:
We propose MAIA, a methodology to evaluate embodied storytelling systems aimed at supporting the child’s creativity. Our approach rests on three key tenets: the study of process instead of objects, imagination as the basis of creativity,... more
We propose MAIA, a methodology to evaluate embodied storytelling systems aimed at supporting the child’s creativity. Our approach rests on three key tenets: the study of process instead of objects, imagination as the basis of creativity, and creativity as recombination. MAIA uses microlevel analyses of enactment videos, the child’s drawings and interview transcripts to generate a ‘broader imagination’ score that can be used for both quantitative and qualitative comparisons. Author
Research Interests:
We present our multimedia Visualization for Situated Temporal Analysis (VisSTA) system that facilitates analysis of multi-modal human communication incorporating video, audio, speech transcriptions, and coded multimodal (e.g. gesture and... more
We present our multimedia Visualization for Situated Temporal Analysis (VisSTA) system that facilitates analysis of multi-modal human communication incorporating video, audio, speech transcriptions, and coded multimodal (e.g. gesture and gaze) data. VisSTA is based on the Multiple Linked Representation strategy and keeps the user temporally situated by ensuring tight linkage among all representational components. The system features multiple representations, which include a hierarchical video-shot organization, a variety of animated graphs, animated multi-tier text transcripts, and an avatar representation. VisSTA is a multivideo system permitting simultaneous playing of multiple synchronized video streams that are time-locked to other data components. An integrated observation database system is included in VisSTA for storing the results of data analysis.
Research Interests:
Despite the great amount of work addressing the issue of embodiment with regards to creativity support technologies for children, there is a paucity of research that presents theoretical bases that are able to explain in depth the process... more
Despite the great amount of work addressing the issue of embodiment with regards to creativity support technologies for children, there is a paucity of research that presents theoretical bases that are able to explain in depth the process of how the generally positive effects on creativity obtained from user studies take place. In this paper, we draw from our experiences conducting studies with children aged 8 to 10 and from the theories of Lev Vygotsky to present a model of the mediated creative process. We further advance that an enactment-based approach to creativity support is beneficial, and describe two studies that advance understanding of the design of enactment-based technology to nurture creativity in children.
Research Interests:
In the simulation of human crowd behavior including evacuation planning, transportation management, and safety engineering in architecture design, the development of pedestrian model for higher behavior fidelity is an important task. To... more
In the simulation of human crowd behavior including evacuation planning, transportation management, and safety engineering in architecture design, the development of pedestrian model for higher behavior fidelity is an important task. To construct plausible facsimiles of real crowd movements, simulations should exhibit human behaviors for navigation, pedestrian decision-making, and social behaviors such as grouping and crowding. The research field is quite mature in some sense, with a large number of approaches that have been proposed to path finding, collision avoidance, and visually pleasing steering behaviors of virtual humans. However, there is still a clear disparity between the variety of approaches and the quality of crowd behaviors in simulations. Many social science field studies inform us that crowds are typically composed of multiple social groups (James, 1953; Coleman and James, 1961; Aveni, 1977). These observations indicate that one component of the complexity of crowd ...
Parkinson’s disease (PD) belongs to a class of neurodegenerative diseases that affect the patient’s speech, motor, and cognitive capabilities. All three deÞcits affect the multimodal communication channels of speech and gesture. We... more
Parkinson’s disease (PD) belongs to a class of neurodegenerative diseases that affect the patient’s speech, motor, and cognitive capabilities. All three deÞcits affect the multimodal communication channels of speech and gesture. We presenta study on the changes in speech pause patterns and gesture holds before and after treatment. We present the results of a pilot study of two Idiopathic PD patients who have undergone Lee Silverman Voice Treatment (LSVT). We show that therewas a consistentchangein the location of pauses with respect to semantic sentential utterance units. After treatment, the number and duration of pauses within sentential units decreased while the inter sentential pauses increased. This indicates reduction in hesitation and increase in speech phrasing. We also found a decrease in the number of sentence repairs and the time spent in repairs. For gesture, we found that non-rest holds intersecting with within-sentence pauses appears to decline after treatment, as does...
Research Interests:
As an input device, fabric holds potential benefits for three dimensional (3D) interaction in the domain of surface design, which includes designing objects from clothing to metalwork. To investigate these benefits, we conducted an... more
As an input device, fabric holds potential benefits for three dimensional (3D) interaction in the domain of surface design, which includes designing objects from clothing to metalwork. To investigate these benefits, we conducted an exploratory study of different users' natural interactions with fabric. During this study, we instructed users to communicate various shapes and surfaces of varying complexity. A prevailing way of communicating shapes proved to be an in-the-air sketch metaphor. Based on this result, we proposed and implemented a system supporting three in-the-air sketch-based input devices: a point, a flexible curve, and a flexible surface. A preliminary feasibility study found that users successfully sketched objects and scenes despite the influence of tracking issues, suggesting lessons learned and relevant constraints for such systems in future work.
Research Interests:
This chapter overviews the state of research in assistive technologies that support teaching and learning for individuals with blindness or severe visual impairment (IBSVI). Education, learning, and information are culturally defined and... more
This chapter overviews the state of research in assistive technologies that support teaching and learning for individuals with blindness or severe visual impairment (IBSVI). Education, learning, and information are culturally defined and designed for efficient communication and access for humans with typical visual and spatial capabilities. Understanding the unintended roadblocks to IBSVI brought on by such culturally defined elements of instruction and information is critical to assisting IBSVI to participate in the education and learning milieux. We divide the assistive technology challenges to the support for classroom instruction and for individual access to informational media. We address different aspects of each of these challenges and discuss various approaches to these aspects.
Overview Though experimental versions of multi-touch screens and tangible user interfaces have been in development in engineering labs for over a decade, the recent commercialization of these technologies available to educators is fairly... more
Overview Though experimental versions of multi-touch screens and tangible user interfaces have been in development in engineering labs for over a decade, the recent commercialization of these technologies available to educators is fairly recent with products such as the ...
Page 1. 946 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 16, NO. 9, SEPTEMBER 1994 Correspondence Vector Field Analysis for Oriented Patterns Chiao-Fe Shu and Ramesh C. Jain ...