The paper proposes a novel technique for robust change detection based upon the integration of in... more The paper proposes a novel technique for robust change detection based upon the integration of intensity and texture differences between two frames. A new texture difference measure based on the relations between gradient vectors is described. The robustness of the measure with respect to noise and illumination changes has been analyzed. Two ways to integrate the intensity and texture differences are proposed. The first combines two measures according to the weightage of texture evidence, while the second takes into additional constraint of smoothness. The parameters of the algorithm are selected automatically. The computational complexity analysis indicates that the proposed technique can run in real-time. Experimental results show that by exploiting both intensity and texture differences for change detection, one can obtain much better segmentation results than using the intensity or structure difference alone.
This paper describes a novel robotic butler, developed by a multi-disciplinary team of researcher... more This paper describes a novel robotic butler, developed by a multi-disciplinary team of researchers. The robotic butler is capable of detecting and tracking human, recognize hand gestures, serving beverages and performs dialog conversation with guest about their interests and their preferences; and providing specific information on the facilities at Fusionopolis building and various technologies used by the robot. The robot employs an event driven dialogue management system (DMS) architecture, speech recognition, ultra wideband, vision understanding and radio frequency identification. All these components and agents that are integrated in the DMS architecture are modular and can be re-used by other applications. In this paper, we will first describe the design concept and the architecture of the robotic butler. Secondly, we will describe in detail the workings of the speech and vision technology as this paper mainly focuses on human-robot interaction aspects of the social robot. Lastly, this paper will highlight some key challenges that were faced during the implementation of speech and vision technology into the robot.
In this paper, we describe the implemented service robot, called FusionBot. The goal of this rese... more In this paper, we describe the implemented service robot, called FusionBot. The goal of this research is to explore and demonstrate the utility of an interactive service robot in a smart home environment, thereby improving the quality of human life. The robot has four main features: 1) speech recognition, 2) object recognition, 3) object grabbing and fetching and 4) communication with a smart coffee machine. Its software architecture employs a multimodal dialogue system that integrates different components, including spoken dialog system, vision understanding, navigation and smart device gateway. In the experiments conducted during the TechFest 2008 event, the FusionBot successfully demonstrated that it could autonomously serve coffee to visitors on their request. Preliminary survey results indicate that the robot has potential to not only aid in the general robotics but also contribute towards the long term goal of intelligent service robotics in smart home environment.
In this paper, we propose a novel Adaboost template to recognize human upper body poses from disp... more In this paper, we propose a novel Adaboost template to recognize human upper body poses from disparity images for natural human robot interaction (HRI). First, the upper body poses of standing persons are classified into seven categories of views. For each category, a mean template, variance template, and percentage template are generated. Then, the template region is divided into positive
... y ) = 0 for the pixels within the window Finally, set B,. Then start to find next human heads... more ... y ) = 0 for the pixels within the window Finally, set B,. Then start to find next human heads inqx ,y ) If nos(x,y) > 0.2 is found ... Thc test images contain various scenes of crowds, including groups of people walking together or standing and talking to cach olhen We alsu captured the ...
In this paper, an effective and efficient methodology to extract visual evidence of suitable scal... more In this paper, an effective and efficient methodology to extract visual evidence of suitable scale for object detection, object-orient scale-adaptive filtering (OOSAF), is proposed. With OOSAF, object extraction from stereo images is formulated as the design of scale-adaptive filters. Based on OOSAF, two methods for human detection from stereo images are developed. One is to detect human objects with close distances to the camera for intelligent human-machine interaction, and the other is to detect human heads in distant crowds for security surveillance. Experiments show that, with OOSAF, efficient solutions for human detection from stereo images could be achieved with high detection rates and low false alarm rates.
... X. Gao, T. Boult, F. Coetzee, and V. Ramesh, “Error analysis of background adaption,” Proc. I... more ... X. Gao, T. Boult, F. Coetzee, and V. Ramesh, “Error analysis of background adaption,” Proc. IEEE Computer Vision Patt. Recog. Conf., pp. 503–510 (2000). L. Li, “Intelligent automated video surveillance,” PhD Thesis, Nanyang Technological Univ., Singapore (2002). ...
Detecting the presence of people and suspicious objects are the essential tasks for security surv... more Detecting the presence of people and suspicious objects are the essential tasks for security surveillance. This paper presents a new real-time system for this purpose. Two motion cues from background subtraction and temporal differencing are employed to not only get reliable motion detection but also identify detected objects in the scene. A fuzzy reasoning technique is developed to detect and locate motion objects from vertical projection of motion cues. The background model is updated on both pixel level and region level to adapt both slow illumination changes and sudden extraneous events. This new background maintenance technique makes the system able to work under varying environments. The system has been run round the clock in real scenes and the performance is very promising
We propose a novel technique for robust change detection based upon the integration of intensity ... more We propose a novel technique for robust change detection based upon the integration of intensity and texture differences between two frames. A new accurate texture difference measure based on the relations between gradient vectors is proposed. The mathematical analysis shows that the measure is robust with respect to noise and illumination changes. Two ways to integrate the intensity and texture differences have been developed. The first combines the two measures adaptively according to the weightage of texture evidence, while the second does it optimally with additional constraint of smoothness. The parameters of the algorithm are selected automatically based on a statistic analysis. An algorithm is developed for fast implementation. The computational complexity analysis indicates that the proposed technique can run in real-time. The experiment results are evaluated both visually and quantitatively. They show that by exploiting both intensity and texture differences for change detection, one can obtain much better segmentation results than using the intensity or structure difference alone
In this paper, a novel study for an automated scene interpretation system, named ELEVIEW, is repo... more In this paper, a novel study for an automated scene interpretation system, named ELEVIEW, is reported to outline the design of the system. It is motivated by the reported crimes that happen inside elevators. The main goal is to investigate techniques that make an ordinary elevator monitoring system intelligent, i.e. see the scene and understand actions that are occurring. The system could filter out normal actions and trigger an alarm once abnormal events are detected. The paper focuses on the system overview, segmentation techniques as well as scenario classification. A double thresholded segmentation is employed to enhance the segmentation outcomes. This paper mainly presents an overview of the system and significant results so far achieved
This paper addresses the problem of background modeling for foreground object detection in comple... more This paper addresses the problem of background modeling for foreground object detection in complex environments. A Bayesian framework that incorporates spectral, spatial, and temporal features to characterize the background appearance is proposed. Under this framework, the background is represented by the most significant and frequent features, i.e., the principal features, at each pixel. A Bayes decision rule is derived for background and foreground classification based on the statistics of principal features. Principal feature representation for both the static and dynamic background pixels is investigated. A novel learning method is proposed to adapt to both gradual and sudden "once-off" background changes. The convergence of the learning process is analyzed and a formula to select a proper learning rate is derived. Under the proposed framework, a novel algorithm for detecting foreground objects from complex environments is then established. It consists of change detection, change classification, foreground segmentation, and background maintenance. Experiments were conducted on image sequences containing targets of interest in a variety of environments, e.g., offices, public buildings, subway stations, campuses, parking lots, airports, and sidewalks. Good results of foreground detection were obtained. Quantitative evaluation and comparison with the existing method show that the proposed method provides much improved results.
This paper proposes a novel method for tracking persons based on the principal colors of human ob... more This paper proposes a novel method for tracking persons based on the principal colors of human objects. First, an efficient human object representation method, principal color representation (PCR), is proposed. Asymmetric similarity measures are then proposed based on the principal color representation. These asymmetric similarity measures could be used to evaluate the matching between two individuals as well as visual evident of an individual in a group. An efficient algorithm for tracking persons as individuals or in groups is then described. The method has been tested using image sequences containing multiple moving persons frequently gathering and separating. Our test results have shown that proposed method has successfully tracked both persons as individuals or in groups, and is robust to illumination changes.
Background subtraction is the first step for video surveillance. Existing methods almost all upda... more Background subtraction is the first step for video surveillance. Existing methods almost all update their background models with a constant learning rate, which makes them not adaptive to some complex situations, e.g., crowded scenes or objects staying for a long time. In this paper, a novel framework which integrates both positive and negative feedbacks to control the learning rate is proposed. The negative feedback comes from background contextual analysis and the positive feedback comes from the foreground region analysis. Two descriptors of global contextual features are proposed and the visibility measures of background regions are derived based on contextual descriptors. Spatial-temporal features of the foreground regions are exploited. Fusing both positive and negative feedbacks, suitable strategy of background updating for specified surveillance task can be implemented. Three strategies for short-term, selective and long-term surveillance have been implemented and tested. Improved results compared with conventional background subtraction have been obtained.
... The experimental results on various complex videos and the quantitative evaluation are presen... more ... The experimental results on various complex videos and the quantitative evaluation are presented in Sec-tion 4. The paper is concluded in Section 5. ... Besides, the foreground object might be converted to be a background object, such as a car moving into a parking lot. ...
The paper proposes a novel technique for robust change detection based upon the integration of in... more The paper proposes a novel technique for robust change detection based upon the integration of intensity and texture differences between two frames. A new texture difference measure based on the relations between gradient vectors is described. The robustness of the measure with respect to noise and illumination changes has been analyzed. Two ways to integrate the intensity and texture differences are proposed. The first combines two measures according to the weightage of texture evidence, while the second takes into additional constraint of smoothness. The parameters of the algorithm are selected automatically. The computational complexity analysis indicates that the proposed technique can run in real-time. Experimental results show that by exploiting both intensity and texture differences for change detection, one can obtain much better segmentation results than using the intensity or structure difference alone.
This paper describes a novel robotic butler, developed by a multi-disciplinary team of researcher... more This paper describes a novel robotic butler, developed by a multi-disciplinary team of researchers. The robotic butler is capable of detecting and tracking human, recognize hand gestures, serving beverages and performs dialog conversation with guest about their interests and their preferences; and providing specific information on the facilities at Fusionopolis building and various technologies used by the robot. The robot employs an event driven dialogue management system (DMS) architecture, speech recognition, ultra wideband, vision understanding and radio frequency identification. All these components and agents that are integrated in the DMS architecture are modular and can be re-used by other applications. In this paper, we will first describe the design concept and the architecture of the robotic butler. Secondly, we will describe in detail the workings of the speech and vision technology as this paper mainly focuses on human-robot interaction aspects of the social robot. Lastly, this paper will highlight some key challenges that were faced during the implementation of speech and vision technology into the robot.
In this paper, we describe the implemented service robot, called FusionBot. The goal of this rese... more In this paper, we describe the implemented service robot, called FusionBot. The goal of this research is to explore and demonstrate the utility of an interactive service robot in a smart home environment, thereby improving the quality of human life. The robot has four main features: 1) speech recognition, 2) object recognition, 3) object grabbing and fetching and 4) communication with a smart coffee machine. Its software architecture employs a multimodal dialogue system that integrates different components, including spoken dialog system, vision understanding, navigation and smart device gateway. In the experiments conducted during the TechFest 2008 event, the FusionBot successfully demonstrated that it could autonomously serve coffee to visitors on their request. Preliminary survey results indicate that the robot has potential to not only aid in the general robotics but also contribute towards the long term goal of intelligent service robotics in smart home environment.
In this paper, we propose a novel Adaboost template to recognize human upper body poses from disp... more In this paper, we propose a novel Adaboost template to recognize human upper body poses from disparity images for natural human robot interaction (HRI). First, the upper body poses of standing persons are classified into seven categories of views. For each category, a mean template, variance template, and percentage template are generated. Then, the template region is divided into positive
... y ) = 0 for the pixels within the window Finally, set B,. Then start to find next human heads... more ... y ) = 0 for the pixels within the window Finally, set B,. Then start to find next human heads inqx ,y ) If nos(x,y) > 0.2 is found ... Thc test images contain various scenes of crowds, including groups of people walking together or standing and talking to cach olhen We alsu captured the ...
In this paper, an effective and efficient methodology to extract visual evidence of suitable scal... more In this paper, an effective and efficient methodology to extract visual evidence of suitable scale for object detection, object-orient scale-adaptive filtering (OOSAF), is proposed. With OOSAF, object extraction from stereo images is formulated as the design of scale-adaptive filters. Based on OOSAF, two methods for human detection from stereo images are developed. One is to detect human objects with close distances to the camera for intelligent human-machine interaction, and the other is to detect human heads in distant crowds for security surveillance. Experiments show that, with OOSAF, efficient solutions for human detection from stereo images could be achieved with high detection rates and low false alarm rates.
... X. Gao, T. Boult, F. Coetzee, and V. Ramesh, “Error analysis of background adaption,” Proc. I... more ... X. Gao, T. Boult, F. Coetzee, and V. Ramesh, “Error analysis of background adaption,” Proc. IEEE Computer Vision Patt. Recog. Conf., pp. 503–510 (2000). L. Li, “Intelligent automated video surveillance,” PhD Thesis, Nanyang Technological Univ., Singapore (2002). ...
Detecting the presence of people and suspicious objects are the essential tasks for security surv... more Detecting the presence of people and suspicious objects are the essential tasks for security surveillance. This paper presents a new real-time system for this purpose. Two motion cues from background subtraction and temporal differencing are employed to not only get reliable motion detection but also identify detected objects in the scene. A fuzzy reasoning technique is developed to detect and locate motion objects from vertical projection of motion cues. The background model is updated on both pixel level and region level to adapt both slow illumination changes and sudden extraneous events. This new background maintenance technique makes the system able to work under varying environments. The system has been run round the clock in real scenes and the performance is very promising
We propose a novel technique for robust change detection based upon the integration of intensity ... more We propose a novel technique for robust change detection based upon the integration of intensity and texture differences between two frames. A new accurate texture difference measure based on the relations between gradient vectors is proposed. The mathematical analysis shows that the measure is robust with respect to noise and illumination changes. Two ways to integrate the intensity and texture differences have been developed. The first combines the two measures adaptively according to the weightage of texture evidence, while the second does it optimally with additional constraint of smoothness. The parameters of the algorithm are selected automatically based on a statistic analysis. An algorithm is developed for fast implementation. The computational complexity analysis indicates that the proposed technique can run in real-time. The experiment results are evaluated both visually and quantitatively. They show that by exploiting both intensity and texture differences for change detection, one can obtain much better segmentation results than using the intensity or structure difference alone
In this paper, a novel study for an automated scene interpretation system, named ELEVIEW, is repo... more In this paper, a novel study for an automated scene interpretation system, named ELEVIEW, is reported to outline the design of the system. It is motivated by the reported crimes that happen inside elevators. The main goal is to investigate techniques that make an ordinary elevator monitoring system intelligent, i.e. see the scene and understand actions that are occurring. The system could filter out normal actions and trigger an alarm once abnormal events are detected. The paper focuses on the system overview, segmentation techniques as well as scenario classification. A double thresholded segmentation is employed to enhance the segmentation outcomes. This paper mainly presents an overview of the system and significant results so far achieved
This paper addresses the problem of background modeling for foreground object detection in comple... more This paper addresses the problem of background modeling for foreground object detection in complex environments. A Bayesian framework that incorporates spectral, spatial, and temporal features to characterize the background appearance is proposed. Under this framework, the background is represented by the most significant and frequent features, i.e., the principal features, at each pixel. A Bayes decision rule is derived for background and foreground classification based on the statistics of principal features. Principal feature representation for both the static and dynamic background pixels is investigated. A novel learning method is proposed to adapt to both gradual and sudden "once-off" background changes. The convergence of the learning process is analyzed and a formula to select a proper learning rate is derived. Under the proposed framework, a novel algorithm for detecting foreground objects from complex environments is then established. It consists of change detection, change classification, foreground segmentation, and background maintenance. Experiments were conducted on image sequences containing targets of interest in a variety of environments, e.g., offices, public buildings, subway stations, campuses, parking lots, airports, and sidewalks. Good results of foreground detection were obtained. Quantitative evaluation and comparison with the existing method show that the proposed method provides much improved results.
This paper proposes a novel method for tracking persons based on the principal colors of human ob... more This paper proposes a novel method for tracking persons based on the principal colors of human objects. First, an efficient human object representation method, principal color representation (PCR), is proposed. Asymmetric similarity measures are then proposed based on the principal color representation. These asymmetric similarity measures could be used to evaluate the matching between two individuals as well as visual evident of an individual in a group. An efficient algorithm for tracking persons as individuals or in groups is then described. The method has been tested using image sequences containing multiple moving persons frequently gathering and separating. Our test results have shown that proposed method has successfully tracked both persons as individuals or in groups, and is robust to illumination changes.
Background subtraction is the first step for video surveillance. Existing methods almost all upda... more Background subtraction is the first step for video surveillance. Existing methods almost all update their background models with a constant learning rate, which makes them not adaptive to some complex situations, e.g., crowded scenes or objects staying for a long time. In this paper, a novel framework which integrates both positive and negative feedbacks to control the learning rate is proposed. The negative feedback comes from background contextual analysis and the positive feedback comes from the foreground region analysis. Two descriptors of global contextual features are proposed and the visibility measures of background regions are derived based on contextual descriptors. Spatial-temporal features of the foreground regions are exploited. Fusing both positive and negative feedbacks, suitable strategy of background updating for specified surveillance task can be implemented. Three strategies for short-term, selective and long-term surveillance have been implemented and tested. Improved results compared with conventional background subtraction have been obtained.
... The experimental results on various complex videos and the quantitative evaluation are presen... more ... The experimental results on various complex videos and the quantitative evaluation are presented in Sec-tion 4. The paper is concluded in Section 5. ... Besides, the foreground object might be converted to be a background object, such as a car moving into a parking lot. ...
Uploads
Papers by Liyuan Li