Robot Vision

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org.

The problem of recursively approximating motion resulting from the Optical Flow (OF) in video thru Total Least Squares (TLS) techniques is addressed. TLS method solves an inconsistent system Gu=z , with G and z in error due to temporal/spatial derivatives, and nonlinearity, while the Ordinary Least Squares (OLS) model has noise only in z. Sources of difficulty involve the non-stationarity of the field, the ill-posedness, and the existence of noise in the data. Three ways of applying the TLS with different noise conjectures to the end problem are observed. First, the classical TLS (cTLS) is introduced, where the entries of the error matrices of each row of the augmented matrix [G;z] have zero mean and the same standard deviation. Next, the Generalized Total Least Squares (GTLS) is defined to provide a more stable solution, but it still has some problems. The Generalized Scaled TLS (GSTLS) has G and z tainted by different sources of additive zero-mean Gaussian noise and scaling [G;z] by nonsingular D and E, that is, D[G;z]E makes the errors iid with zero mean and a diagonal covariance matrix. The scaling is computed from some knowledge on the error distribution to improve the GTLS estimate. For moderate levels of additive noise, GSTLS outperforms the OLS, and the GTLS approaches. Although any TLS variant requires more computations than the OLS, it is still applicable with proper scaling of the data matrix.

Line Follower Robot is one of autonomous mobile robot form that has mission following autonomously the guided line. In its design and implementation, problems that should be solved are system of robot vision, architecture of hardware including electronics and mechanics, and organization of software for knowledge base and real time control. The goal of this thesis is to design and construct a Line Follower Robot by using microcontroller AT89C51 and infrared sensor. The mechanical system of the robot adopts maneuverability system of common four wheel automobile.Organization of software uses timed event methode for saving microcontroller's timer. Knowledge base of the robot contains codes of action done by robot. This is based on information from sensors. Methode to transform the information becoming action uses look up table methode.

This paper presents a novel approach for angular positioning of a robotic elbow movement and reducing the error response using kalman filter. In recent years, there is an increasing trend of research in robotics application to be place in terrain, sky, and ocean. Robotic elbow movement is introduced in this paper for mimicking human arm to help doing functions such as welding, painting, manufacturing and production assembly, welding, screwing, etc. Simulation study is carried out to verify the effectiveness of the proposed method. The system developed in this work was done with ISIS. A random noise is introduced in the system making the angular positioning movement of the DC motor unstable. Kalman filter is applied and is able to correct itself. The desirable response is met and the error response of the angular positioning of a DC motor was smoothened. In conclusion, the application of kalman filter improves the accuracy and response of angular position of the robotic elbow.

Vision-based robot navigation systems allow a robot to explore and to navigate in its environment in a way that facilitates path planning and goal-oriented tasks. The vision sensor is mainly used for obstacle detection and avoidance, object detection and tracking, and interaction with users. Usually these systems do not depend solely on vision sensors but use other sensors like sonar and laser range finder. The paper considers an important issue for mobile robots navigation. This issue is the detection of obstacles in front of the robot within a corridor. We proposed and evaluated three algorithms for obstacle detection within a corridor environment using image processing techniques.

We present a new method to translate videos to commands for robotic manipulation using Deep Recurrent Neural Networks (RNN). Our framework first extracts deep features from the input video frames with a deep Convolutional Neural Networks (CNN). Two RNN layers with an encoder-decoder architecture are then used to encode the visual features and sequentially generate the output words as the command. We demonstrate that the translation accuracy can be improved by allowing a smooth transaction between two RNN layers and using the state-of-the-art feature extractor. The experimental results on our new challenging dataset show that our approach outperforms recent methods by a fair margin. Furthermore, we combine the proposed translation module with the vision and planning system to let a robot perform various manipulation tasks. Finally, we demonstrate the effectiveness of our framework on a full-size humanoid robot WALK-MAN.

IET indexes its books and journal in SCOPUS and IEEE Xplore.
Computer Vision (CV) and Sensors play a decisive role in the operation of Unmanned Aerial Vehicle (UAV), but there exists a void when it comes to analysing the extent of their impact on the entire UAV system. In general, the fact that a UAV is a Cyber-Physical System (CPS) is not taken into account. In this proposal, we propose to expand on earlier books covering the use of CV and sensing in UAVs. Among other things, an entirely autonomous UAV can help to (i) obtain information about the environment, (ii) work for an extended period of time without human interference, (iii) move either all or part of itself all over its operating location devoid of human help and (iv) stay away from dangerous situations for people and their possessions. A Vision System (VS) entails the way CV data will be utilized, the appropriate architecture for total avionics integration, the control interfaces, and the UAV operation. Since the VS core is its sensors and cameras, multi-sensor fusion, navigation, hazard detection, and ground correlation in real time are important operational aspects that can benefit from CV knowledge and technology. This book will aim to collect and shed some light on the existing information on CV software and hardware for UAVs as well as pinpoint aspects that need additional thinking. It will list standards and a set of prerequisites (or lack of them thereof) when it comes to CV deployment in UAVs. The issue of data fusion takes a centre place when the book explores ways to deal with sensor data and images as well as their integration and display. The best practices to fuse image and sensor information to enhance UAV performance by means of CV can greatly improve all aspects of the corresponding CPS. The CPS viewpoint can improve the way UAVs interact with the Internet of Things (IoT), use cloud computing, meet communications requirements, implement hardware/software paradigms necessary to handle video streaming, incorporate satellite data, and combine CV with Virtual/Augmented Realities.

VOLUME 1-CONTROL AND PERFORMANCE: This tome explores how sensors and computer vision technologies are used in unmanned aerial vehicles for the navigation, control, stability, reliability, guidance, fault detection, self-maintenance, strategic replanning and reconfiguration of the entire system. It helps analyse the manner UAVs interact with the Internet of Things (IoT), use cloud computing, meet communications requirements, implement hardware/software paradigms necessary to handle still imagery, video streaming, incorporate satellite data, and combine computer vision with virtual/augmented realities (VR/AR).NB: This is planned to be the companion volume of Estrela, Hemanth, Saotome (Eds) / Imaging and Sensing for Unmanned Aerial Vehicles: Volume 2-Deployment and Applications

This paper pursues the best multiclass classification strategy for pose-based 3D human motion recognition using Extreme Learning Machines (ELM). Such classification task is one of the most difficult classification problem because the pose is not unique and the same pose might be randomly distributed inside any unrelated and absolutely different activities. In this study, bag of poses are selected as features and several multiclass classification strategies commonly used in binary classifiers such as Support Vector Machines (SVM) are adopted into the ELM implementation and then compared with non-binary classifications of ELM and binary classifications of SVM. A number of multiclass strategies such as One-Against-All (OAA), One-Against-One (OAO), Directed Acyclic Graph (DAG), hierarchical binary tree, and OAO-3Tree are evaluated and analysed. We found that the OAO-3Tree strategy using Max-Win vote fusion of labeled output function gives the best result.

This paper discusses the control strategies for robot manipulators to track moving targets based on realtime vision guidance. The work is motivated by some new demands of vision guided robot manipulators in which the workpieces being manipulated are in complex motion and the widely adopted look-then-move control strategy cannot give satisfactory performance. In order to serve industrial applications, the limited sensing and actuation capabilities have to be considered properly. In our work, a cascade control structure is introduced. The sensor and actuator limits are dealt with by consecutive modules in the controller respectively. In particular, the kinematic visual servoing (KVS) module is discussed in detail. It is a kinematic controller that generates reference trajectory in real-time. Sliding control is used to give a basic Jacobian-based design. Constrained optimal control is applied to address the actuator limits. validation is conducted through simulation and experiment.

According to several authors in psychology and neurosciences, our ability to perceive affordances is subtended by motor simulation mechanisms. Such mechanisms provide dynamic representations of feasible actions, thus enabling to scale the surrounding structures on the behavioural repertoire and capacities supported by our body. This attractive hypothesis has been taken up in robotics, to build intelligent systems able to determine in advance if a given action would be successful given the current state of the environment and their own skills.
Several arguments however suggest that the motor simulation framework is not sufficient to explain affordance perception: (i) it rests on a misunderstanding of what affordances are: not actions that are currently feasible, but actions that are possible; (ii) it is computationally unrealistic: motor simulation is too costly in terms of computational resources to explain how one can access prospectively to actions that are potentiated by surrounding structures; (iii) it only covers the part of the perceptual field within the scope of our attention, but the affordances we perceive do not reduce to the object or state of affairs our attention is focused on at time t; (iv) it can only work if a first layer of affordances is available: motor simulation cannot explain affordance perception, because its very functioning presupposes such perception. Other mechanisms must consequently be hypothesized.

We present a new method to detect object affordances in real-world scenes using deep Convolutional Neural Networks (CNN), an object detector and dense Conditional Random Fields (CRF). Our system first trains an object detector to generate bounding box candidates from the images. A deep CNN is then used to learn the depth features from these bounding boxes. Finally, these feature maps are post-processed with dense CRF to improve the prediction along class boundaries. The experimental results on our new challenging dataset show that the proposed approach outperforms recent state-of-the-art methods by a substantial margin. Furthermore, from the detected affordances we introduce a grasping method that is robust to noisy data. We demonstrate the effectiveness of our framework on the full-size humanoid robot WALK-MAN using different objects in real-world scenarios.

Optical flow algorithms are difficult to apply to robotic vision applications in practice because of their extremely high computational and frame rate requirements. In most cases, traditional general purpose processors and sequentially executed software cannot compute optical flow in real time. In this paper, a tensor-based optical flow algorithm is developed and implemented using field programmable gate array (FPGA) technology. The resulting algorithm is significantly more accurate than previously published FPGA results and was specifically developed to be implemented using a pipelined hardware structure. The design can process 640 × 480 images at 64 fps, which is fast enough for most real-time robot navigation applications. This design has low resource requirements, making it easier to fit into small embedded systems. Error analysis on a synthetic image sequence is given to show its effectiveness. The algorithm is also tested on a real image sequence to show its robustness and limitations. The resulting limitations are analyzed and an improved scheme is then proposed. It is then shown that the performance of the design could be substantially improved with sufficient hardware resources.

This paper presents a Kalman filter (KF)-integrated optical flow method to measure the velocity of mobile robots using a downward-looking camera. Tests conducted earlier by the authors have shown that currently available differential optical flow methods (X. Song, L. D. Seneviratne, K. Althoefer, and Z. Song, "Vision-based velocity estimation for unmanned ground vehicles," Int. J. Inf. Acquis., vol. 4, no. 4, pp. 303-315, 2007) require large image overlap for accurate velocity estimation. This constraint significantly limits the usefulness of this approach in practical applications. To overcome the problem of dealing with large image displacements, a KF is incorporated to efficiently predict the image transformations. Reducing the feature search area, the KF enables the differential optical flow method to rapidly converge and give accurate velocity estimates. The proposed method has been validated on a linear test rig under laboratory conditions and on a mobile platform in an outdoor field. Test results show good performance of the proposed method in velocity measurements with large image displacements. With this improvement, the required image overlap for feature tracking can be reduced approximately from 80% to 20%, resulting in a fourfold increase of the maximum measurable velocity of the mobile platform. The proposed method has good potential in velocity sensing for mobile robots, particularly in cases, where GPS and inertial measurement unit fail or are unavailable.

—Human Action recognition research is an interesting and active filed of research in the current years. Human Action Recognition (HAR) has many potential and promising applications, in such fields as security, surveillance, professional sports, and human-computer interaction. One of the ongoing challenges of HAR is the creation of methods that are subject invariant, that is methods of action recognition that is not influenced by the appearance of the object in question. One the methods that have been quite popular in recent years are methods that are based on the kinematic features of the action to be recognized. Such features are based on the optical flow of the pixel in time and include features such as the divergence, curl (vorticity) among others. These features are proven to be subject invariant and can easily be calculated in a frame by frame basis. In this study, we present an analysis of feature classification techniques for action classification based on such kinematic features. In this paper, we build a multilayer neural network model trained using Restricted Boltzmann Machines (RBM) that achieves 70% cross-validation accuracy on the Weizmann dataset using kinematic features.

For the motion control of industrial robots, the end-effector performance is of the ultimate interest. However, industrial robots are generally only equipped with motor-side encoders. Accurate estimation of the end-effector position and velocity is thus difficult due to complex joint dynamics. To overcome this problem, this paper presents an optical sensor based on position sensitive detector (PSD), referred as PSD camera, for direct end-effector position sensing. PSD features high precision and fast response while being cost-effective, thus is favorable for real-time feedback applications. In addition, to acquire good velocity estimation, a kinematic Kalman filter (KKF) is applied to fuse the measurement from the PSD camera with that from inertial sensors mounted on the end-effector. The performance of the developed PSD camera and the application of the KKF sensor fusion scheme have been validated through experiments on an industrial robot.

Vision-based sensors (VBSs) provide several advantages to unmanned aircraft systems (UASs) primarily due to a large amount of data they are able to capture, and their reduced size, weight, power, and cost compared to other state-of-the-art sensors. A number of vision-based navigation (VBN) methods have emerged recently, which aim to maximise state-estimation performance and reduce reliance on the global navigation satellite system. This chapter identifies and describes some of the most popular visual navigation strategies for the UAS to acquaint the reader with this important field of study. VBN methods presented here include visual servoing, optical flow-based state estimation, visual odometry and terrain referenced visual navigation. Reference system architectures and relevant mathematical models for these methods are presented to facilitate a more in-depth understanding. A review of these methods and their applications to various UAS use-cases is conducted, focussing primarily on seminal work in this domain. The limitations of this sensing modality are also presented, along with a discussion of future trends including multi-spectral imaging and biomimetic systems to inform the reader of key gaps and research avenues in this field.

We present a modified Temporal Deep Belief Networks (TDBN) for human motion analysis and synthesis by incorporating Sparse Encoding Symmetric Machines (SESM) improvement on its pre-training. SESM consisted of two important terms: regularization and sparsity. In this paper, we measure the effect of these two terms on the smoothness of synthesized (or generated) motion. The smoothness is measured as the standard deviation of five bones movements with three motion transitions. We also address how these two terms influence the free energy and reconstruction error profiles during pre-training of the Restricted Boltzmann Machines (RBM) layers and the Conditional RBM (CRBM) layers. For this purpose, we compare gait transitions by bifurcation experiments using four different TDBN settings: original TDBN; modified-TDBN(R): a TDBN with only regularization constraint; modified-TDBN(S): a TDBN with only sparsity constraint; and modified-TDBN(R+S): a TDBN with regularization plus sparsity constraints. These experiments shows that the modified-TDBN(R+S) reaches lower energy faster in RBM pre-training and reach lower reconstruction error in the CRBM training. Even though the smoothness of the synthesized motion from the modified-TDBN approaches is slightly less smooth than the original TDBN, they are more responsive to the action command to change a motion (from run to walk or vice versa) while preserving the smoothness during motion transitions without incurring much overhead computation time.

In this paper the authors will apply a mathematical system, the Conformal Geometric Algebra (CGA), to propose applications in computer vision and robotics. The CGA keeps our intuition and insight of the problem's geometry at hand, besides helping us to reduce the computational problems' burden.

In this paper, we apply MCMCLDA (Multi-class Markov Chain Latent Dirichlet Allocation) model to classify abnormal activity of students in an examination. Abnormal activity in exams is defined as a cheating activity. We compare the usage of Harris3D interest point detector and a human joints detector, MODEC (Multimodal Decomposable Models), as the feature detector. Experiment results show that using MODEC to detect arm joints and head location as interest point gives better performance in accuracy and computational time than Harris3D when classifying cheating activity. MODEC suffers low accuracy due to its inability to differentiate elbow and wrist when the object wears clothes with indistinguishable colors. Meanwhile, Harris3D detects too many irrelevant interest point to recognize cheating activity reliably.

Consider the problem of visually finding an object in a mostly unknown space with a mobile robot. It is clear that all possible views and images cannot be examined in a practical system. Visual attention is a complex phenomenon; we view it as a mechanism that optimizes the search processes inherent in vision. Here, we de- scribe a particular example

Line Follower Robot is one of autonomous mobile robot form that has mission following autonomously the guided line. In its design and implementation, problems that should be solved are system of robot vision, architecture of hardware including electronics and mechanics, and organization of software for knowledge base and real time control. The goal of this thesis is to design and construct a Line Follower Robot by using microcontroller AT89C51 and infrared sensor. The mechanical system of the robot adopts maneuverability system of common four wheel automobile.Organization of software uses timed event methode for saving microcontroller's timer. Knowledge base of the robot contains codes of action done by robot. This is based on information from sensors. Methode to transform the information becoming action uses look up table methode.

This paper describes a fast plane extraction algorithm for 3D range data. Taking advantage of the point neighborhood structure in data acquired from 3D sensors like range cameras, laser range finders and Microsoft Kinect, it divides the plane-segment extraction task into three steps. The first step is a 2D line segment extraction from raw sensor data, interpreted as 2D data, followed by a line segment based connected component search. The final step finds planes based on connected segment component sets. The first step inspects 2D sub spaces only, leading to a line segment representation of the 3D scan. Connected components of segments represent candidate sets of coplanar segments. Line segment representation and connected components vastly reduce the search space for the plane-extraction step. A region growing algorithm is utilized to find coplanar segments and their optimal (least square error) plane approximation. Region growing contains a fast plane update technique in its core, which combines sets of co-planar segments to form planar elements. Experiments are performed on real world data from different sensors.

Reliable perception is required in order for robots to operate safely in unpredictable and complex human environments.  However, reliability of perceptual inference algorithms has been poorly studied so far.  These algorithms capture uncertain knowledge about the world in the form of probabilistic belief distributions.  A number of Monte Carlo and deterministic approaches have been developed, but their efficiency depends on the degree of smoothness of the beliefs.  In the real world, the smoothness assumption often fails, leading to unreliable perceptual inference results.

Motivated by concrete robotics problems, we propose two novel perceptual inference algorithms that explicitly consider local non-smoothness of beliefs and adapt to it.  Both of these algorithms fall into the category of iterative divide-and-conquer methods and hence scale logarithmically with desired accuracy.  The first algorithm is termed Scaling Series.  It is an iterative Monte Carlo technique coupled with annealing.  Local non-smoothness is accounted for by sampling strategy and by annealing schedule.  The second algorithm is termed GRAB, which stands for Guaranteed Recursive Adaptive Bounding.  GRAB is an iterative adaptive grid algorithm, which relies on bounds.  In this case, local non-smoothness is captured in terms of local bounds and grid resolution.  Scaling Series works well for beliefs with sharp transitions, but without many discontinuities.  GRAB is most appropriate for beliefs with many discontinuities.  Both of these algorithms far outperform the prior art in terms of reliability, efficiency, and accuracy.  GRAB is also able to guarantee that a quality approximation of the belief is produced.

The proposed algorithms are evaluated on a diverse set of real robotics problems: tactile perception, autonomous driving, and mobile manipulation.  In tactile perception, we localize objects in 3D starting with very high initial uncertainty and estimating all 6 degrees of freedom.  The localization is performed based on tactile sensory data.  Using Scaling Series, we obtain highly accurate and reliable results in under 1 second.  Improved tactile object localization contributes to manufacturing applications, where tactile perception is widely used for workpiece localization.  It also enables robotic applications in situations where vision can be obstructed, such as rescue robotics and underwater robotics.

In autonomous driving, we detect and track vehicles in the vicinity of the robot based on 2D and 3D laser range finders.  In addition to estimating position and velocity of vehicles, we also model and estimate their geometric shape.  The geometric model leads to highly accurate estimates of pose and velocity for each vehicle.  It also greatly simplifies association of data, which are often split up into separate clusters due to occlusion. The proposed Scaling Series algorithm greatly improves reliability and ensures that the problem is solved within tight real time constraints of autonomous driving. 

In mobile manipulation, we achieve highly accurate robot localization based on commonly used 2D laser range finders using the GRAB algorithm.  We show that the high accuracy allows robots to navigate in tight spaces and manipulate objects without having to sense them directly.  We demonstrate our approach on the example of simultaneous building navigation, door handle manipulation, and door opening.  We also propose hybrid environment models, which combine high resolution polygons for objects of interest with low resolution occupancy grid representations for the rest of the environment.  High accuracy indoor localization contributes directly to home/office mobile robotics as well as to future robotics applications in construction, inspection, and maintenance of buildings.

Based on the success of the proposed perceptual inference algorithms in the concrete robotics problems, it is our hope that this thesis will serve as a starting point for further development of highly reliable perceptual inference methods.

This paper proposes an MCU based robotic elbow movement  control.  In recent years, a lot of research has been developed  in alignment with mimicking body movement especially in the  field of medical and industrial applications. Robotic movement has different implementations. The usual method is through PID controller. However, implementation is complicated,  response is slow, and movement control is delayed. Speed is  also a problem causing the human arm and robotic arm not  synchronized with their movement. In this paper, the fuzzy logic method is substituted to PID control. The MCU is used in the implementation of robotic elbow movement. The experimental results are now carried out to demonstrate the  effectiveness of the proposed technique and method. Results  show that the proposed method is preferable and better over the conventional PID from response, delay, and also the movement control through three variable speed controls. Also,  the  movement control of the robotic elbow do synchronized  with the human arms with negligible delay. In conclusion, the  application of microcontroller does improve the movement of  the robotic elbow. The switching response is faster and the delay is minimized caused by the PID control.

Recent advances in computer vision on the one hand, and imaging technologies on the other hand, have opened up a number of interesting possibilities for robust 3D scene labeling. This paper presents contributions in several directions to improve the state-of-the-art in RGB-D scene labeling. First, we present a novel combination of depth and color features to recognize different object categories in isolation. Then, we use a context model that exploits detection results of other objects in the scene to jointly optimize labels of co-occurring objects in the scene. Finally, we investigate the use of social media mining to develop the context model, and provide an investigation of its convergence. We perform thorough experimentation on both the publicly available RGB-D Dataset from the University of Washington as well as on the NYU scene dataset. An analysis of the results shows interesting insights about contextual object category recognition, and its benefits.

In this paper, we present a method to navigate a mobile robot using a webcam. This method determines the shortest path for the robot to transverse to its target location, while avoiding obstacles along the way. The environment is first captured as an image using a webcam. Image processing methods are then performed to identify the existence of obstacles within the environment. Using the Voronoi Diagrams VD(s) method, locations with obstacles are identified and the corresponding Voronoi cells are eliminated. From the remaining Voronoi cells, the shortest path to the goal is identified. The program is written in MATLAB with the Image Processing toolbox. The proposed method does not make use of any other type of sensor other than the webcam.

İnsanlar ve diğer canlılar mutluluk, üzüntü, korku, öfke gibi duygulara sahiptir. Peki, robotların bu tür duyguları var mıdır veya duygusal robot teori gelecekte mümkün olabilecek midir? Robotlar gittikçe daha akıllı hale gelen varlıklar ve insanlarla etkileşimlerinde büyük bir yol kat ettiler. Ancak bir robotu duygusal hale getirmek için neler gerekir ve bu kapasiteye sahip olmalarını hiç ister miyiz?

Shape matching and point correspondence recovering play a fundamental role in applications like pattern and object recognition, shape classification, image alignment and registration, visual information data mining, and many other computer vision related areas, which recently became active research disciplines due to the increasing needs in many practical fields. This article introduces the Contour-Point Signature (CPS), a novel technique that produces features that are invariant to translation, scaling, and rotation. The approach make use of CPS to find the correspondences of points extracted from the contours of two similar shapes and to establish a relation to map arbitrary points from one shape to another.