Robot Vision

This paper presents an obstacle detection and avoidance of mobile robot using stereo camera for indoor environment. Block matching algorithm is solved the correspondence problem occurred in comparing stereo images (left and right sensors of the camera). The algorithm uses Sum of Absolute Differences (SAD). Left image works as a reference block to the right image and the output is disparity mapping or depth maps with the left coordinate system. A pair of camera or stereo vision baseline is based on horizontal configuration. The block matching technique is briefly described with the performance of its output. The curve fitting tool would determine the range of each obstacle detected in disparity mapping. The programming activities are using Matlab software starting from capturing images until navigation of mobile robot.

Recent advancements in laser and visible light sensor technology allows for the collection of photorealistic 3D scans of large scale spaces. This enables the technology to be used in real world applications such as crime scene investigation. The 3D models of the environment obtained with a 3D scanner capture visible surfaces but do not provide semantic information about salient features within the captured scene. Later processing must convert these raw scans into salient scene structure. This paper describes ongoing research into the generation of semantic data from the 3D scan of a crime scene to aid forensic specialists in crime scene investigation and analysis.

This paper presents a robotic vision system that automatically retrieves and positions surgical instruments during robotized laparoscopic surgical operations. The instrument is mounted on the end-effector of a surgical robot which is controlled by visual servoing. The goal of the automated task is to safely bring the instrument at a desired 3-D location from an unknown or hidden position. Light Emitting Diodes are attached on the tip of the instrument and a speci£c instrument-holder £tted with optical £bers is used to project laser dots on the surface of the organs. These optical markers are detected in the endoscopic image and allow to localize the instrument with respect to the scene. The instrument is recovered and centered in the image plane by means of a visual servoing algorithm using feature errors in the image. With this system, the surgeon can specify a desired relative position between the instrument and the pointed organ. The relationship between the velocity screw of the surgical instrument and the velocity of the markers in the image is estimated online and, for safety reasons, a multi-stages servoing scheme is proposed. Our approach has been successfully validated in a real surgical environment by performing experiments on living tissues in the surgical training room of IRCAD.

A research area in Computer Vision focuses on the identification of articulated objects, such as human actions and movements of the hand, which can be used in human-computer interaction, surveillance, and other tracking systems. Two problems arise: identify when two articulated objects in different stances are in the same class of objects, and differentiate the distinct positions of the same object. In both cases, it is necessary to know how correspond the different points or regions of such objects standing in different attitudes. This article presents the Contour-Point Signature; a point descriptor that allows to establish a method to achieve the better matching of points between two figures, and to thus obtain a transformation which relates them. With this descriptor, we can achieve more accurate shape features and implement more efficient retrieval under multi-resolution. In addition, CPS is robust to rigid translation, scaling, rotation and independent of the origin point. A measure of dissimilarity between two figures for classifying various human postures in a video sequence is also presented.

This paper deals with the mechanical aspects of robot vision and focuses on the design and implementation of a high-performance system. The system is called CeDAR (Cable Drive Active vision Robot). As part of the research at the Robotic Systems Laboratory (RSL) to develop real-world vision systems, CeDAR is designed for speed and accuracy through its novel use of cables in zerobacklash transmissions and a parallel mechanical architecture. By optimising the design for agility, the system is able to a carry a large payload (700g) while achieving its performance specification of five 90°saccades per second with an angular repeatability of 0.01°in all three axes.

This paper devises a new multicriteria image based controller for the control of six degrees of freedom (PUMA560) robotic arm, based upon Linear Matrix Inequality (LMI). The aim lies in developing such a method that neither involves camera calibration parameters nor inverse kinematics. The approach adopted in this paper includes transpose Jacobian control; thus, inverse of the Jacobian matrix is no longer required. The proposed controller allows stabilizing the camera despite the unknown value of the target point depth. To make sure that the features remain in the camera field of view, and to restrict the controller's input using some bounds, visibility and kinematic constraints are introduced in the form of LMIs. By invoking the Lyapunov's direct method, closed loop stability of the system is ensured. Simulation results are shown for three different cases, which exhibit the system stability and convergence even in the presence of large errors, and present the comparative analysis of both i.e., systems with and without visibility and kinematic constraints.

For many decades automatic facial expression recognition has scientifically been considered a real challenging problem in the fields of pattern recognition or robotic vision. The current research aims at proposing Relevance Vector Machines (RVM) as a novel classification technique for the recognition of facial expressions in static images. The aspects related to the use of Support Vector Machines are also presented. The data for testing were selected from the Cohn-Kanade Facial Expression Database. We report 90.84% recognition rates for RVM for six universal expressions based on a range of experiments. Some discussions on the comparison of different classification methods are included.

Motion vision (visual odometry, the estimation of camera egomotion) is a well researched field, yet has seen relatively limited use despite strong evidence from biological systems that vision can be extremely valuable for navigation. The limited use of such vision techniques has been attributed to a lack of good algorithms and insufficient computer power, but both of those problems were resolved as long as a decade ago. A gap presently yawns between theory and practice, perhaps due to perceptions of robot vision as less reliable and more complex than other types of sensing. We present an experimental methodology for assessing the realworld precision and reliability of visual odometry techniques in both normal and extreme terrain. This paper evaluates the performance of a mobile robot equipped with a simple vision system in common outdoor and indoor environments, including grass, pavement, ice, and carpet. Our results show that motion vision algorithms can be robust and effective, and suggest a number of directions for further development.

A pulsed time-of-flight laser range finder with a 1 GHz avalanche photo diode ͑APD͒ receiver and a laser pulser with ϳ35 ps pulse width has been developed and tested. The receiver channel is constructed using a silicon ASIC chip and a commercially available silicon APD placed on a hybrid ceramic susbstrate. The laser pulser utilizes a single heterostructure laser operating in Q-switching mode. It is shown that the single-shot precision of the complete laser range finder is ϳ2.1 mm ͑ value͒ at best. The nonaccuracy in the distance range of 0.5-34.5 m was ϳϮ2 mm excluding errors caused by the statistical variations and long-term instability. The single-shot precision is clearly better than the single-shot precision of the earlier laser range finders with ϳ100-200 MHz bandwidths. Also, two types of optics, coaxial and paraxial, were tested. The linearity of the coaxial optics was better, especially with a long ͑4 m͒ receiver fiber. Some possible applications of the laser range finder utilizing ps level pulses are, for example, fast three-dimensional vision in industrial environments and structure analysis of materials.

Developing easy to use, intuitive interfaces is crucial to introduce robotic automation to many small medium sized enterprises (SMEs). Due to their continuously changing product lines, reprogramming costs exceed installation costs by a large margin. In addition, traditional programming methods for industrial robots is too complex for an inexperienced robot programmer, thus external assistance is often needed. In this paper a new incremental multimodal language, which uses augmented reality (AR) environment, is presented. The proposed language architecture makes it possible to manipulate, pick or place the objects in the scene. This approach shifts the focus of industrial robot programming from coordinate based programming paradigm, to object based programming scheme. This makes it possible for non-experts to program the robot in an intuitive way, without going through rigorous training in robot programming.

In this paper, we present a solution and experimental results for real-time control of manipulation tasks considering the alignment of a robots end effector relative to some reference. The developed controller is applicable to industrial manipulators and is based on the pseudoinverse redundancy resolution method. The application considered is the employment of two industrial robots in an offshore remote inspection and maintenance system. A leader robot is controlled freely from onshore, and a follower robot uses an attached camera to provide the onshore operator with live video feed of the ongoing operation. Robot manipulators constitute flexible camera platforms, compared to e.g. simple pan/tilt units, for monitoring offshore operations. We develop a controller for the follower robot such that automatic camera tracking is achieved using pseudoinverse redundancy resolution control. A minimal task space parametrization relying on stereographic projection is constructed which achieves relative end effector alignment tracking without introducing representational or algorithmic singularities. It is shown that singular configurations will only in special cases affect the closed loop behavior. The controller is applicable to tasks such as spray painting or polishing on curved surfaces. The control approach is experimentally validated on two Kuka KR-16 industrial robot manipulators.

A comprehensive approach to the design and implementation of multi-robots cooperative systems is described. It focuses on a design process that uses the Unified Modeling Language and on a detailed ontology description with the goal of sharing the knowledge on environments that robots can acquire through the use of their vision sub-system. We base the implementation of our robotics vision system on agents inserted in a generic multi-level architectures. The first objective of this work is to provide a framework ...

Vision-based robot navigation systems allow a robot to explore and to navigate in its environment in a way that facilitates path planning and goal-oriented tasks. The vision sensor is mainly used for obstacle detection and avoidance, object detection and tracking, and interaction with users. Usually these systems do not depend solely on vision sensors but use other sensors like sonar and laser range finder. The paper considers an important issue for mobile robots navigation. This issue is the detection of obstacles in front of the robot within a corridor. We proposed and evaluated three algorithms for obstacle detection within a corridor environment using image processing techniques.

We would like to invite you to join this exciting new project as a chapter contributor. Since this is a textbook, a great deal of this chapter entails a survey on the topic under the paradigm of cyber-physical systems, what can be done onboard and remotely, the distributed nature of the system and some exercises on futurology (anticipating trends can shed some light on upcoming designs). IET will bring great visibility to your work. You are welcome to suggest another topic/chapter title if you feel it would be more suitable. Each chapter should be around 20-25 pages each and can be submitted as a Word or Latex File. The IET will send you additional information (formatting, permission form, etc.) with the contributor's agreement once you have agreed to contribute to the book. Visit http:// www.theiet.org/resources/author-hub/books/index.cfm to get all information you need as a contributor to an IET research-level book. Each book is expected to have a total number of 500 printed pages (based on approximately 550 words per page with a 20% allowance for figures and tables). We have included a tentative schedule and list of topics below. If this is something you would consider, please send me the title of your chapter, a short description/abstract of the chapter content, and your full contact details. We will expect original content and new results for this book. You can, of course, reuse published material but the percentage of material reuse for the chapter should be less than 40%. The IET will run a piracy software on the full manuscript to control that you are including original material and will reject chapters who contain a large amount of already-published material so please do take this into consideration.

Currently, a major difficulty for the widespread use of robots in assembly and material handling comes from the necessity of feeding accurately positioned workpieces to robots. ``Bin picking'' techniques help reduce this constraint. This paper presents the application of matched filters for enabling robots with vision to acquire workpieces randomly stored in bins. This approach complements heuristic methods already reported. The concept of matched filter is an old one. Here, however, it is redefined to take into account robot end-effector features, in terms of geometry and mechanics. In particular, the proposed filters match local workpiece structures where the robot end-effector is likely to grasp successfully and hold workpieces. The local nature of the holdsites is very important as computation costs are shown to vary with the fifth power of structure size. In addition, the proposed filters tend to have a narrow angular bandwidth. An example, which features a parallel-jaw hand is developed in detail, using both statistical and Fourier models. Both approaches concur in requiring a very small number of filters (typically four), even if a good orientation accuracy is expected (two degrees). Success rates of about 90 percent in three or fewer attempts have been experimentally obtained on a system which includes a small minicomputer, a 128 × 128 pixel solidstate camera, a prototype Cartesian robot, and a ``universal'' parallel-jaw hand.

This paper presents a Kalman filter (KF)-integrated optical flow method to measure the velocity of mobile robots using a downward-looking camera. Tests conducted earlier by the authors have shown that currently available differential optical flow methods (X. Song, L. D. Seneviratne, K. Althoefer, and Z. Song, "Vision-based velocity estimation for unmanned ground vehicles," Int. J. Inf. Acquis., vol. 4, no. 4, pp. 303-315, 2007) require large image overlap for accurate velocity estimation. This constraint significantly limits the usefulness of this approach in practical applications. To overcome the problem of dealing with large image displacements, a KF is incorporated to efficiently predict the image transformations. Reducing the feature search area, the KF enables the differential optical flow method to rapidly converge and give accurate velocity estimates. The proposed method has been validated on a linear test rig under laboratory conditions and on a mobile platform in an outdoor field. Test results show good performance of the proposed method in velocity measurements with large image displacements. With this improvement, the required image overlap for feature tracking can be reduced approximately from 80% to 20%, resulting in a fourfold increase of the maximum measurable velocity of the mobile platform. The proposed method has good potential in velocity sensing for mobile robots, particularly in cases, where GPS and inertial measurement unit fail or are unavailable.

This paper describes a fast plane extraction algorithm for 3D range data. Taking advantage of the point neighborhood structure in data acquired from 3D sensors like range cameras, laser range finders and Microsoft Kinect, it divides the plane-segment extraction task into three steps. The first step is a 2D line segment extraction from raw sensor data, interpreted as 2D data, followed by a line segment based connected component search. The final step finds planes based on connected segment component sets. The first step inspects 2D sub spaces only, leading to a line segment representation of the 3D scan. Connected components of segments represent candidate sets of coplanar segments. Line segment representation and connected components vastly reduce the search space for the plane-extraction step. A region growing algorithm is utilized to find coplanar segments and their optimal (least square error) plane approximation. Region growing contains a fast plane update technique in its core, which combines sets of co-planar segments to form planar elements. Experiments are performed on real world data from different sensors.

This paper describes two humanoid robots developed in the Humanoid Robotics Institute, Waseda University. Hadaly-2 is intended to realize information interaction with humans by integrating environmental recognition with vision, conversation capability (voice recognition, voice synthesis), and gesture behaviors. It also possesses physical interaction functions for direct contact with humans and behaviors that are gentle and safe for humans. WABIAN is a robot with a complete human configuration that is capable of walking on two legs and ...

For many desirable applications of vision guided industrial robots, real-time visual servoing is necessary but also challenging. Difficulty comes from the limited sampling rate and response time of typical machine vision systems equipped on industrial robots. These factors are addressed as the dynamics of visual sensing. In addition, robot dynamics should also be fully considered when designing the control law. Considering these aspects, this paper presents a control scheme of visual servoing. A dual-rate adaptive tracking filter is presented to compensate the visual sensing dynamics. Based on the compensated vision feedback, the techniques of multi-surface sliding control and dynamic surface control are used to formulate a two-layer control law for target tracking. System kinematics and dynamics are decoupled and dealt with by the two layers of the control law respectively. The proposed method is validated through experiments on a SCARA robot.

We kindly ask you to participate in this research devoted to the adoption of service robots by travel, tourism and hospitality companies. By completing the questionnaire, you will be eligible to participate in an opportunity for one of five electronic 100-dollar gift cards, redeemable as Amazon gift vouchers, among other options. Once you have completed the survey, a separate link will be sent to you allowing you to register for the opportunity. The researchers will not be able to identify your responses to the survey questions and the identifying information you will leave for the purposes of the opportunity for the gift vouchers.

For questions about your rights as a research subject, please contact the Director, Office of Research Integrity, Ball State University, Muncie, IN 47306, (765) 285-5070 or at irb@bsu.edu. IRB Protocol Number: 1194315-1

Please complete the questionnaire if only you are at least 18 years of age.
Thank you in advance for your participation!

Here is the link to the survey:

https://bsu.qualtrics.com/jfe/form/SV_4PdOOi6ERWHw97v

Sincerely,

Dr. Craig Webster, Ball State University, USA, email: cwebster3@bsu.edu
Prof. Stanislav Ivanov, Varna University Management, Bulgaria, email: stanislav.ivanov@vumk.eu

One of the major issues humans deal with on a regular basis is their own (at times, of their possessions) safety from various external agencies like car crashes, or homicide threats due to concealed weapons. To tackle this issue, a robotic vision and image processing application could be developed and trained to recognize situations where human lives could be at hazard. For this proposal, a portable robot equipped with a set of cameras (infrared + normal vision as well as a LIDAR) is being considered, to be able to detect imminent crashes in moving vehicles or detect if there are concealed weapons on a personnel, all in real-time. The robot would survey human surroundings 24*7 and give appropriate warnings/ signals in case of danger. This robot could further be developed to perform preventive actions if a situation of alert is detected thus, acting as a trained bodyguard. With the present advances in technology, it is possible to develop such robotic applications that will make our lives much safer and easier.

Deep learning and image processing are two areas of great interest to academics and industry professionals alike. The areas of application of these two disciplines range widely, encompassing fields such as medicine, robotics, and security and surveillance. The aim of this book, 'Deep Learning for Image Processing Applications', is to offer concepts from these two areas in the same platform, and the book brings together the shared ideas of professionals from academia and research about problems and solutions relating to the multifaceted aspects of the two disciplines. The first chapter provides an introduction to deep learning, and serves as the basis for much of what follows in the subsequent chapters, which cover subjects including: the application of deep neural networks for image classification; hand gesture recognition in robotics; deep learning techniques for image retrieval; disease detection using deep learning techniques; and the comparative analysis of deep data and big data. The book will be of interest to all those whose work involves the use of deep learning and image processing techniques

Convolutional neural networks (CNNs) represent an interesting method for adaptive image processing, and form a link between general feedforward neural networks and adaptive filters. Two dimensional CNNs are formed by one or more layers of two dimensional filters, with possible non-linear activation functions and/or down-sampling. CNNs possess key properties of translation invariance and spatially local connections (receptive fields). The present paper presents a description of the convolutional network architecture, and an application to a practical image processing application on a mobile robot. A CNN is used to detect and characterize cracks on an autonomous sewer inspection robot. The filter sizes used in all layers of the CNN used here was 5x5, and the common activation function used was a log-sigmoid. The network was trained using a dataset of 39 still image 320x240 pixel frames sampled from a pre-recorded sewer pipe inspection video. The results support the notion that data-based adaptive image processing methods such as CNNs are useful for image processing, or other applications where the input arrays are large, and spatially / temporally distributed. Further refinements of the CNN architecture, such as the implementation of separable filters, or extensions to three dimensional (ie video) processing, are suggested.

ABSTRACT This work presents a system for the automatic selective harvesting of asparagus in open field being developed in the framework of the Italian National Project on Robotics. It is composed of a mobile robot, equipped with a suitable manipulator, and driven by a stereo-vision module. In this paper we discuss in detail the problems related to the vision module.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org.

In the robotic community Robotic Arm is one of the popular ideas. Robotic arms are very usual in industries where they are mainly used in assembly lines in manufacturing plants. Building a Robotic Arm is a difficult process and involves complex programming for beginners. Once we put our full focus on the thing and even if our thoughts were the same in the at start , we found it rather easy as all it takes was some time and tinkering. The robotic arm we have built in this project is fully automated to do tasks. Apart from the Microcontroller we have used Arduino Nano and servo motors, all other components. This robotic arm can color sort and for that we have used sensors.The main motto of this project is to design an efficient system that pick up right color of objects and put it down at right place to the minimizing the cost of the products, optimizes productivity and decreasing human mistakes. This paper presents an application to differentiate colored objects with a robotic arm. This robotic arm can pick different colored objects and sorts them in specified cups. It contacts with color sensor TCS 3200 and various motor modules in real time to detect the correct color object and to control the movement of arm.

The autonomous delivery robot is meant to be a substitute for a goods delivery person. The delivery robot is capable of navigating through a cluttered space environment from a home location to a destination point while avoiding obstacles in the process. It uses an ultrasonic sensor to detect if anything has been placed inside the bin and only once when something is placed inside, the robot starts moving from a position to another. Bluetooth beacons are used to represent the start, end and mid-point which will be used by the robot to differentiate the specified locations. Bluetooth beacons will act as a terminal for the robot to detect. The autonomous robot will move on an undesignated path i.e. an unmarked route unlike the archaic robots only capable of moving inside a marked black lane. The robot will be trained and multiple simulations are run on it with the help of the DonkeyCar library so it can successfully avoid obstacles and relay the path with efficacy, making the delivery fast and efficient. A Pi camera is used which will help rectify the unmarked desired path over which the robot has to move.

A new technique for three-dimensional (3D) camera calibration for machine vision metrology using off-the-shelf TV cameras and lenses is described. The two-stage technique is aimed at efficient computation of camera external position and orientation relative to object reference coordinate system as well as the effective focal length, radial lens distortion, and image scanning parameters. The two-stage technique has advantage in terms of accuracy, speed, and versatility over existing state of the art. A critical review of the state of the art is given in the beginning. A theoretical framework is established, supported by comprehensive proof in five appendixes, and may pave the way for future research on 3D robotics vision. Test results using real data are described. Both accuracy and speed are reported. The experimental results are analyzed and compared with theoretical prediction. Recent effort indicates that with slight modification, the two-stage calibration can be done in real time.

IET indexes its books and journal in SCOPUS and IEEE Xplore.
Computer Vision (CV) and Sensors play a decisive role in the operation of Unmanned Aerial Vehicle (UAV), but there exists a void when it comes to analysing the extent of their impact on the entire UAV system. In general, the fact that a UAV is a Cyber-Physical System (CPS) is not taken into account. In this proposal, we propose to expand on earlier books covering the use of CV and sensing in UAVs. Among other things, an entirely autonomous UAV can help to (i) obtain information about the environment, (ii) work for an extended period of time without human interference, (iii) move either all or part of itself all over its operating location devoid of human help and (iv) stay away from dangerous situations for people and their possessions. A Vision System (VS) entails the way CV data will be utilized, the appropriate architecture for total avionics integration, the control interfaces, and the UAV operation. Since the VS core is its sensors and cameras, multi-sensor fusion, navigation, hazard detection, and ground correlation in real time are important operational aspects that can benefit from CV knowledge and technology. This book will aim to collect and shed some light on the existing information on CV software and hardware for UAVs as well as pinpoint aspects that need additional thinking. It will list standards and a set of prerequisites (or lack of them thereof) when it comes to CV deployment in UAVs. The issue of data fusion takes a centre place when the book explores ways to deal with sensor data and images as well as their integration and display. The best practices to fuse image and sensor information to enhance UAV performance by means of CV can greatly improve all aspects of the corresponding CPS. The CPS viewpoint can improve the way UAVs interact with the Internet of Things (IoT), use cloud computing, meet communications requirements, implement hardware/software paradigms necessary to handle video streaming, incorporate satellite data, and combine CV with Virtual/Augmented Realities.

VOLUME 1-CONTROL AND PERFORMANCE: This tome explores how sensors and computer vision technologies are used in unmanned aerial vehicles for the navigation, control, stability, reliability, guidance, fault detection, self-maintenance, strategic replanning and reconfiguration of the entire system. It helps analyse the manner UAVs interact with the Internet of Things (IoT), use cloud computing, meet communications requirements, implement hardware/software paradigms necessary to handle still imagery, video streaming, incorporate satellite data, and combine computer vision with virtual/augmented realities (VR/AR).NB: This is planned to be the companion volume of Estrela, Hemanth, Saotome (Eds) / Imaging and Sensing for Unmanned Aerial Vehicles: Volume 2-Deployment and Applications

We present a new method to translate videos to commands for robotic manipulation using Deep Recurrent Neural Networks (RNN). Our framework first extracts deep features from the input video frames with a deep Convolutional Neural Networks (CNN). Two RNN layers with an encoder-decoder architecture are then used to encode the visual features and sequentially generate the output words as the command. We demonstrate that the translation accuracy can be improved by allowing a smooth transaction between two RNN layers and using the state-of-the-art feature extractor. The experimental results on our new challenging dataset show that our approach outperforms recent methods by a fair margin. Furthermore, we combine the proposed translation module with the vision and planning system to let a robot perform various manipulation tasks. Finally, we demonstrate the effectiveness of our framework on a full-size humanoid robot WALK-MAN.

In this paper the authors will apply a mathematical system, the Conformal Geometric Algebra (CGA), to propose applications in computer vision and robotics. The CGA keeps our intuition and insight of the problem's geometry at hand, besides helping us to reduce the computational problems' burden.

The conference was organized in three simultaneous tracks: "Intelligent Control Systems and Optimization", "Robotics and Automation" and "Systems Modeling, Signal Processing and Control". The book is based on the same structure. Although ICINCO 2004 received 311 paper submissions, from 51 different countries in all continents, only 115 where accepted as full papers. From those, only 29 were selected for inclusion in this book, based on the classifications provided by the Program Committee. The selected papers also reflect the interdisciplinary nature of the conference. The diversity of topics is an importante feature of this conference, enabling an overall perception of several important scientific and technological trends. These high quality standards will be maintained and reinforced at ICINCO 2005, to be held in Barcelona, Spain, and in future editions of this conference. Furthermore, ICINCO 2004 included 6 plenary keynote lectures and 2 tutorials, given by internationally recognized researchers. Their presentations represented an important contribution to increasing the overall quality of the conference, and are partially included in the first section of the book.

This paper discusses the control strategies for robot manipulators to track moving targets based on realtime vision guidance. The work is motivated by some new demands of vision guided robot manipulators in which the workpieces being manipulated are in complex motion and the widely adopted look-then-move control strategy cannot give satisfactory performance. In order to serve industrial applications, the limited sensing and actuation capabilities have to be considered properly. In our work, a cascade control structure is introduced. The sensor and actuator limits are dealt with by consecutive modules in the controller respectively. In particular, the kinematic visual servoing (KVS) module is discussed in detail. It is a kinematic controller that generates reference trajectory in real-time. Sliding control is used to give a basic Jacobian-based design. Constrained optimal control is applied to address the actuator limits. validation is conducted through simulation and experiment.