Abstract
This report proposes a novel approach to learn from demonstrations and classify contextual tasks the human operator executes by remotely controlling a mobile robot with joystick, aiming to assist mobile robot teleoperation within a shared autonomy system in a task-appropriate manner. The proposed classifier is implemented with the Gaussian Process (GP). GP is superior in uncertainty estimation when predicting class labels (i.e. the introspective capability) over other state-of-art classification methods, such as Support Vector Machine (SVM), which is probably the most widely used approach on this topic to date. Moreover, to keep the learned model sparse to limit the amount of storage and computation required, full GP is approximated with a state-of-art Sparse Online Gaussian Process (SOGP) algorithm, to maintain scalability to large datasets without compromising classification performance. The proposed approach is extensively evaluated on real data and verified to outperform the baseline classifiers both in classification accuracy and uncertainty estimation in predicting class labels, while maintaining sparsity and real-time property to scale with large datasets. This demonstrates the feasibility of the proposed approach for online use in real applications.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Alkan, A., Günay, M.: Identification of emg signals using discriminant analysis and svm classifier. Expert Syst. Appl. 39(1), 44–47 (2012)
Allen, J.E., Guinn, C.I.: Mixed-initiative interaction. IEEE Intell. Syst. Appl. 14(5), 14–23 (1999)
Argall, B.D., Chernova, S., Veloso, M., Browning, B.: A survey of robot learning from demonstration. Robot. Auton. Syst. 57(5), 469–483 (2009)
Berczi, L.P., Posner, I., Barfoot, T.D.: Learning to assess terrain from human demonstration using an introspective gaussian-process classifier. In: 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 3178–3185. IEEE (2015)
Bidder, O.R., Campbell, H.A., Gómez-Laich, A., Urgé, P., Walker, J., Cai, Y., Gao, L., Quintana, F., Wilson, R.P.: Love thy neighbour: automatic animal behavioural classification of acceleration data using the k-nearest neighbour algorithm. PloS one 9(2), e88,609 (2014)
Billings, C.E.: Aviation automation: The search for a human-centered approach (1997)
Britton, N., Yoshida, K., Walker, J., Nagatani, K., Taylor, G., Dauphin, L.: Lunar micro rover design for exploration through virtual reality tele-operation. In: Field and Service Robotics, pp. 259–272. Springer (2015)
Burridge, R.R., Hambuchen, K.A.: Using prediction to enhance remote robot supervision across time delay. In: 2009. IROS 2009. IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5628–5634. IEEE (2009)
Chang, C.C., Lin, C.J.: Libsvm: a library for support vector machines. ACM Trans. Intell. Syst. Technol. (TIST) 2(3), 27 (2011)
Csató, L., Opper, M.: Sparse on-line gaussian processes. Neural Comput. 14(3), 641–668 (2002)
Dragan, A., Srinivasa, S.S.: Formalizing assistive teleoperation. R: SS (2012)
Ellis, K., Kerr, J., Godbole, S., Lanckriet, G., Wing, D., Marshall, S.: A random forest classifier for the prediction of energy expenditure and type of physical activity from wrist and hip accelerometers. Physiol. Measur. 35(11), 2191 (2014)
Ferber, J.: Multi-agent systems: an introduction to distributed artificial intelligence, vol. 1. Addison-Wesley, Reading (1999)
Fong, T., Thorpe, C.: Vehicle teleoperation interfaces. Autonomous Robots 11(1), 9–18 (2001)
Gao, M., Oberlȧnder, J., Schamm, T., Zȯllner, J.M.: Contextual Task-Aware Shared Autonomy for Assistive Mobile Robot Teleoperation. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE (2014)
Gao, M., Oberlȧnder, J., Schamm, T., Zȯllner, J.M.: Shared autonomy for assisted mobile robot teleoperation by recognizing operator intention as contextual task. In: 2014 IEEE International Conference on Robotics and Biomimetics (ROBIO). IEEE (2014)
Gao, M., Schamm, T., J.Marius, Z.: Contextual task recognition to assist mobile robot teleoperation with introspective estimation using gaussian process. In: 2016 IEEE International Conference on Autonomous Robot Systems and Competitions. IEEE (2016)
Gao, M., Schamm, T., Zȯllner, J.M.: Multiple contextual task recognition for sharing autonomy to assist mobile robot teleoperation. In: 2015 International Conference on Intelligent Robotics and Application (IRA). Springer (2015)
Gao, M., Schamm, T., Zȯllner, J.M.: Online and incremental contextual task learning and recognition for sharing autonomy to assist mobile robot teleoperation. In: 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO). IEEE (2015)
Geng, Y., Chen, J., Fu, R., Bao, G., Pahlavan, K.: Enlighten wearable physiological monitoring systems: On-body rf characteristics based human motion classification using a support vector machine. IEEE Trans. Mob. Comput. 15(3), 656–671 (2016)
Griffiths, P.G., Gillespie, R.B.: Sharing control between humans and automation using haptic interface: primary and secondary task performance benefits. Human Factors: J. Human Factors Ergon. Soc. 47(3), 574–590 (2005)
Grimmett, H., Paul, R., Triebel, R., Posner, I.: Knowing when we don’t know: Introspective classification for mission-critical decision making. In: 2013 IEEE International Conference on Robotics and Automation (ICRA), pp. 4531–4538. IEEE (2013)
Grimmett, H., Triebel, R., Paul, R., Posner, I.: Introspective classification for robot perception. Int. J. Robot. Res. 35(7), 743–762 (2016)
Grollman, D.H., Jenkins, O.C.: Sparse incremental learning for interactive robot control policy estimation. In: 2008. ICRA 2008. IEEE International Conference on Robotics and Automation, pp. 3315–3320. IEEE (2008)
Grollman, D.H., Jenkins, O.C.: Incremental learning of subtasks from unsegmented demonstration. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 261–266. IEEE (2010)
Guo, S., Pang, M., Gao, B., Hirata, H., Ishihara, H.: Comparison of semg-based feature extraction and motion classification methods for upper-limb movement. Sensors 15(4), 9022–9038 (2015)
Hauser, K.: Recognition, prediction, and planning for assisted teleoperation of freeform tasks. Auton. Robot. 35(4), 241–254 (2013)
Kapoor, A., Grauman, K., Urtasun, R., Darrell, T.: Gaussian processes for object categorization. Int. J. Comput. Vis. 88(2), 169–188 (2010)
Keskinpala, H.K., Adams, J.A., Kawamura, K.: PDA-based human-robotic interface. In: 2003. IEEE International Conference on Systems, Man and Cybernetics, vol. 4, pp. 3931–3936. IEEE (2003)
Khan, A.M., Tufail, A., Khattak, A.M., Laine, T.H.: Activity recognition on smartphones via sensor-fusion and kda-based svms. International Journal of Distributed Sensor Networks (2014)
Kohlbrecher, S., Meyer, J., von Stryk, O., Klingauf, U.: A flexible and scalable slam system with full 3d motion estimation. In: Security and Rescue Robotics (SSRR) Proceedings of IEEE International Symposium on Safety. IEEE (2011)
Lee, D.H., Noh, K.W., Park, S.H., Lee, J.M.: Accurate velocity control based on the distance of a mobile robot and obstacle using a haptic joystick. In: 2012 12th International Conference on Control, Automation and Systems (ICCAS), pp. 1649–1654. IEEE (2012)
Liarokapis, M.V., Artemiadis, P.K., Katsiaris, P.T., Kyriakopoulos, K.J., Manolakos, E.S.: Learning human reach-to-grasp strategies: Towards emg-based control of robotic arm-hand systems. In: 2012 IEEE International Conference on Robotics and Automation (ICRA), pp. 2287–2292. IEEE (2012)
Lin, M.H., Song, K.T.: Design and experimental study of a shared-controlled omnidirectional mobile platform. In: 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 3579–3584. IEEE (2014)
Lozano-Perez, T., Cox, I.J., Wilfong, G.T.: Autonomous robot vehicles. Springer Science & Business Media, Berlin (2012)
Nielsen, C.W., Goodrich, M.A., Ricks, R.W.: Ecological interfaces for improving mobile robot teleoperation. IEEE Trans. Robot. 23(5), 927–941 (2007)
Okada, Y., Nagatani, K., Yoshida, K., Tadokoro, S., Yoshida, T., Koyanagi, E.: Shared autonomy system for tracked vehicles on rough terrain based on continuous three-dimensional terrain scanning. J. Field Robot. 28(6), 875–893 (2011)
Paul, R., Triebel, R., Rus, D., Newman, P.: Semantic categorization of outdoor scenes with uncertainty estimates using multi-class gaussian process classification. In: Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on, pp. 2404–2410. IEEE (2012)
Pei, L., Liu, J., Guinness, R., Chen, Y., Kuusniemi, H., Chen, R.: Using ls-svm based motion recognition for smartphone indoor wireless positioning. Sensors 12(5), 6155–6175 (2012)
Pitzer, B., Styer, M., Bersch, C., DuHadway, C., Becker, J.: Towards perceptual shared autonomy for robotic mobile manipulation. In: Proceedings - IEEE International Conference on Robotics and Automation, pp. 6245–6251. IEEE (2011). https://doi.org/10.1109/ICRA.2011.5980259
Rasmussen, C.E.: Gaussian processes for machine learning (2006)
Sa, I., Hrabar, S., Corke, P.: Inspection of pole-like structures using a visual-inertial aided vtol platform with shared autonomy. Sensors 15(9), 22,003–22,048 (2015)
Secchi, C., Franchi, A., Bulthoff, H.H., Robuffo Giordano, P.: Bilateral control of the degree of connectivity in multiple mobile-robot teleoperation. In: 2013 IEEE International Conference on Robotics and Automation (ICRA), pp. 3645–3652. IEEE (2013)
Sheridan, T.B.: Telerobotics, automation and human supervisory control. The MIT press, Cambridge (1992)
Tambe, M., Scerri, P., Pynadath, D.V.: Adjustable autonomy for the real world. J. Artif. Intell. Res. 17(1), 171–228 (2002)
Varkey, J.P., Pompili, D., Walls, T.A.: Human motion recognition using a wireless sensor-based wearable system. Personal Ubiquit. Comput. 16(7), 897–910 (2012)
Vozar, S.E.: A framework for improving the speed and performance of teleoperated mobile manipulators. Ph.D. thesis, US Army (2013)
Wu, W., Dasgupta, S., Ramirez, E.E., Peterson, C., Norman, G.J.: Classification accuracies of physical activities using smartphone motion sensors. J. Med. Internet Res. 14(5), e130 (2012)
Yu, N., Wang, K., Li, Y., Xu, C., Liu, J.: A haptic shared control algorithm for flexible human assistance to semi-autonomous robots. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5241–5246. IEEE (2015)
Author information
Authors and Affiliations
Corresponding author
Additional information
The authors would like to acknowledge the prior publication at 2016 ICARSC conference, and thank all test participants. The authors sincerely appreciate the constructive comments made by the editors and the reviewers, which significantly helps improve this report.
Rights and permissions
About this article
Cite this article
Gao, M., Zöllner, J.M. Sparse Contextual Task Learning and Classification to Assist Mobile Robot Teleoperation with Introspective Estimation. J Intell Robot Syst 93, 571–585 (2019). https://doi.org/10.1007/s10846-017-0681-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10846-017-0681-8