Abstract
Object grasping is a crucial task for robots, inspired by nature, where humans can flexibly grasp any object and detect whether it is slipping from grasp or not, more by the sense of touch than vision. In this work we present a bionic gripper with an Edge-AI device that is able to dexterously grasp the handled objects, sense and predict their slippage. In this paper, a bionic gripper with tactile sensors and a time-of-flight sensor is developed. We propose a LSTM model which is used to detect (incipient) slip/slippage, where a 6 degree-of-freedom robot manipulator is used for data collection and testing. The aim of this paper is to develop an efficient slip detection system which we can deploy on the edge device on our gripper, so it can be a stand-alone product that can be attached to almost any robotic manipulator. We have collected a dataset, trained the model and achieved a slip detection accuracy of 95.34%. Due to the efficiency of our model we were able to implement the slip detection on an edge device. We use the Nvidia Jetson AGX Orin development board to show the inference/prediction in a real-time scenario. We demonstrate in the our experiments how the on-gripper slip detection capability allows more robust grasping as the grip force is adjusted in response to a slippage.
Y. Nassar and M. Radke—Equal contribution.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
ESP32-DevKitC. https://www.espressif.com/en/products/devkits/esp32-devkitc. Accessed 20 Oct 2023
Force-sensing linear potentiometer. https://www.pololu.com/product/2730. Accessed 25 Oct 2023
Jetson Orin board. https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-orin. Accessed 25 Oct 2023
Scikit-learn. https://scikit-learn.org. Accessed 01 Sept 2023
Why ROS2. https://design.ros2.org/articles/why_ros2.html. Accessed 24 Oct 2023
Chen, W., Khamis, H., Birznieks, I., Lepora, N.F., Redmond, S.J.: Tactile sensors for friction estimation and incipient slip detection-toward dexterous robotic manipulation: a review. IEEE Sens. J. 18(22), 9049–9064 (2018)
Crooks, W., Vukasin, G., O’Sullivan, M., Messner, W., Rogers, C.: Fin ray® effect inspired soft robotic gripper: from the robosoft grand challenge toward optimization. Front. Robot. AI 3, 70 (2016)
Dong, S., Ma, D., Donlon, E., Rodriguez, A.: Maintaining grasps within slipping bounds by monitoring incipient slip. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 3818–3824. IEEE (2019)
Dong, S., Yuan, W., Adelson, E.H.: Improved gelsight tactile sensor for measuring geometry and slip. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 137–144. IEEE (2017)
Francomano, M.T., Accoto, D., Guglielmelli, E.: Artificial sense of slip-a review. IEEE Sens. J. 13(7), 2489–2498 (2013)
Gao, J., Huang, Z., Tang, Z., Song, H., Liang, W.: Visuo-tactile-based slip detection using a multi-scale temporal convolution network. arXiv preprint arXiv:2302.13564 (2023)
Graves, A., Graves, A.: Long short-term memory. In: Supervised Sequence Labelling with Recurrent Neural Networks, pp. 37–45 (2012)
James, J.W., Lepora, N.F.: Slip detection for grasp stabilization with a multifingered tactile robot hand. IEEE Trans. Rob. 37(2), 506–519 (2020)
Johansson, R.S., Vallbo, A.B.: Tactile sensibility in the human hand: relative and absolute densities of four types of mechanoreceptive units in glabrous skin. J. Physiol. 286(1), 283–300 (1979)
Li, J., Dong, S., Adelson, E.: Slip detection with combined tactile and visual information. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 7772–7777. IEEE (2018)
Medsker, L., Jain, L.C.: Recurrent Neural Networks: Design and Applications. CRC Press (1999)
Meier, M., Patzelt, F., Haschke, R., Ritter, H.J.: Tactile convolutional networks for online slip and rotation detection. In: Villa, A.E.P., Masulli, P., Pons Rivero, A.J. (eds.) ICANN 2016. LNCS, vol. 9887, pp. 12–19. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-44781-0_2
Yan, G., Schmitz, A., Funabashi, S., Somlor, S., Tomo, T.P., Sugano, S.: SCT-CNN: a spatio-channel-temporal attention CNN for grasp stability prediction. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 2627–2634. IEEE (2021)
Yan, G., Schmitz, A., Tomo, T.P., Somlor, S., Funabashi, S., Sugano, S.: Detection of slip from vision and touch. In: 2022 International Conference on Robotics and Automation (ICRA), pp. 3537–3543. IEEE (2022)
Yousef, H., Boukallel, M., Althoefer, K.: Tactile sensing for dexterous in-hand manipulation in robotics—a review. Sens. Actuators A: Phys. 167(2), 171–187 (2011)
Yu, Y., Si, X., Hu, C., Zhang, J.: A review of recurrent neural networks: LSTM cells and network architectures. Neural Comput. 31(7), 1235–1270 (2019)
Yuan, W., Li, R., Srinivasan, M.A., Adelson, E.H.: Measurement of shear and slip with a gelsight tactile sensor. In: 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 304–311. IEEE (2015)
Zapata-Impata, B.S., Gil, P., Torres, F.: Learning spatio temporal tactile features with a ConvLSTM for the direction of slip detection. Sensors 19(3), 523 (2019)
Acknowledgements
This work is partially supported by a grant of the EFRE and MWK ProFö-R &D program, no. FEIH_ProT_2517820 and MWK32-7535-30/10/2.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Nassar, Y. et al. (2024). BiGSiD: Bionic Grasping with Edge-AI Slip Detection. In: Filipe, J., Röning, J. (eds) Robotics, Computer Vision and Intelligent Systems. ROBOVIS 2024. Communications in Computer and Information Science, vol 2077. Springer, Cham. https://doi.org/10.1007/978-3-031-59057-3_10
Download citation
DOI: https://doi.org/10.1007/978-3-031-59057-3_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-59056-6
Online ISBN: 978-3-031-59057-3
eBook Packages: Computer ScienceComputer Science (R0)