2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020
As the number of robots in our daily surroundings like home, office, restaurants, factory floors,... more As the number of robots in our daily surroundings like home, office, restaurants, factory floors, etc. are increasing rapidly, the development of natural human-robot interaction mechanism becomes more vital as it dictates the usability and acceptability of the robots. One of the valued features of such a cohabitant robot is that it performs tasks that are instructed in natural language. However, it is not trivial to execute the human intended tasks as natural language expressions can have large linguistic variations. Existing works assume either single task instruction is given to the robot at a time or there are multiple independent tasks in an instruction. However, complex task instructions composed of multiple inter-dependent tasks are not handled efficiently in the literature. There can be ordering dependency among the tasks, i.e., the tasks have to be executed in a certain order or there can be execution dependency, i.e., input parameter or execution of a task depends on the ou...
2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), 2020
This paper presents a novel architecture to detect social groups in real-time from a continuous i... more This paper presents a novel architecture to detect social groups in real-time from a continuous image stream of an ego-vision camera. F-formation defines social orientations in space where two or more person tends to communicate in a social place. Thus, essentially, we detect F-formations in social gatherings such as meetings, discussions, etc. and predict the robot’s approach angle if it wants to join the social group. Additionally, we also detect outliers, i.e., the persons who are not part of the group under consideration. Our proposed pipeline consists of – a) a skeletal key points estimator (a total of 17) for the detected human in the scene, b) a learning model (using a feature vector based on the skeletal points) using CRF to detect groups of people and outlier person in a scene, and c) a separate learning model using a multi-class Support Vector Machine (SVM) to predict the exact F-formation of the group of people in the current scene and the angle of approach for the viewin...
We have developed a robotics ontology, OntoScene, that extends IEEE CORA [5] and SemNav [1] ontol... more We have developed a robotics ontology, OntoScene, that extends IEEE CORA [5] and SemNav [1] ontologies. Contrary to the prior work that lacked usage of ontology in scene understanding, the proposed system uses OntoScene to figure out objects and their relations in a scene and create a scene graph for aid in various cognitive robotic tasks where object localization, scene graph generation is important. This work positions semantic web technology as a key enabler in robotic tasks.
2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019
A robot as a coworker or a cohabitant is becoming mainstream day-by-day with the development of l... more A robot as a coworker or a cohabitant is becoming mainstream day-by-day with the development of low-cost sophisticated hardware. However, an accompanying software stack that can aid the usability of the robotic hardware remains the bottleneck of the process, especially if the robot is not dedicated to a single job. Programming a multi-purpose robot requires an on the fly mission scheduling capability that involves task identification and plan generation. The problem dimension increases if the robot accepts tasks from a human in natural language. Though recent advances in NLP and planner development can solve a variety of complex problems, their amalgamation for a dynamic robotic task handler is used in a limited scope. Specifically, the problem of formulating a planning problem from natural language instructions is not studied in details. In this work, we provide a non-trivial method to combine an NLP engine and a planner such that a robot can successfully identify tasks and all the...
2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020
As the number of robots in our daily surroundings like home, office, restaurants, factory floors,... more As the number of robots in our daily surroundings like home, office, restaurants, factory floors, etc. are increasing rapidly, the development of natural human-robot interaction mechanism becomes more vital as it dictates the usability and acceptability of the robots. One of the valued features of such a cohabitant robot is that it performs tasks that are instructed in natural language. However, it is not trivial to execute the human intended tasks as natural language expressions can have large linguistic variations. Existing works assume either single task instruction is given to the robot at a time or there are multiple independent tasks in an instruction. However, complex task instructions composed of multiple inter-dependent tasks are not handled efficiently in the literature. There can be ordering dependency among the tasks, i.e., the tasks have to be executed in a certain order or there can be execution dependency, i.e., input parameter or execution of a task depends on the ou...
Robots in our daily surroundings are increasing day by day. Their usability and acceptability lar... more Robots in our daily surroundings are increasing day by day. Their usability and acceptability largely depend on their explicit and implicit interaction capability with fellow human beings. As a result, social behavior is one of the most sought-after qualities that a robot can possess. However, there is no specific aspect and/or feature that defines socially acceptable behavior and it largely depends on the situation, application, and society. In this article, we investigate one such social behavior for collocated robots. Imagine a group of people is interacting with each other and we want to join the group. We as human beings do it in a socially acceptable manner, i.e., within the group, we do position ourselves in such a way that we can participate in the group activity without disturbing/obstructing anybody. To possess such a quality, first, a robot needs to determine the formation of the group and then determine a position for itself, which we humans do implicitly. The theory of ...
2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), 2019
The number of robots deployed in our daily surroundings is ever-increasing. Even in the industria... more The number of robots deployed in our daily surroundings is ever-increasing. Even in the industrial setup, the use of coworker robots is increasing rapidly. These cohabitant robots perform various tasks as instructed by co-located human beings. Thus, a natural interaction mechanism plays a big role in the usability and acceptability of the robot, especially by a non-expert user. The recent development in natural language processing (NLP) has paved the way for chatbots to generate an automatic response for users’ query. A robot can be equipped with such a dialogue system. However, the goal of human-robot interaction is not focused on generating a response to queries, but it often involves performing some tasks in the physical world. Thus, a system is required that can detect user intended task from the natural instruction along with the set of pre- and post-conditions. In this work, we develop a dialogue engine for a robot that can classify and map a task instruction to the robot’s ca...
2018 3rd International Conference On Internet of Things: Smart Innovation and Usages (IoT-SIU), 2018
Driving style analysis and road anomaly detection have a remarkable impact on road safety. They d... more Driving style analysis and road anomaly detection have a remarkable impact on road safety. They directly influence road accidents and have been a vital area of research in order to address road safety problems. In this paper, a system called D&RSense have been proposed that uses GPS and accelerometer of smartphones to categorize driving style of drivers, assess the road quality as well as to give real-time warnings to drivers in order to make driving safer. D&RSense does the categorization through detection of driving events like acceleration and braking and road anomalies like bumps and potholes by using the popular machine learning technique, Support Vector Machine (SVM) and gives real-time warning and instructions to drivers using a locally running Fast Dynamic Time Warping (FastDTW) algorithm. Extensive experiments have been conducted to evaluate the effectiveness of the proposed system.
IEEE International Conference on Advanced Networks and Telecommunications Systems (ANTS), 2017
Noise pollution in urban areas is a subject of grave concern and it is being recognized globally ... more Noise pollution in urban areas is a subject of grave concern and it is being recognized globally in different countries and cities. People are facing many health-related problems because of this. Therefore, in the proposed work, we envisioned to tackle the challenge of acquiring real time and spatially fine-grained noise pollution data with a community-driven sensing infrastructure. Mobile crowdsourcing over smartphones presents a new paradigm for collecting context aware sensing data of a vast area like a city. Thus, the proposed system exploits the power of mobile crowdsourcing. The proposed system monitors the present noise level in the surroundings of the user and also generates city's noise pollution footprints. The noise map reflects the real-time pollution scenario of the city which changes with time. The prototype of the system has been evaluated with extensive experiments based on crowdsourced sensing data collected by volunteers in Kolkata city.
Proceedings of the Workshop Program of the 19th International Conference on Distributed Computing and Networking, 2018
Integration of the physical world with the computerized world has led to the manifestation of Cyb... more Integration of the physical world with the computerized world has led to the manifestation of Cyber-Physical Systems (CPSs) in an attempt to build a better and smarter world. In this paper, such a CPS named D&RSense has been proposed to promote smart transportation in order to make travelling more comfortable and safe. By studying driving patterns of drivers, D&RSense can get valuable insights to their braking and accelerating styles which can help to give them real-time warnings when they drive aggressively. Detection of rash driving prone areas across the city can help to recommend which areas of the city need stricter surveillance. D&RSense involves smartphones of commuters and utilizes their accelerometer and GPS sensors to detect driving events like braking and acceleration as well as poor road conditions like bumps and potholes by applying the ensemble learning method for classification, Random Forest (RF). The accuracy of the same has been compared to other supervised machine learning classifiers like Naïve Bayes, k-Nearest Neighbours (k-NN), Decision Trees (DT), Support Vector Machine (SVM) and Artificial Neural Networks (ANN). Rash-driving prone areas and poor road segments during the course of the experiment have been plotted using Density-based spatial clustering of applications with noise (DBSCAN) algorithm. Effectiveness of the proposed application has been evaluated through extensive testing.
GPS is one of the most used services in any location-based app in our smartphone, and almost a qu... more GPS is one of the most used services in any location-based app in our smartphone, and almost a quarter of all Android apps available in the Google Play store are using this GPS. There are many apps which require monitoring your locations in a continuous fashion because of the application's nature, and those kinds of apps consume the highest power from the smartphones. Because of the high-power draining nature of this GPS, we hesitate to take part in different crowd-sourced applications which are very much important for the smart city realization as maximum of these applications use GPS in real time or in a very frequent manner for the realization of participatory sensing in a smart city scenario. To resolve this, we have introduced an energy-efficient context-aware approach which utilizes user's mobility information from the user's context and as well smartphone's sensing values from the inbuilt accelerometer, magnetometer, and gyroscope of the smartphone to provide us a very close estimation of the present location of the user without using continuous GPS. It is an energy-efficient solution without sacrificing the accuracy compared to energy saving which will boost the crowd to take part in the smartphone-based crowd-sourced applications that depend on participatory sensing for the smart city environment.
The 27th International Symposium on Robot and Human Interactive Communication (IEEE RO-MAN), 2018
A robot as a companion or co-worker is not an emerging concept anymore, but a reality. However, o... more A robot as a companion or co-worker is not an emerging concept anymore, but a reality. However, one of the major barriers to this realization is the seamless interaction with the robots that includes both explicit and implicit interaction. In this work, we assume a use-case where a human and a robot together carry a heavy object in a co-habitat (home or workplace/factory). Two human beings while doing such a work understands each other without explicit (vocal) interaction. To realize such behavior, the robot must understand the fatigue state of the human co-worker to enable seamless work experience and ensure safety. In this article, we present DeFatigue, a non-intrusive fatigue state detection mechanism. We assume that the robot's hand is equipped with a force sensor. Based on the change of force from the human side while carrying the object, DeFatigue is able to determine the fatigue state without instrumenting the human being with an additional sensor (internally or externally). Moreover, it detects the fatigues state on-the-fly (online) as well as it does not require any (user-specific) training. Based on our experiments with 18 test subjects, fatigue state detection by DeFatigue overlaps with the ground truth for 85.18% of the cases whereas it deviates 4.09s (on average) for the remaining cases.
2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020
As the number of robots in our daily surroundings like home, office, restaurants, factory floors,... more As the number of robots in our daily surroundings like home, office, restaurants, factory floors, etc. are increasing rapidly, the development of natural human-robot interaction mechanism becomes more vital as it dictates the usability and acceptability of the robots. One of the valued features of such a cohabitant robot is that it performs tasks that are instructed in natural language. However, it is not trivial to execute the human intended tasks as natural language expressions can have large linguistic variations. Existing works assume either single task instruction is given to the robot at a time or there are multiple independent tasks in an instruction. However, complex task instructions composed of multiple inter-dependent tasks are not handled efficiently in the literature. There can be ordering dependency among the tasks, i.e., the tasks have to be executed in a certain order or there can be execution dependency, i.e., input parameter or execution of a task depends on the ou...
2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), 2020
This paper presents a novel architecture to detect social groups in real-time from a continuous i... more This paper presents a novel architecture to detect social groups in real-time from a continuous image stream of an ego-vision camera. F-formation defines social orientations in space where two or more person tends to communicate in a social place. Thus, essentially, we detect F-formations in social gatherings such as meetings, discussions, etc. and predict the robot’s approach angle if it wants to join the social group. Additionally, we also detect outliers, i.e., the persons who are not part of the group under consideration. Our proposed pipeline consists of – a) a skeletal key points estimator (a total of 17) for the detected human in the scene, b) a learning model (using a feature vector based on the skeletal points) using CRF to detect groups of people and outlier person in a scene, and c) a separate learning model using a multi-class Support Vector Machine (SVM) to predict the exact F-formation of the group of people in the current scene and the angle of approach for the viewin...
We have developed a robotics ontology, OntoScene, that extends IEEE CORA [5] and SemNav [1] ontol... more We have developed a robotics ontology, OntoScene, that extends IEEE CORA [5] and SemNav [1] ontologies. Contrary to the prior work that lacked usage of ontology in scene understanding, the proposed system uses OntoScene to figure out objects and their relations in a scene and create a scene graph for aid in various cognitive robotic tasks where object localization, scene graph generation is important. This work positions semantic web technology as a key enabler in robotic tasks.
2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019
A robot as a coworker or a cohabitant is becoming mainstream day-by-day with the development of l... more A robot as a coworker or a cohabitant is becoming mainstream day-by-day with the development of low-cost sophisticated hardware. However, an accompanying software stack that can aid the usability of the robotic hardware remains the bottleneck of the process, especially if the robot is not dedicated to a single job. Programming a multi-purpose robot requires an on the fly mission scheduling capability that involves task identification and plan generation. The problem dimension increases if the robot accepts tasks from a human in natural language. Though recent advances in NLP and planner development can solve a variety of complex problems, their amalgamation for a dynamic robotic task handler is used in a limited scope. Specifically, the problem of formulating a planning problem from natural language instructions is not studied in details. In this work, we provide a non-trivial method to combine an NLP engine and a planner such that a robot can successfully identify tasks and all the...
2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020
As the number of robots in our daily surroundings like home, office, restaurants, factory floors,... more As the number of robots in our daily surroundings like home, office, restaurants, factory floors, etc. are increasing rapidly, the development of natural human-robot interaction mechanism becomes more vital as it dictates the usability and acceptability of the robots. One of the valued features of such a cohabitant robot is that it performs tasks that are instructed in natural language. However, it is not trivial to execute the human intended tasks as natural language expressions can have large linguistic variations. Existing works assume either single task instruction is given to the robot at a time or there are multiple independent tasks in an instruction. However, complex task instructions composed of multiple inter-dependent tasks are not handled efficiently in the literature. There can be ordering dependency among the tasks, i.e., the tasks have to be executed in a certain order or there can be execution dependency, i.e., input parameter or execution of a task depends on the ou...
Robots in our daily surroundings are increasing day by day. Their usability and acceptability lar... more Robots in our daily surroundings are increasing day by day. Their usability and acceptability largely depend on their explicit and implicit interaction capability with fellow human beings. As a result, social behavior is one of the most sought-after qualities that a robot can possess. However, there is no specific aspect and/or feature that defines socially acceptable behavior and it largely depends on the situation, application, and society. In this article, we investigate one such social behavior for collocated robots. Imagine a group of people is interacting with each other and we want to join the group. We as human beings do it in a socially acceptable manner, i.e., within the group, we do position ourselves in such a way that we can participate in the group activity without disturbing/obstructing anybody. To possess such a quality, first, a robot needs to determine the formation of the group and then determine a position for itself, which we humans do implicitly. The theory of ...
2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), 2019
The number of robots deployed in our daily surroundings is ever-increasing. Even in the industria... more The number of robots deployed in our daily surroundings is ever-increasing. Even in the industrial setup, the use of coworker robots is increasing rapidly. These cohabitant robots perform various tasks as instructed by co-located human beings. Thus, a natural interaction mechanism plays a big role in the usability and acceptability of the robot, especially by a non-expert user. The recent development in natural language processing (NLP) has paved the way for chatbots to generate an automatic response for users’ query. A robot can be equipped with such a dialogue system. However, the goal of human-robot interaction is not focused on generating a response to queries, but it often involves performing some tasks in the physical world. Thus, a system is required that can detect user intended task from the natural instruction along with the set of pre- and post-conditions. In this work, we develop a dialogue engine for a robot that can classify and map a task instruction to the robot’s ca...
2018 3rd International Conference On Internet of Things: Smart Innovation and Usages (IoT-SIU), 2018
Driving style analysis and road anomaly detection have a remarkable impact on road safety. They d... more Driving style analysis and road anomaly detection have a remarkable impact on road safety. They directly influence road accidents and have been a vital area of research in order to address road safety problems. In this paper, a system called D&RSense have been proposed that uses GPS and accelerometer of smartphones to categorize driving style of drivers, assess the road quality as well as to give real-time warnings to drivers in order to make driving safer. D&RSense does the categorization through detection of driving events like acceleration and braking and road anomalies like bumps and potholes by using the popular machine learning technique, Support Vector Machine (SVM) and gives real-time warning and instructions to drivers using a locally running Fast Dynamic Time Warping (FastDTW) algorithm. Extensive experiments have been conducted to evaluate the effectiveness of the proposed system.
IEEE International Conference on Advanced Networks and Telecommunications Systems (ANTS), 2017
Noise pollution in urban areas is a subject of grave concern and it is being recognized globally ... more Noise pollution in urban areas is a subject of grave concern and it is being recognized globally in different countries and cities. People are facing many health-related problems because of this. Therefore, in the proposed work, we envisioned to tackle the challenge of acquiring real time and spatially fine-grained noise pollution data with a community-driven sensing infrastructure. Mobile crowdsourcing over smartphones presents a new paradigm for collecting context aware sensing data of a vast area like a city. Thus, the proposed system exploits the power of mobile crowdsourcing. The proposed system monitors the present noise level in the surroundings of the user and also generates city's noise pollution footprints. The noise map reflects the real-time pollution scenario of the city which changes with time. The prototype of the system has been evaluated with extensive experiments based on crowdsourced sensing data collected by volunteers in Kolkata city.
Proceedings of the Workshop Program of the 19th International Conference on Distributed Computing and Networking, 2018
Integration of the physical world with the computerized world has led to the manifestation of Cyb... more Integration of the physical world with the computerized world has led to the manifestation of Cyber-Physical Systems (CPSs) in an attempt to build a better and smarter world. In this paper, such a CPS named D&RSense has been proposed to promote smart transportation in order to make travelling more comfortable and safe. By studying driving patterns of drivers, D&RSense can get valuable insights to their braking and accelerating styles which can help to give them real-time warnings when they drive aggressively. Detection of rash driving prone areas across the city can help to recommend which areas of the city need stricter surveillance. D&RSense involves smartphones of commuters and utilizes their accelerometer and GPS sensors to detect driving events like braking and acceleration as well as poor road conditions like bumps and potholes by applying the ensemble learning method for classification, Random Forest (RF). The accuracy of the same has been compared to other supervised machine learning classifiers like Naïve Bayes, k-Nearest Neighbours (k-NN), Decision Trees (DT), Support Vector Machine (SVM) and Artificial Neural Networks (ANN). Rash-driving prone areas and poor road segments during the course of the experiment have been plotted using Density-based spatial clustering of applications with noise (DBSCAN) algorithm. Effectiveness of the proposed application has been evaluated through extensive testing.
GPS is one of the most used services in any location-based app in our smartphone, and almost a qu... more GPS is one of the most used services in any location-based app in our smartphone, and almost a quarter of all Android apps available in the Google Play store are using this GPS. There are many apps which require monitoring your locations in a continuous fashion because of the application's nature, and those kinds of apps consume the highest power from the smartphones. Because of the high-power draining nature of this GPS, we hesitate to take part in different crowd-sourced applications which are very much important for the smart city realization as maximum of these applications use GPS in real time or in a very frequent manner for the realization of participatory sensing in a smart city scenario. To resolve this, we have introduced an energy-efficient context-aware approach which utilizes user's mobility information from the user's context and as well smartphone's sensing values from the inbuilt accelerometer, magnetometer, and gyroscope of the smartphone to provide us a very close estimation of the present location of the user without using continuous GPS. It is an energy-efficient solution without sacrificing the accuracy compared to energy saving which will boost the crowd to take part in the smartphone-based crowd-sourced applications that depend on participatory sensing for the smart city environment.
The 27th International Symposium on Robot and Human Interactive Communication (IEEE RO-MAN), 2018
A robot as a companion or co-worker is not an emerging concept anymore, but a reality. However, o... more A robot as a companion or co-worker is not an emerging concept anymore, but a reality. However, one of the major barriers to this realization is the seamless interaction with the robots that includes both explicit and implicit interaction. In this work, we assume a use-case where a human and a robot together carry a heavy object in a co-habitat (home or workplace/factory). Two human beings while doing such a work understands each other without explicit (vocal) interaction. To realize such behavior, the robot must understand the fatigue state of the human co-worker to enable seamless work experience and ensure safety. In this article, we present DeFatigue, a non-intrusive fatigue state detection mechanism. We assume that the robot's hand is equipped with a force sensor. Based on the change of force from the human side while carrying the object, DeFatigue is able to determine the fatigue state without instrumenting the human being with an additional sensor (internally or externally). Moreover, it detects the fatigues state on-the-fly (online) as well as it does not require any (user-specific) training. Based on our experiments with 18 test subjects, fatigue state detection by DeFatigue overlaps with the ground truth for 85.18% of the cases whereas it deviates 4.09s (on average) for the remaining cases.
Uploads
Papers by pradip pramanick