Computer vision-based technologies play a key role in precision livestock farming, and video-base... more Computer vision-based technologies play a key role in precision livestock farming, and video-based analysis approaches have been advocated as useful tools for automatic animal monitoring, behavior analysis, and efficient welfare measurement management. Accurately and efficiently segmenting animals’ contours from their backgrounds is a prerequisite for vision-based technologies. Deep learning-based segmentation methods have shown good performance through training models on a large amount of pixel-labeled images. However, it is challenging and time-consuming to label animal images due to their irregular contours and changing postures. In order to reduce the reliance on the number of labeled images, one-shot learning with a pseudo-labeling approach is proposed using only one labeled image frame to segment animals in videos. The proposed approach is mainly comprised of an Xception-based Fully Convolutional Neural Network (Xception-FCN) module and a pseudo-labeling (PL) module. Xception-...
Deep learning-based video segmentation methods can offer a good performance after being trained o... more Deep learning-based video segmentation methods can offer a good performance after being trained on the large-scale pixel labeled datasets. However, a pixel-wise manual labeling of animal images is challenging and time consuming due to irregular contours and motion blur. To achieve desirable tradeoffs between the accuracy and speed, a novel one-shot learning-based approach is proposed in this article to segment animal video with only one labeled frame. The proposed approach consists of the following three main modules: guidance frame selection utilizes “BubbleNet” to choose one frame for manual labeling, which can leverage the fine-tuning effects of the only labeled frame; Xception-based fully convolutional network localizes dense prediction using depthwise separable convolutions based on one single labeled frame; and postprocessing is used to remove outliers and sharpen object contours, which consists of two submodules—test time augmentation and conditional random field. Extensive experiments have been conducted on the DAVIS 2016 animal dataset. Our proposed video segmentation approach achieved mean intersection-over-union score of 89.5% on the DAVIS 2016 animal dataset with less run time, and outperformed the state-of-art methods (OSVOS and OSMN). The proposed one-shot learning-based approach achieves real-time and automatic segmentation of animals with only one labeled video frame. This can be potentially used further as a baseline for intelligent perception-based monitoring of animals and other domain-specific applications.11The source code, datasets, and pre-trained weights for this work are publicly [Online]. Available: https://github.com/tengfeixue-victor/One-Shot-Animal-Video-Segmentation.
2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2011
This paper presents and compares two different approaches to integrating sensor information from ... more This paper presents and compares two different approaches to integrating sensor information from an Iner- tial Measuring Unit (IMU), Global Positioning System (GPS) receiver and monocular vision camera mounted to a low- flying Unmanned Aerial Vehicle (UAV) for building large-scale 3D terrain reconstructions. Both approaches utilise a statisti- cally optimal bundle adjustment formulation that incorporates Vision, IMU and GPS observations
2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2005
... intelligent guidance scheme for a UAV for improving navigation integrity when oper-ating over... more ... intelligent guidance scheme for a UAV for improving navigation integrity when oper-ating over unknown terrain ... MWMG Dissanayake, P. Newman, S. Clark, HF Durrant-Whyte, M. Csorba, ASolution to the Simultaneous Localization and Map Building (SLAM) Problem, IEEE ...
2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422)
This paper presents results of the application of simultaneous localisation and map building (SLA... more This paper presents results of the application of simultaneous localisation and map building (SLAM) for an uninhabited aerial vehicle (UAV). Single vision camera and inertial measurement unit (IMU) are installed in a UAV platform. The data taken from a flight test is used to run the SLAM algorithm. Results show that both the map and the vehicle uncertainty are corrected even though the model of the system and observation are highly non-linear. The results, however, also indicate that further work of observability and the relationship between vehicle model drift and the number and the location of landmarks need to be further analysed given the highly dynamic nature of the system.
In this paper we present an approach to tree recognition and localisation in orchard environments... more In this paper we present an approach to tree recognition and localisation in orchard environments for tree-crop applications. The method builds on the natural structure of the orchard by first segmenting the data into individual trees using a Hidden Semi-Markov Model. Second, a descriptor for representing the characteristics of the trees is introduced, allowing a Hidden Markov Model based matching method to associate new observations with an existing map of the orchard. The localisation method is evaluated on a dataset collected in an almond orchard, showing good performance and robustness both to segmentation errors and measurement noise.
2012 IEEE International Conference on Robotics and Automation, 2012
This paper introduces an unsupervised graph cut based object segmentation algorithm, ShadowCut, f... more This paper introduces an unsupervised graph cut based object segmentation algorithm, ShadowCut, for robotic aerial surveillance applications. By exploiting the spatial setting of the aerial imagery, ShadowCut algorithm differs from state-of-the-art object segmentation algorithms ([1] [2] [3] [4] [5]) by not requiring a large number of labelled training data set, nor constant user interaction ([6] [7] [8]). In this paper
Future unmanned aerial vehicle (UAV) applications will require high-accuracy localisation in envi... more Future unmanned aerial vehicle (UAV) applications will require high-accuracy localisation in environments in which navigation infrastructure such as the Global Positioning System (GPS) and prior terrain maps may be unavailable or unreliable. In these applications, long-term operation requires the vehicle to build up a spatial map of the environment while simultaneously localising itself within the map, a task known as
Abstract Text: Advances in technology could reduce time farmers spend on repetitive tasks. In pas... more Abstract Text: Advances in technology could reduce time farmers spend on repetitive tasks. In pasture-based systems, herding cows from grazing areas to the milking parlour is a repetitive task ideally suited to automation. We conducted a field study at Sydney University’s dairy farm to determine the behavioural response of dairy cows to a remotely operated unmanned ground vehicle (UGV) across time. Twenty milking cows were separated from the main herd at 0830 h and offered 0.5 ha of an ad-libitum kikuyu pasture allocation (50 kg DM/cow to ground level). A pre-defined figure eight route was determined for the UGV within this area. The UGV entered the pasture allocation at 0900 h and traversed this route at a target speed of 2.5 km/h, with the whole procedure repeated 6 times at 15 min intervals. The 0.5 ha was virtually split into four sectors. Four observers monitored cows exiting or entering each sector. Data were analysed by REML, where Cows out = Fixed (Robot (presence/absence) *...
In this paper we demonstrate a co-operative path-planning algorithm for multi-vehicle simultaneou... more In this paper we demonstrate a co-operative path-planning algorithm for multi-vehicle simultaneous localisation and mapping (SLAM) that uses information-based measures to maximize the accuracy of a feature map which is constructed from terrain observation made by each vehicle. The SLAM algorithm is distributed amongst the vehicles where each vehicle shares locally built map information via a central communications node. This information is used to assist in localisation which in turn increases the accuracy of the map information each vehicle provides. Each vehicle communicates to the central node potential trajectories it can take and the associated map information it will provide. The central communications node then co-ordinates the actions of each platform such as to maximise the accuracy of the globally constructed map. The vehicles each perform SLAM using a combination of on-board inertial sensors and an on-board terrain sensor. Results are presented using a 6-DoF simulation of several UAVs over an initially unexplored terrain.
2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012
ABSTRACT This paper presents a novel approach to depth estimation from monocular images that is b... more ABSTRACT This paper presents a novel approach to depth estimation from monocular images that is based on the Gaussian Derivative Process (GDP) formulation. We use an inverse depth parametrisation and learn the mapping from image pixel coordinates to the inverse depth of the corresponding scene point given an estimate of the relative camera motion. We show that information about the geometry of the measurements can be integrated into Gaussian Process (GP) models and learnt jointly with the measurements. We provide a novel formulation of the inverse depth and its derivatives and learn their joint distribution. Experimental results are presented using synthesised examples and real monocular images captured from an Unmanned Aerial Vehicle (UAV). Results show improvement in depth estimation over standard Gaussian Process Regression (GPR). This improvement is presented by a reduction in the GP depth prediction errors and the predictive variance. Finally, we show mathematically that this improvement is due to the augmented derivative covariance terms and the correlations between the inverse depth and the derivatives.
The use of a multi-rotor unmanned air vehicle (UAV) in image acquisition tasks is promising for t... more The use of a multi-rotor unmanned air vehicle (UAV) in image acquisition tasks is promising for three-dimensional (3D) object modeling. Such an autonomous data acquisition system can be useful to handle the geometric complexity of objects such as trees and the inherent difficulties of image capture. In this paper, we address the problem of view planning for a camera-equipped multi-rotor UAV to acquire an adequate set of images that leads to more detailed and complete knowledge of the 3D tree model. The proposed algorithm based on shape-from-silhouette methods incorporates both expected new visual information and vehicle movement. Occupancy estimation for volumetric object model serves as a baseline measure of new information. The outlined approach determines next best views across the viewpoint space bounded by the sensor coverage and the capability of the UAV with minimal a priori knowledge of the object. Simulation studies conducted with virtual reality environments show the effectiveness of the algorithm.
Computer vision-based technologies play a key role in precision livestock farming, and video-base... more Computer vision-based technologies play a key role in precision livestock farming, and video-based analysis approaches have been advocated as useful tools for automatic animal monitoring, behavior analysis, and efficient welfare measurement management. Accurately and efficiently segmenting animals’ contours from their backgrounds is a prerequisite for vision-based technologies. Deep learning-based segmentation methods have shown good performance through training models on a large amount of pixel-labeled images. However, it is challenging and time-consuming to label animal images due to their irregular contours and changing postures. In order to reduce the reliance on the number of labeled images, one-shot learning with a pseudo-labeling approach is proposed using only one labeled image frame to segment animals in videos. The proposed approach is mainly comprised of an Xception-based Fully Convolutional Neural Network (Xception-FCN) module and a pseudo-labeling (PL) module. Xception-...
Deep learning-based video segmentation methods can offer a good performance after being trained o... more Deep learning-based video segmentation methods can offer a good performance after being trained on the large-scale pixel labeled datasets. However, a pixel-wise manual labeling of animal images is challenging and time consuming due to irregular contours and motion blur. To achieve desirable tradeoffs between the accuracy and speed, a novel one-shot learning-based approach is proposed in this article to segment animal video with only one labeled frame. The proposed approach consists of the following three main modules: guidance frame selection utilizes “BubbleNet” to choose one frame for manual labeling, which can leverage the fine-tuning effects of the only labeled frame; Xception-based fully convolutional network localizes dense prediction using depthwise separable convolutions based on one single labeled frame; and postprocessing is used to remove outliers and sharpen object contours, which consists of two submodules—test time augmentation and conditional random field. Extensive experiments have been conducted on the DAVIS 2016 animal dataset. Our proposed video segmentation approach achieved mean intersection-over-union score of 89.5% on the DAVIS 2016 animal dataset with less run time, and outperformed the state-of-art methods (OSVOS and OSMN). The proposed one-shot learning-based approach achieves real-time and automatic segmentation of animals with only one labeled video frame. This can be potentially used further as a baseline for intelligent perception-based monitoring of animals and other domain-specific applications.11The source code, datasets, and pre-trained weights for this work are publicly [Online]. Available: https://github.com/tengfeixue-victor/One-Shot-Animal-Video-Segmentation.
2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2011
This paper presents and compares two different approaches to integrating sensor information from ... more This paper presents and compares two different approaches to integrating sensor information from an Iner- tial Measuring Unit (IMU), Global Positioning System (GPS) receiver and monocular vision camera mounted to a low- flying Unmanned Aerial Vehicle (UAV) for building large-scale 3D terrain reconstructions. Both approaches utilise a statisti- cally optimal bundle adjustment formulation that incorporates Vision, IMU and GPS observations
2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2005
... intelligent guidance scheme for a UAV for improving navigation integrity when oper-ating over... more ... intelligent guidance scheme for a UAV for improving navigation integrity when oper-ating over unknown terrain ... MWMG Dissanayake, P. Newman, S. Clark, HF Durrant-Whyte, M. Csorba, ASolution to the Simultaneous Localization and Map Building (SLAM) Problem, IEEE ...
2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422)
This paper presents results of the application of simultaneous localisation and map building (SLA... more This paper presents results of the application of simultaneous localisation and map building (SLAM) for an uninhabited aerial vehicle (UAV). Single vision camera and inertial measurement unit (IMU) are installed in a UAV platform. The data taken from a flight test is used to run the SLAM algorithm. Results show that both the map and the vehicle uncertainty are corrected even though the model of the system and observation are highly non-linear. The results, however, also indicate that further work of observability and the relationship between vehicle model drift and the number and the location of landmarks need to be further analysed given the highly dynamic nature of the system.
In this paper we present an approach to tree recognition and localisation in orchard environments... more In this paper we present an approach to tree recognition and localisation in orchard environments for tree-crop applications. The method builds on the natural structure of the orchard by first segmenting the data into individual trees using a Hidden Semi-Markov Model. Second, a descriptor for representing the characteristics of the trees is introduced, allowing a Hidden Markov Model based matching method to associate new observations with an existing map of the orchard. The localisation method is evaluated on a dataset collected in an almond orchard, showing good performance and robustness both to segmentation errors and measurement noise.
2012 IEEE International Conference on Robotics and Automation, 2012
This paper introduces an unsupervised graph cut based object segmentation algorithm, ShadowCut, f... more This paper introduces an unsupervised graph cut based object segmentation algorithm, ShadowCut, for robotic aerial surveillance applications. By exploiting the spatial setting of the aerial imagery, ShadowCut algorithm differs from state-of-the-art object segmentation algorithms ([1] [2] [3] [4] [5]) by not requiring a large number of labelled training data set, nor constant user interaction ([6] [7] [8]). In this paper
Future unmanned aerial vehicle (UAV) applications will require high-accuracy localisation in envi... more Future unmanned aerial vehicle (UAV) applications will require high-accuracy localisation in environments in which navigation infrastructure such as the Global Positioning System (GPS) and prior terrain maps may be unavailable or unreliable. In these applications, long-term operation requires the vehicle to build up a spatial map of the environment while simultaneously localising itself within the map, a task known as
Abstract Text: Advances in technology could reduce time farmers spend on repetitive tasks. In pas... more Abstract Text: Advances in technology could reduce time farmers spend on repetitive tasks. In pasture-based systems, herding cows from grazing areas to the milking parlour is a repetitive task ideally suited to automation. We conducted a field study at Sydney University’s dairy farm to determine the behavioural response of dairy cows to a remotely operated unmanned ground vehicle (UGV) across time. Twenty milking cows were separated from the main herd at 0830 h and offered 0.5 ha of an ad-libitum kikuyu pasture allocation (50 kg DM/cow to ground level). A pre-defined figure eight route was determined for the UGV within this area. The UGV entered the pasture allocation at 0900 h and traversed this route at a target speed of 2.5 km/h, with the whole procedure repeated 6 times at 15 min intervals. The 0.5 ha was virtually split into four sectors. Four observers monitored cows exiting or entering each sector. Data were analysed by REML, where Cows out = Fixed (Robot (presence/absence) *...
In this paper we demonstrate a co-operative path-planning algorithm for multi-vehicle simultaneou... more In this paper we demonstrate a co-operative path-planning algorithm for multi-vehicle simultaneous localisation and mapping (SLAM) that uses information-based measures to maximize the accuracy of a feature map which is constructed from terrain observation made by each vehicle. The SLAM algorithm is distributed amongst the vehicles where each vehicle shares locally built map information via a central communications node. This information is used to assist in localisation which in turn increases the accuracy of the map information each vehicle provides. Each vehicle communicates to the central node potential trajectories it can take and the associated map information it will provide. The central communications node then co-ordinates the actions of each platform such as to maximise the accuracy of the globally constructed map. The vehicles each perform SLAM using a combination of on-board inertial sensors and an on-board terrain sensor. Results are presented using a 6-DoF simulation of several UAVs over an initially unexplored terrain.
2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012
ABSTRACT This paper presents a novel approach to depth estimation from monocular images that is b... more ABSTRACT This paper presents a novel approach to depth estimation from monocular images that is based on the Gaussian Derivative Process (GDP) formulation. We use an inverse depth parametrisation and learn the mapping from image pixel coordinates to the inverse depth of the corresponding scene point given an estimate of the relative camera motion. We show that information about the geometry of the measurements can be integrated into Gaussian Process (GP) models and learnt jointly with the measurements. We provide a novel formulation of the inverse depth and its derivatives and learn their joint distribution. Experimental results are presented using synthesised examples and real monocular images captured from an Unmanned Aerial Vehicle (UAV). Results show improvement in depth estimation over standard Gaussian Process Regression (GPR). This improvement is presented by a reduction in the GP depth prediction errors and the predictive variance. Finally, we show mathematically that this improvement is due to the augmented derivative covariance terms and the correlations between the inverse depth and the derivatives.
The use of a multi-rotor unmanned air vehicle (UAV) in image acquisition tasks is promising for t... more The use of a multi-rotor unmanned air vehicle (UAV) in image acquisition tasks is promising for three-dimensional (3D) object modeling. Such an autonomous data acquisition system can be useful to handle the geometric complexity of objects such as trees and the inherent difficulties of image capture. In this paper, we address the problem of view planning for a camera-equipped multi-rotor UAV to acquire an adequate set of images that leads to more detailed and complete knowledge of the 3D tree model. The proposed algorithm based on shape-from-silhouette methods incorporates both expected new visual information and vehicle movement. Occupancy estimation for volumetric object model serves as a baseline measure of new information. The outlined approach determines next best views across the viewpoint space bounded by the sensor coverage and the capability of the UAV with minimal a priori knowledge of the object. Simulation studies conducted with virtual reality environments show the effectiveness of the algorithm.
Uploads
Papers by Salah Sukkarieh