1. Introduction
Pigs are one of the most important animals in the world, and their meat plays an important role in people’s daily consumption and nutritional needs [
1,
2]. At present, many countries are encouraging pig welfare farming in which the breeding goal is respecting and improving the wellbeing of pigs [
3]. Pig welfare farming emphasizes addressing the behavioral, physiological and psychological needs of pigs, and it also focuses on providing an appropriate environment, feeding conditions and management measures to improve the health and quality of pigs’ lives.
Currently, more and more animals are slaughtered each year for food production. The growing market demand for animal products has promoted the development of intensive animal husbandry, and motivated farmers to increase the number of animals in the herds without more resources. To meet the market demand and take good care of each animal, automated tools become a good choice [
4,
5].
During the breeding process, weight is an important index to measure the growth and development of pigs [
6]. Regularly recording pigs’ weights is helpful to understand their growth progress, health status and development. In addition, it is also helpful to judge the feeding effect and quality, so that the farmer can timely adjust feeding management and then improve breeding efficiency [
7]. In terms of marketing, pig weight is an important indicator for buyers and sellers to evaluate the quality and value of pigs.
At present, the pig weight measurement methods include direct and indirect methods. The direct method is that the employees of the pig farm directly rush the pigs to the scale for weighing [
8,
9]. Although this is probably the most accurate measurement, it requires a lot of manpower and time, and also leads to the cost increase of pig breeding [
10,
11]. Furthermore, excessive manual contact may induce the spread of swine fever in pigs, affecting the health of pigs and the quality of pork.
With the development of computer vision (CV), automatic non-contact pig weight estimation has attracted more attention [
12,
13]. Computer vision-based pig weight estimation can reduce the pressure of manual intervention and the labor costs, so it can promote healthy development of farms [
14]. The key steps of computer vision-based pig weight estimation are as follows. Firstly, the contours of a pig’s back are obtained by image processing. Secondly, features are extracted based on the obtained contours, including body length and width of the pig, and the perimeter and area of the pig’s back in the vertical view image [
15]. Finally, body weight is estimated according to the obtained features.
In the image processing stage, the core problem is how to segment the contours of the animal in the complex livestock environment. The initial segmentation is based on the threshold method. All pixels are divided into object and background according to the gray value and threshold. Although the threshold method is straightforward to implement, its effectiveness is often limited. Then, graph partitioning is developed to solve the segmentation with complex scenes [
16,
17,
18]. However, graph partitioning easily leads to blurred boundaries, and performs poor on the noise or complex texture images. With the development of deep learning, object detection methods have been applied to extract pig contours [
19]. Cang et al. designed an improved Faster R-CNN to estimate the weight of the pig by vertical view images, which integrated pig detection and the live weight regression branch into an end-to-end network [
20]. Tengtrairat et al. extracted the contours of tilapia by three object detection methods (Faster R-CNN, RetinaNet, and YOLO) as well as the instance segmentation method Mask R-CNN [
21]. They concluded that Mask R-CNN had the highest accuracy in extracting the contours.
Since Mask R-CNN uses the RoI Align process, it can select the high-level features. It outputs not only the bounding box, but also the pixel-level mask of objects. The features of each position can be extracted more accurately, and it avoids the problem of feature deviation and can improve the accuracy. In addition, Mask R-CNN is faster, more adaptable and more flexible. So, we adopt Mask R-CNN to extract the contours of pig’s back, and focused on solving the problems of contour error caused by illumination and noise.
In order to estimate the live weight accurately, the proper features should be extracted. The most commonly used features are body length and body width. In addition, the perimeter and area of the pig’s back image, eccentricity, axis length, body height and volume may also be useful [
22,
23]. Pezzuolo et al. acquired the top-down and side-looking depth images of live pigs, extracted the features of body length, heart perimeter and body height, and then built a multiple quadratic regression model with a goodness of fit of 0.9942 and an MAE of 0.48 kg [
9]. Subsequently, Zhang et al. used a multi-output regression convolutional neural network to estimate the weight and size of the pigs by the shoulder width, shoulder height, hip width, hip height and body length. The correlation coefficient between the estimated results and the actual results was between 0.9879 and 0.9973 [
24]. Li et al. used a single frame surface point cloud to measure the body length, body height and body width of pigs and built a model to estimate the weight of pigs [
25].
Although there are many features that have been extracted for weight prediction, image-based feature information remains to be mined. In addition, the pig body image is curved in some images, so if the features extracted from the images are used directly for weight prediction, there will be prediction errors. So how to correct the features is also a challenge.
Regarding the difficult issues mentioned above, this paper proposed a new pig weight estimation method that combines Mask R-CNN and ensemble regressors. The innovations of this paper are as below:
(1) Mask R-CNN was used to extract the contours of the pig accurately by using RoI Align better. We converted the resulting image into a binary image. It solved the problem of uneven illumination on the image.
(2) We extracted the body size information and image features, and used the actual measured body size information to correct the body length and hip width. Furthermore, the depth information, which is the height from the camera to the pig back, was also used as a feature, which solved the problem of body bending. Finally, the rationality of the features was analyzed.
(3) A large number of regressors and deep learning methods were used to make predictions on three different kinds of features.
4. Conclusions
This paper proposed a new pig weight estimation method based on a deep learning and regression method. In total, 1505 back images of 39 pigs obtained by Azure kinect DK were used in the numerical experiments. To solve the problem of uneven illumination, the mask images were converted to binary images so that the edges and details of the target object were more clear. In order to solve the problem of the pig body bending, we extracted the image feature Dev and corrected PBL, PHW and Hdep using actual information and XGBoost. In addition, we analyzed the rationality of the extracted features. In the model stage, 10 machine learning methods in total were used to estimate the live weight of pigs in three different strategies. All three strategies provide good predictions of body weight. The one with the highest prediction accuracy is XGBoost, with an MAE of 0.389, RMSE of 0.576, MAPE of 0.318% and of 0.995. The results show that our method is accurate and reliable in weight estimation. Furthermore, the combination of Mask R-CNN and the ensemble regression method XGBoost is the best one in our article. We also recommend using the Mask R-CNN + RFR method because it has fairly high precision in each strategy.
We also need to restrict the movement of the pigs in our experiments. We have requirements in the shooting site that the pigs pass through the corridor one at a time. At the same time, we do not take full advantage of the side-looking information we collect from the pigs. Our method is only suitable for fattening pigs weighing 100–150 kg, and models and parameters need to be re-tested in other weight ranges. In the next stage, we plan to design a model to test the weight of pigs in all weight ranges and focus on the above issues.