Spray Quality Assessment on Water-Sensitive Paper Comparing AI and Classical Computer Vision Methods ^†

Abstract

Precision agriculture seeks to optimize crop yields while minimizing resource use. A key challenge is achieving uniform pesticide spraying to prevent crop damage and environmental contamination. Water-sensitive paper (WSP) is a common tool used for assessing spray quality, as it visually registers droplet impacts through color change. This work introduces a smartphone-based solution for capturing WSP images within vegetation, offering a tool for farmers to assess spray quality in real-world conditions. To achieve this, two approaches were explored: classical computer vision techniques and machine learning (ML) models (YOLOv8, Mask-RCNN, and Cellpose). Addressing the challenges of limited real-world data and the complexity of manual annotation, a programmatically generated synthetic dataset was employed to enable sim-to-real transfer learning. For the task of WSP segmentation within vegetation, YOLOv8 achieved an average Intersection over Union of 97.76%. In the droplet detection task, which involves identifying individual droplets on WSP, Cellpose achieved the highest precision of 96.18%, in the presence of overlapping droplets. While classical computer vision techniques provided a reliable baseline, they struggled with complex cases. Additionally, ML models, particularly Cellpose, demonstrated accurate droplet detection even without fine-tuning.

1. Introduction

As the world’s population continues to grow, the agricultural industry widely uses pesticides to increase the productivity of existing land, raising environmental and health concerns. Pesticides, which protect crops from pests and diseases, are vital for sustaining crop yields. However, their effectiveness heavily depends on precise application methods to minimize waste and ensure targeted deposition. Despite efforts to promote non-chemical alternatives, chemical pesticides remain prevalent. This has led to regulatory measures such as the European Commission’s Sustainable Use of Pesticides Regulation, aimed at reducing pesticide risks by 2030.

In this context, remote sensing techniques can be used as a tool to address environmental and agricultural challenges, which are aligned with the Sustainable Development Goals (SDGs) of the UN 2030 Agenda. These remote sensing technologies offer precise monitoring of pesticide application and efficiency while reducing environmental risks [1].

Inconsistent application of pesticides in crop management can lead to significant economic, environmental, and health risks. These include uneven protection, weed resistance, and increased pesticide use [2]. Achieving optimal droplet patterns is crucial to reduce drift and ensure accurate product deposition. In the last few years, techniques for calculating important metrics, which are key factors in reducing wasted spray, have evolved from manual processes to automated solutions such as DropLeaf [3] and SmartSpray [4]. However, the growing need for more efficient tools highlights the potential for artificial intelligence and machine learning (ML) techniques to advance spray quality assessment and optimize agricultural practices.

The primary objective of this work was to design and develop a portable system that uses a smartphone’s integrated camera to photograph water-sensitive paper (WSP) and automatically assess spray quality for agricultural applications. Two strategies were implemented for droplet detection: one using classical computer vision (CCV) techniques and the other employing machine learning (ML) models, both producing comparable results with a similar workflow. The system offloads computational tasks to an external server for efficiency, with a synthetic dataset created to improve ML model training. The developed system enables accurate and scalable droplet analysis, offering a practical tool for farmers to optimize pesticide application and promote sustainable agricultural practices.

WSPs have been a fundamental tool for agricultural spray evaluation for over 40 years, used in both aerial and ground applications [5,6]. The use of sensitive colorimetric papers to monitor surface ozone mixing ratios dates back to 1900. Notably, this technique was applied in Athens, Greece, between 1901 and 1940, resulting in a unique historical ozone series. This record provides a benchmark for comparing historical surface ozone levels with current measurements, offering insights into long-term atmospheric trends [7].

The tools used include semi-rigid papers, coated with bromothymol blue on one side, available in various sizes. The coating appears yellow when dry but turns different shades of brown, blue, and purple when it comes into contact with water droplets. These stains create a significant contrast with the dry yellow background, making it easier to assess the dispersion of the droplets [2,3,8]. However, this method faces limitations such as the inability to measure droplets smaller than 50 µm [3,6], sensitivity to high humidity [4,8], and inaccuracies with high coverage [5]. These limitations require alternative methods for accurate spray analysis.

The evaluation of droplet sizes commonly uses volume median diameter (VMD), which expresses drop size based on the volume of liquid sprayed. The spray is divided into two equal parts based on the sprayed volume, meaning that 50% of the total volume is made up of drops with diameters larger than the VMD value, and the other 50% is made up of droplets with diameters smaller than the VMD value [2,9]. The relative span factor (RSF) is another metric commonly used. It is a dimensionless parameter indicative of the uniformity of the drop size distribution. RSF provides a practical way to compare various drop size distributions. High RSF values indicate wider drop size distributions and lower RSF values indicate less variation among drop sizes [2,3,9,10].

The primary data required from the WSP images are the diameters of the droplets. The diameter is a fundamental parameter that directly influences the calculations of VMD, RSF, and coverage percentage. Precise measurement of droplet diameters allows for a detailed assessment of the spray pattern and effectiveness. From the image, the most reliable measurement acquired from all the segmentation methods is the area. By using the area, the diameter can be calculated by approximating the droplet shape to a perfect circle and using the relation between area and diameter. Additionally, to calculate the real-world measurements of each droplet, the server-side application must obtain the real-world size of the paper to be analyzed in micrometers, which is then used to establish the true ratio between the measurements in pixels and micrometers:

$$d_{\mu m}=2\times \sqrt{{\displaystyle \frac{A_{px}}{\pi }}}\times {\displaystyle \frac{widt{h}_{\mu m}}{widt{h}_{px}}}.$$

(1)

Here, $d_{\mu m}$ is the diameter in micrometers, $A_{px}$ is the area of the droplet in pixels, $widt{h}_{px}$ is the measurement of the width of the WSP in pixels, and $widt{h}_{\mu m}$ is the measurement of the width of the WSP in micrometers.

Micrometers are the standard unit of measurement for droplet diameters, given the scale and precision required in agricultural spray analysis. Pesticide droplets are typically very small, often ranging from a few micrometers to several hundred micrometers.

In the field of agricultural spray quality analysis, a variety of vision-based techniques are employed to evaluate droplet deposition on WSP. CCV methods, like DepositScan [11] and SnapCard [12], have long been used to assess spray coverage and droplet size. Recent advances include methods by Özlüoymak [13] and Xun [14], which address specific challenges, such as overlapping droplets, and enhance accuracy. These approaches highlight the evolution of classical techniques to better meet the needs of agricultural applications.

On the other hand, ML methods have been introduced to further refine spray quality assessment. Techniques by Chen et al. [15], involving deep convolutional neural networks (CNNs), and Yang et al. [16], focusing on methods for UAV-based detection, improve accuracy in detecting and segmenting droplets, particularly in complex scenarios involving overlapping droplets. Additionally, systems like DDAS [17] incorporate environmental data to enhance prediction accuracy.

The integration of these advanced methodologies demonstrates a significant shift toward more precise tools in agricultural spray quality analysis. Notably, this study stands out from earlier research through its unique approach of employing synthetic annotated datasets alongside AI models, enhancing accuracy and adaptability in real-world conditions.

2. Data and Methods

The main goal of the system is to develop an Android application tool for real-time assessment of spray quality on WSP. The client-side Android application captures an image of the WSP and sends it to the server side. The server analyzes the image, provides droplet statistics, and sends the results back to the client. The server-side application uses a dual-method algorithm that can use CCV or ML techniques to accurately segment water-sensitive paper and droplets. The primary objective of both algorithms is to detect and separate overlapped droplets, ensuring that droplets that overlap are counted as separate entities for statistical purposes. Both of the developed methods follow an equivalent logic, as follows: detect the WSP, remove distortion, segment individual droplets, and calculate the WSP statistics. This dual-algorithm approach allows us to directly compare distinct methods of analyzing WSPs. For a visual representation of the proposed solution, refer to Figure 1.

2.1. Water-Sensitive Paper Datasets

The project requires two distinct datasets to develop algorithms for (i) the detection of WSP in the vegetation and (ii) the identification of individual droplets on that WSP. The datasets developed are designed according to their task.

The standard WSP image contains over 1000 droplets that require manual labeling to establish an accurate ground truth. This is challenging due to pixel-level segmentation requirements and the risk of incorrect labels. To overcome this, an algorithm was created to generate synthetic data that replicate real-world processes and automatically generate ground-truth annotations. This approach saves time, ensures consistency, and improves annotation accuracy—factors that are vital for effective ML model training. Its main objective is to effectively apply transfer learning during model training by containing various elements extracted from real datasets to improve accuracy when models are presented with real data.

Two distinct sets of WSP images were provided, forming the basis for the analysis and study of the visual aspect of real spraying on a WSP. The images were also used to test the application of the implemented algorithms. Given the task of manually labeling each droplet, only two images were properly annotated for testing purposes.

2.1.1. Real Water-Sensitive Paper Dataset

The first set of images consists of 127 images captured with a smartphone in field conditions, where WSPs were placed on pink backgrounds for contrast. However, the dataset lacked sufficient variation and quantity of images to train a CNN effectively. The second set of images comprises PDF files of WSPs stapled to white paper, labeled with details like location and height of placement. GIMP was used to extract the individual WSPs. The color variability across these two sets of images highlights the challenges of accurately detecting droplets under different conditions.

The number of images provided is insufficient to effectively train a CNN and the provided images offer limited variation in backgrounds and WSP placement. As a result, new photographs were taken and were added to the dataset. Images of the synthetic dataset at the time were printed and placed against various backgrounds and photographed, resulting in a more varied and comprehensive dataset. It is important to note that the printed images represented an early version of the synthetic dataset developed. Since then, the dataset has undergone multiple upgrades, significantly improving the appearance and accuracy of the droplets. However, the images from the initial dataset were already available, so it was more efficient to retain them for annotation purposes that solely focused on identifying the paper.

The images were captured using two separate smartphones in multiple locations, at different times of the day, and under varying lighting conditions. This resulted in a total of 160 images, ensuring diverse environmental conditions and lighting variations. Sample images depicting the variety of challenges are shown in Figure 2.

The annotation process for the WSP segmentation task dataset involved delineating the location of the WSP in the various backgrounds using Roboflow’s smart polygon tool, creating a final dataset with 278 images from both synthetic and real WSPs. For droplet annotation, an initial automatic segmentation was performed with the Cellpose model, followed by manual verification using Label Studio. Due to the inefficiency of online labeling tools for displaying large quantities of labels in one single image, these were divided into 512 by 512 pixel squares to manage the annotation workload. Currently, only two images have been annotated to evaluate the algorithms’ performance in real-world conditions.

2.1.2. Synthetic Water-Sensitive Paper Dataset

A synthetic dataset was created to address the challenges of data acquisition and annotation in WSP studies, with the primary goal of facilitating transfer learning during model training. The dataset incorporates elements from real data to improve accuracy when applied to real-world scenarios. To be effective, the dataset must meet several criteria: it should accurately reflect human perception, include real elements from actual datasets, mimic real-world droplet patterns, offer diverse content to avoid overfitting, and provide automatic annotation for droplet locations and spray quality metrics.

The dataset was generated by analyzing WSP visual characteristics and combining real-world elements like droplet shapes and background designs. While droplet shapes and segmentation are key, the dataset does not prioritize replicating the exact distribution of droplets. The average time to create one image in the synthetic dataset is approximately 41.2 s. Since the dataset only needs to be created once, further optimization was deemed unnecessary.

The following points detail the methodology used to generate the synthetic WSP images:

• Distribution of the number of droplets per image: The number of droplets per image in the synthetic dataset is predetermined by configurable values of mean and standard deviation. The distribution follows a normal pattern, capturing variability in the dataset. This ensures that each image reflects a controlled yet varied range of droplet counts, enhancing the dataset’s ability to represent diverse spraying scenarios;
• Size distribution of droplets: The algorithm uses the Rosin–Rammler distribution to determine droplet sizes in the synthetic dataset. This distribution is a widely used function that expresses drop size distribution with two parameters, representative diameter and a measure of drop size dispersion, and is useful for single-peaked results [10,18]. In this study, droplet size is represented as the area of the polygon describing each droplet in pixels.
• Image resolution: The datasets were created with three different image resolutions to test the model’s ability to generalize across varying image qualities. This simulates real-world conditions where image quality can vary due to different devices or settings. To maintain consistency, droplet sizes were adjusted according to the image resolution.
• Yellow background: A radial gradient background is created for each WSP image, with the center of the gradient randomly positioned within the image. The gradient transitions between two randomly selected shades of yellow, chosen from a predefined list derived from the real dataset, are given in Figure 3. To enhance realism, 10,000 pixels within each image are randomly selected. The intensity or value of these pixels is decreased by 10 units. This intentional alteration introduces imperfections into the background, mimicking real-world scenarios where images may exhibit small irregularities or imperfections.
• Droplet color: Real colors from two distinct datasets were used to fill the droplets, with each color palette applied separately to enhance variation and realism. Figure 4 represents the colors used.
• Droplet shape: Droplet sizes are determined using the Rosin–Rammler distribution for realistic probabilistic sizing. Shapes are selected from a set of 25,404 real droplet shapes, captured from pre-annotated images of the real dataset. Each droplet shape is based on the polygon coordinates of real droplets, scaled to fit within bounding rectangles for easier placement in the final image.
• Droplet placement and coloring: Droplets are all placed on randomly generated coordinates of the image. Then, overlapping droplets are identified and their polygons are joined. To reduce the computational load of checking overlaps, a sliding window technique is used, limiting comparisons to droplets within the same window. After all shapes are placed and overlaps are identified, the algorithm colors each droplet pixel by pixel, using real dataset hues. A coloring technique named “shape burst” is used, which darkens droplet edges and lightens toward the center. There are three main sets of colors: inner, middle, and outer colors, which are used given the distance of the pixel to the closest edge of the polygon. Colors are blended smoothly from outer to inner areas to avoid harsh lines, with smaller droplets exhibiting less middle and inner coloring, mimicking real-world observations where smaller droplets appear darker with less variation.

Figure 5 provides a direct visual comparison between a synthetic droplet and a droplet from the real dataset it aims to replicate. Figure 6 illustrates some examples of the synthetic images created.

The current synthetic dataset, generated by a highly parameterized program, can easily generate more synthetic images. These images can be used to achieve sim-to-real transfer learning of ML models and to have (synthetic) images labeled automatically. We should note that the images can be improved further so that they become even closer to human eye perception, but they are used as shown in Figure 6.

2.2. Segmentation and Droplet Classification

The primary objective of this study is the instance segmentation of droplets on WSP, which is essential for calculating spray quality metrics. Accurate droplet segmentation allows for precise measurement of droplet sizes and patterns, which directly impact spray analysis. The process involves two types of segmentation: WSP segmentation, which isolates the WSP from the background to ensure correct proportions for droplet analysis, and droplet segmentation, which identifies and segments each droplet on the WSP for accurate measurement of their sizes and distribution.

2.2.1. Segmentation of the Water-Sensitive Paper

One of the primary challenges in segmenting the WSP is distinguishing it from the background while mitigating image distortions. Variations in lighting, background clutter, and the paper’s inherent texture can obscure the WSP’s edges and details. To address this, the segmentation algorithm must be robust enough to differentiate the WSP from various backgrounds and correct for any distortions that might affect the accuracy of droplet size measurement.

The CCV method for segmenting WSP involves several key steps:

• Color analysis: The image is preprocessed with a Gaussian blur to reduce noise and smooth minor details. A 3D color histogram identifies the most prevalent color, which helps in creating a mask to isolate the WSP from the background. Canny edge detection is used to find contours based on the prominent colors.
• Thresholding: The masked image is converted to grayscale, simplifying it for thresholding. A binary threshold is applied, creating a black-and-white image that emphasizes edges and features for easier contour detection.
• Contour detection: Using OpenCV’s findContours function, the largest contour, assumed to be the WSP, is detected. The convex hull of the largest contour is computed to approximate the boundary of the WSP.
• Perspective transformation: Corner points of the WSP are identified, and the Euclidean distances between them are measured. A perspective transformation matrix is applied to correct any distortion, resulting in a properly aligned WSP image.

The ML method for segmenting WSP utilizes YOLOv8, a state-of-the-art object detection model known for its speed and accuracy. This model was chosen due to its real-time processing capability and high accuracy in handling complex images with varying backgrounds. The steps taken to integrate the model into the algorithm are described in the following list:

• Preprocessing: Images were resized to 640 pixels while preserving the aspect ratio, and auto-orientation adjustments were made to ensure consistent alignment.
• Data augmentation: To enhance the diversity of the training data and improve model robustness, various augmentation techniques were applied using Roboflow, such as horizontal and vertical flips, hue adjustment, changes in brightness, and the introduction of blur to simulate lens imperfections and noise in the image. Care was taken to ensure that all augmented images resembled real-world images to accurately represent the task at hand.
• Dataset splitting: The dataset was divided into training, validation, and testing subsets to ensure comprehensive model evaluation. Training images accounted for 70% of the dataset, while validation and testing accounted for 10% and 20%, respectively.
• Training parameters: A small model size was chosen as a compromise between maintaining sufficient detail for accurate segmentation and ensuring processing speed. Values of 0.2 and 0.5 for dropout and weight decay, respectively, were chosen to prevent the model from overfitting. The model trained for 200 epochs, since initial trials showed that a lower count was insufficient for effective convergence.
• Perspective transformation: Once the YOLOv8 model was trained, it was applied to segment the WSP in the image. Using its predicted mask, the WSP was isolated from the background. The results were then used to apply a perspective transformation, aligning the segmented WSP correctly in the final output image.

2.2.2. Segmentation of the Droplets in Water-Sensitive Paper

Accurately distinguishing between single and overlapping droplets is a significant challenge in droplet segmentation. Overlapping droplets can visually merge, complicating the task of counting and measuring each droplet individually. Effective segmentation techniques must separate these overlapping droplets to ensure precise size measurements and reliable statistical calculations, which means the algorithms must be capable of applying instance segmentation. This enables a direct comparison with all other methods developed, as both execute the same steps and return the same resulting information, the individual segmented droplets.

A significant challenge in this project was to find a model capable of instance segmentation, as most existing CNN models focus on the task of semantic segmentation. To address this, the search was expanded to include models trained to implement cell segmentation, as the tasks performed are similar to the tasks at hand for this study.

The CCV approach for droplet segmentation employs a multi-step process to accurately delineate droplets from the background, which are described in the following list as well as the visual representation in Figure 7:

• Thresholding: The segmentation begins by blurring the image to reduce noise, enhancing clarity. Otsu’s thresholding method is then applied to automatically separate droplets from the background. The Otsu method [19] stands out as particularly effective in the domain of this study [20,21].
• Edge detection: OpenCV’s findContours function is applied to the thresholded image, with an inversion step to ensure contours are detected as white lines against a black background.
• Contour classification: Detected contours are classified into three categories: single circle, which is a clearly defined circular droplet, single ellipse, which includes droplets resembling ellipses, and overlapped droplets if they do not fit into the other two categories. The algorithm uses circularity and aspect ratio measurements to differentiate between these classifications.
• Droplet instance segmentation: For circles and ellipses, the area is calculated and the information is saved. For shapes classified as overlapping droplets, the Hough Circle Transform [22] is applied to detect potential circles in the shape. The functions return a considerable amount of circles all clutched together. To further refine the circle detection, the KMeans method of the class cluster of sklearn is applied, with the idea of grouping similar circles by clustering and removing outliers. This helps in managing overlapping droplets by consolidating multiple detections into a single, more accurate representation. The final step is to remove circles that contribute negatively to the overall IoU (Intersection over Union) score. This IoU score is computed by comparing the threshold image of the detected shape in the ROI (region of interest) with the detected circles minus one of them. If the value of the IoU is better without a certain shape, then it is filtered out of the final detected circles. The visual representation of this pipeline is described in Figure 8.

The ML approach for droplet segmentation addresses the challenge of distinguishing overlapping droplets by leveraging instance segmentation models. Several models were evaluated for instance segmentation. Traditional models like UNet were limited due to the use of image masks, which fail to differentiate overlapping droplets. YOLOv8 and Mask R-CNN were selected as they handle instance segmentation effectively, accommodating overlapping objects through textual labels. YOLOv8 and Mask R-CNN were retrained with a synthetic dataset to optimize their performance for droplet segmentation.

Cellpose was not retrained due to label format incompatibilities and time constraints. It requires non-overlapping labels, which conflicted with the synthetic dataset created for YOLOv8. Additionally, detailed label format information was not readily available, complicating the manual annotation process. Despite this, the results from Cellpose are included.

One other limitation of Cellpose is its current reliance on the input of the average diameter of the droplets for each image, which affects its performance when applied across images with varying resolutions. The average diameter is not determined by the user but is internally set within the code with a value of 10. This approach ensures that computation time and model accuracy are optimized without requiring user input, which might introduce inconsistencies.

• Dataset splitting: The dataset was divided into training (70%), validation (10%), and testing (20%) sets, similar to the previous segmentation model.
• Image resizing: Images were resized into quadrants with a maximum size of 320 × 320 pixels to balance detail and computational efficiency, keeping processing times manageable. This size was chosen for accurate droplet detection while maintaining real-time processing capabilities. The resizing also aimed to minimize the number of droplets per image, improving detection accuracy. Both images and labels were divided, resulting in a final dataset of 5000 distinct images.
• Data augmentation: Data augmentation was not implemented as the synthetic dataset already provided diverse droplet shapes and sizes.
• YOLOv8 training parameters: The YOLOv8 medium model for segmentation was chosen to create a balance between accuracy and computational efficiency. This model was the largest model that could be used with the computational constraints of the computer used during the development of this project. It was trained for 300 epochs to ensure convergence.
• Mask R-CNN training parameters: The model was trained for 100 epochs, as testing showed it converged before reaching this limit, ensuring sufficient training without overfitting. A small learning rate was selected to ensure smooth convergence, as larger updates could destabilize the complex Mask R-CNN architecture. A confidence threshold of 0.4 was set to balance prediction certainty and model sensitivity, enabling accurate droplet detection while controlling false positives.

The segmentation algorithms were integrated into the server-side application to ensure uniformity in handling and presenting results. Regardless of the segmentation method used, outputs were stored in the same format (the YOLOv8 polygon label format), allowing the same code to generate statistics and visual classifications, ensuring consistency and easy comparison between methods.

For CCV algorithms, entire images can be processed directly, returning a list of detected droplets.

ML models, trained on smaller images, require the captured images to be divided into smaller sections for accurate processing, similar to the images used during the training process.

When droplets appear at the boundaries of these divided sections, a post-processing step merges droplets detected at the edges by comparing neighboring sections, ensuring accurate droplet counting. With the final droplet predictions, relevant statistics are calculated and displayed along with annotated images.

2.3. Android Application

The Android application was developed using the programming languages Kotlin and Java, with the UI layout design in XML. It comprises four distinct pages, which are represented in Figure 9:

• Initial page: This page contains a single button to start the camera;
• Camera page: This page has two buttons—one to take a picture and another to choose an image from the smartphone’s gallery;
• Configuration page: After selecting an image, this page requests practical information about the paper’s dimensions and allows the user to choose the segmentation algorithm. The information required is given to the user through the hint feature of Android Studio;
• Results page: This final page displays the results of the droplet detection, color-coded to help the user verify if the image was correctly identified, along with metrics used to evaluate the spray quality of the water-sensitive paper.

Architecture and Web Service

To address the computational demands associated with image segmentation, a web service was developed using Python and Flask to handle processing tasks, transferring them from the smartphone to a server. Communication between the smartphone and the server was facilitated through a REST API, where the server received images, processed them using the designated segmentation model, and returned results, including detected objects and relevant statistics. Data were exchanged between the client and the server in structured JSON packages.

This server–client architecture extends the usability of the system to a broad range of Android devices, as it minimizes the computational and energy demands on the smartphone. Centralizing processing tasks on the server also simplifies model updates and code maintenance. The evolution of machine learning models can be managed entirely server-side, allowing for continuous improvements without requiring changes to the client-side application.

While processing on the smartphone would reduce dependency on server communication, it would significantly increase battery consumption and potentially limit the system’s compatibility with lower-end devices. In contrast, the proposed architecture introduces a trade-off, that is, it requires constant online communication and server availability. This dependency necessitates a well-maintained server infrastructure that is capable of handling real-time requests reliably. To mitigate the challenges posed by intermittent connectivity in remote agricultural settings, an additional feature was implemented, allowing users to select images stored locally on the smartphone for processing. This enhancement increases the system’s flexibility, particularly in remote agricultural settings with intermittent connectivity.

The Android app and web service provide an efficient platform for real-time analysis of WSP metrics, addressing the identified market gap.

3. Results

3.1. Segmentation of Water-Sensitive Paper Results

The YOLOv8 model trained for 2 h and 30 min over 200 epochs. Throughout the training, the model automatically computed segmentation metrics, providing insights into its performance.

Figure 10 and Figure 11 show how the training and validation losses evolved across epochs. These losses reflect the accuracy of the model in predicting bounding boxes (box loss), pixel-level segmentation (segmentation loss), and object classification (classification loss). Upon completing the training epochs, the algorithm conducts a final validation round, computing precision vs. recall at varying confidence thresholds (steps of 0.001), as depicted in Figure 12.

The steady decline in losses demonstrates the model’s learning progress, with convergence observed as losses stabilize near zero, indicating minimal benefit from further training. Additionally, the precision vs. recall graph reflects high recall and precision across varying confidence thresholds. Overall, YOLOv8 demonstrates high accuracy, precision, and recall in segmenting droplets on water-sensitive paper, making it a dependable tool for accurate statistical analysis.

The validation of both the algorithms developed for the segmentation of the water-sensitive paper is conducted by calculating the IoU metric using both methods on the testing images, which were put aside when the training of YOLOv8 took place. These images include a multitude of backgrounds, with the WSP not always being centered or the main focus of the image, assuming the worst type of user for the images.

The results, presented in

Table 1. The metric evaluation of algorithms used for segmenting water-sensitive paper. The values in bold highlight the best results for each metric.

Method	IoU	Time (ms)
YOLOv8	0.9776	0.2801
CCV	0.3664	0.0357

, show that YOLOv8 significantly outperformed the CCV approach in terms of IoU. CCV’s reliance on fixed parameters made it less adaptable to diverse image characteristics, such as variations in shape, color, and WSP orientation, leading to inferior performance compared to YOLOv8.

3.2. Segmentation of Droplets

For the droplet segmentation evaluation, a dedicated algorithm was developed to ensure consistent evaluation across methods. It standardized segmentation prediction results into the YOLOv8 label format and matched predicted annotations with ground-truth annotations based on the highest IoU.

The following two dataset formats were used to study how input image structure affects segmentation algorithm performance: full unaltered images and images divided into 320 × 320 pixel quadrants. This matches the format used to train YOLOv8 and Mask R-CNN models. This division simulates real-world use of the algorithm developed, where images are cropped to improve segmentation results, and it must be able to handle droplets at image boundaries. The datasets include synthetic (SD) and real (RD) images, with the synthetic dataset featuring 30 images and producing 860 quadrant images, and the real dataset consisting of 2 manually annotated images, generating 55 quadrant images. Despite the small size of the real dataset, its inclusion is vital for testing the algorithms under real-world conditions and evaluating the transfer learning performance.

YOLOv8 trained for 5 h and 15 min over 300 epochs. As previously mentioned, YOLOv8 generates its own graphs to present the losses, which are presented in Figure 13, Figure 14 and Figure 15.

The steady decrease in both training and validation losses indicates effective learning and good generalization capabilities of the YOLOv8 model. The alignment between the trends of training and validation losses suggests that the model is not overfitting. By the end of 300 epochs, all loss functions appear to reach stability, although the final values are not as low as desired. This implies that further training may not yield significant improvements in model performance.

The Mask R-CNN model saves log files for each epoch, which enables the graphical visualization of the train and validation losses using TensorBoard. Figure 16 and Figure 17 illustrate the evolution of the different training and validation losses across the epochs for the final Mask R-CNN model, which was specifically retrained for droplet segmentation.

Both graphs show a rapid decrease in losses related to segmentation, bounding box regression, and classification in both the training and validation phases. This downward trend indicates that the model is effectively learning the task of droplet detection, with each component improving as the training progresses. The final values are considerably low, suggesting that the model has converged and is capable of accurately predicting droplets with minimal errors in both bounding box placement and segmentation masks.

Ideally, a precision–recall curve, which is represented in Figure 18, would be closer to the top right corner, indicating high precision and recall simultaneously. The sharp decline in precision in both graphs from YOLOv8 and Mask R-CNN shows that the models’ ability to maintain accuracy declines rapidly as they try to identify more droplets. This makes the model more cautious when trying to detect droplets in an image, as it prefers to refrain from making a guess rather than guess wrong.

The algorithms developed were evaluated on the segmentation task and the statistical performance task, to determine which model is the best for the objectives of this study.

Table 2. Algorithm results for calculating droplet segmentation metrics on water-sensitive paper. The best and worst values for each metric are highlighted in green and red, respectively.

Method	Dataset	Precision	Recall	F1 Score	mAP50	mAP50-95	Time (s)
YOLOv8	RDS	0.4081	0.2758	0.3292	0.1408	0.0335	0.5317
	RDF	0.3922	0.2808	0.3273	0.1553	0.0323	14.6225
	SDS	0.5706	0.6183	0.5935	0.4153	0.1447	2.5819
	SDF	0.5254	0.1732	0.2606	0.1155	0.0377	6.9409
Mask R-CNN	RDS	0.7074	0.2423	0.361	0.1501	0.0416	13.1807
	RDF	0.6613	0.2694	0.3829	0.1919	0.0552	300.5127
	SDS	0.9449	0.2178	0.354	0.3108	0.1423	13.0516
	SDF	0.9368	0.1825	0.3054	0.221	0.0982	168.3657
Cellpose	RDS	0.9618	0.8083	0.8784	0.7846	0.522	7.637
	RDF	0.9577	0.8272	0.8877	0.7818	0.4353	129.2765
	SDS	0.9284	0.7819	0.8489	0.7346	0.3225	7.2934
	SDF	0.9249	0.7833	0.8482	0.7129	0.2692	75.3545
CCV	RDS	0.788	0.8507	0.8182	0.6776	0.3095	0.1108
	RDF	0.7425	0.8586	0.7963	0.6134	0.2509	11.1281
	SDS	0.8508	0.7279	0.7846	0.6376	0.4491	0.1808
	SDF	0.837	0.7279	0.7786	0.6202	0.3323	7.2448

displays the performance metrics of the four segmentation methods, YOLOv8, Mask R-CNN (MRCNN), Cellpose, and CCV, on real (RDS, RDF) and synthetic (SDS, SDF) datasets. Figure 19 illustrates the visual segmentation results of each method.

Two dataset formats were used to study how the structure of input images affects algorithm performance. The first format used full, unaltered images, while the second divided the images into smaller quadrants, each with a maximum dimension of 320 × 320 pixels. This division mimics the format used to train the YOLOv8 and Mask R-CNN models. The reason for using both formats is based on how the segmentation algorithms are used in real-world scenarios. When users apply the segmentation model, their input images are cropped to match the format used for YOLOv8 training as a way to obtain better results. Therefore, the evaluation performed also seeks to determine how well the algorithms handle droplets located on the boundaries between adjacent image sections.

The datasets themselves consist of synthetic (SD) and real (RD) images. The synthetic dataset includes 30 images (125,923 droplets), constituting 20% of the total dataset—SDF. After dividing the full synthetic images into quadrants, a total of 860 smaller images were created—SDS. The real dataset, on the other hand, is composed of two manually annotated images (2754 droplets), which are adequate for pixel-wise segmentation. These were carefully verified for accuracy but remain subject to potential human error—RDF. When divided, these real images generate 55 quadrant images—RDS. Despite the small size of the real dataset, its inclusion is vital to test the algorithms under real-world conditions, where segmentation challenges may differ from those in synthetic environments, as well as to analyze the transfer learning efficacy.

YOLOv8 had the lowest precision, struggling with low-resolution images and small droplet sizes, as well as overlapping droplets, making it more suited for object detection rather than precise segmentation. Cellpose excelled overall, particularly with real datasets, effectively handling overlapping droplets and achieving the best precision and recall, although its need for user input on droplet size affected consistency. CCV produced excellent visual segmentation and was robust across different droplet sizes and shapes, but it was less effective at handling overlapping droplets compared to Cellpose. Mask R-CNN showed average performance, with an overly conservative approach leading to frequent missed detections and lower confidence scores, which impacted its overall effectiveness.

To evaluate the quality of the developed segmentation methods, comparisons were made with existing smartphone applications. Three apps—DropLeaf, SmartSpray, and SnapCard—were identified and were freely available, but only DropLeaf allowed image uploads from the gallery, making it the sole option for a fair comparison. A virtual machine running Android 7.1.2 was set up to use DropLeaf, and a sample of 30 images from the testing dataset was selected to make the comparison manageable. Since each application offers different metrics, the comparison focused on calculating the relative error between the ground-truth values (derived from the developed segmentation algorithm) and those provided by DropLeaf, particularly for metrics like RSF, droplet count, coverage percentage, and representative diameters.

Table 3. Relative error of each metric used to evaluate the spray quality of a WSP. The best and worst values for each metric are highlighted in green and red, respectively.

Method	Dataset	Number of Droplets	VMD	RSF	Coverage Percentage
YOLOv8	RDS	0.4602	0.6190	0.4312	0.4274
	RDF	0.3570	0.1522	0.4739	0.2057
	SDS	0.1985	0.2229	0.2881	0.3957
	SDF	0.2053	0.1540	0.3022	0.4398
Mask R-CNN	RDS	0.7230	0.6846	1.0269	0.6062
	RDF	0.7274	0.1609	0.6883	0.6422
	SDS	0.6808	0.1878	0.2660	0.7082
	SDF	0.7845	0.0799	0.1750	0.7908
Cellpose	RDS	0.1291	0.4483	0.1581	0.1645
	RDF	0.0894	0.1181	0.0923	0.1034
	SDS	0.1763	0.2247	0.1748	0.2695
	SDF	0.1912	0.2317	0.1694	0.2925
CCV	RDS	0.1811	0.3965	0.3957	0.3922
	RDF	0.1229	0.0636	1.3136	0.3030
	SDS	0.1822	0.1645	0.3866	0.0905
	SDF	0.1938	0.1206	0.4009	0.0843
DropLeaf	RDS	-	-	-	-
	RDF	0.6096	0.7362	0.0916	0.5223
	SDS	-	-	-	-
	SDF	0.9667	0.3738	8.5897	0.2639

represents the relative errors calculated given the statistics each method presents.

Conservative models like YOLOv8 and Mask R-CNN tend to not detect most droplets, resulting in lower droplet counts and higher relative errors in coverage and droplet number, whereas Cellpose and CCV strike a better balance in droplet detection. Cellpose consistently delivers low errors, excelling in overlapping droplet segmentation, while CCV competes well in simpler datasets but tends to overestimate droplet counts, leading to higher errors in VMD and RSF. YOLOv8 performs well in synthetic datasets but struggles with real-world complexity. Mask R-CNN’s tendency to under-segment affects its RSF values. DropLeaf shows the highest relative errors, especially in sub-sampled datasets, highlighting the need for recalibration. Synthetic datasets, being more controlled, generally produce fewer segmentation challenges, as evidenced by the lower errors compared to real-world datasets across all models.

Synthetic datasets tend to present fewer challenges for segmentation compared to real-world datasets due to their uniform characteristics. This is evident from the generally lower errors in the statistics and overall higher precision in the segmentation observed in synthetic datasets compared to real-world datasets across all models.

4. Discussion

In this study, both CCV techniques and ML-based methods were employed to address the challenge of droplet segmentation on WSP. Each approach offered unique strengths and presented distinct challenges, ultimately contributing to a comprehensive understanding of the segmentation process.

The CCV method in the WSP segmentation, which detects the WSP based on image color histograms, performs well only when the WSP is centered and occupies most of the image. Two example images highlight the contrasting performance of the methods in Figure 20. In one example, the CCV method achieved an IoU of 97.34%, while in another, it only reached 8.59%. Conversely, YOLOv8 showed more consistent results, with IoU values of 97.03% and 93.78%, respectively, demonstrating its robustness and adaptability to various image conditions.

In the droplet segmentation algorithms, Cellpose achieved the highest precision and recall for detecting droplets in a WSP. However, its current reliance on the user to input the average diameter of the droplets for each image affects its performance when applied across images with varying droplet sizes and resolutions. CCV managed to segment droplets of different sizes and shapes with ease, proving to be robust across various image conditions.

The conservative approach of both Mask R-CNN and YOLOv8, although leading to more cautious predictions, did not benefit the final segmentation outcome as much as a more aggressive approach like that of Cellpose or CCV. Furthermore, Mask R-CNN and YOLOv8 required dividing the image into quadrants, which can introduce errors when stitching results together. Full image analysis, as conducted by Cellpose and CCV, had lower precision due to the larger error margin with bigger images.

In the detection of overlapping droplets, CCV demonstrated an adequate ability to identify the presence of overlaps. However, challenges in accurately segmenting individual droplets were observed. The method predominantly produced circular shapes that aimed to encompass the detected regions as a whole. Occasionally, empty spaces between droplets were also misclassified as droplets, particularly when the negative area of the shapes appeared circular, indicating potential areas for refinement in segmentation accuracy.

YOLOv8 showed promising results in handling certain scenarios but faced difficulties with the segmentation of small droplets and clustered arrangements. When droplets were closely grouped, the model sometimes produced square-like masks that did not capture the full extent of individual droplets. In cases of overlapping droplets, the model occasionally divided shapes into smaller sections that did not consistently represent the actual droplet structure. These observations suggest room for improvement in the model’s ability to cohesively recognize and segment overlapping regions.

Mask R-CNN demonstrated strong potential by covering most of the pixels belonging to droplet shapes when segmentation was successful. However, accurately segmenting overlapping droplets remained a challenge. The results typically did not show significant overlap, indicating that further training or adjustments might enhance the model’s ability to handle such scenarios. Given its proven success in other research contexts, Mask R-CNN holds promise for achieving more robust performance with additional optimization.

The loading times of YOLOv8, Mask R-CNN, and Cellpose posed challenges for real-time applications, with significant delays in model initialization. CCV demonstrated faster segmentation times for smaller images, making it more suitable for real-time analysis. In practical applications, the choice of model may depend on the trade-offs between speed and precision, with each offering distinct advantages depending on the specific requirements of the droplet segmentation task.

When comparing the results from the real and synthetic datasets, it became clear that the transfer learning applied to the Mask R-CNN and YOLOv8 models had limited success. For Mask R-CNN, the real dataset, comprising 2754 droplets, achieved a precision score of 70.74%, while the synthetic dataset, containing 125,923 droplets, showed stronger performance, achieving an F1 score of 94.49%. This finding implies that the synthetic dataset was well-constructed for Mask R-CNN, enabling the model to adapt effectively. However, when evaluating YOLOv8, the results highlight its inability to accurately detect very small droplets. Performance was substandard for both datasets, with a precision of 57.06% on the synthetic dataset and only 40.81% on the real dataset.

This outcome can be attributed to several factors. The synthetic images may not have accurately represented real-world conditions, creating a mismatch between the training and testing environments; the small size and limited diversity of the real dataset likely restricted the models’ ability to generalize from synthetic to real images; variations in lighting, texture, and droplet appearance between the two datasets may have further hindered the effectiveness of transfer learning.

Additionally, the real dataset yielded a precision score of 96.18% with Cellpose. In contrast, the synthetic dataset achieved a slightly lower precision of 92.84%. These results suggest that the synthetic dataset is, in fact, ambitious, and Cellpose is able to detect real droplets more easily than synthetic ones.

The synthetic datasets developed in this study offer significant potential for improving applications beyond droplet segmentation. Their controlled generation of diverse shapes, sizes, and spatial distributions can benefit fields like medical imaging, where the segmentation of irregular biological structures (e.g., cells, tissues) often poses challenges. Similarly, in industrial applications like defect detection on surfaces or particle analysis in materials science, synthetic datasets can provide comprehensive training scenarios for machine learning models, enabling them to adapt better to real-world variability. By refining synthetic dataset creation to closely mirror specific application conditions, this approach can facilitate model generalization. This ultimately leads to more robust, high-performing algorithms across a wide range of domains and potentially benefits other applications.

5. Conclusions and Future Work

This study employed both CCV techniques and ML-based methods to tackle droplet segmentation on WSP, revealing strengths and challenges for each approach.

One of the core objectives was to develop a portable system for in-field use that could accurately assess spray quality using a smartphone. This goal was achieved through the successful integration of an Android application, which communicates with an external server via a REST API for efficient processing. As a result, the system is both practical for in-field use and adaptable for future improvements, fulfilling the goal of enhancing spray quality evaluation in real agricultural settings.

Another significant objective was the comparison of classical computer vision techniques and machine learning models for droplet detection. The study successfully implemented both strategies, ensuring a comprehensive accuracy, efficiency, and scalability analysis.

For the WSP segmentation task (finding the paper), YOLOv8 demonstrated strong performance with high precision scores, indicating effective learning and strong generalization. The model’s convergence over 200 epochs suggests it effectively learned the necessary features for accurate segmentation, achieving over 97% in key metrics.

For the droplet segmentation task, Cellpose delivered the best overall performance, especially on real-world datasets, excelling in segmenting overlapping droplets. However, the need for inputting droplet diameters for each image was a limitation, especially with images of extreme resolutions. CCV techniques also performed well, particularly in detecting droplets of different sizes and shapes, although they struggled with overlapping droplets.

ML-based methods, such as YOLOv8 and Cellpose, offer significant advantages over CCV techniques in both tasks. While CCV is effective for simpler tasks and provides strong visual results, it struggles with complex scenarios such as overlapping droplets. However, ML models can learn intricate patterns and relationships from the data, allowing them to better handle these challenges. Additionally, ML approaches are more adaptable and capable of improving performance with additional training or data augmentation, whereas CCV methods typically rely on fixed algorithms that may not generalize as well across different datasets or conditions.

Creating a synthetic dataset to overcome the challenges of manual annotation also represented a significant point in the development of the project. The synthetic data, derived from real-world elements, enabled transfer learning, allowing the machine learning models to generalize their performance effectively. The success of this strategy demonstrates that the solution provides an alternative to the labor-intensive manual annotation task, fulfilling the objective of streamlining the dataset creation process and accelerating the development of segmentation models.

The authors offer the organized and annotated dataset, including synthetic and real data, for future use by visiting the Zenodo project web page (https://zenodo.org/records/13995950 accessed on 24 January 2025).

The application offers real-time functionality through cellphone communication, enabling immediate processing and statistics. In scenarios where the connection is unavailable, users can capture standard photographs and process them later when a connection is restored.

For optimal performance, the WSP must have a clear contrast with its background, for accurate segmentation and analysis. While the application incorporates algorithms to correct a limited degree of perspective distortion, it does not address camera lens distortion. This design choice simplifies the app’s usability but relies on the inherent photogrammetric quality of the smartphone’s camera. Furthermore, it supports images with varying spatial resolutions, provided the WSP occupies a substantial portion of the frame. Based on empirical observations, the authors estimate that the processing performs reliably if at least 50% of the image pixels correspond to the WSP. Despite these constraints, the application’s capabilities make it a practical and accessible tool for precision agriculture.

While this study has successfully developed a system for droplet segmentation and spray quality assessment using a smartphone, several areas could be improved in the future. These enhancements could further improve the system’s accuracy and practical application, and also address some of the limitations encountered during this study. The following key areas are proposed for future research and development:

• Data augmentation: Applying data augmentation techniques to enhance the robustness of ML models, particularly for complex scenarios like overlapping droplets.
• Overlap handling: Investigating advanced techniques or post-processing methods to improve the separation of overlapping droplets.
• Synthetic and real-world data balance: Expanding the training dataset to include a higher proportion of real-world images, improving model generalizability across varied conditions.
• Retraining ML models: Retraining Cellpose and similar models with both synthetic and real-world datasets for better performance and adaptability.
• Resource optimization: Leveraging more powerful computational resources to explore larger models and higher-resolution images, which could enhance segmentation accuracy.

To increase the practical utility of these findings, future work could add features to the application to benefit real-world deployment and ease of usage. Key enhancements include integrating automated preprocessing features such as adaptive contrast adjustments and perspective correction to address environmental challenges such as poor lighting and uneven surfaces. Implementing offline functionality for machine learning models on mobile devices would remove reliance on network connectivity, making the tool more viable in remote agricultural settings. Additionally, it would also be of interest to expand platform compatibility (e.g., iOS) and make the application available in major app stores.

This article presents findings from Inês Simões’s master’s thesis [23], completed as part of her graduation requirements at University of Porto.