Identifying Rhodamine Dye Plume Sources in Near-Shore Oceanic Environments by Integration of Chemical and Visual Sensors

Abstract

This article presents a strategy for identifying the source location of a chemical plume in near-shore oceanic environments where the plume is developed under the influence of turbulence, tides and waves. This strategy includes two modules: source declaration (or identification) and source verification embedded in a subsumption architecture. Algorithms for source identification are derived from the moth-inspired plume tracing strategies based on a chemical sensor. The in-water test missions, conducted in November 2002 at San Clemente Island (California, USA) in June 2003 in Duck (North Carolina, USA) and in October 2010 at Dalian Bay (China), successfully identified the source locations after autonomous underwater vehicles tracked the rhodamine dye plumes with a significant meander over 100 meters. The objective of the verification module is to verify the declared plume source using a visual sensor. Because images taken in near shore oceanic environments are very vague and colors in the images are not well-defined, we adopt a fuzzy color extractor to segment the color components and recognize the chemical plume and its source by measuring color similarity. The source verification module is tested by images taken during the CPT missions.

1. Introduction

Autonomous underwater vehicles (AUVs) with chemical plume tracing (CPT) capabilities would be valuable in searching for deep sea hydrothermal vents, finding unexploded ordnance in near-shore oceanic environments, and monitoring pollutants or localizing sources of hazardous chemicals in a harbor. Factors that complicate CPT in natural environments (near-shore ocean conditions) include significant filament intermittency and chemical plume meander, and flow variations with both location and time.

Over the last decade there has been interest in developing bio-inspired strategies for CPT in natural environments. Belanger and Willis presented plume tracing strategies, including counter-turning strategies, intended to mimic moth behavior and analyzed the performance in a computer simulation [1]. Li et al. evaluated and optimized the moth-inspired plume tracing strategies using a simulated plume with significant meander and intermittency of plume puffs [2]. Grasso and Basil presented biomimetic strategies, inspired by lobsters, crayfishes, and crab, and used a lobster-inspired robot for locating an odor source [3]. Minagawa et al. presented a crayfish robot employing flow induced by waving to locate a chemical source [4]. Lochmatter and Martinoli tested the bio-inspired algorithms for tracking odor plumes in a laminar wind field [5]. Swarm-based CPT algorithms inspired by insects, such as a group of moths or an ant colony, are discussed in [6–8]. Most engineering-oriented algorithms for CPT are comprehensively reviewed in [9].

Most of the existing studies have mainly focused on the plume tracing issue and validated their algorithms in a small operation area in a range of centimeters to a few meters or in a simulated environment, but they lacked detailed reports of CPT experiments in natural environments on a large scale. The tracking of a chemical plume in the natural environment, such as near-shore oceanic conditions, where the plume is developed under the influence of turbulence, tides and waves, challenges CPT algorithms on how to cope with the intermittency of filaments and significant plume meander. The lack of correlation between the fluid flow direction and the plume long axis often causes an AUV to lose contact with the plume because the instantaneous fluid flow direction within a plume with significant meander is not always aligned with the plume's long axis [10]. Conducting CPT missions in near-shore ocean environments on a large scale has to consider the full spectrum of field behavior that include plume finding, plume following, maneuvers to recontact a lost plume, and declaration that the source has been found [2]. Li et al. proposed a subsumption architecture for CPT missions [11], consisting of these four fundamental behaviors: finding the plume, maintaining the plume, reacquiring the plume, and declaring the source location or identifying the source location.

Our research mainly focuses on the plume source identification issue in real world scenarios. The strategy discussed herein includes two modules: Source declaration (or identification) and source verification. This article presents the experimental results on identifying rhodamine dye plume sources in near-shore oceanic environments to validate two variations of the source identification algorithms SIZ_T and SIZ_F described in [12]. For the in-water tests, the SIZ_T algorithm was implemented on a REMUS vehicle, developed by the Oceanographic Systems Laboratory at the Woods Hole Oceanographic Institute [11,13], as shown in Figure 1. In order to validate the SIZ_F algorithm [14], Shenyang Institute of Automation (SIA), Chinese Academy of Sciences, developed a new AUV plume tracer and an alternative strategy for generating a rhodamine dye plume in near-ocean environment. Figure 2 shows the mission MSN171639 conducted at Dalai Bay (China) in October 2010.

In the field tests, several strategies were used to verify the declared chemical source, e.g., for the experiments in 2002, divers observed the declaration activities with help of an additional underwater camera system, for the experiments of 2003, side scanners generated sonar images to verify the declared chemical source, and for the experiments of 2010, an operator visually verified the declared chemical source. For further operations to deal with the declared plume source, the source needs to be automatically or interactively confirmed or inspected using a visual system. However, images taken in near shore ocean environments are very vague and colors in the images are not well-defined since the propagation of light underwater is affected by scattering, refraction, and absorption. In order to deal with these uncertainties, we adopt a fuzzy color extractor to segment the color components and recognize the chemical plume and its source by measuring color similarity.

This paper is organised as follows: In Section 2, we present the subsumption architecture for chemical plume tracing. In Section 3, we discuss the source declaration module derived from a moth-inspired plume tracing strategy. In Section 4, we report the in-water tests for declaring rhodamine dye plume source locations, mainly focusing on the most recent tests conducted in October 2012. In Section 5, we discuss the fuzzy color segmentation based verification module. In Section 6, we draw conclusions.

2. Subsumption Architecture for Plume Tracing

A subsumption architecture [15] provides a bottom-up method of implementing a behavior-based control system layer-by-layer, so the architecture is suited to integrate the low-level bio-inspired algorithms with high-level machine intelligence algorithms. For example, the subsumption architecture for the CPT mission in Figure 3 has easily expanded from having four behaviors in [11] to having six behaviors in [16]: Finding The Plume (Find-Plume), maintaining the plume (Maintain-Plume), reacquiring the plume (Reacquire-Plume), identifying the source location (Declare-Source), avoiding an obstacle (Avoid-Obstacle), and constructing a plume map (Construct-Plume-Map). The architecture uses inhibition or suppression mechanisms to coordinate potential conflicts of the commands from the various behaviors. Consequently, the outputs from a higher layer override or subsume the output generated by behaviors in the next lower layer. Maintain-Plume and Reacquire-Plume are abstracted by the methods of location of pheromone-emitting females by flying male moths [17,18]. Find-Plume is abstracted from fluid mechanics forces [19,20]. Construct-Plume-Map and Avoid-Obstacle are discussed in [16] and [21], respectively. Different plume tracing tests in a laboratory, for CPT missions in near-shore ocean environments the go-home module controls the vehicle to go back the home location where the vehicle is launched for pick-up, after the chemical source is identified.

The objective of the source identification is to declare the chemical source location during a CPT mission. For biological entities (e.g., moths), the conclusion of identifying the pheromone source may still be a mystery. Instead, while moth plume tracing relies primarily on a sensed pheromone, the final determination on the location of the female moth could be based on data from multiple sensors, including vision, tactile, or even auditory cues [22]. This article discusses how to implement the source declaration and source verification modules embedded in the subsumption architecture. The source declaration module uses plume events detected by a chemical sensor, in combination with measured AUV locations and fluid flow directions, to identify the plume source location, while the source verification module uses fuzzy color segmentation algorithms to recognize the chemical plume and its source in an image taken when the source is declared.

3. Source Declaration Module

This section briefly presents the source identification algorithms derived from the two moth-inspired behaviors: Maintain-Plume and Reacquire-Plume. Maintain-Plume is broken down into Track-In and Track-Out activities due to intermittency of a chemical plume transported in a fluid flow environment [12]. An AUV alternatively utilizes Maintain-Plume and Reacquire-Plume in making progress towards the source location in the up-flow direction. In a typical scenario of plume tracing, the AUV activates Track-In once it detects the chemical, e.g., the activities during ΔT₁ and ΔT₃ in Figure 4(a). It continues Track-Out when it loses contact with the chemical within λ seconds, e.g., the activity during ΔT₂ in Figure 4(a). After λ seconds, it switches to Reacquire-Plume for casting the plume, e.g., the activity after ΔT₄ in Figure 4(a).

A chemical detection point at which the AUV loses contact with the chemical plume for λ seconds is defined as a LCDP, e.g., point (x_last, y_last) at T_last in Figure 4(a). In our applications, the coordinates of (x_last, y_last) are specified in a coordinate system with the origin defined by the center of an operation area. We use a Cloverleaf trajectory with the center (x_last, y_last) to implement the Reacquire-Plume behavior, as shown in Figure 4(b). Note that one leaf is aligned with the down-flow direction for the AUV to rediscover the chemical when it has passed the source location. During a Reacquire-Plume activity, the AUV either detects the chemical or completes the Cloverleaf trajectory N_re times (N_re = 2 or 3 for the in-waters). The length of each leaf of the Cloverleaf trajectory is constrained to be larger than the AUV turning radius about 15 meters. If N_re repetitions are completed without a chemical detection, the AUV reverts to Find-Plume. Here, we define a LCDP node by:

struct LCDP_Node
{
double	Tlast, xlast, ylast;
double	conc, fdir, fmag;
double	xflow, yflow
};

where T_last is the time when the LCDP is detected, (x_last, y_last) are the coordinates of the AUV at T_last, conc is the chemical concentration at (x_last, y_last) and T_last, (f_dir,f_mag) are the flow direction and magnitude at (x_last, y_last) and T_last, and (x_flow, y_flow) are the coordinates in a new coordinate system defined according to the current flow direction. For convenience, we also use (x_last, y_last) to represent the LCDP in the following discussion.

In the moth-inspired CPT strategies, the chemical sensor works as a “binary detector”. The Boolean value is “1” if the chemical concentration is above the threshold. Otherwise, the Boolean value is set to “0”. The threshold value was chosen as conc > 4% of the full scale (i.e., 0.2 V) based on an analysis of chemical sensor data from the REMUS operating in San Diego Bay in the absence of the chemical. In this scenario, the sensor readings were pure noise, but never surpassed 0.2 V. The proposed algorithms make the source identification decision based on the number of LCDPs in SIZ instead of their concentrations. LCDPs provide important information about plume traversal distances between Reacquire-Plume activities. The LCDPs are separated along the axis of the plume when the AUV is far from the source location, while the LCDPs are clustered in the vicinity of the source when the AUV is approaching the source location. The AUV usually exits the plume and moves up flow from the source when it traces the plume to the source location. When this situation occurs, the AUV also activates Reacquire-Plume to rediscover the plume on a Cloverleaf trajectory. As a result of the frequent switching between Maintain-Plume and Reacquire-Plume, the AUV generates a pattern with a number of Cloverleaf trajectories in the vicinity of the source location, as shown in Figure 4(b). Such a distribution of the LCDPs is employed to facilitate development of the source identification algorithm.

The AUV detects a new LCDP and inserts its node into the priority queue when it switches its behaviors from Maintain-Plume to Reacquire-Plume. The queue sorts the LCDP nodes in a new coordinate system, defined in order of the current up-flow direction, f_dir + 180 °. Its x axis is aligned with the f_dir direction, and its origin is located at (x_last, y_last). The algorithm maps each LCDP in the queue into the new coordinate system by:

$$\left[\begin{array}{c}{x}_{\mathit{\text{flow}}}\\ y_{\mathit{\text{flow}}}\end{array}\right]=\left[\begin{array}{cc}cos(f_{\mathit{\text{dir}}}+180°)& -sin(f_{\mathit{\text{dir}}}+180°)\\ sin(f_{\mathit{\text{dir}}}+180°)& +cos(f_{\mathit{\text{dir}}}+180°)\end{array}\right]\left[\begin{array}{c}{x}_{\mathit{\text{last}}}\\ y_{\mathit{\text{last}}}\end{array}\right]$$

(1)

The x_flow components determine the LCDP nodes’ priorities according to the current up-flow direction. The smallest x_flow has the highest priority. The more LCDPs the priority queue accumulates, the more information about the source location the AUV gathers. We use LCDPs to develop the source identification algorithms SIZ_T and SIZ_F.

Algorithm 1 lists the pseudo code of the SIZ_T algorithm which keeps updating the N_dec most recent LCDPs during CPT missions, and sorts them in the order of time serials using the priority queue. The algorithm constructs the SIZ_T size by:

$$\begin{array}{cc}\begin{array}{cc}{x}_{\mathit{\text{last}}}^{(min)}& =min\left\{x_{\mathit{\text{last}}}^{(t(\text{i})}\right\}\\ x_{\mathit{\text{last}}}^{(max)}& =max\left\{x_{\mathit{\text{last}}}^{t(\text{i})}\right\}\\ y_{\mathit{\text{last}}}^{(min)}& =min\left\{y_{\mathit{\text{last}}}^{t(\text{i})}\right\}\\ y_{\mathit{\text{last}}}^{(max)}& =max\left\{y_{\mathit{\text{last}}}^{t(\text{i})}\right\}\end{array}& \text{I}=1,\cdots N_{\mathit{\text{dec}}}\end{array}$$

(2)

where a superscript t indicates that the queue sorts the N_dec LCDPs in time series. When the AUV approaches the odor source, the distances between the LCDPs become smaller, i.e., the SIZ_T size becomes smaller. During CPT missions, the SIZ_T algorithm dynamically checks the diagonal:

$$R=\sqrt{{\left(x_{\mathit{\text{last}}}^{\left(\mathit{\text{max}}\right)}-x_{\mathit{\text{last}}}^{\left(\mathit{\text{min}}\right)}\right)}^2+{\left(y_{\mathit{\text{last}}}^{\left(\mathit{\text{max}}\right)}-y_{\mathit{\text{last}}}^{(\min \mathit{)}}\right)}^2}$$

(3)

when R≤ ε_T, SIZ_T identifies the mean:

$$\begin{array}{cc}{x}_{\mathit{\text{last}}}^{(\text{M})}& ={\sum }_{\text{I}=1}^{N_{\mathit{\text{dec}}}}{x}_{\mathit{\text{last}}}^{t(\text{i})}/N_{\mathit{\text{dec}}}\\ y_{\mathit{\text{last}}}^{(\text{M})}& ={\sum }_{\text{I}=1}^{N_{\mathit{\text{dec}}}}{y}_{\mathit{\text{last}}}^{t(\text{i})}/N_{\mathit{\text{dec}}}\end{array}$$

(4)

as the source location. The SIZ_T algorithm needs three parameters: the criterion, ε_T, for checking the SIZ_T size, the integer, N_dec, indicating the constant number of LCDPs maintained in the queue, and the initial value, N_ini, identifying the priority queue for accumulation of at least N_ini LCPDs before starting source identification. The parameters ε_T and N_dec are adjustable and crucial to achieving the desired performance of source identification.

Algorithm 1. Pseudo Code of SIZ_T algorithm.

ALGORITHM SIZ_T(Q[ 1,… Ndec ] )

//Identifying the source location by SIZ_T algorithm

//Input: Priority queue Q[ 1,… Ndec ]

//Output: Status of source identification

if ( Ndec ≥ Nini )

Sort Q in the order of the time serials

for i ← 1 to Ndec do

P [i] ← Q[i ] // P is a list

Calculate $(x_{\mathit{\text{last}}}^{(\text{max})},y_{\mathit{\text{last}}}^{(\text{max})})$ and in Equation (2)

Calculate the diameter of SIZ_T in Equation (3)

if the diameter ≤ εT

return$(x_{\mathit{\text{last}}}^{(\text{M})},y_{\mathit{\text{last}}}^{(\text{M})})$ as the source location

else

return no source location identified

else

return no source location identified

Algorithm 2 lists the pseudo code of the SIZ_F algorithm which maintains all LCDPs in the order of the current up-flow direction using the priority queue. SIZ_F holds a constant size, ε_F, and makes the source identification by the following iterative construct: First, SIZ_F calculates $(x_{\mathit{\text{last}}}^{(\text{M})},y_{\mathit{\text{last}}}^{(\text{M})})$ of all the LCDPs; Second, SIZ_F find the point, p_max, with the largest distance to $(x_{\mathit{\text{last}}}^{(\text{M})},y_{\mathit{\text{last}}}^{(\text{M})})$:

$$D_{\mathit{\text{max}}}=\mathit{\text{max}}\left\{\sqrt{{\left(x_{\mathit{\text{last}}}^{f(\text{i})}-x_{\mathit{\text{last}}}^{(\text{M})}\right)}^2+{\left(y_{\mathit{\text{last}}}^{f(\text{i})}-x_{\mathit{\text{last}}}^{(\text{M})}\right)}^2}\right\}(\text{I}=1,2,\dots N_{\mathit{\text{all}}})$$

(5)

from the priority queue, where a superscript f indicates that the LCDPs are sorted in the order of the most recent up-flow direction, and N_all is the total number of LCDPs detected during a CPT mission. If D_max is greater than, ε_F, SIZ_F removes the LCDP with p_max from the set of LCDPs. These calculations repeat until all remaining LCDPs are close enough to $(x_{\mathit{\text{last}}}^{(\text{M})},y_{\mathit{\text{last}}}^{(\text{M})})$, i.e., they all are located inside SIZ_F. If the number of the remaining LCDPs is greater than N_min, SIZ_F identifies its most up-flow LCDP as the odor source. The SIZ_F algorithm has three parameters: the SIZ_F size, ε_F, the initial value, N_ini, and the integer, N_min, which indicates the minimum number of LCDPs remaining inside SIZ_F for the source identification. The SIZ_F algorithm also has two the adjustable parameters ε_F and N_min. The parameter, N_ini, defined in the algorithms works as a filter to block some invalid LCPDs only when N_dec or N_min is very small.

Algorithm 2. Pseudo Code of SIZ_F algorithm.

ALGORITHM SIZ_F ( Q[1,… Nall] )

//Identifying the source location by SIZ_F algorithm

//Input: Priority queue Q[1,… Nall]

//Output: Status of source identification

if ( Nall ≥ Nini )

Sort Q in the order of the current up-flow direction

L[1,… Nall] ← Q[1,… Nall]; n1 ← Nall // L is a list

status ← false

while n1 ≥ Nmin do

Calculate $(x_{\mathit{\text{last}}}^{(\text{M})},y_{\mathit{\text{last}}}^{(\text{M})})$ of all LCDPs in the priority queue;

Find pmax with Dmax in Equation (5)

if Dmax > εF

remove pmax from L; n1←n1-1

else

status ← true; break

if status = true

return $(x_{\mathit{\text{last}}}^{f(1)},y_{\mathit{\text{last}}}^{f(1)})$ as the source location

else

return no source location identified

else

return no source location identified

We evaluate the SIZ algorithms using the simulated plume [23,24]. The simulated plume model achieves significant computational simplification relative to turbulence models, but it was designed to maintain the plume characteristics that significantly complicate the plume tracing problems (intermittency, meander, and varying flow) caused by natural flow fluid. Instead of adjusting the Reynolds numbers, it controls a filament release rate (5–10 filaments/s) to simulate filament intermittency and addresses the meandering nature of the plume as a key factor complicating the plume tracing. It also manipulates flow variation to challenge the CPT strategies. An operation area is specified by [0, 100] × [−50, 50] in meters. The simulation time step is 0.01 s and the mean fluid velocity is 1 m/s. A source location is chosen as (20, 0) in meters for checking the accuracy of the identified source locations, but it is unknown to the vehicle during CPT test runs. The simulation environment is set below: first, the filament release rate is 5 filaments/s because a low release rate may result in significant plume intermittency, which often causes the vehicle to lose contact with the plume.

The SIZ_T condition is stronger than the SIZ_F condition as SIZ_T requires the AUV to generate a few of the most recent LCDPs close enough in the vicinity of the odor source. However, it is not always true because the AUV may exit both sides of plumes in the vicinity of the source location due to the width of chemical plumes, the variation of flow directions, the intermittency of filaments, or the vehicle's mechanical restrains. For source declaration, SIZ_F needs a few of the most up-flow LCDPs close enough so it is suggested to use SIZ_F to identify a static plume source since the odor source located in the up-flow direction always makes the AUV progress up-flow.

4. In-Water Tests for Declaring Rhodamine Dye Plume Sources

A type of SIZ_T algorithms is implemented on the REMUS vehicle for the CPT missions. The initial values of SIZ_T were chosen ε_T =3 and N_dec = 3 prior to the 2002 CPT missions. Eight of the first ten runs aborted due to vehicle issues, including start-up problems, incorrect time settings, equipment failure, etc. The other two test runs performed plume tracing well, but source declaration did not occur as both ε_T and N_dec were too small. We empirically changed to ε_T =4 and N_dec = 6 (our Monte Carlo evaluations in [25] show this set of parameters has a success rate of 96.8%). These settings successfully declared the source locations of rhodamine dye plumes, as shown in Figure 5, for the last seven test runs at the San Clemente Island (California, USA) in November 2002 labeled as MSN007r2–MSN010r3. Figure 1(a) shows the mission MSN007r2 that documented over a distance of 411 meters from the first detection point to the identified source location. In the in-water tests in April 2003 at San Clemente Island, the same settings successfully identified the source location on seven of eight experiments. The experiments included ground truth confirmation of identified source locations via sidescan sonar with 8–17 m accuracy. These settings were also successfully used for the in-water tests conducted in June 2003 in Duck (North Carolina, USA) in an operation area of 367 × 1,094 m (bigger than 60 football fields). Two types of missions were of interest during this set of experiments. The first mission type involved a single chemical source in the operation area. The second mission type may contain a few chemical sources in the operation area. The two types of CPT mission successfully identified the source locations with ground truth confirmation via sidescan sonar with 6–31 m accuracy [13]. Figure 1(b) shows the MSN003 mission at Duck in June 2003. This mission tracked the chemical plume for 976 m between the first detection point and the declared source location, with 13 m accuracy.

The most recent in-water tests for identifying rhodamine dye plume sources were conducted in October 2010 at Dalian Bay (China). For this set of field tests, we implemented the SIZ_F algorithm on the AUV developed by SIA. The parameters ε_F =6 and N_min= 6 in SIZ_F are optimized in [12] prior to the 2012 CPT missions. The AUV shown in Figure 6 is equipped with multiple sensors, including a depth sensor, an underwater fluorometer, and a Doppler Velocity Log (DVL), etc. Figure 7 shows the fluorometer Cyclops-7 produced by Turner Designs, Inc. (Sunnyvale, CA, USA). Its technical specifications can be found at http://www.turnerdesigns.com/products/submersible/cyclops-7. The AUV uses the fluorometer to detect rhodamine dye plume concentration. For the in-water tests, we set the sampling rate of the fluorometer as 10 Hz and choose the 0–10 ug/L measurement scale in which the fluorometer outputs 5 VDC corresponding to a rhodamine dye concentration of 10 μg/L. Figure 8 shows the workhorse navigator 1200K DVL produced by Teledyne RD Instruments (Poway, CA, USA), and its technical specifications can be found at http://www.rdinstruments.com/pdfs/wh_navigator_ds_lr.pdf. The DVL measures the AUV speed relative to the sea bottom based on which we use the dead reckoning method to estimate AUV positions as well as fluid speed and orientation.

For the in-water tests conducted in October 2010 at Dalian Bay (China), we developed an alternative strategy to generate rhodamine dye plumes on the sea surface, instead of on the seabed. The rhodamine dye is pumped up to the sea surface through a wound drainage plastic pipe. The pump was placed on a small boat that is anchored to the sea bottom. The plume source is 1.0 meters × 0.5 meters and the chemical source is submerged 1.5 meters below the sea surface. The rate of release of the rhodamine dye is about 1–2 g/min. Figure 9 shows a snapshot of the chemical plume taken at a location about 350 meters from its source. The scenario in Figure 9 provides the following information: first, a significant plume meander appears when the plume is propagated over a long distance. Changes in fluid flow direction cause the plume to meander (forming a snakelike path). Because the fluid flow direction and magnitude change, spatially and temporally, the instantaneous fluid flow direction within the plume often will not point toward the plume's source nor be coincident with the plume's centerline. To our knowledge, most of existing plume tracing algorithms do not explicitly discuss how to deal with the plume meander issue. Second, fluid flow waves affect the rhodamine dye plume so that the chemical concentration distribution is not uniform. The rhodamine dye plume in Figure 8 shows the chemical concentration at some locations where the plume looks like it is broken and the color is very low, but at other locations with a large dye area the level of deep red color is very high. The problem with local concentration maxima significantly challenges source declaration algorithms which are investigated in laboratory environments by searching for the maximum proximity of a plume with a uniform concentration distribution. For identifying the source locations, the AUV started tracing the rhodamine dye plumes developed on the sea surface. In this case, we visually observe the AUV maneuvers and confirm the identified chemical source locations via a GPS. Two among the six test runs aborted due to equipment failure instead of the SIZ_F settings. The mission, labeled as MSN171639, successfully tracked the chemical plume to its source location and identified the source locations with accuracy 8.38 m relative to the source locations confirmed by the GPS, as shown in Figure 2. The declared source location accuracies of the missions, labeled as MS10171617, MS10191502, and MS10191536, are 15.77 m, 12.33 m, and 28.42 m, respectively. The work in [12] developed a time metric to estimate the identification time cost (N_min−1)*κ which is free from plume mission initial positions. κ is about 94.5 s, determined by the vehicle velocity and turn radius. The identification time costs’ average is close to the estimated one. The source declaration algorithms allow little room for the improvement of time cost if only a single chemical sensor is used.

Neutral buoyancy of the chemical or stratification of the flow will often result in the rhodamine dye plume of limited vertical extent, which may be approximated as a two dimensional (2-D) problem. In the field tests, no test runs failed due to the 3D problem. For searching for hydrothermal vents in oceans, the 3D issue has to be considered.

5. Source Verification Module

For further operations to deal with the declared plume source, the source needs to be automatically or interactively confirmed or inspected using a visual system. Images taken in near shore ocean environments are very vague and colors in the images are not well-defined since the propagation of light underwater is affected by scattering, refraction, and absorption. However, most of the existing techniques [26–28] provide crisp segmentation of images, where each pixel is classified into a unique subset. This classification may not be effective for extracting a rhodamine dye plume and its source in underwater conditions because colors in the images taken in near shore ocean environments are not well-defined.

In order to deal with uncertainty, we use a fuzzy logic based iterative algorithm to segment color components of the rhodamine dye plume and its source [29]. In this study, colors of an image are described in the RGB space. The color of each pixel p(m, n) denoted by p(m, n)_RGB is processed to separate its red, green, and blue components {p(m, n)_R, p(m, n)_G, p(m, n)_B}. The fuzzy color extractor extracts a cluster of colors based on a defined color pattern (CP or CP_RGB). The CP_RGB is either directly defined by its RGB components (CP_R, CP_G, CP_B) or determined by a pixel in the image. The color component differences between p(m, n)_RGB and CP_RGB are calculated as follows:

$$\{\begin{array}{l}\mathit{\text{dif}}{(\text{M},n)}_{\text{R}}=p{(\text{M},n)}_{\text{R}}-{\text{CP}}_{\text{R}}\hfill \\ \mathit{\text{dif}}{(\text{M},n)}_G=p{(\text{M},n)}_G-{\text{CP}}_G,0\le \text{M}<\text{M},0\le n<N,\hfill \\ \mathit{\text{dif}}{(\text{M},n)}_B=p{(\text{M},n)}_B-{\text{CP}}_B\hfill \end{array}$$

(6)

where M and N indicate the size of an array which holds the image. The following fuzzy rules are applied to dif (m, n)_R, dif (m, n)_G, and dif (m, n)_B:

If	dif (m, n)R and dif (m, n)G and dif (m, n)B are Zero
Then	p(m, n) is Matched
If	dif (m, n)R or dif (m, n)G or dif(m, n)B is Negative or Positive
Then	p(m, n) is Unmatched

Both rules indicate that the pixel, p(m, n), belongs to the object to be extracted, if the Euclidean distances between p(m, n)_RGB and CP_RGB along the three axes in RGB coordinate system are small enough; otherwise, p(m, n) does not belong to the object. Figure 10(a) shows the membership functions (μ_N(x), μ_Z(x), μ_P(x)) for the input fuzzy variables (Negative, Zero, Positive) defined by:

$${\mu }_{\text{N}}(x)=\{\begin{array}{l}1-255\le x<-{\alpha }_2\hfill \\ \frac{(x+{\alpha }_1)}{({\alpha }_1-{\alpha }_2)}-{\alpha }_2\le x<-{\alpha }_1\hfill \\ 0-{\alpha }_1\le x\le -255\hfill \end{array}$$

(7)

$${\mu }_{\text{Z}}(x)=\{\begin{array}{l}0-255\le x<-{\alpha }_2\hfill \\ \frac{(x+{\alpha }_2)}{({\alpha }_2-{\alpha }_1)}-{\alpha }_2\le x<-{\alpha }_1\hfill \\ 1-{\alpha }_1\le x<-{\alpha }_1\hfill \\ \frac{(x+{\alpha }_1)}{({\alpha }_2-{\alpha }_1)}{\alpha }_1\le x<-{\alpha }_2\hfill \end{array}$$

(8)

$${\mu }_{\text{p}}(x)=\{\begin{array}{l}0-255\le x<{\alpha }_1\hfill \\ \frac{(x+{\alpha }_1)}{({\alpha }_2-{\alpha }_1)}{\alpha }_1\le x<{\alpha }_2\hfill \\ 1{\alpha }_2\le x\le 255\hfill \end{array}$$

(9)

Figure 10(b) shows the membership functions (μ_M(x), μ_U(x)) for the output fuzzy variables (Matched, Unmatched) defined by:

$${\mu }_{\text{M}}(x)=\{\begin{array}{l}\frac{({\mathrm{\rho }}_{\text{M}}-x)}{{\mathrm{\rho }}_{\text{M}}}0\le x<{\mathrm{\rho }}_{\text{M}}\hfill \\ 0{\mathrm{\rho }}_{\text{M}}\le x\le 255\hfill \end{array}$$

(10)

$${\mu }_{\text{U}}(x)=\{\begin{array}{l}00\le x<{\mathrm{\rho }}_{\text{U}}\hfill \\ \frac{(x-{\mathrm{\rho }}_{\text{U}})}{(255-{\mathrm{\rho }}_{\text{U}})}{\mathrm{\rho }}_{\text{U}}\le x\le 255\hfill \end{array}$$

(11)

where ρ_M + ρ_U = 255. Based on dif (m, n)_R, dif (m, n)_G, and dif (m, n)_B, the fuzzy rules produce the weight w_m for Matched and the weight w_u for Unmatched by:

$$\begin{array}{c}{w}_{\text{M}}=\mathit{\text{min}}\{w_{\text{M}(\text{R})},w_{\text{M}(\text{G})},w_{\text{M}(\text{B})}\}\\ w_{\text{U}}=\mathit{\text{max}}\{w_{\text{U}(\text{R})},w_{\text{U}(\text{G})},w_{\text{U}(\text{B})}\}\end{array}$$

(12)

Figure 10(b) also shows the produced areas in the output domain while w_m and w_u cutting μ_M(x) and μ_U(x). An output value, Δρ_F, is calculated by the centroid defuzzification method:

$$\Delta {\mathrm{\rho }}_F=\frac{\int {\mu }_{\text{out}}(x)xdx}{\int {\mu }_{\text{out}}(x)dx}$$

(13)

where μ_out (x) represents the envelope function of the areas cut by w_m and w_u in fuzzy output domain. If Δρ_F < δ, p(m, n) is extracted; otherwise, p(m, n) is not extracted, where δ is a threshold. The fuzzy color extractor can be understood as a mapping operator between Euclidean distances {dif(m, n)_R, dif(m, n)_G, dif(m, n)_B} in the RGB space and a difference Δρ_F in the intensity space under a fuzzy metric. We use the procedure FuzzyColorSeg to the extract specified color from a color image.

FuzzyColorSeg( IS )

CPRGB ← GetCP( IS );

[IU, IM] ← FCE( IS, CPRGB );

return [IU, IM];

FuzzyColorSeg invokes two procedures: GetCP and FCE. GetCP generates a CP_RGB for FCE which splits a source image I_S into two sub-images: one sub-image, I_M, holds the matched colors; and the other sub-image, I_U, remains the unmatched colors.

The difficulty of segmenting the chemical plume and its source from images taken in ocean environments lies in the definition of their color patterns, since colors in the vague images are not well defined due to illumination variance and fluid advection affects. Now, we discuss an iterative procedure to extract the rhodamine dye plume and its source from such a color image taken in near-shore oceanic environments. For convenience, we define two groups of the sub-images ${\text{I}}_{\text{M}}^{(\text{i})}$ and ${\text{I}}_{\text{U}}^{(\text{i})}$ split by FCE. ${\text{I}}_{\text{M}}^{(\text{i})}$ contains matched colors after the ith step extraction, while ${\text{I}}_{\text{U}}^{(\text{i})}$ contains unmatched colors. The original image ${\text{I}}_{\text{U}}^{(0)}$ is the initial source image to the procedure, and ${\text{I}}_{\text{U}}^{(\text{i})}$ is the source image for the (i + 1)th step segmentation. We have the relationship ${\text{I}}_{\text{U}}^{(\text{i})}={\text{I}}_{\text{U}}^{(\text{i}+1)}\cup {\text{I}}_{\text{M}}^{(\text{i}+1)}$. In our application, a default color is defined as white with (255, 255, 255) in the RGB space. FCE generates ${\text{I}}_{\text{M}}^{(\text{i})}$ by moving the matched pixels from the source image to an image initialized by white color, and generates ${\text{I}}_{\text{U}}^{(\text{i})}$ by setting the extracted pixels in the source image to white color. Obviously, selecting color patterns becomes the key issue of extracting desired objects from an image. We employ some well-defined colors as reference colors to generate CPs. To our knowledge, the color components of the rhodamine dye plume are close to the red color, and those of the chemical source are dark and close to the blue color. Therefore, we define the two reference colors: RED_RGB with (255, 0, 0) and DARK-BLUE_RGB with (BLACK_RGB+BLUE_RGB)/2 = (0, 0, 128) to generate color patterns by using GetCP( I_s ):

RefColor ← REDRGB or DARK-BLUERGB;

‖CPRGB‖ ← ∞;

for m ← 0 to M-1 do

for n ← 0 to N-1 do

Dis ← ‖RefColor - p(m, n)RGB‖

if Dis < ‖CPRGB‖

CPRGB ← p(m, n)RGB;

return CPRGB;

where I_s is an original image to be processed, M and N determine the size of I_s. In order to extract color components of the chemical plume, RED_RGB is assigned to RefColor, and the procedure returns a CP_RGB that holds the color components of the pixel in the image with the shortest Euclidean distance to the reference color – RED_RGB. ${\text{I}}_{\text{U}}^{(0)}$ is the original image–“odor”, as shown in Figure 11(a), and ${\text{I}}_{\text{M}}^{(1)}$ is initialized as an empty image with white. FuzzyColorSeg takes ${\text{I}}_{\text{U}}^{(0)}$ as an input and passes ${\text{I}}_{\text{U}}^{(0)}$ to GetCP. GetCP defines RED_RGB as RefColor and returns a CP_RGB (106, 106, 231) for ${\text{I}}_{\text{U}}^{(0)}$. FCE uses the CP_RGB to split ${\text{I}}_{\text{U}}^{(0)}$ –“odor” into two sub-images ${\text{I}}_{\text{U}}^{(1)}$ and ${\text{I}}_{\text{M}}^{(1)}$. ${\text{I}}_{\text{M}}^{(1)}$ is produced by moving the extracted pixels from ${\text{I}}_{\text{U}}^{(0)}$ to the empty image and holds color components closely matching to the CP_RGB. ${\text{I}}_{\text{U}}^{(1)}={\text{I}}_{\text{U}}^{(0)}-{\text{I}}_{\text{M}}^{(1)}$ is produced by setting the extracted pixels to white in ${\text{I}}_{\text{U}}^{(0)}$. ${\text{I}}_{\text{M}}^{(1)}$ is the first segmented image for the chemical plume, as shown in Figure 11(b). The chemical plume in the image is not completely extracted since the rhodamine dye colors are distorted due to environmental illumination variation and fluid advection affects. The further segmentation needs ${\text{I}}_{\text{U}}^{(1)}$ as a new source image for the next step segmentation. FuzzyColorSeg takes ${\text{I}}_{\text{U}}^{(1)}$ as its input, and GetCP gets a new CP_RGB with (156, 136, 243) for ${\text{I}}_{\text{U}}^{(1)}$. FCE uses the updated CP_RGB to split the image ${\text{I}}_{\text{U}}^{(1)}$ into two sub-images ${\text{I}}_{\text{U}}^{(2)}$ and ${\text{I}}_{\text{M}}^{(2)}$. Similarly, ${\text{I}}_{\text{M}}^{(2)}$ is produced by moving the extracted pixels from ${\text{I}}_{\text{U}}^{(1)}$ to an empty image and holds the color components closely matching to the updated CP_RGB. ${\text{I}}_{\text{M}}^{(2)}$ represents the second segmented image for the chemical plume, as shown in Figure 11(c). ${\text{I}}_{\text{U}}^{(2)}$ is generated by setting the extracted pixels from ${\text{I}}_{\text{U}}^{(1)}$ to white. At the ith step, FuzzyColorSeg split ${\text{I}}_{\text{U}}^{(\text{i})}$ into ${\text{I}}_{\text{U}}^{(\text{i}+1)}$ and ${\text{I}}_{\text{M}}^{(\text{i}+1)}$. Figure 11(d) displays the segmented image ${\text{I}}_{\text{M}}^{(3)}$ at the third step. The union operation ${\text{I}}_{\text{plume}}={\text{I}}_{\text{M}}^{(1)}\cup {\text{I}}_{\text{M}}^{(2)}\cup {\text{I}}_{\text{M}}^{(3)}$ generates the segmented chemical plume, as shown in Figure 11(e).

Similarly, we use FuzzyColorSeg to extract the color components of the chemical source from the image. In this case, DARK-BLUE_RGB is assigned to RefColor to generate a CP_RGB. FCE uses this CP_RGB split ${\text{I}}_{\text{U}}^{(3)}$ into two sub-images ${\text{I}}_{\text{U}}^{(4)}$ and ${\text{I}}_{\text{M}}^{(4)}$, as shown in Figure 11(f). This procedure continues four steps to extract the color components of the chemical source, as shown in Figure 11(f–j). The union operation ${\text{I}}_{\text{source}}={\text{I}}_{\text{M}}^{(4)}\cup {\text{I}}_{\text{M}}^{(5)}\cup {\text{I}}_{\text{M}}^{(6)}\cup {\text{I}}_{\text{M}}^{(7)}$ generates the extracted color components of the chemical source shown in Figure 11(k).

The extracted colors in ${\text{I}}_{\text{M}}^{(\text{i})}$ can be understood as a fuzzy color cluster under the given membership function. For segmenting colors in the images in Figure 11, we choose the parameters of the membership functions as follows: α₁ = 35, α₂ = 200, ρ_M = 20, ρ_U = 235, and ɛ = 80. In this study, we consider two criteria for clustering the colors extracted by FCE [30]: The first is based on the distances of the CPs to their associated reference color, and the second based on the relative distances between the CPs. For the first criterion, we take the CP_RGB with the shortest distance to the given reference color as a pivot. Then, we calculate the differences between distances of the pivot and any other CPs in this group: Δ₁ = ‖CP_RGB‖−‖Pivot_RGB‖. If Δ₁ > δ₁, its corresponding CP_RGB is excluded from this group. For the second criterion, we construct a graph based on the distances between the CPs in terms of the given reference color. Then, we start with the pivot in the graph to calculate the shortest distances using Floyd's algorithms. The CP_RGB is removed from this graph, if its shortest distance is greater than δ₂. Both the criteria ensure that the CPs in each group are close to their pivot enough. In this study, we select δ₁ and δ₂ as 40 and 45.

6. Conclusions

This article discusses a strategy for identifying a chemical source in a near-shore and ocean environment by integration of chemical and visual sensors. The two source identification algorithms SIZ_T and SIZ_F are abstracted from the moth-inspired plume tracing strategies presented in [2]. While animal plume tracing relies primarily on sensed pheromones, the final determination of the plume source location could be based on data from multiple sensors, including vision, tactile or even auditory cues [22]. Typically, olfactory-based mechanisms proposed for biological entities combine a large-scale orientation behavior based in part on olfaction with a multisensor local search in the vicinity of the source.

Extracting the chemical plume and its source from an image taken in underwater conditions is difficult since the color images taken in near-shore oceanic environments are very vague and the objects’ colors in the image are significantly distorted from their natural colors due to dim illumination conditions and flow fluid influence. In order to deal with uncertainty, segmenting color components of the chemical plume and its source is based on the fuzzy color extractor. The fuzzy color extractor is directly extended from the fuzzy gray-level extractor, which was applied to recognize while line landmarks on roads for autonomous vehicle navigation [31,32]. The verification module was used to test five images captured from the video made during the in-water of 2002 and accomplished the plume source extraction. Evaluations of the module in different environment conditions are planned to test its robustness and effectiveness further. The autonomous underwater vehicle for our ongoing project on searching for hydrothermal vents in oceans will carry an underwater camera and a lightning system to get images during operation. An alternative sensor, e.g., sidescan sonar, can be used to confirm the declared source location if an image of the source is unavailable during CPT missions.

For the in-water tests, the CPT algorithm has to be able cope with significant plume meander. The simulation studies show that the moth-inspired CPT strategy is effective for tracing the plume with significant meander so we selected it for implementation. By considering time-consumption and cost of the field tests, no other algorithms are selected for the in-water tests.

The source identification performance can be improved if the visual sensor based source identification model checks each LDCP. This visual sensor based system has a number of potential applications in our on-going project, e.g., it will guide a manipulator to get samples at thermal vents. We will investigate the fuzzy color segmentation algorithm by considering location and texture information. In our further research, we will also improve the source declaration performance by adaptively calculating the parameters of SIZ algorithms and consider the algorithms for tracking of the plume in 3D environment.