A Method for Supervisory Control of Manipulator of Underwater Vehicle

Abstract

A novel method for supervisory control of multilink manipulators mounted on underwater vehicles is considered. This method is designed to significantly increase the level of automation of manipulative operations, by the building of motion trajectories for a manipulator working tool along the surfaces of work objects on the basis of target indications given by the operator. This is achieved as follows: The operator targets the camera (with changeable spatial orientation of optical axis) mounted on the vehicle at the work object, and uses it to set one or more working point on the selected object. The geometric shape of the object in the work area is determined using clouds of points obtained from the technical vision system. Depending on the manipulative task set, the spatial motion trajectories and the orientation of the manipulator working tool are automatically set using the spatial coordinates of these points lying on the work object surfaces. The designed method was implemented in the C++ programming language. A graphical interface has also been created that provides rapid testing of the accuracy of overlaying the planned trajectories on the mathematically described surface of a work object. Supervisory control of an underwater manipulator was successfully simulated in the V-REP environment.

1. Introduction

Underwater unmanned vehicles (UUV) equipped with multilink manipulators (MM) are widely used in various research and technological operations in the world’s oceans. They facilitate the cleaning of underwater structures [1], diagnostics of underwater pipelines and cables [2,3], operations with subsea valve systems [3,4,5], collection of scientific material [6,7,8], archaeological surveys [9], cleaning of ship hulls from fouling [10], etc.

Modern trends in the design and operation of UUV are aimed at increasing their functional capabilities on the basis of novel technologies and methods for recognizing surrounding objects and for automatically building trajectories and modes of motion for both the UUV, and MM mounted on them [11,12,13,14]. Currently, the vast majority of underwater technological and research manipulative operations are performed only in a manual mode by specially trained UUV operators. Experienced operators successfully set trajectories for MM working tools [15,16,17]; nevertheless, it is very difficult for them to quickly and accurately solve a multitude of complex problems while having no direct contact with a work object and using only video images. This reduces the efficiency of their operation and increases the probability of errors.

This problem was also faced during deep-sea research expeditions in the Sea of Japan, Bering Sea, and Sea of Okhotsk, with the deployment of a Sub-Atlantic Comanche 18 remotely operated vehicle (ROV) equipped with a Schilling Orion 7P manipulator [18,19,20]. The routine collection of sediment (Figure 1), geological rock, and bacterial mat samples very often led to operator fatigue within a short period of time, to a significant increase in operation time, to errors in operations, and even to damage to working tools and samplers.

During manipulative operations, an UUV is exposed to undesirable dynamic impacts from the surrounding aquatic environment and the moving MM. These impacts shift the UUV position from the work object and complicate the control of the MM. To solve these problems, there are systems [21,22,23] that allow holding an UUV performing a manipulative operation in the desired spatial position with great accuracy. Moreover, if an MM has a greater number of degrees of mobility than is required for operations, the system [24] generates control signals for its actuators in such a way that this MM has the least impact on the UUV. These systems are indispensable for providing accurate operations in the UUV hovering mode. However, to facilitate manual control of MM, additional supervisory control systems are required.

To increase the efficiency of manipulative operations and reduce the load on operators, computer systems have been created [25] that allow operations to be performed on-line in a supervisory mode. In this control mode, the operator sets only simple target indications for the MM. Based on these indications, control signals for all the MM actuators are generated in a real-time mode. Some components of such systems have been tested on the ROV Jason [26]. To simplify the manual control of MMs, a strategy with the use of the P300 Brain–Computer Interface is proposed [27]. However, all the systems considered do not allow identification of the shape of the work objects or automatically building complex spatial motion trajectories of MM working tools over the surface of these objects.

With the development of technical vision systems (TVS) and environmental object recognition technologies, a new trend has arisen to increase the level of automation of many manipulative operations. The approach proposed in [28] is the use of a stereo camera mounted on the MM gripper, which allows determining the position and orientation of a known object with a pre-installed “Fiducial marker”. This work suggested automatic generation of MM control signals based on the obtained data that ensure firm gripping of objects with the correct orientation of the gripper. However, applying special markers to underwater objects is often impossible, which significantly limits the practical feasibility of this approach.

To facilitate spatial perception of work objects by the MM operator, the article [29] provides a method that allows composing a map of image depth with the use of two stereo cameras mounted on the UUV. This provides the operator with a three-dimensional (3D) view of the environment, which can be used to control the MM in an augmented reality mode. The approach considered in [30] is based on 3D scanning with two multibeam sonars, while the algorithm in [31] provides rapid reconstruction of the scanned surface, with filtering out noise in the data transferred from these echo-sounders. The above listed methods can be successfully applied to accurately determine the spatial positions and shapes of many underwater objects.

However, the existing systems and methods, while facilitating the operator’s perception of the surrounding underwater environment and simplifying the process of manual control of the MM, do not provide sufficiently automated building of the desired motion trajectories of the MM along the surface of underwater object and with the subsequent automatic performing of these trajectories with its working tools. Thus, the process of complex manipulative operations still requires MM operators to constantly control and perform necessary operative actions. This leads to operator fatigue within a short period of time, to a significant increase in the time needed for even standard operations, and often to serious errors and emergencies.

Due to the significant increase in the level of automation of numerous (especially standard) manipulative operations and work using UUV, the present study addresses the issue of developing a new method and algorithms to perform even complex underwater manipulative operations with a MM in a supervisory mode. The designed method should provide rapid building of motion trajectories for MM working tools on the basis of target indications given by the operator. These trajectories, built in an automatic mode, should be accurately passed by the MM working tool with the required orientation relative to the work objects. In this case, the operator, by targeting the optical axis of the video camera mounted on the rotary platform of the UUV, will set the start and end points of the trajectories of the working tool motion and the boundaries of the work object area on which these trajectories should then be projected, taking into account the shape and spatial position of this object. To facilitate the operator’s manipulations in the process of supervisory target indication, a graphical interface will be created, which, among other things, will allow rapid testing of the accuracy of overlaying the built trajectories on the mathematically described area of the work object’s surface.

2. Specifics of the Method for Supervisory Control of MM

Using this method (see Figure 2), the geometric shapes and positions of the objects in the work area are first determined using clouds of points obtained from the TVS. Then the operator targets the camera (with changeable spatial orientation of the optical axis) mounted on the UUV at the work object and uses it to set one or more working point on the selected object. These points are set by pressing a button on the control panel or the graphical interface. Depending on the manipulative task set, the spatial motion trajectories and orientation of the MM working tool are automatically set on the basis of the spatial coordinates of these points lying on the surfaces of the work object.

Figure 3 shows a diagram of real-time supervisory control of MM 1 mounted on UUV 2 in the process of the sampling of a surface sediment layer using multibeam sonar 3. This sonar forms a point cloud to build a mathematical model of the seabed surface 4, which is used for the subsequent forming of the trajectory of the sediment sampler motion. The operator, moving the optical axis 5 of the camera, sets s target points $A_i$ ($i=\overline{1,\hspace{0.17em}s}$), which are the points of intersection of this axis with the bottom surface at the most interesting places for sediment sampling. The motion trajectory 6 of the sediment sampler is built automatically by connecting the points $A_i$. Its vector $\overrightarrow{a}$ during movement along the trajectory (see Figure 3) should always be perpendicular to the bottom surface, and the vector $\overrightarrow{n}$ of the normal line to $\overrightarrow{a}$ should be directed to the next target point $A_{i+1}$.

Similarly, the proposed method of supervisory control can be used to perform more complex manipulative operations, according to operator’s target indications.

3. Building of Motion Trajectories for a MM Working Tool Performing Operations on a Flat Bottom Surface

As numerous deep-sea studies [18,32] have shown, sampling of sediment layers and geological rocks is frequently carried out on a relatively flat seabed surface. Such a surface can be represented by an averaged plane [32]. To do this, it is sufficient to use point clouds of points obtained with a Doppler velocity log (DVL). A DVL has at least three hydroacoustic antennas and allows measurement of the distance from their location on the UUV hull to the bottom, along all the axes of these antennas with a high accuracy.

Therefore, in the coordinate system (CS) $xyz$, rigidly fixed on the UUV hull, this plane can be described by the equation in the normal form [33]:

$$ax+by+cz=d,$$

(1)

where $x$, $y$, and $z$ are the axes of the CS, with origin point O located at the UUV center of buoyancy; the $x$ axis coincides with the horizontal, longitudinal axis of the UUV; the $z$ axis coincides with its vertical axis; the y axis forms the right-hand triple with them; $a$, $b$, and $c$ are the directional cosines of the unit vector $\overrightarrow{m}=[a,\hspace{0.17em}b,\hspace{0.17em}c]$ of the normal line to the plane (Equation (1)), which coincides with the vector $\overrightarrow{a}$; and $d$ is the distance from the origin of the CS $xyz$ to this plane. The elements $\overrightarrow{m}$ and $d$ are calculated using the cloud of points belonging to the bottom surface, formed by DVL, according to the method described in [32].

To form the motion trajectory of the MM working tool in the CS $xyz$, it is necessary to calculate the coordinates of the target points $A_i$ (see Figure 3). If the camera is targeted at point $A_i$, the position of this camera in the CS $xyz$ is determined by the preset point $P_i=[P_{ix},P_{iy},P_{iz}]$ located on the optical axis, and its orientation is determined by the unit vector ${\overrightarrow{d}}_i=[d_{ix},d_{iy},d_{iz}]$. In this case, the optical axis of the camera, coinciding with the vector ${\overrightarrow{d}}_i$ that extends from the point $P_i$ towards the bottom plane (Equation (1)), is described by the following equation:

$$A_i=P_i+t_i{\overrightarrow{d}}_i,$$

(2)

where $t_i$ is the parameter that varies within the limits $[0,\infty )$. The value of the parameter $t_i$ at the point of intersection of the axis (Equation (2)) with the plane (Equation (1)) can be obtained by substituting Equation (2) in Equation (1) [34]:

$$a(P_{ix}+t_i{d}_{ix})+b(P_{iy}+t_i{d}_{iy})+c(P_{iz}+t_i{d}_{iz})-d=0$$

(3)

$$t_i=\frac{-(a{P}_{ix}+b{P}_{iy}+c{P}_{iz}-d)}{a{d}_{ix}+b{d}_{iy}+c{d}_{iz}}$$

(4)

The coordinates of the point $A_i$ at the intersection of the camera optical axis with the bottom plane are determined by substituting the found value of the parameter $t_i$ in Equation (2):

$$A_i=P_i+\frac{-(a{P}_{ix}+b{P}_{iy}+c{P}_{iz}-d)}{a{d}_{ix}+b{d}_{iy}+c{d}_{iz}}{\overrightarrow{d}}_i$$

(5)

After the operator points the camera’s optical axis at the next target point $A_{i+1}$, the value of the parameter $t_{i+1}$ is calculated using Equation (3) for the new point $P_{i+1}$ and vector ${\overrightarrow{d}}_{i+1}$. The coordinates of the rest of the target points are determined in the same way. As a result, the motion trajectory of the MM working tool is represented in the form of its sequential movements through all target points. The vector ${\overrightarrow{n}}_i$ of the tool is always parallel to the vector $\overrightarrow{A_i{A}_{i+1}}$, connecting the passed, and the next, target points.

4. Building of Motion Trajectories for a MM Working Tool in Case of Rough Bottom Topography

If the bottom surface has a rough topography, then it is necessary to use 3D multibeam sonars [35,36], which provide a high accuracy for formation of point clouds on the bottom surface in the area of the UUV operation. These clouds and other known methods are used [31,37] to build a triangulation model (see Figure 4), which is a multitude of triangular shapes stitched together. The trajectory of the MM working tool, passing through the target points $A_i$ set by the operator on the rough seabed surface, is formed on the basis of this model. Each kth triangle in the triangulation model is set in the CS $xyz$ as coordinates of three vertices $V_{k0},\hspace{0.17em}{V}_{k1},\hspace{0.17em}{V}_{k2}\in R^3$, where $k=\overline{1,g}$ and $g$ is the number of triangles.

To find the coordinates of each target point $A_i$ at the intersection of the camera optical axis and the obtained triangulation surface in this CS, first, the triangle currently intersected by the axis (Equation (2)) should be found among all $g$ triangles. For this, the Möller–Trumbore algorithm [38] is used. Based on the coordinates of the point $P_i$, the vector ${\overrightarrow{d}}_i$, and the vertices $V_{k0},\hspace{0.17em}{V}_{k1},\hspace{0.17em}{V}_{k2}$ of all $g$ triangles, this algorithm identifies the kth triangle through which the axis (Equation (2)) passes. In this case, the coordinates of the point $A_i$ in the specified CS relative to the vertices of the kth triangle in vector form can be determined as follows:

$$A_{ik}=w_{ik}{V}_{k0}+u_{ik}{V}_{k1}+v_{ik}{V}_{k2}$$

(6)

where $u_{ik},v_{ik},w_{ik}$ are the barycentric coordinates of the point $A_i$ relative to the vertices of the kth triangle ($u_{ik}+v_{ik}+w_{ik}=1$).

By expressing $w_{ik}=1-(u_{ik}+v_{ik})$ and substituting the equation of the axis (Equation (2)) in Equation (4), we obtain a system of equations with respect to the parameters $t_{ik}, u_{ik}, v_{ik}$:

$$P_i+t_{ik}{\overrightarrow{d}}_i=(1-(u_{ik}+v_{ik}))V_{k0}+u_{ik}{V}_{k1}+v_{ik}{V}_{k2}$$

(7)

which can be expressed in the matrix form:

$$\left[\begin{array}{ccc}-{\overrightarrow{d}}_i& V_{k1}-V_{k0}& V_{k2}-V_{k0}\end{array}\right]\hspace{0.17em}\left[\begin{array}{c}{t}_{ik}\\ u_{ik}\\ v_{ik}\end{array}\right]=\left[P_i-V_{k0}\right]$$

(8)

Values of the parameters $t_{ik}, u_{ik}, v_{ik}$ are obtained by solving Equation (5) [38]:

$$\hspace{0.17em}\left[\begin{array}{c}{t}_{ik}\\ u_{ik}\\ v_{ik}\end{array}\right]=\frac{1}{({\overrightarrow{d}}_i\times (V_{k2}-V_{k0}))\cdot (V_{k1}-V_{k0})}\left[\begin{array}{c}((P_i-V_{k0})\times (V_{k1}-V_{k0}))\cdot (V_{k2}-V_{k0})\\ ({\overrightarrow{d}}_i\times (V_{k2}-V_{k0}))\cdot (P_i-V_{k0})\\ ((P_i-V_{k0})\times (V_{k1}-V_{k0}))\cdot {\overrightarrow{d}}_i\end{array}\right]$$

(9)

where ($\cdot$) is the dot product of vectors, and ($\times$) is the vector product of vectors.

The triangle k with the vertices $V_{k0},\hspace{0.17em}{V}_{k1},\hspace{0.17em}{V}_{k2}$, which intersects the axis (Equation (2)), is found with the following conditions [38]:

$$0\le u_{ik}\le 1$$

(10)

$$0\le v_{ik}\le 1$$

(11)

$$u_{ik}+v_{ik}\le 1$$

(12)

and the coordinates of the point $A_i$ of intersection of the camera optical axis with the kth triangle are determined by substituting the found parameter $t_{ik}$ from Equation (6) in Equation (2):

$$A_i=P_i+\frac{((P_i-V_{k0})\times (V_{k1}-V_{k0}))\cdot (V_{k2}-V_{k0})}{({\overrightarrow{d}}_i\times (V_{k2}-V_{k0}))\cdot (V_{k1}-V_{k0})}{\overrightarrow{d}}_i$$

(13)

For the subsequent building of the motion trajectory for the MM working tool passing through all the obtained points $A_i$, this trajectory is represented as a sequence of transects of the triangulation surface, which are the vertical profiles of its parts between neighboring target points. These profiles can be represented as a set of points of intersection of the triangulation triangles sides and the plane formed by the vector $\overrightarrow{A_i{A}_{i+1}}$, connecting neighboring target points, and the unit vector $\overrightarrow{z}$ parallel to the axis $z$ in the CS $xyz$ (see Figure 4).

To build the motion trajectory for the MM working tool passing through the points of the triangles’ intersection with the plane formed by the vectors $\overrightarrow{A_i{A}_{i+1}}$ and $\overrightarrow{z}$, it is necessary to calculate which of all g triangles intersect the specified plane, and calculate the coordinates of the points of these intersections. When the fact that a side of the kth triangle intersects the specified plane has been established, the signs of the distances $p_{ikh}$ from the hth vertex (h = 0, 1, 2) to the plane formed by the vectors $\overrightarrow{A_i{A}_{i+1}}$ and $\overrightarrow{z}$ are compared for each pair of its vertices by the algorithm [28]. The parameters $p_{ikh}$ are calculated by the formula $p_{ikh}={\overrightarrow{r}}_i\cdot \overrightarrow{A_i{V}_{kh}},$ ${\overrightarrow{r}}_i=\frac{\overrightarrow{A_i{A}_{i+1}}\times \overrightarrow{z}}{\Vert \overrightarrow{A_i{A}_{i+1}}\times \overrightarrow{z}\Vert }$, where $\overrightarrow{A_i{V}_{kh}}$ is the vector connecting the target point $A_i$ with the hth vertex $V_{kh}$ of the kth triangle.

If values of $p_{ikh}$ have the same signs ($p_{ik0}{p}_{ik1}>0$ and $p_{ik0}{p}_{ik2}>0$ or $p_{ik0}{p}_{ik1}<0$ and $p_{ik0}{p}_{ik2}<0$) for any pair of the respective vertices of the kth triangle, then the plane does not intersect the kth triangle. If $p_{ikh}$ have different signs ($p_{ik0}{p}_{ik1}>0$ and $p_{ik0}{p}_{ik2}<0$ or $p_{ik0}{p}_{ik1}<0$ and $p_{ik0}{p}_{ik2}>0$), then the point $T_{ike}$, through which the MM working tools should move, is lying on the side of the kth triangle formed by this pair of vertices (in general, the sides of each triangle can have no more than two points of intersection with the plane). The coordinates of the point $T_{ike}$ for the side formed by the vertices $V_{k0}$ and $V_{k1}$ are determined by the formula [34]: $T_{ike}=V_{k0}+t_{ike}\frac{\overrightarrow{V_{k0}{V}_{k1}}}{\Vert \overrightarrow{V_{k0}{V}_{k1}}\Vert }$, where $t_{ike}=\frac{-({\overrightarrow{r}}_i\cdot \overrightarrow{O{V}_{k0}}+\left|\overrightarrow{O{A}_i}\cdot {\overrightarrow{r}}_i\right|)}{{\overrightarrow{r}}_i\cdot \overrightarrow{V_{k0}{V}_{k1}}},$ $\overrightarrow{V_{k0}{V}_{k1}}$ is the vector connecting the vertices $V_{k0}$ and $V_{k1}$ of the kth triangle [34], and $\left|\overrightarrow{O{A}_i}\cdot {\overrightarrow{r}}_i\right|$ is the distance from the plane formed by the vectors $\overrightarrow{A_i{A}_{i+1}}$ and $\overrightarrow{z}$ to the origin O of the CS $xyz$.

The coordinates of the second point $T_{ike}$ on one of the two remaining sides of the kth triangle, formed by the vertices $V_{k0}$ and $V_{k1}$ or $V_{k1}$ and $V_{k2}$, are calculated in a similar way. If the equality $p_{ikh}=0$ is true, then the hth vertex of the kth triangle is lying on the specified plane. In this case, $T_{ike}=V_{kh}$.

As a result, there is a set of points $T_{ike}$, through which the MM should move its working tool. Thus, at the points $T_{ik1}$ and $T_{ik2}$, as well as during movement between these points, the vector ${\overrightarrow{a}}_{ik}$ is always perpendicular to the plane of the kth triangle, to which these points belong. Therefore, in the CS $xyz$, the vector ${\overrightarrow{a}}_{ik}$ is always perpendicular to the plane defined by the three vertices of the kth triangle.

To build the desired trajectory of the working tool, the resulting set of points $T_{ike}$ with their respective vectors ${\overrightarrow{a}}_{ik}$ are transformed into a sequence of points $T_j$ with vectors ${\overrightarrow{a}}_j$ located between the adjacent target points $A_i$ and $A_{i+1}$, where j is the sequential number of the point in the sequence of the working tool’s motion. In this case, the vector ${\overrightarrow{n}}_j$ of this tool is always parallel to the vector $\overrightarrow{T_j{T}_{j+1}}$ connecting the passed, and the next, points of the set. As a result, the obtained sequence of points $T_j$ and their respective vectors ${\overrightarrow{a}}_j$ and ${\overrightarrow{n}}_j$ form the motion trajectory of the working tool (see Figure 5).

5. Building of Complex Trajectories for MM Working Tools to Move Over the Surface of Underwater Objects

To perform complex manipulative operations, including those with subsea valve systems, cleaning silt and fouling, measurements of pipeline wall thickness, etc., the working tool of MM should move along complex spatial trajectories. The shape of such work objects is usually well known. Therefore, the desired motion trajectories of the MM can be built in advance, taking into account the requirements of certain technological operations. In this case, the operator can visually estimate the area of the underwater object on which the specified trajectory is to be overlaid. Using a video camera, the operator indicates the start and end points, with the desired trajectory between them automatically projected from the computer memory on the specified surface of the work object, taking into account the spatial position of this object [39].

When the UUV hovers over an object located in the scanning field of the TVS, the desired trajectory of the working tool can be immediately set in the CS $xyz$ of the UUV in the form of a sequence of points $B_j$ or in an analytical form. If this trajectory is set in the plane $Oxy$ as a function $y=f(x)$, then it should be projected on the surface of the object in the direction of the unit vector $\overrightarrow{o}$ that coincides with the negative direction of the $z$ axis of the CS $xyz$. The operator targets the camera optical axis, setting the start $A_1$ and end $A_2$ points belonging to the object and determining the beginning and end of the working tool’s movement along the trajectory. The coordinates of these points are calculated using Equation (10).

To project the trajectory on a sector of the object’s triangulation surface confined between the points $A_1$ and $A_2$, this trajectory, set in analytical form, is first converted into a sequence of auxiliary points $B_j.$ For this, the coordinates of the points $B_j$ are calculated as follows:

$$\begin{array}{c}{B}_{jx}=B_{(j-1)x}+g\\ B_{jy}=f(B_{jx})\\ B_{jz}=0\end{array}$$

(14)

where $f(B_{jx})$ is the confirmed value of the y coordinate from x obtained by the analytical expression $y=f(x)$, $B_{1x}=0$; g is the step along the x axis, which determines the number of auxiliary points $B_j$ (determined by the parameters of this trajectory and the degree of detail of the scanned object). An increase in the number of points $B_j$ enhances the accuracy of the trajectory formation but requires an increase in the number of calculations. To take into account the spatial position and orientation of certain areas of the work objects when projecting points $B_j$, first, all the vectors connecting the origin O of the CS $xyz$ with the points $B_j$ should be rotated at an angle $\alpha =\angle (x,\overrightarrow{A_1{A}_2})$ around the $z$ axis. Then the second rotation should be performed around the $y$ axis of the CS $xyz$ at an angle $\phi$ confined between the vector $\overrightarrow{A_1{A}_2}$ and the plane $Oxy$. Afterwards, a linear transfer of the obtained vectors is performed along the x and y axes by the values $A_{1x}$ and $A_{1y}$, respectively. As a result, the coordinates of the sought points ${B^{\prime }}_j$ will be determined using the following equation:

$$\left[\begin{array}{c}{B^{\prime }}_{jx}\\ {B^{\prime }}_{jy}\\ {B^{\prime }}_{jz}\end{array}\right]=R\left[\begin{array}{c}{B}_{jx}\\ B_{jy}\\ B_{jz}\end{array}\right]+\left[\begin{array}{c}{A}_{1x}\\ A_{1y}\\ 0\end{array}\right]$$

(15)

$$\begin{array}{l}R=R_{z\alpha }{R}_{y\phi }=\left[\begin{array}{ccc}\cos \alpha & -\sin \alpha & 0\\ \sin \alpha & \cos \alpha & 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{ccc}\cos \phi & 0& \sin \phi \\ 0& 1& 0\\ -\sin \phi & 0& \cos \phi \end{array}\right]\\ =\left[\begin{array}{ccc}\cos \phi \cos \alpha & -\sin \alpha & \sin \phi \cos \alpha \\ \cos \phi \sin \alpha & \cos \alpha & \sin \phi \sin \alpha \\ -\sin \phi & 0& \cos \phi \end{array}\right],\end{array}$$

(16)

where $R_{z\alpha }$ and $R_{y\phi }$ are the matrices of elementary rotations around the $z$ and $y$ axes of the CS $xyz$, respectively, and $R$ is the matrix of complex rotation [40].

The intersections of the projection rays extending from the points $B_j$ in the direction of the vector $\overrightarrow{o}$ with the triangulation surface model form a sequence of points $T_j$ (see Figure 6). The sequence of these points describes the desired motion trajectory of the working tool, located between the points $A_1,A_2$ indicated by the operator.

The coordinates of the points $T_j$ are determined using the Möller–Trumbore algorithm [38] with Equation (11) taken into account in the following form:

$$T_j={B^{\prime }}_j+\frac{(({B^{\prime }}_j-V_{k0})\times (V_{k1}-V_{k0}))\cdot (V_{k2}-V_{k0})}{(\overrightarrow{o}\times (V_{k2}-V_{k0}))\cdot (V_{k1}-V_{k0})}\overrightarrow{o}$$

(17)

The vectors ${\overrightarrow{a}}_j$ and ${\overrightarrow{n}}_j$, corresponding to each point $T_j$, are calculated using the algorithm described in Section 3. To find the point $T_j$, it is necessary to determine the intersections of each triangle in the triangulation grid by the ray extending from the point ${B^{\prime }}_j$. In case of a large number of projecting rays or triangles, the search for the intersections of each triangle by each ray (starting from the first one) requires a lot of computational resources. To reduce the number of necessary computational operations (including false ones, where the selected triangle has no intersection points with a certain ray), when searching for the point $T_{j+1}$, first, a test is carried out to check if the ray extending from the point ${B^{\prime }}_{j+1}$ intersects the triangle where the point $T_j$ is located. If there is no intersection, then the search for the intersected triangle begins in the normal mode, starting from the first one.

6. Software Implementation and Results of the Study of the Method for Supervisory Control of an MM Mounted on a UUV

The developed method for supervisory control of an MM was implemented in the C++ programming language. The input data for the created program is a set of points on the underwater object surface, obtained using TVS and fixed in 3D space. The coordinates of the points $P_i$ and vectors ${\overrightarrow{d}}_i$, specified by the operator, that define the position of the camera optical axis in the CS $xyz$ are input in the same program.

The created program allows building triangulation models for surfaces of underwater objects using the algorithm [31]. After the operator specifies the points $P_i$ and vectors ${\overrightarrow{d}}_i$, the coordinates of the target points $A_i$ and the sequence of points $T_j$ (see Figure 5 and Figure 6) and their respective vectors ${\overrightarrow{a}}_j$ and ${\overrightarrow{n}}_j$ are calculated for a certain MM working tool using this program implementation.

For a more precise description of the underwater objects’ shape, and also for the subsequent control of the formation of MM motion trajectories, a graphical interface was created using the OpenGL Core libraries (see Figure 7), which displays the triangulation surface of the object being observed, the trajectory projected on this surface, and the video image coming from the camera in real-time mode. Using the resulting image, the operator can check the correct construction of the automatically generated trajectory before starting a manipulative operation.

Since the method is based on spatial geometric transformations (finding intersections of the camera’s optical axis with the triangulation surface and with the subsequent projection of the desired trajectories on this surface), its accuracy depends only on the accuracy of point clouds obtained from the TVS and on the features of the triangulation algorithms used for these points. Therefore, the method can easily be implemented both in systems for semi-natural modeling and testing [41] and for an actual UUV, regardless of the kinematic and dynamic features of these vehicles and their MM.

The proposed method was tested by numerical simulation in the V-REP environment. A scenario of one of the manipulative operations for cleaning a continuous pipeline from fouling is shown in Figure 8a. A model of multibeam sonar with a resolution of 32 × 32 dots and a unidirectional scanning pattern was used as the TVS. The sequence of points $T_j$ and their respective vectors ${\overrightarrow{a}}_j$ and ${\overrightarrow{n}}_j$ of the MM working tool, formed by the program, was transferred to the Matlab/Simulink via the UDP protocol. In Matlab/Simulink, the resulting sequence was smoothed using a third-order parametric B-spline [42]. Program control signals for the electric drives of the respective degrees of MM mobility were generated using this sequence. During the simulation, we used a well-tested mathematical model of a UUV [43] with a mounted MM based on the PUMA kinematic model, which has the first five degrees of freedom (3 translatory and 2 orientation). The main parameters of the UUV and its MM are given in

Table 1. Parameters of the UUV and MM.

Parameters		Values (unit)
UUV	L × H × W	1.2 × 1.5 × 0.9 (${\mathrm{m}}^3$)
	Mass	117 (kg)
Link 1	Length	0.05 (m)
	Mass	0.4 (kg)
Link 2	Length	0.5 (m)
	Mass	4 (kg)
Link 3	Length	0.5 (m)
	Mass	4 (kg)
Link 4	Length	0.1 (m)
	Mass	0.5 (kg)

. All parameters of the kinematic and dynamic model of the UUV are provided in [44,45]. The mathematical model of the underwater MM used and its parameters are described in detail in [46]. This model includes a recurrent algorithm for solving the inverse dynamics problem, which takes into account not only the joint coupling between degrees of mobility of the MM, but also the effects of a viscous environment on its moving links, as well as the experimentally determined parameters of these effects [47]. To stabilize the UUV in the hovering mode, a combined system [21,48] was used, which makes it possible to compensate for the impacts of marine currents and the dynamic impacts from moving the MM, calculated in real time, with the UUV thrusters.

To visualize the MM motions over the surface of work objects, the data (generalized coordinates of MM, and linear and angular displacements of the UUV relative to the initial point of its stabilized positioning) obtained during the simulation of the complex dynamic model of the UUV with MM were sent from Matlab/Simulink back to V-REP by the UDP protocol. The results of the visualization of the actual technological manipulative operation performed are shown in Figure 8a,b.

An assessment of the relative positions of the pipeline surface and the MM working tool was made using Figure 8b. On the basis of the analysis carried out, it became possible to automatically generate and then, using the proposed method, pass through the complex motion trajectories, thus, precisely performing quite complex manipulative operations with the working tool. The operator gave only the target indications and checked the correctness of the working tool’s automatically built motion trajectories along the surface of the work object by means of the designed graphical interface. This greatly simplified and facilitated the operator’s work. Implementation of the program tools for the proposed method should not pose any serious difficulties, since these tools do not have significant computational complexity and can be installed on standard on-board computers.

7. Conclusions

In the present study, we addressed the problem of developing a new method for supervisory control of complex manipulative operations, which can significantly increase the speed and simplify complex work and operations by automating the entire technological process, while reducing the mental load on the MM operator. A significant increase in the efficiency of manipulative operations was achieved by precisely building complex curved trajectories of the MM working tool’s motion on the basis of operator target indications. The trajectories built are accurately performed by the MM with the required orientation of the working tools in an automatic mode. The operator, by targeting the optical axis of the video camera mounted on the UUV rotary platform, sets the start and end points of the motion trajectories for these tools and the boundaries of specified areas of the work objects. Then the trajectories, formed by taking into account the shape and spatial position of the work areas, are projected onto them.

The developed method of supervisory control was implemented in the C++ programming language. The specifics of application of the method have been tested through numerical simulation in the V-REP environment. A graphical interface has been created that provides rapid testing of the accuracy of overlaying the planned trajectories on the mathematically described surface of the work object. The results of the simulation of actual technological operations with the use of an MM mounted on a UUV confirmed the high efficiency of the proposed method and the simplicity of its practical application.

Further work will focus on creating methods and algorithms for intellectual support of operators’ activities during implementation of supervisory control modes for UUV with MM, while taking into account the limited bandwidth of hydroacoustic communication channels. Herewith, the issues of providing a dialog mode for MM control will be ad-dressed, factors and conditions that could potentially cause emergency situations identified, and timely measures to prevent them taken by generating warnings and recommendations for UUV operators.