Received: 27 October 2019 Revised: 24 February 2020 Accepted: 26 May 2020

DOI: 10.1002/rnc.5106

RESEARCH ARTICLE

Robust control design for teleoperation of multiple mobile

manipulators under time delays

Usman Ahmad Ya-Jun Pan Henghua Shen

Department of Mechanical Engineering,

Dalhousie University, Halifax, Canada Summary
This article addresses a novel multilateral teleoperation control scheme for
Correspondence
single-master multiple slave systems, which can be extended to n masters and
Ya-Jun Pan, Department of Mechanical
Engineering, Dalhousie University, n slaves without the loss of generality, where the master is a m degrees of free-
Halifax, Nova Scotia, Canada. dom (DOF) manipulator arm and the slaves are m DOF mobile manipulators.
Email: yajun.pan@dal.ca
The human operator operates the master robot to remotely control the slaves
handling a target object. The master position signal is transmitted to the slave
side to generate a desired object trajectory as well as the reference mobile base
velocity. An adaptive robust controller is designed for the slaves to follow the
desired trajectory from the master, which not only provides the excellent trajec-
tory tracking but also optimize the internal force distribution of the object. A null
space controller is designed for the mobile platforms of the mobile manipulators
to achieve the velocity consensus while achieving the main task of object trans-
portation. The novel control design uses the transmission of the environmental
force feedback over the communication channel by the estimated parameters
of the environment, which helps retain the stability of the overall system. The
environmental force is predicted on the master side based on the estimated envi-
ronmental parameters. The proposed control design can simultaneously achieve
the objectives of stability, synchronization, and optimal internal force distribu-
tion. The simulation results of a single-master and three slaves teleoperation
system validate the efficacy of the proposed control algorithm.

KEYWORDS
adaptive robust control, communication delay, force prediction, mobile manipulators, multilateral
teleoperation, null space control

1 I N T RO DU CT ION

The ever increasing demand of industrial applications like heavy object manipulation, safe and stable operation, and com-
plex task performances require a well-designed control system to achieve all these goals. Bilateral teleoperation systems
are sometime not suitable for such applications and there comes the need to extend these systems to multilateral teleop-
eration. Fixed base manipulators have limited workspace and dexterity. Teleoperation of mobile manipulators emerges
as a decent solution and provides enough flexibility of achieving task execution goals with guaranteed stability and safety
of operation. In addition, the mobility of a robotic manipulator not only maximizes the task generality of the system but

Int J Robust Nonlinear Control. 2020;1–19. wileyonlinelibrary.com/journal/rnc © 2020 John Wiley & Sons, Ltd. 1
2 AHMAD et al.

also provides an increased task space in structured or unstructured environments. However, this comes at the cost of
additional control design challenges.
Multilateral teleoperation of mobile manipulators becomes complicated due to increased number of human operators,
master robots, slave robots, and the transmission of signals over the communication channel. The multilateral teleoper-
ation control design poses challenges in handling time delays, dynamics of the master and slave robots, the realism of
the teleoperation system, and the internal forces of the target object. The control algorithms for the mobile manipula-
tors have to deal with redundant DOFs and simultaneous or decoupled control of the robotic arm and the mobile base.
As we leverage the increased task space and dexterity of the mobile manipulator system, we face the challenges of high
dimensionality and complexity in the control design.1,2
Teleoperation control of mobile manipulators has attracted researchers over the past few years. A formation control
design for the teleoperation of nonholonomic mobile manipulators under constant delays was proposed in the work.3
One of the limitations of the research was that the slave mobile manipulators were only controlled in the constrained
workspace of the master robot with constant delays.
An adaptive controller was proposed in the work4 to remotely control a mobile manipulator operating in an
unbounded workspace. A velocity command from a joystick is sent over the communication channel to the slave side
and the slave local controller makes the slave end-effector converge to the master pose. A neural adaptive control scheme
was developed in the work5 for the single-master multiple slave teleoperation system of mobile manipulators. The time
delays and the dead zone uncertainties were taken into account for the controller design of the mobile manipulators
cooperatively handling an object. A linear matrix inequality (LMI) based model reference neural control approach was
implemented to make the tracking errors converge to zero.
A time domain passivity approach (TDPA) was proposed in the work6 for the bilateral teleoperation of a mobile
manipulator. The human operator controlled the speed of the mobile base via a haptic device and the force feedback is
transmitted to the human operator when the robot interacts with the environment. The passivity observer and passiv-
ity controller make sure that the system remains passive during the operation to avoid instability. A switching control
mechanism was proposed in the work7 where the human operator can either control the position of the end-effector
or the speed of the mobile base depending on the requirement of the task. A hybrid control approach was presented
in the work8 where a teleoperation control without sensor feedback was realized by applying the torque observers. In
the work,9 the authors proposed a proportional derivative (PD) like scheme to control the velocity of the end-effector
of the mobile manipulator and the dynamics control and secondary control actions were achieved in the operational
space. In all of the abovementioned research, the control design either deals with the control of a single-mobile manip-
ulator or with the bilateral teleoperation. Absolute transparency based control schemes were discussed in the work10
that analyzed different control criterion applied to bilateral teleoperation of mobile robots. Absolute transparency, real-
ism of the teleoperation system, was tested for various control approaches applicable for mobile robot teleoperation. All
the aforementioned control designs either deal with the motion tracking or the teleoperation control of a single-mobile
manipulator.
Fuzzy control has also been widely applied to the teleoperation systems. A lot of work has been carried out in
recent years discussing fuzzy control ideas with applications to robot teleoperation. A bimanual teleoperation sys-
tem was considered in the work,11 where a predictive control method was proposed without taking into consideration
any force/torque or acceleration sensors. A combination of a position predictor and force predictor was developed
to predict the position states of the remote side and the contact forces. The evaluation and suppression of the sys-
tem uncertainties were carried out using an adaptive fuzzy control scheme, which was based on LMIs. A similar
idea was implemented in the work12 with impedance based internal force control. An adaptive teleoperation control
method based on a radial basis function (RBF) neural network (NN) was proposed in the work,13 where the system
stability and position tracking were analyzed by the estimation errors of RBF-NN. In the work,14 a fuzzy observer
based hybrid force/position Takagi Sugeno method was presented for bimanual teleoperation. The system structure
consisted of position observers as well as force/torque estimators. All the abovementioned methods presented teleoper-
ation control schemes bypassing the costly requirements of sensors thus making control implementation much simpler
and efficient.
In this article, we propose a novel multilateral teleoperation control of multiple mobile manipulators handling an
object. An integrated control design, null space control, adaptive robust control (ARC), and environment force predic-
tion, is proposed for the trajectory tracking of the end-effectors and the mobile platforms of the mobile manipulators.
In the communication channel, the master position signal is transmitted to the slave side and an optimal processing is
carried out to generate the desired object trajectory and the reference mobile base velocity. The trajectory tracking of the
AHMAD et al. 3

end-effector is achieved with the help of an adaptive robust controller, which not only ensures the stability of the system
but also provides synchronization and optimized internal force distribution. The motion tracking of the mobile platforms
is achieved by decoupling the system dynamics into the task space and the null space. The null space controller provides
the velocity consensus of the mobile platforms with a given reference velocity. While the mobile manipulators handle the
object, the environmental parameters are estimated online and are transmitted over the communication channel to the
master side. The environment force is predicted based on the estimated environment parameters and the human oper-
ator is able to feel the force feedback without any delay thus improving the transparency of the system. To the best of
authors’ knowledge, this is the first time that a control design scheme for multilateral teleoperation of multiple mobile
manipulators is presented with the control objectives of stability, synchronization, and internal force distribution. The
main contributions of this work are as follows.

1. Propose a novel control design for a single-master multiple slave teleoperation system, which can be easily extended
to n masters and n slaves, to cooperatively handle a target object in a remote environment.
2. Achieve the control objectives of stability, synchronization, and force distribution while the end-effectors and the
mobile platforms are controlled separately.
3. Predict the undelayed environment reaction force on the master side in such a way that the communication channel
passivity is naturally avoided.
4. Deal with arbitrary time delays in multilateral communication channels as well as in the graph topology over which
the slaves are connected.

In order to achieve the abovementioned control objectives, a combination of adaptive robust control and null
space control is proposed to realize a multilateral teleoperation system, which not only guarantees the stability but
avoids the trade-off limitation between the robust stability and transparency. In contrast to other control methods like
impedance-based control for teleoperation, ARC proved to be more efficient in terms of performance criteria of stability,
synchronization, and internal force distribution.

2 PRELIMINARIES

In a network of multiple agents connected over a communication network, the signal sharing or information exchange can
be described by a communication graph. A graph (, ) can be defined as a finite set () = {vi , … , vn }, with elements
called nodes or vertices, including a set () ⊂  ×  with elements defined as edges, which is an ordered pair of distinct
nodes.15-17 The in-degree di of a vertex v ∈ () is the number of edges that have this vertex as a head. An undirected graph
is a graph for which all the edges (vi , vj ) ∈ () and (vj , vi ) ∈ () otherwise it is a directed graph. A strongly connected
graph is a directed graph in which any two vertices in () can be joined by a path. The robots in a networked system
can be represented as the vertices of a graph and the information exchange among them can be defined by a weighted
graph as w = (, , ), where (w ) = {wij }, j ∈  , where wij is the weight of the edge from vj to vi and  is the finite
set of the robots sharing the information. Given the edge weights wij , a communication graph can be represented by a
connectivity or adjacency matrix  = [wij ] as,
{
wij > 0, if (vi , vj ) ∈ 
=
wij = 0, otherwise.

The diagonal in-degree matrix  = diag[di ] is used to define the Laplacian matrix  of the graph and is given by  =
 − . It is noted that  has row sum zero. The weights of the edges are not always identical so the weighted Laplacian
for the directed graph can be defined as,

⎧∑
⎪ wij , if i = j
⎪j∈
w = ⎨ −w , if j ∈ 
ij
⎪
⎪ 0, otherwise.
⎩
4 AHMAD et al.

3 S Y ST E M D E SCR IPT ION

3.1 Master manipulator

A nonredundant p-link master manipulator can be described in the joint space as,

M(qm )q̈ m + C(qm , q̇ m )q̇ m + G(qm ) = um + Jm

T
(qm )Fhc , (1)

where qm ∈ Rp represents the joint angle of the manipulator, M(qm ) ∈ Rp×p is the inertia matrix of the manipulator, which
is symmetric positive definite, C(qm , q̇ m ) ∈ Rp is the Centripetal and Coriolis torque, and G(qm ) ∈ Rp is the gravitational
𝜕x
torque. The control input is represented as um ∈ Rp , the Jacobian matrix is Jm (qm ) = 𝜕qm ∈ Rp×p , where xm = Φm (qm ) ∈ Rp
m
represents the angle and the position of the end-effector of the manipulator. The force which is applied by the human
operator to the end-effector of the master manipulator is represented by Fhc ∈ Rp . The dynamics in (1) can be rewritten
in Cartesian space as,

Mc (qm )̈xm + Cc (qm , q̇ m )ẋ m + Gc (qm ) = umc + Fhc , (2)

−1 ̇
−T
where Mc = Jm −1
MJm , Cc = Jm
−T −1
CJm −T
− Jm MJm −1
J m Jm , Gc = Jm
−T −T
G, and umc = Jm um .

3.2 Object and environment

The dynamics of the object can be expressed as,

∑
n
Mo (xo )̈xo + Co (xo , ẋ o )ẋ o + Go (xo ) = Fi − Fe , (3)
i=1

where xo ∈ Rp is describing the position of the object. The inertia matrix of the object is M o (xo ), Co (xo , ẋ o ) is the Centripetal
and Coriolis torque, and Go represents the gravitational torque. The force exerted on the object by the end-effector of the
slaves is represented by F i and F e is the reaction force of the environment. The environment reaction force, under the
flexible contact model, can be written as,18

Fe = Be ẋ o + Ke xo + Ce = 𝜃eT 𝜑e (xo , ẋ o ), (4)

where 𝜃e = [Be , Ke , Ce ]T are unknown parameters and 𝜑e = [ẋ o , xo , 1] is the regressor signal for the object. This model can
consider 𝜃e = 0, which essentially means the free motion of the slave side. There exists a kinematic chain between the
object and the end-effectors of the slaves, which can be expressed as,19

xo = Φo (xsi ), ẋ o = Li (xsi )ẋ si , fi = LTi Fi , (5)

𝜕xo (xsi )
where Li is the transformation matrix and Li = 𝜕xsi
. The constraints imposed on the movement of the object and the
slaves due to the kinematic chain can be represented as,18

xo = xo (xsi ) = xo (xsi (qsi )) = Ψ(qsi ), Ai = Li Jsi ,

ẍ o = Ai q̈ si + Ȧ i q̇ si , ẋ o = Ai (qsi )q̇ si , (6)

3.3 Slave mobile manipulators

The dynamics of a networked system of n-link N mobile manipulator slaves with i = 1, … N can be described in Euler
Lagrange (EL) form as follows.5,20,21
Msi (qsi )q̈ si + Csi (qsi , q̇ si )q̇ si + Gsi (qsi ) + fi (qsi ) = Bi (qsi )𝜏si , (7)
AHMAD et al. 5

where M si (qsi ) ∈ Rn×n is the inertia matrix, which is a symmetric bounded positive definite matrix; Csi (qsi , q̇ i ) ∈ Rn×n is the
[ ]T [ ]T
Centripetal and Coriolis matrix; Gsi (qsi ) ∈ Rn is the gravitational force vector; fi (qsi ) = finT fihT = (ATi (qip )𝜆i )T 0 ∈
[ ]T
Rn is the generalized constraint force, where 𝜆i = 𝜆in 𝜆ih is the Lagrangian multiplier, 𝜆in considers the nonholo-
nomic constraint and 𝜆ih considers the holonomic constraints, and Ai (qip ) is the kinematic constraint matrix. Bi (qi ) ∈ Rn×m
is a full-rank input transformation matrix and also assumed to be known, 𝜏si ∈ Rm is the control input to the system.
[ ]T
qsi = qTip qTia ∈ Rn is the vector of generalized coordinates. qip denotes the generalized coordinates of the wheeled
mobile platform and qia denotes the generalized coordinates of the manipulator arm.2 The terms, for brevity dropping
the subscript s, can be further represented as,
[ ] [ ]
Mip Mipa Cip Cipa
Mi (qi ) = , Ci (qi , q̇ i ) =
Miap Mia Ciap Cia
[ ] [ ] [ ]
Gip Bip 0 𝜏ip
Gi (qi ) = , Bi (qi ) = , 𝜏i = ,
Gia 0 Bia 𝜏ia

where M ip and M ia describe the inertia matrices for the mobile platform and the manipulator arm, respectively. M ipa and
M iap describe the coupling inertia matrices of the mobile platform and the manipulator arm. Cip and Cia are Centripetal
and Coriolis torques for the mobile platform and the manipulator arm, respectively. Cipa and Ciap are the coupling Cen-
tripetal and Coriolis torques of the mobile platform and the manipulator arm. Gip and Gia are the gravitational forces of
the mobile platform and the manipulator arm, respectively. Bip and Bia denote the input transformation matrices of the
mobile platform and the manipulator arm, respectively. 𝜏ip and 𝜏ia are the control input of the mobile platform and the
robotic arm. The coordinates of the mobile platform can be described as qip = [xi yi 𝜃ip ]T , where xi ,yi are the coordinates
of the center of the mobile platform and 𝜃ip is the orientation or the heading angle of the mobile platform. The nonholo-
nomic kinematic constraint for the midpoint of the wheel axle where the manipulator arm is mounted can be expressed
as,
ẋ i sin 𝜃ip − ẏ i cos 𝜃ip = 0. (8)

The constraint in (8) can also be written as,

Ai (qip )q̇ ip = 0, Ai (qip ) = [sin 𝜃ip − cos 𝜃ip 0]. (9)

Suppose there are l numbers of nonintegrable and independent velocity constraints and it is assumed to have the full
rank l. The mobile platform here is assumed to be completely nonholonomic and we can write Ai (qip ) matrix of (9) as,
[ ]T
Ai (qip ) = AT1 (qip ) AT2 (qip ) AT3 (qip ) … ATl (qip ) .

The nonholonomic generalized constraint forces are:

fin = (ATi (qip )𝜆in )T . (10)

Hi (qip ) ∈ n×m is a matrix with rank being m formed by a set of smooth and linearly independent vectors spanning
the null space of matrix Ai (qip ), that is,
HiT (qip )ATi (qip ) = 0, (11)

where Hi (qip ) = [H1 (qip ) H2 (qip ) … Hnp −l (qip )]. Note that here HiT Hi is of full rank. According to (9) and (11), the
first-order velocity kinematic model of a nonholonomic mobile platform which is also called the steering system can be
written in the following form,
q̇ ip = Hi (qip )𝛼i , (12)

where 𝛼i is an auxiliary function 𝛼i ∈ 2 and called the steering velocity of the kinematic system. 𝛼i includes the linear and
angular velocities of the wheeled mobile platform, as 𝛼i = [vi 𝜔i ]T , or the right wheel velocity and the left wheel velocity
6 AHMAD et al.

FIGURE 1 Schematic diagram of the differential-driven

mobile manipulator3

of the mobile platform as 𝛼i = [𝜃iR 𝜃iL ]T . We can rewrite (12) in the specific kinematic form, in terms of linear and angular
velocities of the wheeled mobile platform and in terms of right and left wheel velocities.

⎡cos 𝜃ip 0⎤ [ ]
⎢ ⎥ vi
q̇ ip = ⎢ sin 𝜃ip 0⎥ , (13)
⎢ ⎥ 𝜔i
⎣ 0 1⎦
⎡ R cos 𝜃ip R
cos 𝜃ip ⎤ [ ]
⎢2 2
⎥ 𝜃iR
q̇ ip = ⎢ R sin 𝜃ip R
sin 𝜃ip ⎥ , (14)
⎥ 𝜃iL
2 2
⎢ R R
⎣ 2D − 2D ⎦

where R is the radius of the wheels and 2D is the distance of the two wheels of the mobile platform, as shown in Figure 1.3
[ ]T
Let 𝜂i = 𝛼iT q̇ Tia . Due to the nonholonomic constraint defined in (9) and (12), there exists a vector 𝜂̇ i , such that,20

q̇ i = Hi (qi )𝜂̇ i . (15)

A reduced dynamic model now can be obtained by substituting (15) and its derivative in (7) and premultiplying
H i (qi )T ,
M𝜂i (qi )𝜂̈ i + C𝜂i (qi , q̇ i )𝜂̇ i + G𝜂i (qi ) = 𝜏𝜂i , (16)

where, M𝜂i (qi ) = HiT (qi )Mi (qi )Hi (qi ), C𝜂i (qi , q̇ i ) = HiT (qi )[Mi (qi )Ḣ i (qi ) + Ci (qi , q̇ i )Hi (qi )], G𝜂i (qi ) = HiT (qi )Gi (qi ), 𝜏𝜂i =
T
Hi (qi )Bi (qi )𝜏i .
In this work, the mobile manipulators are subjected to the following assumptions.
Assumption 1. The mobile platform is driven with two wheels operated by two independent motors or actuators and
the manipulator arm is fixed on this platform. The mobile manipulator is considered to be redundant and operating away
from singularity.
Assumption 2. It is considered that 𝜂̇ i = [𝜃̇ iR 𝜃̇ iL 𝜃̇ 1 … 𝜃̇ k ]T , where 𝜃̇ j is the angular velocity of the manipulator joints
and j = 1, … k.
In the task space, X i contains all the variables that are required to define the task of the end-effector then the kinematic
equation can be written as,
Xma = fi (𝜂i ), (17)

where f i (.) being a nonlinear transformation describes the relation between the joint space of the system and the
operational space of the end-effector.20 The velocity of the end-effector can be derived by differentiating (17),

Ẋ ma = Jma (𝜂i )𝜂̇ i ,

AHMAD et al. 7

𝜕f (𝜂 )
where Jma (𝜂i ) = 𝜕𝜂
i i
is the nonsquare Jacobian matrix of the end-effector and is noninvertible due to the redundancy of
i
the system. The redundancy of the system can be utilized to achieve additional tasks. A user defined kinematic function
for the motion of the mobile platform can be defined as,

Xmp = gi (𝜂i ). (18)

𝜕gi (𝜂i )
By differentiating (18), we have, Ẋ mp = Jmp (𝜂i )𝜂̇ i where the Jacobian matrix is Jmp (𝜂i ) = 𝜕𝜂i
. We can use the extended
T T T
position Xi = [Xma Xmp ] to accomplish the secondary task while simultaneously achieving the primary task. Now the
mobile manipulator system has an extended differential kinematic model as given by,
[ ]
Jma (𝜂i )
Ẋ i = 𝜂̇ i = J(𝜂i )𝜂̇ i , (19)
Jmp (𝜂i )

where J(𝜂i ) is the extended square Jacobian matrix of the whole system and for the sake of brevity, we will use J as J(𝜂i ).
From (19), we can have,
𝜂̇ i = J −1 Ẋ i , 𝜂̈ i = J −1 Ẍ i − J −1 JJ
̇ −1 Ẋ i . (20)

We get the extended operational space dynamic equation of the system, similar to (2), by substituting (20) into (16)
and premultiplying J −T as follows,
Mxi (qi )Ẍ i + Cxi (qi , q̇ i )Ẋ i + Gxi (qi ) = 𝜏xi , (21)

where Mxi (qi ) = J −T M𝜂i (qi )J −1 , Cxi (qi , q̇ i ) = J −T [−M𝜂i (qi )J −1 J̇ + C𝜂i (qi , q̇ i )]J −1 , Gxi (qi ) = J −T G𝜂i (qi ), 𝜏xi = J −T 𝜏𝜂i . The
combined dynamics of the slaves and the object can be written as,

MS (xo , qsi )̈xo + CS (xo , ẋ o , qsi , q̇ si )ẋ o + GS (xo , qsi ) = uS − Fe , (22)

∑n ∑n ∑n
where MS = Mo + i=1 A−T Msi A−1 , CS = Co + i=1 (A−T Csi A−1 − A−T Msi A−1 Ȧ i A−1 ), GS = Go + i=1 A−T Gsi , uS =
∑n i i i i i i i i
i=1 Ai 𝜏si .
−T

The dynamics in (2) and (22) have the following properties.

Property 1. The matrices M c and M S are symmetric positive definite and there exist positive constants 𝜇m1 , 𝜇m2 , 𝜇o1 and
𝜇o2 such that 𝜇m1 I ≤ Mc ≤ 𝜇m2 I, and 𝜇o1 I ≤ MS ≤ 𝜇o2 I, where I is an n×n identity matrix.
Property 2. A part of the nonlinear system dynamics depend linearly on dynamic parameter vectors 𝜑m and 𝜑o such
that,
Mc (qm )̈xm,r + Cc (qm , q̇ m )ẋ m,r + Gc (qm ) = 𝜑m (qm , q̇ m , ẋ m,r , ẍ m,r )T 𝜃m , (23)

MS (xo , qsi )̈xo,r + CS (xo , ẋ o , qsi , q̇ si )ẋ o,r + GS (xo , qsi ) = 𝜑o (xo , ẋ o , qsi , q̇ si , ẋ o,r , ẍ o,r )T 𝜃o , (24)

where 𝜃m and 𝜃o are the unknown parameters of the master, slaves, and the object and 𝜑m and 𝜑o are the regressor
matrices. The reference velocities and accelerations are represented by ẋ m,r , ẍ m,r , ẋ o,r , and ẍ o,r .
Property 3. Under a proper definition of the matrices, the matrices Ṁ c − 2Cc , Ṁ S − 2CS are skew symmetric.
Assumption 3. If we define the estimate and estimation error of the parameters as 𝜃̂ and 𝜃̃ where 𝜃̃ = 𝜃̂ − 𝜃, then we
assume that the extent of the uncertainty in parameters is known such that,

Δ
𝜃 ∈ Ω𝜃 = {𝜃 ∶ 𝜃min ≤ 𝜃 ≤ 𝜃max }, (25)

where 𝜃min and 𝜃max are known constant vectors or scalars.

Assumption 4. The signals F hc and F e are measurable and the position and velocities of the master and slaves are also
available.
8 AHMAD et al.

FIGURE 2 Control architecture of SMMS teleoperation system

The control objectives of this work include:

Stability: To synthesize the input such that the robust stability is achieved in the presence of communication delays.
Synchronization: To make the object trajectory follow the desired trajectory precisely minimizing the tracking error.
Force distribution: To control the internal force of the slaves to have an optimized distribution of the slave inputs.

4 CO N T RO L ARCHIT ECT U R E

This work proposes a novel structure of the communication channel for teleoperation where the slave side receives the
signal xm and the master side receives the environment parameters 𝜃e , which are estimated online. Unlike traditional
teleoperation systems, the signal we receive at master side contain estimated values of the environmental parameters
instead of the force signal F e , which naturally avoids the passivity problem of the communication channel. At the master
side, a force predictor F̂ e is designed to predict the environmental force using the estimated environmental parameters
𝜃̂ e and the signals xm and ẋ m . The human input signal Fhc is used to generate the desired master trajectories xm , ẋ m , and
ẍ m . The signal received at the slave side xm is optimally processed to generate the desired object trajectory xod for the
end-effector of the slave control, and a reference velocity signal is also generated for the mobile base of the slaves. An
adaptive robust slave controller uS is designed to minimize the tracking error xo − xod . Finally, the internal force control
is implemented to obtain the slave control inputs. A general schematic of the control architecture is shown in Figure 2.

4.1 Parameter estimation

All the unknown parameters 𝜃e , 𝜃o , and 𝜃m are estimated using a projection-type adaptation law.22 The parameter 𝜃̂ is
estimated as,

𝜃̂̇ = P𝜃̂ (Γ𝛾), 𝜃̂ ∈ Ω𝜃 , (26)

where Γ is a positive definite matrix and 𝛾 is an adaptation parameter.

⎧ • ̈ or nT • ≤ 0,
if 𝜃̂ ∈ Ω
⎪( ) 𝜃̂
P𝜃̂ (•) = ⎨ T
n𝜃̂ n ̂ (27)
𝜃 ̂
⎪ I − Γ nT̂ ΓnT̂ • 𝜃 ∈ 𝛿Ω𝜃 and n𝜃̂ • > 0.
T
⎩ 𝜃 𝜃
AHMAD et al. 9

where Ω ̈ and 𝛿Ω𝜃 are the interior and the boundary of Ω𝜃 and nT denotes the unit normal vector at 𝜃̂ ∈ 𝛿Ω𝜃 . This
𝜃̂
projection-type adaptation law has the following properties.
̂ ∈ Ω𝜃 , ∀t. This means
Property 4. The parameter estimates are always within the known bounded set Ω𝜃 , that is, 𝜃(t)
̂
that from Assumption 4, 𝜃min ≤ 𝜃(t) ≤ 𝜃max , ∀t.
Property 5. It holds that,
𝜃̃ (Γ−1 P𝜃̂ (Γ𝛾) − 𝛾) ≤ 0,
T
∀𝛾. (28)

1
For the estimation of 𝜃e , a filter Hf (s) = (𝛾f s+1)2
is used on both sides of (4).

Fef = 𝜑Tef 𝜃e , (29)

where F ef and 𝜑ef are the filtered values of F e and 𝜑e . If we define the prediction output F̂ ef = 𝜑Tef 𝜃̂ e , then the prediction
error can be defined as,
𝜖 = F̂ ef − Fef = 𝜑Tef 𝜃̃ e . (30)

Now 𝜃̂̇ e can be designed as,

𝜃̂̇ e = P𝜃̂ (Γe 𝛾e ), (31)

⎧ Γe 𝜑ef 𝜑T Γe
⎪𝛼Γe − 1+𝜈𝜑T Γef 𝜑 if 𝜆max (Γe (t)) ≤ 𝜁M ,
Γ̇ e = ⎨ ef e ef (32)
⎪ 0 otherwise.
⎩
𝜑ef 𝜖
𝛾e = , (33)
1 + 𝜈𝜑Tef Γe 𝜑ef

where 𝛼 is the forgetting factor, 𝜈 ≥ 0 and 𝜁M is the bound on ||Γe (t)||.

Lemma 1. The parameter estimation 𝜃̂ e always stay within a known bound when the least square estimation algorithm
(31) is used: 𝜃emin ≤ 𝜃̂ e (t) ≤ 𝜃emax , ∀t. In addition, when there are only uncertainties in the parameters, the persistent excitation
(PE) condition given below is satisfied,22

(t+T)
𝜑ef 𝜑Tef d𝜏 ≥ 𝜉I, ∀t > t0 for some T, 𝜉 > 0, (34)
∫t

then 𝜃̂ e converges to its true value 𝜃e .

4.2 Controller design for the master side

This section presents a novel adaptive and robust controller design for the master manipulator. As the estimates of
the environmental parameters are transmitted over the communication channel, the master side receives 𝜃̂ e (t − T) and
predicts the force F̂ e as,

F̂ e = 𝜃̂ e (t − T(t))𝜑e (km Φo (xm ), km Li ẋ m ).

T
(35)

The master manipulator is required to follow a desired trajectory xmdt , which is designed as follows,

Md (xmd )̈xmd + Cd (xmd , ẋ md )ẋ md + Gd (xmd ) = 𝜑m (xmd , ẋ md , ẍ md )T 𝜃d = L−T ̂ ̇

i Fhc − kf 𝜃 e (t − T(t))𝜑e (km xmd , km x md ), (36)

xmdt = Φ−1
o (xmd ), (37)
10 AHMAD et al.

where xmd is the virtual object trajectory, kf > 0 is the force scaling factor, and 𝜃d is the target parameter vector of the
impedance behavior, which is chosen to satisfy xmd . The objective to design the controller is to ensure that xm tracks the
desired trajectory xmdt precisely.
Define sm as follows:
sm = ė m + km1 em , em = xm − xmdt , (38)

where km1 > 0 is a diagonal matrix. Differentiate (38) and note (2),

Mc ṡ m + Cc sm = umc + Fhc − 𝜑m (qm , q̇ m , ẋ m,r , ẍ m,r )T 𝜃m , (39)

Now, the controller can be designed as follows:

umc = 𝜑m (qm , q̇ m , ẋ m,r , ẍ m,r )T 𝜃̂ m − Fhc + uf − Kp sm

𝜃̂̇ m = P𝜃̂ m (Γm 𝛾m ), 𝛾m = 𝜑m (qm , q̇ m , ẋ m,r , ẍ m,r )sm , (40)

where uf − K p sm is the robust control law, the term 𝜑m (qm , q̇ m , ẋ m,r , ẍ m,r )T 𝜃̂ m − Fhc is the model compensation for tracking,
K p is chosen to be a symmetric positive definite matrix, and uf will be designed later. Substituting (40) into (39) and
rearranging the terms,
Mc ṡ m + Cc sm = −Kp sm + uf + 𝜑m (qm , q̇ m , ẋ m,r , ẍ m,r )T 𝜃̃ m . (41)

Now, uf can be designed to satisfy the conditions below.

(a) sTm (uf + 𝜑m (qm , q̇ m , ẋ m,r , ẍ m,r )T 𝜃̃ m ) ≤ 𝜎m ,

(b) sTm uf ≤ 0, (42)

where 𝜎m > 0 is a small design parameter.

Theorem 1. For a desired trajectory, the controller in (40) guarantees that all the signals of the master side are bounded.

Proof. Choose a Lyapunov function,

1 T
Vm = s M c sm . (43)
2 m

The derivative of V m is:

1
V̇ m = sTm Mc ṡ m + sTm Ṁ c sm = sTm (Mc ṡ m + Cc sm ) = −sTm Kp sm + sTm (uf + 𝜑Tm 𝜃̃ m ) ≤ −𝜆m Vm + 𝜎m . (44)
2

Taking integral on both sides of (44),

𝜎m
Vm ≤ e−𝜆m t Vm (0) + [1 − e−𝜆m t ]. (45)
𝜆m

where 𝜆m = 2𝜎min (Kp )∕𝜇m2 and 𝜎min (•) is the minimum eigenvalue of a matrix. Thus, sm is bounded and em and ė m are
also bounded from (38). Using the adaptation law (26), 𝜃̂ m is bounded thus the control input umc is bounded. As xmdt , ẋ mdt ,
and ẍ mdt are bounded, xm and ẋ m are also bounded. ▪

4.3 Controller design for the slave side

The controller design for the slaves can be divided into two parts, where the first controller deals with the control of
the mobile platform and the second one deals with the trajectory tracking of the end-effectors. We take advantage of
the decoupling of the operational and null space of the mobile manipulators to design the controllers.23 The motion of
the mobile manipulators is decoupled in the operational space (end-effector) and the null space (mobile platform). This
AHMAD et al. 11

decoupling of the operational and null space allows the prioritization of the tasks in the event of a disagreement between
the operational and null space. Once the null space controller is designed, an adaptive robust controller (ARC) will be
designed for the end-effectors of the slaves for trajectory tracking.

4.3.1 Decoupling of null and operational space

Assuming that the operational space consists of the x-y coordinates of the end-effector, from (19) we have the general
solution of the velocity as,
𝜂̇ i = J ⋆ (𝜂i )Ẋ ma + Ni 𝜂̇ i , (46)

where J ⋆ (𝜂i ) is the pseudoinverse of J(𝜂i ) and Ni = I − J ⋆ (𝜂i )J(𝜂i ) is the null space. Differentiating (46),

⋆
𝜂̈ i = J ⋆ (𝜂i )Ẍ ma + J̇ (𝜂i )Ẋ ma + Ni 𝜂̈ i + Ṅ i 𝜂̇ i . (47)

A weighted pseudoinverse can be defined as,

J ◦ = M𝜂−1
i
J T (JM𝜂−1
i
J T )−1 . (48)

Now, the input torque and the operational space force relation can be given as,

𝜏𝜂i = J T fi + NiT 𝜏ji (49)

where f i is the operational space force and 𝜏ji is the vector of joint torques. Under this decoupling, the acceleration in the
operational space is not affected by the torque in the null space. Considering J ⋆ = J ◦ , the null space can be rewritten as
Ni◦ = I − J ◦ J.
Now by defining f ei as the external forces acting on the mobile manipulator, (16) can be rewritten as,

M𝜂i (qi )𝜂̈ i + C𝜂i (qi , q̇ i )𝜂̇ i + G𝜂i (qi ) = 𝜏𝜂i + fei . (50)

Using (46), (47) along with I = J T (𝜂i )J ⋆T (𝜂i ) + Ni , (50) becomes,

J T fi + NiT 𝜏ji = T1 + T2 + T3 , (51)

where T 1 ,T 2 ,T 3 are the terms defined as,23

T1 = J T J ◦T (M𝜂i (qi )J ◦ (Ẍ ma − J̇ 𝜂̇ i ) + C𝜂i (qi , q̇ i )𝜂̇ i + G𝜂i (qi ) − fei ),

where T 1 contains all the operational space forces.

T2 = NiT (M𝜂i (qi )Ni 𝜂̈i + (C𝜂i (qi , q̇ i )𝜂̇ i + G𝜂i (qi )) − fei ),

T 2 contains all the null space torques.

T3 = (J T J ◦T M𝜂i (qi )Ni 𝜂̈i + NiT M𝜂i (qi )J ◦ (Ẍ − J̇ 𝜂̇ i )),

and T 3 contains all the coupling torques and forces.

The decoupling of the internal and end-effector motion can be obtained by making T 3 = 0 so that J ◦T M𝜂i (qi )Ni 𝜂̈i = 0
and NiT M𝜂i (qi )J ◦ = 0.24 As J ◦ in (48) is dynamically consistent so the above equations are always satisfied. The substitution
of (46) and (47) into (50) with the premultiplication of J ◦T and using (48), the decoupling of (21) can be obtained as,

Mx◦i (qi )Ẍ ma + Cx◦i (qi , q̇ i )Ẋ ma + G◦xi (qi ) = Fi◦ + Fe◦ , (52)
12 AHMAD et al.

where Mx◦i (qi ) = J ◦T M𝜂i (qi )J ◦ , Cx◦i (qi , q̇ i )Ẋ = J ◦T C𝜂i (qi , q̇ i )𝜂̇ i − Mx◦i (qi )J̇ 𝜂̇ i , G◦xi (qi ) = J ◦T G𝜂i (qi ), Fi◦ = J ◦T 𝜏𝜂i , and Fe◦ = J ◦T fei .
The control input to decouple the operational and null space now can be designed as,

◦
𝜏𝜂i = J T Mx◦i (qi )(uOi − J̇ 𝜂̇ i ) + C𝜂i (qi , q̇ i )𝜂̇ i + J T Fe◦ + Ni◦T M𝜂i (uNi + J̇ Ẋ ma ) + G𝜂i (qi ), (53)

where uOi = uS and uNi are the operational and null space control inputs, respectively. The substitution of (53) into (52)
with the premultiplication of the controller with JM𝜂−1
i
(qi ) gives the operational space closed loop dynamics as,

uOi = Ẍ ma . (54)

Now, by premultiplying (52) with NM𝜂−1

i
(qi ) for 𝜏𝜂i , the null space can be given as,
T
−N 𝜂̈i + NuNi + N J̇◦ Ẋ ma = −NM𝜂−1
i
(qi )N fei . (55)

T T
From this we have that N fei = N J T Fe◦ = (JN)Fe◦ = 0.17 Hence, (55) can be rewritten as,
◦
𝜂̈i = J̇ Ẋ ma + uNi . (56)

As we have achieved the decoupled dynamics in (54) and (56), the distributed coordination control of the slaves can
easily be designed.

4.3.2 Null space control

The additional objective of keeping a formation of the mobile manipulators can be achieved in the null space by control-
ling the position of the mobile platform. The input command for the mobile platform wheels and the arm joints can be
considered as the null space controller in (56),
uNi = [𝜏pi 𝜏ai ]T . (57)

It should be noted that the end-effector controller will be designed later so we make 𝜏ai = 0. As in Reference 25, the
equations of motion of mobile platforms with the kinematic constraints take the form,

M𝜂pi 𝜂̈ pi + C𝜂pi 𝜂̇ pi = B𝜂pi 𝜏pi , (58)

T
where M𝜂pi = Hpi T
Mpi Hpi , C𝜂pi = Hpi (Cpi Hpi + Mpi Ḣ pi , and B𝜂pi = Hpi
T
Bpi . Now 𝜏pi can be designed as,

𝜏pi = B−1
𝜂pi M𝜂pi u𝜂pi + B𝜂pi C𝜂pi 𝜂̇ pi ,
−1
(59)

where u𝜂pi = Jpi−1 (upi − J̇ pi 𝜂̇ i ) is defined as the change in control input for the mobile platform in the null space. It should
be noted that the subscripts mp and pi are being used interchangeably in this work for the mobile platform. Now upi can
be designed as,
upi = Ẍ pi . (60)

The mobile platform can be controlled to track a desired trajectory Xpid with the controller upi = Ẍ pi + kpv (Ẋ pi −
d d

Ẋ pi ) + kbp (Xpid − Xpi ), where kpv and kbp are positive control gains. The desired trajectory Xpid is generated from the signal
xm (t−T(t)) by applying a similar filter to generate xod . The error dynamics ëpi = −kpv ė pi − kbp e − pi when epi = Xpi − Xpid
are stable and the mobile base can track Xpid without any effect on the task space control. As we consider different scenar-
ios for the formation control and consensus of mobile manipulators, one of the scenarios is to have the velocity consensus
of the agents. For this case, the control input can be defined as,
∑ wij
upi = [Ẋ pj (t − Td ) − Ẋ pi ] = upiv , (61)
j∈i
di
AHMAD et al. 13

where N i is the set of all the agents and T d is the delay in the communication topology.
Theorem 2. Consider the null space dynamics in (56) and the mobile platform dynamics in (58) with the control inputs
(57) and (59). The controller (61) guarantees the velocity consensus of the mobile manipulators.

Proof. Consider a positive semidefinite functional candidate as,

[ ]
∑ 1 ̇T ̇ 1
t
Ẋ pi (𝜎)Ẋ pi (𝜎)d𝜎 ,
T
V= X pi X pi + wij (62)
i∈Nj ,Nn
2 2 ∫t−Td

where N j and N n are the sets of the connected and disconnected agents in the communication topology. Taking time
derivative of V along (60) and (61), we get
∑ 1
V̇ = [Ẋ pi upiv + wij (Ẋ pi (t)Ẋ pi (t) − Ẋ pi (t − Td )T Ẋ pi (t − Td )].
T T
(63)
i∈Nj ,Nn
2

By following the proof in the work26 and using (61), we get

𝜆 ∑∑ ̇
V̇ Nj = − ||X pj (t − Td ) − Ẋ pi (t)||2 ≤ 0.
2di i∈N j∈N
j n

Hence, V̇ = V̇ Nj ≤ 0. Again following the work,26 we conclude that all the mobile platform velocities match such that the
limt→∞ (Ẋ pi (t − Td ) − Ẋ pi (t)) = 0. ▪

4.3.3 ARC for slaves

According to the proposed control architecture, the slave side receives the delayed position signal xm (t−T(t)) of the mas-
ter. The desired object trajectory xod can be obtained by an optimal processing on the master position signal. A filter,
1
Hr (s) = (𝜏 s+1) 2
can be used with xod = Hr (s)[km Φo (xm (t − T(t)))], which can simultaneously provide the desired velocity
r
and acceleration of the object. Define ss as follows:

ss = ė s + ks1 es , es = xo − xod , (64)

where ks1 > 0 is a diagonal matrix. Now differentiating (64) and noting (22),

MS ṡ s + CS ss = uS − Fe − 𝜑o (xo , ẋ o , qsi , q̇ si , ẋ o,r , ẍ o,r )T 𝜃o , (65)

where ẋ o,r = ẋ od − ks1 es and ẍ o,r = ẍ od − ks1 ė s . Now the slave controller is designed as follows:

uS = −Kd ss + ur + 𝜑o (xo , ẋ o , qsi , q̇ si , ẋ o,r , ẍ o,r )T 𝜃̂ o + Fe

𝜃̂̇ o = P𝜃̂ o (Γo 𝛾o ), 𝛾o = −𝜑o (xo , ẋ o , qsi , q̇ si , ẋ o,r , ẍ o,r )ss , (66)

where −K d ss + ur is the robust control law, the term 𝜑o (xo , ẋ o , qsi , q̇ si , ẋ o,r , ẍ o,r )T 𝜃̂ o + Fe is the model compensation for track-
ing, K d is chosen to be a symmetric positive definite matrix, and ur will be designed later. Substituting (66) into (65) and
rearranging the terms,
MS ṡ s + CS ss = −Kd ss + ur + 𝜑o (xo , ẋ o , qsi , q̇ si , ẋ o,r , ẍ o,r )T 𝜃̃ o . (67)

Now, ur can be designed to satisfy the conditions below.

( )
(a) sTs ur + 𝜑o (xo , ẋ o , qsi , q̇ si , ẋ o,r , ẍ o,r )T 𝜃̃ o ≤ 𝜎s ,
(b) sTs ur ≤ 0, (68)
14 AHMAD et al.

where 𝜎s > 0 is a small design parameter.

Theorem 3. The controller (66) can guarantee that all the slave side signals are bounded if the received signal xm (t−T(t))
is bounded.

Proof. Choose a Lyapunov function,

1 T
Vs = ss M S ss . (69)
2

The derivative of V s is:

1
V̇ s = sTs MS ṡ s + sTs Ṁ S ss = sTs (MS ṡ s + CS ss ) = −sTs Kd ss + sTs (ur + 𝜑To 𝜃̃ o ) ≤ −𝜆s Vs + 𝜎s . (70)
2

Taking integral on both sides of (70),

𝜎s
Vs ≤ e−𝜆s t Vs (0) + [1 − e−𝜆s t ], (71)
𝜆s

where 𝜆s = 2𝜎min (Kd )∕𝜇o2 and 𝜎min (•) is the minimum eigenvalue of a matrix. Thus, ss is bounded and es and ė s are also
bounded from (64). Using the adaptation law (26), 𝜃̂ o is bounded thus the control input uS is bounded. As xm (t−T(t)) is
bounded and H r (s) is stable then the trajectories xod , ẋ od , and ẍ od are bounded, xo and ẋ o are also bounded. ▪
∑n
There exists the redundancy of the slave controller (66) with uS = i=1 A−T i
usi , where usi are the input to the slaves.
The inputs usi with the internal force distribution are designed as follows:19

[us1 … usn ] = A‡s uS + Fint , (72)

where A‡s = Q−1 ATs [As Q−1 ATs ], As = [A−T −T

1 … An ], and Q is a weighting matrix, which is used to optimize the norm of the
control inputs. The internal force F int is chosen as As F int = 0 so that it has no effect on the motion of the object. Another
possible selection of F int can be as follows:

Fint = [AT1 − AT2 AT3 … ATn−1 − ATn ]fd , (73)

where f d is an arbitrary predefined force vector.

Remark 1. Theorem 1 guarantees the stability of the master side without being dependent on the stability of the slave side.
Theorems 2 and 3 guarantee the stability of the slave side when the signal transmitted from the master side is bounded.
Remark 2. Unlike the conventional teleoperation, the master side receives the estimated values of the environmental
parameters and the environmental force is predicted. When we have 𝜃̂ e → 𝜃e in Lemma 1 and tracking performance
of xo (t) → [km Φo (xm (t − T(t)))], then we achieve F̂ e ↔ Fe (t + T(t)), which provides the human operator the feeling of
operation without any delay.
Remark 3. Theorem 2 guarantees that given a reference velocity, the mobile platforms of the manipulators achieve the
velocity consensus. Theorem 3 guarantees that es = xo − xod will be converging to the known bound when the param-
eters are unknown. An optimized distribution of the control input for slaves can be achieved with internal force
control (72).

5 S I M UL AT ION R E SU LT S

In the simulation, a 2-DOF master robotic arm is operated by the human operator. The position signal of the master
manipulator is sent over the communication channel to control three mobile manipulators acting as slaves to han-
dle an object. The parameters of the 2-DOF mobile manipulators are given in Table 1. The parameter values of the
master manipulator are m1 = 1 kg, m2 = 1 kg, I1 = 0.2 kg m2 , I2 = 0.2 kg m2 , lci = 0.05 m, lc2 = 0.05 m, L1 = 0.1 m, L2 =
0.1 m. The parameters of the object are M o = diag{5,1},Co = {0,0},Go = {0,5 * 9.8}. The kinematic chain between the
object and the slave manipulators is given by xo = x1 = diag(−1,1,−1) × x2 . The parameters in (4) are chosen to be
AHMAD et al. 15

T A B L E 1 Simulation parameters
Value Value

Mass of platform 32 kg Mass of link 1 3 kg

Mass of link 2 3 kg Inertia of platform 1 kg m2
Inertia of link 1 1 kg m2 Inertia of link 2 1 kg m2
Radius of wheel 0.5 m Length of link 1 1.5 m
Length of link 2 1.5 m Gravitational force 9.8 m s−2

FIGURE 3 Graph topology 1

Be = [50,−30]T ,K e = [50,10]T , and Ce = [0,0]T . For the control design, the projection-type estimation algorithm param-
eters are 𝛼 = 0.02, 𝜁m = 5000, and 𝜈 = 0.1. The initial estimates and the lower and upper bounds of the parame-
ter variations are chosen to be 𝜃e (0) = [80, 0, 80, 3, 0, 0]T , 𝜃emin = [0, −40, 0, 0, 0, 0]T , and 𝜃emax = [100,100, 80, 80, 0, 0]T .
The filter H f (s) is chosen to have 𝛾f = 0.005. The scaling gains are km = 10 and kf = 0.1, respectively. The weight-
ing matrix for the internal force control is Q=diag{1,1} and f d = 0. The slave gains are ks1 = diag{50,150} and
K d = diag{250,375}. The filter H r (s) is chosen to have 𝜏r = 0.005 in order to generate the desired object trajectory.
The target parameter of the master side is chosen as 𝜃d = [2, 0.4]T , which means that M d = diag{2,2},Cd = diag{0,0}
and Gd = [0,2 * 9.8]T . The master parameters are Γm = diag{2, 2, 4, 4}, km1 = diag{150,150}, and K p = diag{75,37.5}. The
gains for the mobile base are chosen to be kpv = 10 and kbp = 14. The time delays in the simulation was tested
up to 1 second.
The simulation considers the teleoperation of a single-master multiple slave system where the slaves are connected
with a communication topology 1 as in Figure 3, where the Laplacian matrix is given by  = [1 0 − 1; −1 1 0; 0 − 1 1].
The in-degree di and the weights wij are chosen to be 1. The agents are strongly connected over a topology and delays
T d are 0.2 second. The main objective is to transport the object to a desired position following a desired trajectory while
the end-effectors grasp the object rigidly. The communication delay between the master and slaves is set to be 0.5 second.
The human inputs are simulated as Fh (1) = 0.5 sin(𝜋t) and Fh (2) = sin(𝜋t).
The desired and actual master trajectories are shown in Figure 4A-D and their corresponding errors are shown
in Figure 4E,F. It can be seen that the tracking errors are very small confirming that the desired trajectory is
perfectly followed by the master. In the master side, the environmental force is predicted based on the esti-
mated environmental parameters. The actual and predicted environmental forces are presented in Figure 5A-D.
It should be noted that although the actual environmental forces are delayed due to communication channel
delay, the human operator is still able to feel the environmental force without any delay. This means that the
proposed controller design not only ensures the stability of the system but also provides good realism in the
teleoperation system.
The estimated environmental parameters of (4) are shown in Figure 6A-D, respectively. It can be observed that
the estimation starts with the initial estimates and converges to the true values of the parameters. This also ensures
that the adaptation laws are working perfectly and the environmental force can be predicted with precision. The esti-
mated values of Ce stay at zero. Under the assumption of flexible contact model, the environment reaction force is
modeled as in (4), where the parameters can be estimated using the regressor signal of the object. There exists a
kinematic chain between the object and the end-effectors, which poses constraints on the object motion. The kine-
matic chain results in constrained movement of the object, thus affecting the convergence of the environmental
estimated parameters to their true values. This can be observed in Figure 6A that the convergence is a bit irregular
in that case.
The desired object trajectory and its errors are shown in Figure 7A-D. It is clear from the results that the object closely
follows the desired trajectory and the task accomplishment of the transportation of the object can be achieved as desired.
The forces exerted on the object by the slave end-effectors are presented in Figure 8E-F, respectively. It can be observed
16 AHMAD et al.

0.25 0.06 0.25

0.04
0.2 0.2
x md(1) (m)

x md(2) (m)

x m(1) (m)
0.02
0.15 0.15
0
0.1 0.1
-0.02

0.05 -0.04 0.05

0 5 10 15 0 5 10 15 0 5 10 15
Time (sec) Time (sec) Time (sec)

(A) (B) (C)

0.06 10-5 10-6

1
5
0.04
0.5
x m(2) (m)

em(1) (m)

em(2) (m)
0.02
0 0
0

-0.5
-0.02
-5
-0.04 -1
0 5 10 15 0 5 10 15 0 5 10 15
Time (sec) Time (sec) Time (sec)

(D) (E) (F)

FIGURE 4 Master trajectories: desired, actual and their error [Colour figure can be viewed at wileyonlinelibrary.com]

2 1 0 1

0 0.5 0.5
-2
F e(1) (N)

F e(2) (N)

F pe(1) (N)

F pe(2) (N)

-2 0 0
-4
-4 -0.5 -0.5
-6
-6 -1 -1

-8 -1.5 -8 -1.5
0 5 10 15 0 5 10 15 0 5 10 15 0 5 10 15
Time (sec) Time (sec) Time (sec) Time (sec)

(A) (B) (C) (D)

FIGURE 5 The contact force F e and its prediction F̂ e [Colour figure can be viewed at wileyonlinelibrary.com]

150 0 80 10

-10 70 6
Be(1)

Be(2)

Ke(1)

Ke(2)

100 4

-20 60 2

50 -30 50 -2
0 5 10 15 0 5 10 15 0 5 10 15 0 5 10 15
Time (sec) Time (sec) Time (sec) Time (sec)

(A) (B) (C) (D)

FIGURE 6 The estimates of environmental parameters 𝜃̂ e [Colour figure can be viewed at wileyonlinelibrary.com]
AHMAD et al. 17

-3 -3
10 10
1.5 1
0.5
0.1
1 0.5
x od(1) (m)

x od(2) (m)

es(1) (m)

es(2) (m)
0.5 0
0 0
0 -0.5

-0.1 -0.5 -1
-0.5
-1 -1.5
0 5 10 15 0 5 10 15 0 5 10 15 0 5 10 15
Time (sec) Time (sec) Time (sec) Time (sec)

(A) (B) (C) (D)

FIGURE 7 The desired object trajectory xod and its error [Colour figure can be viewed at wileyonlinelibrary.com]

400 800 300

200 600 200

400 100
fs1(1) (N)

fs1(2) (N)

fs2(1) (N)
0
200 0
-200
0 -100
-400 -200 -200

-600 -400 -300

0 5 10 15 0 5 10 15 0 5 10 15
Time (sec) Time (sec) Time (sec)
(A) (B) (C)

300 600 400

200 400

100 200 200

fs2(2) (N)

fs3(1) (N)

fs3(2) (N)

0 0

-100 -200 0

-200 -400

-300 -600 -200

0 5 10 15 0 5 10 15 0 5 10 15
Time (sec) Time (sec) Time (sec)
(D) (E) (F)

FIGURE 8 The forces exerted on the object [Colour figure can be viewed at wileyonlinelibrary.com]

5 4 4
X position X position X position
4 Y position Y position Y position
Tracking error (m)

Tracking error (m)

Tracking error (m)

3 3
3
2 2
2
1 1
1

0 0 0

-1 -1 -1
0 5 10 15 0 5 10 15 0 5 10 15
Time (sec) Time (sec) Time (sec)
(A) (B) (C)

FIGURE 9 Mobile platform position errors [Colour figure can be viewed at wileyonlinelibrary.com]
18 AHMAD et al.

from the object trajectory that the slaves stay still for few seconds in the start of the simulation and then start moving
the object to follow the desired trajectory. This affects the forces exerted on the object by the slaves. As the object starts
moving, the forces are being exerted on the object by the end-effectors. It should also be noted that the practical internal
force of the object can essentially be just the difference of all the forces exerted on the object. It is clearly seen that the
forces exerted on the object are almost of the same level and they operate on the object synchronously. This allows the
optimized distribution of internal forces avoiding any pull and drag effect of the manipulators. The tracking errors of the
positions of the mobile platforms of the slaves are shown in Figure 9A-C. The mobile platforms regulate their position
during the task accomplishment to achieve the prioritized mission of the object transportation. The results of mobile
platforms show some errors but as the platforms are only controlled in the null space so these errors are acceptable.

6 CO N C LUSION S

We proposed a novel control scheme for the teleoperation of single-master multiple slave system where the master
was considered to be a robotic manipulator, whereas the slaves were the mobile manipulators. A human operator
operated the master robot to remotely control the slaves to handle a target object. A desired master trajectory was
generated at the master side so the master tracked that trajectory and the master position signal was transmitted
to the slave side over the communication channel. An optimal processing was carried out on the position signal
of the master to generate the desired object trajectory and the reference velocity signal for the mobile platforms. A
robust adaptive controller and a null space controller were designed for the slaves and an internal force distribu-
tion was implemented. The slaves interacted with the target object and exerted forces on it. The force feedback signal
of traditional teleoperation was replaced by the estimated environmental parameters to avoid the passivity problem
of the communication channel. The environment force was effectively predicted using the estimated environmental
parameters. The control design achieved the control objectives of stability, synchronization, and force distribution.
The simulation results of one master three slaves teleoperation system validated the effectiveness of the proposed
scheme.

ACKNOWLEDGEMENT
This work was supported by Natural Sciences and Engineering Research Council (NSERC), Canada, and Nova Scotia
Provincial Government, Canada.

ORCID
Henghua Shen https://orcid.org/0000-0001-5330-4735

REFERENCES
1. Siciliano B, Khatib O. Springer Handbook of Robotics. New York, NY: Springer; 2016.
2. Li Z, Ge-Shuzhi S. Fundamentals in Modeling and Control of Mobile Manipulators. Boca Raton: CRC Press; 2013.
3. Palafox OM, Spong MW. Bilateral Teleoperation of a Formation of Nonholonomic Mobile Robots Under Constant Time Delay. IEEE;
2009:2821-2826.
4. Hernandez T, Nuño E, Alanis AY. Teleoperation of Mobile Manipulators with Non-Holonomic Restrictions. IEEE; 2016:1-6.
5. Li Z, Su C-Y. Neural-adaptive control of single-master–multiple-slaves teleoperation for coordinated multiple mobile manipulators with
time-varying communication delays and input uncertainties. IEEE Trans Neural Netw Learn Syst. 2013;24(9):1400-1413.
6. Farkhatdinov I, Ryu J-H, Poduraev J. A Feasibility Study of Time-Domain Passivity Approach for Bilateral Teleoperation of Mobile
Manipulator. IEEE; 2008:272-277.
7. Farkhatdinov I, Ryu J-H. Switching of Control Signals in Teleoperation Systems: Formalization and Application. IEEE; 2008:353-358.
8. Lasnier A, Murakami T. Hybrid Sensorless Bilateral Teleoperation of Two-Wheel Mobile Manipulator with Underactuated Joint. IEEE;
2010:347-352.
9. Santiago DD, Slawiñski E, Mut VA. Stable delayed bilateral teleoperation of mobile manipulators. Asian J Control. 2017;19(3):1140-1152.
10. Slawinski E, Mut VA, Fiorini P, Salinas LR. Quantitative absolute transparency for bilateral teleoperation of mobile robots. IEEE Trans
Syst Man Cybern Part A Syst Humans. 2012;42(2):430-442.
11. Lu Z, Huang P, Liu Z. Predictive approach for sensorless bimanual teleoperation under random time delays with adaptive fuzzy control.
IEEE Trans Ind Electron. 2017;65(3):2439-2448.
12. Lu Z, Huang P, Liu Z. Relative impedance-based internal force control for bimanual robot teleoperation with varying time delay. IEEE
Trans Ind Electron. 2019;67(1):778-789.
AHMAD et al. 19

13. Cheng R, Huang P, Lu Z. Adaptive Teleoperation Control Method Based on RBF-Neural Networks and Performance Analysis. IEEE;
2017:493-498.
14. Lu Z, Huang P, Liu Z, Chen H. Fuzzy-observer-based hybrid force/position control design for a multiple-sampling-rate bimanual
teleoperation system. IEEE Trans Fuzzy Syst. 2018;27(7):1383-1396.
15. Godsil C, Royle GF. Algebraic Graph Theory. New York: Springer Science & Business Media; 2013.
16. Liu Y-C, Chopra N. Synchronization of networked mechanical systems with communication delays and human input. J Dyn Syst Measur
Control. 2013;135(4):041004.
17. Liu Y-C, Chopra N. Controlled synchronization of heterogeneous robotic manipulators in the task space. IEEE Trans Robot.
2012;28(1):268-275.
18. Chen Z, Pan Y-J, Gu J. Integrated adaptive robust control for multilateral teleoperation systems under arbitrary time delays. Int J Robust
Nonlinear Control. 2016;26(12):2708-2728.
19. Sirouspour S, Setoodeh P. Multi-Operator/Multi-Robot Teleoperation: An Adaptive Nonlinear Control Approach. IEEE; 2005:1576-1581.
20. Boukattaya M, Jallouli M, Damak T. On trajectory tracking control for nonholonomic mobile manipulators with dynamic uncertainties
and external torque disturbances. Robot Autonomous Syst. 2012;60(12):1640-1647.
21. Li Z, Li J, Kang Y. Adaptive robust coordinated control of multiple mobile manipulators interacting with rigid environments. Automatica.
2010;46(12):2028-2034.
22. Yao B. Integrated Direct/Indirect Adaptive Robust Control of SISO Nonlinear Systems in Semi-Strict Feedback Form. IEEE; 2003:3020-3025.
23. White GD, Bhatt RM, Krovi VN. Dynamic redundancy resolution in a nonholonomic wheeled mobile manipulator. Robotica.
2007;25(2):147-156.
24. Nemec B, Zlajpah L. Force control of redundant robots in unstructured environment. IEEE Trans Ind Electr. 2002;49(1):233-240.
25. White GD, Bhatt RM, Tang CP, Krovi VN. Experimental evaluation of dynamic redundancy resolution in a nonholonomic wheeled mobile
manipulator. IEEE/ASME Trans Mechatron. 2009;14(3):349-357.
26. Liu Y-C. Distributed synchronization for heterogeneous robots with uncertain kinematics and dynamics under switching topologies.
J Frankl Inst. 2015;352(9):3808-3826.

How to cite this article: Ahmad U, Pan Y-J, Shen H. Robust control design for teleoperation of multiple mobile
manipulators under time delays. Int J Robust Nonlinear Control. 2020;1–19. https://doi.org/10.1002/rnc.5106