Hierarchical Control

In schizophrenia, disturbances of cognitive control have been associated with impaired functional specialization within the lateral prefrontal cortex (LPFC), but little is known about the functional interactions between specialized LPFC subregions. Here, we addressed this question with a recent model that describes the LPFC functioning as a cascade of control processes along a rostrocaudal axis, whereby anterior frontal regions influence the processing in posterior frontal regions to guide action selection on the basis of the temporal structure of information.

The search for the neural substrate of vertebrate action selection has focused on structures in the forebrain and midbrain, and particularly on the group of sub-cortical nuclei known as the basal ganglia. Yet, the behavioural repertoire of decerebrate and neonatal animals suggests the existence of a relatively self-contained neural substrate for action selection in the brainstem. We propose that the medial reticular formation (mRF) is the substrate's main component and review evidence showing that the mRF's inputs, outputs and intrinsic organization are consistent with the requirements of an action-selection system. The internal architecture of the mRF is composed of interconnected neuron clusters. We present an anatomical model which suggests that the mRF's intrinsic circuitry constitutes a small-world network and extend this result to show that it may have evolved to reduce axonal wiring. Potential configurations of action representation within the internal circuitry of the mRF are then assessed by computational modelling. We present new results demonstrating that each cluster's output is most likely to represent activation of a component action; thus, coactivation of a set of these clusters would lead to the coordinated behavioural response observed in the animal. Finally, we consider the potential integration of the basal ganglia and mRF substrates for selection and suggest that they may collectively form a layered/hierarchical control system.

Proponents of Int,elligent Vehicle/Highway System or IVHS see it as a new technology which will make a major change in highway transportation. Control, communication and computing technologies will be combined into an IVHS system that can significantly increase safety and highway capacity without new roads having to be built. This paper outlines key features of one highly aut,omated IVHS system, shows how core driver decisions are improvedt proposes a basic IVHS control system architecture, and offers a design of some control s~bsyst~ems. It also summarizes some experimental work. We hope that the paper will stimulate interest in IVHS among control engineers. 'This table applies to a typical commute trip. Additional decisions are involved in trips by drivers of commercial vehicles. ' A study commissioned by the European Prometheus program projects a decrease in travel time of 10 ?4 if all vehicles are equipped with a route guidance system.

This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers.

This paper considers the problem of pursuit evasion games (PEGs), where the objective of a group of pursuers is to chase and capture a group of evaders in minimum time with the aid of a sensor network. The main challenge in developing a real-time control system using sensor networks is the inconsistency in sensor measurements due to packet loss, communication delay, and false detections. We address this challenge by developing a real-time hierarchical control system, named LochNess, which decouples the estimation of evader states from the control of pursuers via multiple layers of data fusion. The multiple layers of data fusion convert noisy, inconsistent, and bursty sensor measurements into a consistent set of fused measurements. Three novel algorithms are developed for LochNess: multisensor fusion, hierarchical multitarget tracking, and multiagent coordination algorithms. The multisensor fusion algorithm converts correlated sensor measurements into position estimates, the hierarchical multitarget tracking algorithm based on Markov chain Monte Carlo data association (MCMCDA) tracks an unknown number of targets, and the multiagent coordination algorithm coordinates pursuers to chase and capture evaders using robust minimum-time control. The control system LochNess is evaluated in simulation and successfully demonstrated using a large-scale outdoor sensor network deployment

A wave of recent behavioral studies has generated a new wealth of parametric observations about serial order behavior. What was a trickle of neurophysiological studies has grown to a steady stream of probes of neural sites and mechanisms underlying sequential behavior. Moreover, simulation models of serial behavior generation have begun to open a channel to link cellular dynamics with cognitive and behavioral dynamics. Here we review major results from prominent sequence learning and performance tasks, namely immediate serial recall, typing, 2 · N, discrete sequence production, and serial reaction time. These tasks populate a continuum from higher to lower degrees of internal control of sequential organization and probe important contemporary issues such as the nature of working-memory representations for sequential behavior, and the development and role of chunks in hierarchical control. The main movement classes reviewed are speech and keypressing, both involving small amplitude movements amenable to parametric study. A synopsis of serial order models, vis-à-vis major 0167-9457/$ -see front matter Ó 2004 Published by Elsevier B.V.

DC and AC Microgrids are key elements to integrate renewable and distributed energy resources as well as distributed energy storage systems. In the last years, efforts toward the standardization of these Microgrids have been made. In this sense, this paper present the hierarchical control derived from ISA-95 and electrical dispatching standards to endow smartness and flexibility to microgrids. The hierarchical control proposed consist of three levels: i) the primary control is based on the droop method, including an output impedance virtual loop; ii) the secondary control allows restoring the deviations produced by the primary control; and iii) the tertiary control manage the power flow between the microgrid and the external electrical distribution system. Results from a hierarchicalcontrolled microgrid are provided to show the feasibility of the proposed approach.

Examines the contemporary discourse on organizational change and transformational leadership in the larger context of a shift from a dominator to a partnership model of social and ideological organization. Traces the historic tension between these ...

In this paper, a microgrid hierarchical control scheme is proposed which includes primary and secondary control levels. The primary level comprises distributed generators (DGs) local controllers. The local controller mainly consists of active and reactive power controllers, voltage and current controllers, and virtual impedance loop. A novel virtual impedance structure is proposed to achieve proper sharing of non-fundamental power among the microgrid DGs. The secondary level is designed to manage compensation of voltage harmonics at the microgrid load bus (LB) to which the sensitive loads may be connected. Also, restoration of LB voltage amplitude and microgrid frequency to the rated values is directed by the secondary level. These functions are achieved by sending proper control signals to the local controllers. The simulation results show the effectiveness of the proposed control scheme.

The concept of microgrid hierarchical control is presented, recently. In this paper, a hierarchical scheme which includes primary and secondary control levels is proposed for islanded microgrids. The primary control level consists of DG local controllers. Local controller of each DG comprises active and reactive power controllers, virtual impedance loop and voltage and current controllers. The secondary level is designed to compensate the voltage unbalance at the load bus (LB) of the islanded microgrid. Also, restoration of LB voltage amplitude and microgrid frequency to the rated values is considered in the secondary level. These functions are achieved by proper control of distributed generators (DGs) interface converters. The presented simulation results show the effectiveness of the proposed control structure in compensating the voltage unbalance and restoring the voltage amplitude and system frequency.

Hierarchical approaches and methodologies are commonly used for control system design and synthesis. Well-known model-based techniques are often applied to solve problems of complex and large-scale control systems. The general philosophy of decomposing control problems into modular and more manageable subsystem control problems applies equally to the growing domain of intelligent and autonomous systems. However, for this class of systems, new techniques for subsystem coordination and overall system control are often required. This article presents an approach to hierarchical control design and synthesis for the case where the collection of subsystems is comprised of fuzzy logic controllers and fuzzy knowledge-based decision systems. The approach is used to implement hierarchical behavior-based controllers for autonomous navigation of one or more mobile robots. Theoretical details of the approach are presented, followed by discussions of practical design and implementation issues. Example implementations realized on various physical mobile robots are described to demonstrate how the techniques may be applied in practical applications involving homogeneous and heterogeneous robot teams.

Soccer robot is a challenging platform for multi-agent research, involving topics such as real-time image processing and control, robot path planning, obstacle avoidance and machine learning. The conventional robot control consists of methods for path generation and path following. When a robot moves away the estimated path, it must return immediately, and while doing so, the obstacle avoidance behavior and the effectiveness of such a path are not guaranteed. So, motion control is a difficult task, especially in real time and high speed control. To achieve good tracking capability, this paper proposes a path following approach based on fuzzy logic. The fuzzy logic control is used for the low level motion of a soccer robot. Firstly, the modeling of the soccer robot is presented. The soccer robot based on MiroSoT Small Size league is a differential-drive mobile robot with non-slipping and pure-rolling. Then, the design of fuzzy controller is describes. To validate the proposed fuzzy algorithm two experiments have been run: in the first experiment, the robot moves toward the static ball and shoots to the opponent goal; in the second, the robot avoid two obstacles and arrive at the desired position and shoots to the opponent goal.

The concept of microgrid hierarchical control is presented recently. In this paper, a hierarchical scheme is proposed which includes primary and secondary control levels. The primary level comprises distributed generators (DGs) local controllers. The local controllers mainly consist of power, voltage and current controllers, and virtual impedance control loop. The central secondary controller is designed to manage the compensation of voltage unbalance at the point of common coupling (PCC) in an islanded microgrid. Unbalance compensation is achieved by sending proper control signals to the DGs local controllers. The design procedure of the control system is discussed in detail and the simulation results are presented. The results show the effectiveness of the proposed control structure in compensating the voltage unbalance.

W orldwide, electrical grids are expected to become smarter in the near future. In this sense, there is an increasing interest in intelligent and flexible microgrids, i.e., able to operate in island or in grid-connected modes. Black start operation, frequency and voltage stability, active and reactive power flow control, active power filter capabilities, and storage energy management are the functionalities expected for these small grids. This way, the energy can be generated and stored near the consumption points, thus increasing the

This paper proposes a new approach to robustly control a 2D under-actuated overhead crane system, where a payload is effectively transported to a destination in real time with small sway angles, given its inherent uncertainties such as actuator nonlinearities and external disturbances. The control law is proposed to be developed by the use of the robust hierarchical sliding mode control (HSMC) structure in which a second-level sliding surface is formulated by two first-level sliding surfaces drawn on both actuated and under-actuated outputs of the crane. The unknown and uncertain parameters of the proposed control scheme are then adaptively estimated by the fuzzy observer, where the adaptation mechanism is derived from the Lyapunov theory. More importantly, stability of the proposed strategy is theoretically proved. Effectiveness of the proposed adaptive fuzzy observer based HSMC (AFHSMC) approach was extensively validated by implementing the algorithm in both synthetic simulations and real-life experiments, where the results obtained by our method are highly promising.

W orldwide, electrical grids are expected to become smarter in the near future. In this sense, there is an increasing interest in intelligent and flexible microgrids, i.e., able to operate in island or in grid-connected modes. Black start operation, frequency and voltage stability, active and reactive power flow control, active power filter capabilities, and storage energy management are the functionalities expected for these small grids. This way, the energy can be generated and stored near the consumption points, thus increasing the

In this paper, hierarchical control techniques is used for controlling a robotic manipulator. The proposed method is based on the establishment of a non-linear mapping between Cartesian and joint coordinates using fuzzy logic in order to direct each individual joint. The hierarchical control will be implemented with fuzzy logic to improve the robustness and reduce the run time computational requirements. Hierarchical control consists of solving the inverse kinematic equations using fuzzy logic to direct each individual joint. A commercial Microbot with three degrees of freedom is utilized to evaluate this methodology. A decentralized fuzzy controller is used for each joint, with a Fuzzy Associative Memories (FAM) performing the inverse kinematic mapping in a supervisory mode. The FAM determines the inverse kinematic mapping which maps the desired Cartesian coordinates to the individual joint angles. The individual fuzzy controller for each joint generates the required control signal to a DC motor to move the associated link to the new position. The proposed hierarchical fuzzy controller is compared to a conventional controller. The simulation experiments indeed demonstate the effectiveness of the proposed method.

Laboratory Virtual Instrumentation and Engineering Workbench (LabVIEW) is a graphical programming tool based on the dataflow language G. Recently, runtime support for a hard real-time environment has become available for LabVIEW, which makes it an option for embedded systems prototyping. Due to its characteristics, the environment presents itself as an ideal tool for both the design and implementation of embedded software. In this paper, we study the design and implementation of embedded software by using G as the specification language and the LabVIEW RT real-time platform. One of the main advantages of this approach is that the environment leads itself to a very smooth transition from design to implementation, allowing for powerful cosimulation strategies (e.g., hardware in the loop, runtime modeling). We characterize the semantics and formal model of computation of G. We compare it to other models of computation and develop design rules and algorithms to propose sound embedded design in the language. We investigate the specification and mapping of hierarchical control systems in LabVIEW and G. Finally, we describe the development of a state-of-the-art embedded motion control system using LabVIEW as the specification, simulation and implementation tool, using the proposed design principles. The solution is state-of-the-art in terms of flexibility and control performance.

In reinforcement learning an agent may explore ineffectively when dealing with sparse reward tasks where finding a reward point is difficult. To solve the problem, we propose an algorithm called hierarchical deep reinforcement learning with automatic sub-goal identification via computer vision (HADS) which takes advantage of hierarchical reinforcement learning to alleviate the sparse reward problem and improve efficiency of exploration by utilizing a sub-goal mechanism. HADS uses a computer vision method to identify sub-goals automatically for hierarchical deep reinforcement learning. Due to the fact that not all sub-goal points are reachable, a mechanism is proposed to remove unreachable sub-goal points so as to further improve the performance of the algorithm. HADS involves contour recognition to identify sub-goals from the state image where some salient states in the state image may be recognized as sub-goals, while those that are not will be removed based on prior knowledge. Our experiments verified the effect of the algorithm.

W orldwide, electrical grids are expected to become smarter in the near future. In this sense, there is an increasing interest in intelligent and flexible microgrids, i.e., able to operate in island or in grid-connected modes. Black start operation, frequency and voltage stability, active and reactive power flow control, active power filter capabilities, and storage energy management are the functionalities expected for these small grids. This way, the energy can be generated and stored near the consumption points, thus increasing the

A multilevel hierarchical control system has been designed and is being applied to chemicalmechanical planarization ͑CMP͒ process control. The current implementation of the control system incorporates closed-loop run-to-run ͑R2R͒ control and open-loop real-time monitoring, and can accommodate inter-cell control. The R2R control element is enabled via a generic cell controller ͑GCC͒ implementation that provides flexible automated control of the process and equipment, multiple control algorithm branches and fuzzy logic decision capability among the branches, simulation capabilities, hardware and software independence, and extensive GUI support for control and data analysis. The R2R element utilizes a linear approximation multivariate control algorithm ͑branch͒ that supports individual exponential weighted moving average ͑EWMA͒ modeling of advices ͑outputs͒, weighting of inputs, granularity, and input bounding. The real-time element of the control system utilizes a partial least squares ͑PLS͒ algorithm to identify real-time equipment input trace patterns and relate these patterns to alarming conditions. The entire control system is designed to provide multivariate control of CMP process removal rate and uniformity. As a result of extensive design of experiments and testing, the R2R control level has been demonstrated to achieve good control of removal rate and fair control of uniformity. The addition of the real-time element is expected to improve process control and reduce R2R process noise, thus leading to a more effective R2R control element.

This paper introduces the development of multiple number of Unmanned Arial Vehicle (UAV) system as a part of BErkeley AeRobot (BEAR) project, highlighting the recent achievements in the design and implementation of rotorcraft-based UAV (RUAV) control system. Based on the experimental flight data, linear system model valid near hover condition is found by applying time-domain numerical methods to experimental flight data. The acquired linear model is used to design feedback controller consisting of inner-loop attitude feedback control, mid-loop velocity feedback control and the outer-loop position control. The proposed vehiclelevel controller is implemented and tested in Berkeley UAV, Ursa Magna 2, and shows superior hovering performance. The vehicle level controller is integrated with higher-level control using a script language framework to command UAV.

Abstract   Different control aspects related to the use of TCSC for stability improvement of power systems are addressed in this paper. A novel hierarchical control designed for both dynamic and steady state stability enhancement is proposed, and a complete analysis is presented of various locally measurable input signals that can be used for the controller. Control strategies to mitigate adverse interactions among the TCSC hierarchical controls are also presented.

This paper presents a novel application of the twotime scale controller for the full envelop flight control of a Rotary wing Unmanned Aerial Vehicle (RUAV). In this paper flapping and servo dynamics, important from a practical point of view, is included in the RUAV model. The two-time scale controller takes advantage of the 'decoupling' of the nonlinear translational and rotation dynamics of the rigid body, resulting in a two-level hierarchical control scheme. The inner loop controller (attitude control) tracks the attitude commands and sets the main rotor thrust vector, while the outer loop controller (position control) tracks the reference position and control the flapping angles and the tail rotor thrust vector. High fidelity RUAV simulation results are used to demonstrate the control performance. Simulation results show acceptable performance of the proposed two-time scale controller. The comparison of control inputs between the proposed two-time scale controller and an already implemented PID controller show that this controller is suitable for practical implementation.

In this study we present a supervisory control system and implement it on a pneumatic system controlled by a PLC. We introduce a step-by-step implementation procedure including a realization methodology of finite state machines by PLCs. A hierarchical control structure is proposed and implemented. The pneumatic system control problem presented in this study is a typical logical Discrete Event System (DES) control problem, therefore it is easy to generalise the methodology for most of the DES control problems.

Features Discusses economic optimization of greenhouse control through mathematical modeling Examines 30 years of scientific research to present a unified framework for efficient decision-making Presents modern methods of control and optimization including classical rule-based and multivariable feedback controllers Utilizes real and experimental examples and novel case discussions such as solar greenhouses Concludes with a discussion of open issues to stimulate

This paper discusses the integration of vision and touch sensors in a coordinate measuring machine (CMM) controller used for dimensional inspection tasks. A real-time hierarchical control system is presented in which a vision system extracts positions of features on a part to be inspected, and then guides a touch probe to efficiently measure these features. The probe is tracked by the vision system as it scans surfaces so that it's motion can be visually servoed. Minimalist sensorderived representations, involving only task-specific information, are used in this process. Although the camera itself remains uncalibrated, a real-time calibration of very limited scope is performed each processing cycle to transform the task-specific image information into 3-D information used as feedback to guide the probe.

Traditional processor scheduling mechanisms in operating systems are fairly rigid, often supporting only one fixed scheduling policy, or, at most, a few "scheduling classes" whose implementations are closely tied together in the OS kernel. This paper presents CPU inheritance scheduling, a novel processor scheduling framework in which arbitrary threads can act as schedulers for other threads. Widely different scheduling policies can be implemented under the framework, and many different policies can coexist in a single system, providing much greater scheduling flexibility. Modular, hierarchical control can be provided over the processor utilization of arbitrary administrative domains, such as processes, jobs, users, and groups, and the CPU resources consumed can be accounted for and attributed accurately. Applications as well as the OS can implement customized local scheduling policies; the framework ensures that all the different policies work together logically and predictably. As a side effect, the framework also cleanly addresses priority inversion by providing a generalized form of priority inheritance that automatically works within and among multiple diverse scheduling policies. CPU inheritance scheduling extends naturally to multiprocessors, and supports processor management techniques such as processor affinity [7] and scheduler activations . Experimental results and simulations indicate that this framework can be provided with negligible overhead in typical situations, and fairly small (5-10%) performance degradation even in scheduling-intensive situations.

This paper presents a novel application of backstepping controller for autonomous landing of a rotary wing UAV (RUAV). This application, which holds good for the full flight envelope control, is an extension of a backstepping algorithm for general rigid body velocity control. The nonlinear RUAV model used in this paper includes the flapping and servo dynamics. The backstepping-based controller takes advantage of the ‘decoupling’ of the translation and rotation dynamics of the rigid body, resulting in a two-step procedure to obtain the RUAV control inputs. The first step is to compute desired thrusts and flapping angles to achieve the commanded position and the second step is to compute control inputs, which achieve the desired thrusts and flapping angles. This paper presents a detailed analysis of the inclusion of a flapping angle correction term in control. The performance of the proposed algorithm is tested using a high-fidelity RUAV simulation model. The RUAV simulation model is based on miniature rotorcraft parameters. The closed-loop response of the rotorcraft indicates that the desired position is achieved after a short transient. The Eagle RUAV control inputs, obtained using high-fidelity simulation results, clearly demonstrate that this algorithm can be implemented on practical RUAVs. Copyright © 2009 John Wiley & Sons, Ltd.

The concept of microgrid hierarchical control is presented recently. In this paper, a hierarchical scheme is proposed which includes primary and secondary control levels. The primary level comprises distributed generators (DGs) local controllers. The local controllers mainly consist of power, voltage and current controllers, and virtual impedance control loop. The central secondary controller is designed to manage the compensation of voltage unbalance at the point of common coupling (PCC) in an islanded microgrid. Unbalance compensation is achieved by sending proper control signals to the DGs local controllers. The design procedure of the control system is discussed in detail and the simulation results are presented. The results show the effectiveness of the proposed control structure in compensating the voltage unbalance.

An anthropomorphic underactuated prosthetic hand, endowed with position and force sensors and controlled by means of myoelectric commands, is used to perform experiments of hierarchical shared control. Three different hierarchical control strategies combined with a vibrotactile feedback system have been developed and tested by able-bodied subjects through grasping tasks used in activities of daily living (ADLs). The first goal is to find a good tradeoff between good grasping capabilities and low attention required by the user to complete grasping tasks, without addressing advanced algorithm for electromyographic processing. The second goal is to understand whether a vibrotactile feedback system is subjectively or objectively useful and how it changes users' performance. Experiments showed that users were able to successfully operate the device in the three control strategies, and that the grasp success increased with more interactive control. Practice has proven that when too much effort is required, subjects do not do their best, preferring, instead, a less-interactive control strategy. Moreover, the experiments showed that when grasping tasks are performed under visual control, the enhanced proprioception offered by a vibrotactile system is practically not exploited. Nevertheless, in subjective opinion, feedback seems to be quite important.

Reinforcement learning is bedeviled by the curse of dimensionality: the number of parameters to be learned grows exponentially with the size of any compact encoding of a state. Recent attempts to combat the curse of dimensionality have turned to principled ways of exploiting temporal abstraction, where decisions are not required at each step, but rather invoke the execution of temporally-extended activities which follow their own policies until termination. This leads naturally to hierarchical control architectures and associated learning algorithms. We review several approaches to temporal abstraction and hierarchical organization that machine learning researchers have recently developed. Common to these approaches is a reliance on the theory of semi-Markov decision processes, which we emphasize in our review. We then discuss extensions of these ideas to concurrent activities, multiagent coordination, and hierarchical memory for addressing partial observability.

A nonlinear model-predictive hierarchical control approach is presented for coordinated ramp metering of freeway networks. The utilized hierarchical structure consists of three layers: the estimation/prediction layer, the optimization layer and the direct control layer. The previously designed optimal control tool AMOC (Advanced Motorway Optimal Control) is incorporated in the second layer while the local feedback control strategy ALINEA is used in the third layer. Simulation results are presented for the Amsterdam ring-road. The proposed approach outperforms uncoordinated local ramp metering and its efficiency approaches the one obtained by an optimal open-loop solution. It is demonstrated that metering of all on-ramps, including freeway-to-freeway intersections, with sufficient ramp storage space leads to the optimal utilization of the available infrastructure.

Intuitive myoelectric prosthesis control is dif cult to achieve due to the absence of proprioceptive feedback, which forces the user to monitor grip pressure by visual information. Existing myoelectric hand prostheses form a single degree of freedom pincer motion that inhibits the stable prehension of a range of objects. Multi-axis hands may address this lack of functionality, but as with multifunction devices in general, serve to increase the cognitive burden on the user. Intelligent hierarchical control of multiple degree-of-freedom hand prostheses has been used to reduce the need for visual feedback by automating the grasping process. This paper presents a hybrid controller that has been developed to enable different prehensile functions to be initiated directly from the user's myoelectric signal. A digital signal processor (DSP) regulates the grip pressure of a new six-degreeof-freedom hand prosthesis thereby ensuring secure prehension without continuous visual feedback.

The development process i.e. assumptions, construction, simulations and analysis of a control strategy for the hybrid drive of the vehicle mounted aerial work platform is presented in the paper. Particular attention is paid to the development of the control system strategy ensuring appropriate energy recuperation by making use of energy stored in the electrochemical form. The control strategy is built up around the concept of bi-level hierarchic control system. The elevation control of the aerial work platform is assumed as the primary goal of the control system. The secondary goal of the control system is formulated in terms of tracking and keeping the charging level of the rechargeable electrochemical accumulator in predefined limits. A control system simulation model is developed in Matlab-Simulink environment. Exemplary results of control system simulations are shown on the example of a hydraulic power unit driving aerial work platform mounted on special vehicle MONTRAKS.

Reinforcement learning is bedeviled by the curse of dimensionality: the number of parameters to be learned grows exponentially with the size of any compact encoding of a state. Recent attempts to combat the curse of dimensionality have turned to principled ways of exploiting temporal abstraction, where decisions are not required at each step, but rather invoke the execution of temporally-extended activities which follow their own policies until termination. This leads naturally to hierarchical control architectures and associated learning algorithms. We review several approaches to temporal abstraction and hierarchical organization that machine learning researchers have recently developed. Common to these approaches is a reliance on the theory of semi-Markov decision processes, which we emphasize in our review. We then discuss extensions of these ideas to concurrent activities, multiagent coordination, and hierarchical memory for addressing partial observability.

Hierarchical approaches and methodologies are commonly used for control system design and synthesis. Well-known model-based techniques are often applied to solve problems of complex and large-scale control systems. The general philosophy of decomposing control problems into modular and more manageable subsystem control problems applies equally to the growing domain of intelligent and autonomous systems. However, for this class of systems, new techniques for subsystem coordination and overall system control are often required. This article presents an approach to hierarchical control design and synthesis for the case where the collection of subsystems is comprised of fuzzy logic controllers and fuzzy knowledge-based decision systems. The approach is used to implement hierarchical behavior-based controllers for autonomous navigation of one or more mobile robots. Theoretical details of the approach are presented, followed by discussions of practical design and implementation issues. Example implementations realized on various physical mobile robots are described to demonstrate how the techniques may be applied in practical applications involving homogeneous and heterogeneous robot teams.

Based on differences in dynamic response times in the crop production process, a hierarchical decomposition of greenhouse climate management is proposed. To a large extent the proposed decomposition builds on the timescale decomposition of singularly perturbed systems commonly found in the literature. Main difference with these existing theoretical concepts is that the proposed decomposition is able to deal with rapidly fluctuating deterministic external inputs or disturbances acting on the fast sub-processes. For an example of economic optimal greenhouse climate management during one lettuce production cycle, the decomposition was successfully evaluated in simulations. Using these favourable results, a hierarchical concept for economic optimal greenhouse climate management is derived and discussed in view of application in horticultural practice.

This paper presents a novel application of backstepping controller for autonomous landing of a rotary wing UAV (RUAV). This application, which holds good for the full flight envelope control, is an extension of a backstepping algorithm for general rigid body velocity control. The nonlinear RUAV model used in this paper includes the flapping and servo dynamics. The backstepping-based controller takes advantage of the ‘decoupling’ of the translation and rotation dynamics of the rigid body, resulting in a two-step procedure to obtain the RUAV control inputs. The first step is to compute desired thrusts and flapping angles to achieve the commanded position and the second step is to compute control inputs, which achieve the desired thrusts and flapping angles. This paper presents a detailed analysis of the inclusion of a flapping angle correction term in control. The performance of the proposed algorithm is tested using a high-fidelity RUAV simulation model. The RUAV simulation model is based on miniature rotorcraft parameters. The closed-loop response of the rotorcraft indicates that the desired position is achieved after a short transient. The Eagle RUAV control inputs, obtained using high-fidelity simulation results, clearly demonstrate that this algorithm can be implemented on practical RUAVs. Copyright © 2009 John Wiley & Sons, Ltd.

e human hand is a complex system, with a large number of degrees of freedom (DoFs), sensors embedded in its structure, actuators and tendons, and a complex hierarchical control. Despite this complexity, the eff orts required to the user to carry out the diff erent movements is quite small (albeit after an appropriate and lengthy training). On the contrary, prosthetic