Chaouki Abdallah, University of New Mexico
Cooperative Learning for Robust Connectivity in Multi-robot Heterogeneous Networks

Abstract: Heterogeneous robotic teams have demonstrated their potential to perform complex missions more efficiently than their homogeneous counterpart. For example, a group made of aerial and ground robots can explore an area of interest faster than employing a single type of these robots. However, the use of autonomous mobile agents is challenging as they need to maintain a robust wireless communication for task coordination. In this work, we are interested in developing strategies to enhance connectivity by exploiting the heterogeneity of a team made of flying and ground robots. To address some of the limitations of operating wirelessly, we propose to merge reinforcement learning techniques with potential field methods to maintain the communication capabilities within the multi-robot network. Numerical simulations show the effectiveness of the proposed cooperative learning algorithm.

Panos J. Antsaklis, University of Notre Dame
Adaptive Stabilization of Uncertain Systems with Model-Based Control and Event-Triggered Feedback Updates

Abstract: This talk considers the problem of stabilization of uncertain systems where feedback measurements are transmitted through a limited bandwidth communication network. An adaptive Model-Based Event-Triggered (MB-ET) control framework is presented where a model of the plant dynamics is implemented in the controller node which provides estimates of the plant states between event-based measurement updates. Further, a sensor collocated Kalman filter is employed in order to estimate the plant parameters which are also transmitted to the controller node and upgrade the plant model. The adaptive MB-ET control framework provides significant improvement in terms of network utilization by allowing longer time intervals between updates. This talk focuses on several applications where the adaptive MB-ET control framework is used in order to stabilize uncertain dynamical systems by estimating unknown system parameters and disturbances while minimizing network communication.

Stjepan Bogdan, The University of Zagreb
Multi-Agent Control in Degraded Communication Environments

Abstract: The goal of this talk is to compute transmission rates and delays that provably stabilize Multi-Agent Systems (MASs) in the presence of disturbances. In other words, given the existing signal delay among the agents and the underlying topology, we determine rates at which information between the agents need to be exchanged such that the MAS of interest is Lp-stable with bias, where this bias accounts for distorted data. In an effort to consider MASs characterized by sets of equilibrium points, the notion of Lp-stability (with bias) and Lp-detectability with respect to a set is employed. Using the Lyapunov-Razumikhin type of arguments, we are able to consider delays that are greater than transmission intervals. The present methodology is applicable to general (not merely single- and double-integrator) heterogeneous linear agents, directed topologies and output feedback. Our results are experimentally verified employing a group of quadcopters.

Sesh Commuri, The University of Oklahoma
Intelligent Control of a Prosthetic Ankle using Gait Recognition

Abstract: In this talk, an Artificial Neural Network (ANN)-based hierarchical controller is proposed that first recognizes the amputees' intent from the actual measured gait data to select a displacement profile for the prosthetic joint, and then adaptively compensates for the unmodeled dynamics and disturbances for closed loop stability with guaranteed tracking performance. Detailed theoretical analysis is carried out to establish the stability and robustness of the proposed approach. The robustness of this approach is demonstrated using actual gait data collected from human subjects. The performance of the controller is studied through numerical simulations in the presence of noise, uncertainties in terrain interaction, disturbance torques, variations in gait parameters, and changes in gait.

Warren Dixon, The University of Florida
Model-Based Reinforcement Learning for Real-Time Approximate Optimal Control

Abstract: This talk shall illustrate implementation of the model-based reinforcement learning (MBRL) architecture for uncertain linearly parameterized control-affine nonlinear systems. A concurrent learning-based system identifier is employed to identify the unknown parameters in the system model. The system identifier is utilized to implement MBRL to solve infinite-horizon set-point regulation, trajectory tracking, and path following problems. Simulation results are provided to illustrate the effectiveness of the MBRL technique.

Rafael Fierro, University of New Mexico
Cooperative Learning for Robust Connectivity in Multi-robot Heterogeneous Networks

Abstract: Heterogeneous robotic teams have demonstrated their potential to perform complex missions more efficiently than their homogeneous counterpart. For example, a group made of aerial and ground robots can explore an area of interest faster than employing a single type of these robots. However, the use of autonomous mobile agents is challenging as they need to maintain a robust wireless communication for task coordination. In this work, we are interested in developing strategies to enhance connectivity by exploiting the heterogeneity of a team made of flying and ground robots. To address some of the limitations of operating wirelessly, we propose to merge reinforcement learning techniques with potential field methods to maintain the communication capabilities within the multi-robot network. Numerical simulations show the effectiveness of the proposed cooperative learning algorithm.

Wassim Haddad, Georgia Institute of Technology
A Mechanistic Neural Field Theory of How Anesthesia Suppresses Consciousness: Synaptic Drive Dynamics, System Stability, Attractors, Partial Synchronization, and the Anesthetic Cascade

Abstract: With the advances in biochemistry, molecular biology, and neurochemistry there has been impressive progress in understanding the molecular properties of anesthetic agents. However, there has been little focus on how the molecular properties of anesthetic agents lead to the observed macroscopic property that denes the anesthetic state, that is, lack of responsiveness to noxious stimuli. In this presentation, we use dynamical system theory to develop a mechanistic mean model for neural activity to study the anesthetic cascade. The proposed synaptic drive firing rate model predicts the conscious-unconscious transition as the implied anesthetic concentration increases, where excitatory neural activity is characterized by a Poincare-Andronov-Hopf bifurcation with the awake state transitioning to a stable limit cycle and then subsequently to an asymptotically stable unconscious equilibrium state. Furthermore, we address the more general question of synchronization of neural activity without mean field assumptions. This is done by focusing on a postulated subset of inhibitory neurons that are not themselves connected to other inhibitory neurons. Finally, several numerical experiments are presented to illustrate the different aspects of the proposed theory.

Guido Herrmann, University of Bristol
An Adaptive Algorithm for Robust, Finite-Time Control and Estimation-Examples in Theory and Practice

Abstract: This talk will present the theoretical framework of an adaptation algorithm which permits a data forgetting mechanism and robustness to bounded disturbances and uncertainty, while achieving convergence to the true parameter in the absence of disturbances. The algorithm can be easily modified to achieve exponential or finite-time convergence. The talk will provide examples of this algorithm in application to a novel adaptive finite-time leader-following control and a novel adaptive approximate optimal tracking control both experimentally validated for a humanoid robot. Newest examples will show how this algorithm can aid the real-time estimation of in-car parameters such as mass and vehicular track gradient for a light passenger vehicle.

João P. Hespanha, University of California, Santa Barbara
Addressing Adaptation and Learning in the Context of MPC and MHE

Abstract: This talk will consider the estimation and control of systems with parametric uncertainty. An approach that combines moving horizon estimation and model predictive control into a single min-max optimization is employed to estimate past and current values of the state, compute a sequence of optimal future control inputs, predict future values of the state, as well as estimate current values of uncertain parameters. This is done by including the state, inputs, and uncertain parameters as optimization variables. Learning the true values of the uncertain parameters requires a sufficiently large number of past measurements and that the system is persistently excited. The true values of the uncertain parameters may change over time, and the optimization computes future control inputs that adapt to changing estimates of the uncertain parameters in order to better control the uncertain system. Several linear and nonlinear examples with parametric uncertainty are discussed and effectively controlled using this combined moving horizon estimation and model predictive control approach.

Jie Huang, The Chinese University of Hong Kong
Certainty Equivalence, Separation Principle, and Cooperative Output Regulation of Multi-Agent Systems by Distributed Observer Approach

Abstract: In this talk, we will first present a more general formulation of the cooperative output regulation problem for linear multi-agent systems that includes some existing versions of the cooperative output regulation problem as special cases. Then, we will describe a more general distributed observer. Finally, we will simplify the proof of the main results by more explicitly utilizing the separation principle and the certainty equivalence principle.

Sarangapani Jagannathan, Missouri University Of Science And Technology
Adaptive Optimal Regulation of a Class of Uncertain Nonlinear Systems using Event Sampled Neural Network Approximators

Abstract: This talk will present an approximate optimal control of nonlinear continuous-time systems in affine form by using the adaptive dynamic programming (ADP) with event sampled state and input vectors. The knowledge of the system dynamics are relaxed by using a neural network (NN) identifier with event sampled inputs. An approximate solution to the Hamilton-Jacobi-Bellman (HJB) equation, by using event sampled NN approximator, is introduced. Subsequently, the NN identifier and approximated value function are utilized to generate the optimal control policy. Both the identifier and value function weights are tuned only at the event sampled instants leading to an aperiodic update scheme. A novel adaptive event sampling condition is designed to determine the sampling instants such that the approximation accuracy and stability are maintained. A positive lower bound on the minimum inter-sample time is guaranteed to avoid accumulation point and the dependence of inter-sample time upon the NN weight estimate is analyzed in detail. The extension of Lyapunov theory is utilized to guarantee the local ultimate boundedness of the resulting nonlinear impulsive dynamical closed-loop system. The net result is the design of event sampled ADP-based controller for nonlinear continuous-time systems. Finally, the above optimal regulator is contrasted with a NN based stabilizing controller with event sampled inputs.

Zhong-Ping Jiang, New York University
Stochastic Adaptive Dynamic Programming for Robust Optimal Control Design

Abstract: In this talk, we present preliminary results on the efforts toward the development of a stochastic adaptive dynamic programming theory for the data-driven, non-model-based design of robust optimal controllers for continuous-time linear stochastic systems. Both signal-dependent noise and additive noise are considered. Two types of optimal control problems, known as the discounted problem and the biased problem, are investigated. Reinforcement learning and adaptive dynamic programming techniques are employed to design stochastic adaptive optimal controllers through online successive approximations of optimal solutions. Rigorous convergence proofs along stability analysis are provided. The effectiveness of the proposed methods is validated by two illustrative practical examples arising from biological motor control and the vehicle suspension control problem.

Zdenko Kovačić, The University of Zagreb
Estimation and Control for Safe Autonomous Vehicle Navigation in Industrial Warehousing Environments

Abstract: In this talk we describe a laser range sensor-based markerless localization of automated guided vehicles (AGV) based on the use of effective SLAM technique, where the accuracy of localization within (1 cm, 0.5 degree) allows its direct use in industrial warehousing conditions. We also describe dynamic routing using grid-based representation of navigation space and lattice-based formation of the planned path. In addition, we describe the decentralized traffic control that allows the simultaneous negotiation, conflict resolution and mission dispatch for a large number of AGVs based on the status of each AGV and service optimization criteria set for certain industrial warehouse environment. The results presented in this talk are confirmed by simulations and experiments carried out with laboratory vehicles (Pioneer DX-3 robots) and industrial vehicles (Euroimpianti laser guided forklifts Skilled 1000 and 1400).

Frank L. Lewis, The University of Texas at Arlington
Recent Advances and New Directions in Integral Reinforcement Learning for Feedback Control

Abstract: This talk will describe recent advances and new directions in RL/ADP for continuous-time systems. The powerful technique of Integral Reinforcement Learning (IRL) developed in 2008 allows applications of RL/ADP to CT systems with incomplete dynamics information while providing rigorous performance guarantees. IRL is based on neurocognitive precepts and involves exploring the system response to a test control policy over an integration time period. IRL is implemented in practice by means of two approximation structures- a critic network to evaluate the applied control policies and an actor network to update the policy.

Zongli Lin, University of Virginia
Discrete-time Coordinated Control of Wheeled Vehicles in the Presence of a Large Communication Delay

Abstract: This talk presents a discrete-time coordinated control design for a multi-agent system, where each agent is a vehicle with nonholonomic dynamics and subject to an arbitrarily large communication delay. A low gain based distributed low control law is derived for each agent on the basis of a gradient of an artificial potential function. We demonstrate using the Lyapunov functional approach that the proposed control laws drive the multi-agent system into the stable flocking behavior. The effectiveness of the proposed control laws is verified through numerical simulation.

Derong Liu, The University of Illinois
Unsupervised Feature Learning for Policy Iteration based on Manifold Regularization

Abstract: In this talk, we shall develop a novel approximate policy iteration reinforcement learning algorithm with unsupervised feature learning based on manifold regularization. The proposed algorithm can automatically learn data-driven smooth basis representations for value function approximation, which can preserve the intrinsic geometry of the state space of Markov decision processes. Moreover, it can provide a direct basis extension for new samples in both policy learning and policy control processes. We evaluate the effectiveness and efficiency of the proposed algorithm on the inverted pendulum task. Simulation results show that this algorithm can learn smooth basis representations and excellent control policies.

Roland Malhamé, École Polytechnique de Montréal
Control of Aggregate Electric Water Heating Loads via Mean Field Games Based Methods

Abstract: With the increasing renewable energy penetration in the grid, load control is also considered as a tool to exploit the storage in dispersed devices naturally present in power systems such as electric water heaters to mitigate generation variability. Tapping into the storage dispersed across the power system is challenging because of the large number of devices that need to be coordinated to produce desirable system level behavior. A hierarchical approach is proposed in this talk where a top level node sets system optimal mean aggregate temperature target trajectory for a specific time frame. For individual electric water heating loads, a Markovian jump-driven model is employed. Locally situated controllers generate control actions through a Mean Field Games (MFG) control algorithm based on local state and mean field dependent cost functions without compromising comfort and safety constraints. Despite the decentralized characteristic of the control design, aggregate load meets the top level node targets asymptotically, thanks to MFG. The corresponding mathematical theory is developed. A numerical algorithm to solve the MFG equations at the individual level to calculate the control signals is provided, and simulation results are given.

Kumpati Narendra, Yale
Hierarchical Adaptive Control of Rapidly Time-Varying Systems

Abstract: The lecture will deal with a novel procedure for combining four distinct methods for the adaptive control of rapidly time-varying systems. These include "Switching", "Switching and Tuning", "Interactive Control" and "Second Level Adaptation". "Switching" and "Switching and Tuning" are currently well known. In interactive control, the time-varying system is observed over a long period of time and the time-variations are suitably classified. Second level adaptation has been shown in recent years to be a fast and accurate method for adaptive control. The lecture will attempt to show how a proper representation of the problem, together with a suitable combination of the methods, can result in a powerful methodology for coping with a variety of distinctly different time-variations.

Robert Newcomb, University of Maryland, College Park
The Stereographic Product of Positive-Real Functions is Positive-Real

Abstract: The circle product, o, has recently been introduced as the key operation in the algebraic structure of stereographic trigonometry. Likewise the positive-real functions have been found to be important in the design of optimal control theory while serving also as the basic functions of circuit synthesis. Here we extend the stereographic circle operation to complex valued functions and show that then z=y1oy2 is positive-real when y1 and y2 are. Several physical interpretations are given, including z being the input impedance when an admittance-based Richards' section of parameter y1 is loaded in the admittance y2. Zero, 0, is the identity, 0ox=x, so that -x is the inverse of x under o, that is (-x)ox=0, -x will not be positive-real when x is; that is the inverse under o need not be positive-real. However, y2=(-y1)oz will be PR when y1 is and z is a Richards' function of y2.

Thomas Parisini, Imperial College, London
Distributed Fault Diagnosis of Large-Scale Interconnected Systems

Abstract: In the talk, an adaptive approximation-based distributed and networked fault diagnosis approach for large-scale nonlinear systems will be dealt with, by exploiting a "divide et impera" approach in which the overall diagnosis problem is decomposed into smaller sub-problems, which can be solved within "local" computation and communication architectures. The proposed formulation incorporates a synchronization methodology with a filtering approach in order to reduce the effect of measurement noise and time delays on the fault detection performance. The proposed approach allows the monitoring of multi-rate systems, where asynchronous and delayed measurements are available, thanks to the introduction of a Virtual Sensor implementing a model-based re-synchronization algorithm. Moreover, a delay compensation strategy for distributed diagnosers is developed. The monitoring architecture exploits an adaptive approximator with learning capabilities for handling uncertainties in the interconnection dynamics. A consensus-based estimator with time-varying weights is introduced, allowing to improve fault detectability in the case of variables shared among more than one subsystem. Furthermore, time-varying threshold functions guaranteeing no false-positive alarms and analytical fault detectability sufficient conditions are derived.

Zhihua Qu, University of Centra Florida
Cooperative Control and Networked Operation of Passivity-Short Systems

Abstract: In this talk, both input-output stability analysis and output-feedback networked control design of nonlinear dynamic systems are investigated. Specifically, the talk focuses upon two classes of passivity- short systems: input-feedforward passivity-short systems and output-feedback passivity-short systems. It is shown that, under an appropriate output feedback control, these classes of passivity-short systems have such desired properties as Lyapunov stability, asymptotic stability and L2 stability. Furthermore, it is shown that these systems can be interconnected and their networked system naturally reaches a consensus while interconnecting strictly passive systems leads to asymptotic stability. Accordingly, a networked operation of nonlinear dynamics systems can be accomplished by making individual systems either passive or passivity-short rather than strictly passive. It is also shown that, by following the proposed design process, networked operation of nonlinear dynamic systems becomes plug-and-play.

Kyriakos G. Vamvoudakis, University of California, Santa Barbara
Model-Free Optimization-Based Algorithms with Applications to Network Security

Abstract: This talk shall be inspired by the human brain and combine interdisciplinary ideas from different fields, i.e. computational intelligence, game theory, control theory, and information theory to develop new self-configuring algorithms for decision and control given the unavailability of model, the presence of enemy components and the possibility of measurement and jamming network attacks. The novel algorithms will be characterized by strong abilities of self-learning and adaptivity and will be implemented locally, and not globally, which implies that fewer sensors will be used for measurements.

Yorai Wardi, Georgia Institute of Technology
Performance Regulation in Discrete Event Dynamic Systems

Abstract: This talk shall present a performance regulation technique in Discrete Event Dynamic Systems (DEDS), recently developed by the authors. The technique is based on an integral feedback controller with variable gain, designed to enhance the stability margins of the closedloop system. The gain is computable by applying the Infinitesimal Perturbation Analysis (IPA) derivative of the plant function, which (for a class of systems) is computable in realtime. A salient feature of the control method is that its performance can be robust to errors of modeling and computation. The talk will summarize the authors' recent work on the subject and present examples from various application domains including queueing networks, manufacturing, traffic control, and throughput tracking in computer cores.

Siva S. Banda, Air Force Research Laboratory, Wright-Patterson Air Force Base, Ohio

Richard D. Braatz, Massachusetts Institute of Technology

Michael Sebek, Czech Technical University in Prague

Marwan A. Simaan, University of Central Florida

Draguna Vrabie, United Technologies Research Center