Keywords: Model/Controller reduction, Reduced order modeling
Abstract: We present a review of some recent contributions to the theory and application of nonlinear model order reduction by moment matching. The tutorial paper is organized in four parts: 1) Moments of Nonlinear Systems; 2) Playing with Moments: Time-Delay, Hybrid, Stochastic, Data-Driven and Beyond; 3) The Loewner Framework; 4) Applications to Optimal Control and Wave Energy Conversion.

Keywords: Reduced order modeling, Delay systems, Stochastic systems
Abstract: We present a systematic technique to extend the notion of moment to general classes of systems. As examples we apply the technique to time-delay systems, stochastic systems, and systems in explicit form. In addition, we provide a numerical algorithm for the computation of moments of nonlinear systems from input-output data.

Keywords: Model/Controller reduction, Reduced order modeling, Nonlinear systems
Abstract: We introduce a model reduction framework for nonlinear systems by generalizing the Loewner framework developed for linear time-invariant (LTI) systems. We start by defining extensions of the Loewner matrices, called Loewner functions, obtained by extending an interconnection-based interpretation of the Loewner matrices. Using these Loewner functions a Loewner equivalent model is developed. This allows determining reduced order models which achieve interpolation in the Loewner sense.

Keywords: Optimal control, Energy systems
Abstract: We show that, besides being a powerful tool for model reduction purposes, the parameterisation of the steady-state response of a system in terms of moments can be useful to approximate optimal control problems (OCPs). In particular, we illustrate this claim by solving the WEC energy-maximising OCP using a moment-based representation, where the moment-based parameterisation is particularly well matched to the application, where steady-state energy-maximisation is of paramount importance, resulting in an efficient control implementation.

Keywords: Energy systems, Adaptive control, Emerging control applications
Abstract: Battery operated devices are common in everyday life. Deciding when a warning for impending battery voltage collapse should be triggered is not trivial. This paper develops an impending battery terminal voltage collapse detection system using a universal adaptive stabilizer (UAS) and a well-known trend filter. This eliminates requiring knowledge of battery model parameter values, or initial state-of-charge (SOC). The proposed approach overcomes the need for extensive training when compared to using neural-networks based techniques. Also, the developed trend filter when used with a UAS, eliminates the need for selecting windows of data to be processed. This is advantageous compared to earlier work, which uses a different trend filtering mechanism, because selection of window sizes is not straightforward. Further, the approach used in this work shows that the UAS based technique is implementable on a cell phone. Associated mathematical results, and experimental data from such an implementation are presented. Additionally, the technique is also applied to other larger capacity Li-ion batteries showing its versatility. The developed technique can also be used to detect when the state-of-health (SOH) of a Li-ion battery is about to enter an unsafe region.

Keywords: Energy systems, Distributed parameter systems, Process Control
Abstract: This article provides a theoretical analysis of the nonlinear dynamics of an ethanol steam reformer, which has the potential to become a key technology for the creation of a hydrogen economy. A set of nonlinear partial differential equations is analyzed that arise from material and energy balances for an ethanol steam reformer. Although all of the governing equations contain derivatives with respect to both space and time, the nonlinear distributed parameter system is shown to be singular by structural rank analysis. A numerical method is proposed for simulation of the singular dynamical system. The character of the singularity is analyzed for both the distributed parameter systems and the lumped parameter system used in its simulation. The nonlinear spatiotemporal and input-output behavior of the system are analyzed, including by calculation of a nonlinearity measure applicable to singular distributed parameter systems. Although some states are highly nonlinear functions of the control inputs, a linear low-order input-output model with uncertainty description is shown to be suitable for controller design.

Keywords: Cooperative control, Optimization algorithms, Optimal control
Abstract: A multi-agent coverage problem is considered with energy-constrained agents. The objective of this paper is to compare the coverage performance between centralized and decentralized approaches. To this end, a near-optimal centralized coverage control method is developed under energy depletion and repletion constraints. The optimal coverage formation corresponds to the locations of agents where the coverage performance is maximized. The optimal charging formation corresponds to the locations of agents with one agent fixed at the charging station and the remaining agents maximizing the coverage performance. We control the behavior of this cooperative multi-agent system by switching between the optimal coverage formation and the optimal charging formation. Finally, the optimal dwell times at coverage locations, charging time, and agent trajectories are determined so as to maximize coverage over a given time interval. In particular, our controller guarantees that at any time there is at most one agent leaving the team for energy repletion.

Keywords: Energy systems, Identification, Machine learning
Abstract: Dynamic physics based modeling of district heating networks has gained importance due to an increased use of renewable energy sources and a transition towards lower temperature district heating networks. The modeling is enhanced by technologies for automatic model generation and co-simulation. These models are in general not suitable for automatic control and optimization methods, due to the complexity of of the model. Moreover, there is no notion of uncertainty in the models, something that can be of importance for decision making, and that can be explicitly accounted for in e.g Bayesian Optimization and Stochastic Nonlinear Model Predictive Control. In this paper a data driven Gaussian process model for the thermal dynamics of the district heating grid is proposed, with a kernel derived using known physics and numerical methods. The model is trained and validated on a realistic first principle simulation model of a district heating pipe. Results show a good correspondence with the output from the training model on a validation dataset, providing explicit propagation of the input uncertainties. It is suggested that the method can be scaled up to larger parts of the grid for use in advanced control and optimization methods.

Keywords: Energy systems, Power systems, Modeling
Abstract: The goal of this paper is to develop a low-order model to represent the coordination of distributed energy resources based on concepts that make the internet work and is known as packetized energy management (PEM). The low-order model includes energy as a state variable together with dynamic opt-out constraints and internal packet request feedback, which in principle turns the model into a PEM virtual battery (PEM-VB) model. The paper focuses on a homogeneous aggregation of electric water heaters (EWHs) under PEM. It is shown that the bottom-up logic of the PEM-VB makes the system observable mainly due to the convenient feeding back of the number of packet requests through the communication channel established between the PEM coordinator and devices enrolled in the scheme. Without such extra information, the system loses the ability to observe the system's stored energy state. A procedure for computing the maximum and minimum energy bounds for the PEM-VB is developed. Moreover, the PEM-VB is explicitly used as the underlying model for an extended Kalman filter observer formulation with the purpose of estimating the energy stored in a simulated ensemble of agent-based EWHs under PEM. The use of PEM-VB is demonstrated in pre-positioning of flexible resources depending upon load forecasts. Finally, conclusions and future directions are provided.

Keywords: Modeling, Optimal control, Energy systems
Abstract: This paper proposes an output power smoothing strategy for a grid-connected wind-hydrogen plant. An Energy Storage System (ESS) composed of an electrolyzer and a fuel cell is used to smooth the fluctuating output power of the wind plant. The aim of this study is to propose a multi-objective optimization model for joint wind farm and energy storage operation, to smooth the wind power output, and to track a load demand subject to a variety of constraints on the system model. Based on a modeling adopting mixed-integer constraints and dynamics, the problem of output power fluctuations and smooth tracking of load demand is solved through the implementation of an optimal controller which follows a sequential optimization technique. We illustrate the effectiveness of the proposed controller with a simulation example, employing real equipment data and wind profiles.

Keywords: Energy systems, Stochastic optimal control, Optimization algorithms
Abstract: This paper presents an analytical method for calculating the operational value of an energy storage device under multi-stage price uncertainties. Our solution calculates the storage value function from price distribution functions directly instead of sampling discrete scenarios, offering improved modeling accuracy over tail distribution events such as price spikes and negative prices. The analytical algorithm offers very high computational efficiency in solving multi-stage stochastic programming for energy storage and can easily be implemented within any software and hardware platform, while numerical simulation results show the proposed method is up to 100,000 times faster than a benchmark stochastic-dual dynamic programming solver even in small test cases. Case studies are included to demonstrate the impact of price variability on the valuation results, and a battery charging example using historical prices for New York City.

Keywords: Numerical algorithms, Queueing systems, Stochastic optimal control
Abstract: The ergodic or long run average cost control problem for diffusions is one of the few classical problems of stochastic control that still eludes a com- pletely satisfactory treatment. This is certainly true for the setting in which the systems to be controlled is modeled by the solutiom of a switching stochastic differential equation with reflection (SSDER). In this paper we set forth a preliminary study for this set- ting in the unidimensional case. Besides carving out a numerical method, our treatment of the ergodic control in this scenario straddles issues of existence and uniqueness of solution of the the SSDER and a verification theorem for the associated HJB equation. We conclude by illustrating the effectiveness of the method considering the control energy consumption in a large parallel processing computer system composed of one queue and several processing stations.

Keywords: Discrete event systems, Supervisory control, Fault tolerant systems
Abstract: A supervisory policy controls a Discrete-Event System (DES) by appropriately disabling a subset of events, known as controllable events, based on the observed event string generated by the supervised DES thus far. We consider supervisory control of DES in the presence of an extraneous fault that renders an arbitrary subset of controllable events to be temporarily uncontrollable. The fault is detected at the first occurrence of a controllable event that was disabled by the supervisor. It is rectified after finitely-many such unintended occurrences of controllable events following which the supervisor regains control of all controllable events and can prevent them from occurring when deemed necessary.

We present a necessary and sufficient condition for the existence of a supervisor that enforces a desired language specification in the paradigm of Ramadge and Wonham, under the fault semantics described above.

We also prove that such a supervisor, if it exists, can always be synthesized if the language of the plant and the specification is regular.

Keywords: Discrete event systems, Supervisory control, Automata
Abstract: In this paper, we study a nonblocking similarity control problem for nondeterministic discrete event systems (DESs), which requires us to synthesize a nonblocking supervisor such that the supervised system is simulated by the given specification. We propose an algorithm that computes a nonblocking supervisor from a possibly blocking one by iteratively removing certain states. To compute a maximally permissive nonblocking supervisor, the algorithm is applied to the supervisor synthesized without imposing nonblockingness. We show that a nonblocking supervisor is generated by the algorithm if and only if there exists a solution to the nonblocking similarity control problem, and the generated nonblocking supervisor is a maximally permissive one.

Keywords: Discrete event systems, Petri nets
Abstract: The problem of checking the consistency property in a P-time event graph (P-TEG) is fundamental and has to be carried out before applying any control strategy. However, no generic algorithm that solves this problem is available yet. In this paper we propose a new, stronger, definition for consistency of P-TEGs that is both more useful and easier to check than standard consistency; bounded consistency. A P-TEG is boundedly consistent if it admits a trajectory in which, for each pair of transitions, the time interval between their k-th firings is bounded for every k. We show that the problem of checking bounded consistency is equivalent to checking the existence of 1-periodic trajectories.

Keywords: Automata, Discrete event systems, Game theory
Abstract: With the increasing sophistication of attacks on cyber-physical systems, deception has emerged as an effective tool to improve system security and safety by obfuscating the attacker?s perception. In this paper, we present a solution to the deceptive game in which a control agent is to satisfy a Boolean objective specified by a co-safe temporal logic formula in the presence of an adversary. The agent intentionally introduces asymmetric information to create payoff misperception, which manifests as the misperception of the labeling function in the game model. Thus, the adversary is unable to accurately determine which logical formula is satisfied by a given outcome of the game. We introduce a model called hypergame on graph to capture the asymmetrical information with one-sided payoff misperception. Based on this model, we present the solution of such a hypergame and use the solution to synthesize stealthy deceptive strategies. Specifically, deceptive sure winning and deceptive almost-sure winning strategies are developed by reducing the hypergame to a two-player game and one-player stochastic game with reachability objectives. A running example is introduced to demonstrate the game model and the solution concept used for strategy synthesis.

Keywords: Cooperative control, Discrete event systems, Optimal control
Abstract: We consider the optimal multi-agent persistent monitoring problem defined on a set of nodes (targets) interconnected through a fixed graph topology. The objective is to minimize a measure of mean overall node state uncertainty evaluated over a finite time interval by controlling the motion of a team of agents. Prior work has addressed this problem through on-line parametric controllers and gradient-based methods, often leading to low-performing local optima or through off-line computationally intensive centralized approaches. This paper proposes a computationally efficient event-driven receding horizon control approach providing a distributed on-line gradient-free solution to the persistent monitoring problem. A novel element in the controller, which also makes it parameter-free, is that it self-optimizes the planning horizon over which control actions are sequentially taken in event-driven fashion. Numerical results show significant improvements compared to state of the art distributed on-line parametric control solutions.

Keywords: Discrete event systems, Linear parameter-varying systems, Nonholonomic systems
Abstract: In this paper, we present a synthesis condition for designing event-triggered dynamic output feedback control of discrete-time polytopic linear parameter-varying (LPV) systems. Here we assume that an event trigger controls the transmission of data from plant sensors to the controller in order to save communication bandwidth. The output feedback control scheme guarantees stability and a bound on the overall ell_2 performance. The synthesis problem is formulated as an LMI problem. The effectiveness of the result is illustrated in a simulated tracking scenario with a non-holonomic vehicle modeled as a kinematic unicycle.

Keywords: Petri nets, Fault diagnosis
Abstract: We propose a method to decide the diagnosability of patterns in labeled Time Petri nets (TPN) that gracefully extends a classic approach for the diagnosability of single faults. Our approach is based on a new technique for computing the language intersection of TPN and on an associated extension of the State Class Graph construction. Our approach has been implemented and we report on some experimental results.

Keywords: Hybrid systems, Switched systems, Automata
Abstract: The paper proposes a model reduction algorithm for linear hybrid systems, i.e., hybrid systems with externally induced discrete events, with linear continuous subsystems, and linear reset maps. The model reduction algorithm is based on balanced truncation. Moreover, the paper also proves an analytical error bound for the difference between the input-output behaviors of the original and the reduced-order model. This error bound is formulated in terms of singular values of the Gramians used for model reduction.

Keywords: Learning, Adaptive control, Machine learning
Abstract: This paper proposes a framework for adaptively learning a feedback linearization-based tracking controller for an unknown system using discrete-time model-free policy-gradient parameter update rules. The primary advantage of the scheme over standard model-reference adaptive control techniques is that it does not require the learned inverse model to be invertible at all instances of time. This enables the use of general function approximators to approximate the linearizing controller for the system without having to worry about singularities. The overall learning system is stochastic, due to the random nature of the policy gradient updates, thus we combine analysis techniques commonly employed in the machine learning literature alongside stability arguments from adaptive control to demonstrate that with high probability the tracking and parameter errors concentrate near zero, under a standard persistency of excitation condition. A simulated example of a double pendulum demonstrates the utility of the proposed theory.

Keywords: Adaptive control, Neural networks, Aerospace
Abstract: The problem of control of a class of nonlinear plants has been addressed by using neural networks together with sliding mode control. We revisit this problem in this paper and propose an adaptive controller based on convex/concave parameterization for adjusting the weights that occur nonlinearly. By using the algorithms that have been proposed previously for adaptive control for nonlinearly parameterized problems, it is shown that global boundedness of all signals can be achieved together with a reduced tracking error. Simulation studies of an aircraft landing problem validate the proposed controller and that it contributes to improved learning of the underlying nonlinearity.

Keywords: Adaptive control, Optimal control
Abstract: In this paper, the adaptive optimal regulation of uncertain linear continuous-time systems with state and input delays is introduced. First, an adaptive identifier is proposed to estimate the system dynamics. Subsequently, by using a quadratic cost function, a Bellman type equation is derived to obtain the optimal regulation via stationarity condition. Finally, the adaptive identifier and the optimal approach are integrated together to design the optimal adaptive regulator in the presence of uncertain system dynamics. The Lyapunov theory is utilized to show the boundedness of the state vector, parameter estimation and identification errors when the initial parameters lie within a compact set. A simulation example is employed to verify the effectiveness of the proposed approach.

Keywords: Adaptive systems, Adaptive control, Machine learning
Abstract: Parameter estimation algorithms using higher order gradient-based methods are increasingly sought after in machine learning. Such methods however, may become unstable when regressors are time-varying. Inspired by techniques employed in adaptive systems, this paper proposes a new variational perspective to derive four higher order tuners with provable stability guarantees. This perspective includes concepts based on higher order tuners and normalization and allows stability to be established for problems with time-varying regressors. The stability analysis builds on a novel technique which stems from symplectic mechanics, that links Lagrangians and Hamiltonians to the underlying Lyapunov stability analysis, and is provided for common linear-in-parameter models.

Keywords: Adaptive control, Machine learning, Statistical learning
Abstract: We consider an idealized version of adaptive control of a MIMO system without state. We demonstrate how rank deficient Fisher information in this simple memoryless problem leads to the impossibility of logarithmic rates of regret. This to some extent resolves an open issue concerning the attainability of logarithmic regret rates in linear quadratic adaptive control. Our analysis rests on a version of the Cramér-Rao inequality that takes into account possible ill-conditioning of Fisher information and a pertubation result on the corresponding singular subspaces. This is used to define a sufficient condition, which we term uniformativeness, for regret to be at least order square root in the samples.

Keywords: Adaptive control, Delay systems, Game theory
Abstract: In this paper, we propose a non-model based strategy for locally stable convergence to Nash equilibrium in a quadratic noncooperative (duopoly) game with arbitrarily delayed player actions. In our noncooperative scenario, the players have access only to their own payoff values, again with delay. The proposed approach is based on the extremum seeking perspective, which has previously been reported for real-time optimization problems by exploring sinusoidal perturbation signals to estimate the Gradient (first derivative) and Hessian (second derivative) of unknown locally quadratic functions. Indeed, this is the first contribution which considers extremum seeking for noncooperative games in the presence of delays. In order to compensate distinct delays in the inputs of the two players, we have employed boundary control via predictor feedback with averaging-based estimates. We apply a small-gain analysis for the resulting Input-to-State Stable hyperbolic PDE-ODE loop as well as averaging theory in infinite dimensions, due to the infinite-dimensional state of the time delays, in order to obtain local convergence results to a small neighborhood of the Nash equilibrium. We quantify the size of these residual sets and corroborate the theoretical results numerically on an example of a two-player game with delays.

Keywords: Adaptive control, Delay systems, Distributed parameter systems
Abstract: We design an adaptive full-state feedback controller to stabilize a one-dimensional reaction-diffusion equation with unknown boundary input delay. An infinite-dimensional representation of the actuator delay is utilized to transform the system into a transport PDE cascading with a reaction-diffusion PDE. A suitable unknown parameter update law is designed and a local stability result is established using the PDE backstepping technique and a Lyapunov argument. Consistent simulation results are provided to support the theoretical result.

Keywords: Adaptive control, Switched systems, Intelligent systems
Abstract: This paper examines the use of reinforcement learning-based controllers to approximate multiple value functions of specific classes of subsystems while following a switching sequence. Each subsystem may have varying characteristics, such as different cost or different system dynamics. Stability of the overall switching sequence is proven using Lyapunov-based analysis techniques. Specifically, Lyapunov-based methods are developed to prove boundedness of individual subsystems and to determine a minimum dwell-time condition to ensure stability of the overall switching sequence. Uniformly ultimately bounded regulation of the states, approximation of the value function, and approximation of the optimal control policy is achieved for arbitrary switching sequences provided the minimum dwell-time condition is satisfied.

Keywords: Aerospace, Constrained control, Control applications
Abstract: In this paper we propose an anti-windup strategy to counteract directionality effects arising in saturated MIMO systems in which independent dynamical subsystems are coupled through a static mixing of the inputs. Since such systems are affected by undesired input cross-couplings when saturation occurs, we propose an anti-windup augmentation scheme built on top of the baseline controller and tailored to achieve satisfactory time-domain performance for reference signals of interest. Motivated by the quadrotor application, in which position control has higher priority over yaw control for safety reasons, we embed in the anti-windup synthesis procedure the possibility to prioritize the level of performance degradation during saturation for the different system outputs.

Keywords: Aerospace, Control applications, Hybrid systems
Abstract: It is well known that small electric Unmanned Aerial Vehicles (UAVs) suffer from low endurance problems. A possibility to extend the range of UAV missions could be to have a carrier drone with several lightweight multirotors aboard, which can take-off from and land on it. In this paper the challenging problem of Air-to-Air Automatic Landing (AAAL) of UAVs is solved by developing a strategy that combines a quasi-time optimal feedback and a hybrid logic to ensure a safe and fast landing. Eventually, the proposed algorithm is validated through experimental activities involving the landing of a small quadcopter on a bigger octocopter used as a carrier.

Keywords: Agents-based systems, Distributed control, Delay systems
Abstract: Consensus algorithms provide a framework for the distributed coordination of a multi-agent system. However, widespread application and deployment of consensus algorithms may be limited in real-world multi-agent coordination problems due to implementation on size, power, and weight constrained vehicles. In this case, limited resources may contribute to delay and packet loss causing algorithm deterioration and violation of performance guarantees. This calls for novel strategies for intelligent resource utilization and computationally simple implementation. Towards this goal, we propose co-regulation strategies for discrete time average consensus under delays allowing dynamic resource utilization while coping with communication limitations. This is done by dynamically adjusting communication frequency to facilitate higher state exchange rates while simultaneously adjusting agents' locations to increase inter-agent connectivity for rapid convergence. We prove that convergence is still guaranteed for co-regulation strategies for discrete time average consensus under bounded delays. In addition, we propose a pause for agents' locations to mitigate adverse behavior caused by delay. To simplify implementation we devise a consensus strategy that decouples the co-regulated consensus from low-level vehicle feedback control. The usability of our proposed system is evaluated through a series of simulations, and we show our proposed co-regulation strategies in fact result in faster convergence time. We evaluate the approach with an outdoor experiment using 4 customized unmanned aircraft systems (UASs).

Keywords: Agents-based systems, Information theory and control, Cooperative control
Abstract: This paper addresses the problem of searching and tracking of an a priori unknown number of indistinguishable targets spread over some geographical area using a fleet of UAVs. State perturbations and measurement noises are assumed to belong to bounded sets. In the monitored geographical area, some false targets (decoys) are present and may be erroneously considered as targets when observed under specific conditions. Moreover, obstacles in the search area constrain the displacements of the targets, alter the UAVs' trajectories, reduce their fields of view, and limit their communications. While the UAVs can detect targets or decoys when observation conditions are satisfied, they cannot identify them individually.

The search process relies on a robust bounded-error estimation approach which aim is to evaluate a set guaranteed to contain the actual states of already localized true targets and a set containing the states of targets still to be discovered. These two sets are used by each UAV to determine their control inputs in a distributed way to minimize future estimation uncertainty.

Simulations involving several UAVs illustrate that the proposed robust set-membership estimator and distributed control laws make it possible to efficiently search and track targets in the presence of decoys in a cluttered area.

Keywords: Air traffic management, Machine learning, Optimal control
Abstract: In this paper, we study the online multi-robot minimum time-energy path planning problem subject to collision avoidance and input constraints in an unknown environment. We develop an online adaptive solution for the problem using integral reinforcement learning (IRL). This is achieved through transforming the finite-horizon minimum time-energy problem with input constraints to an approximate infinite-horizon optimal control problem. To achieve collision avoidance, we incorporate artificial potential fields into the approximate cost function. We develop an IRL-based optimal control strategy and prove its convergence. The theoretical results are verified through simulation studies.

Keywords: Autonomous systems, Aerospace, Lyapunov methods
Abstract: This paper presents a control system for a quadrotor unmanned aerial vehicle transporting a payload connected by a link. This system is considered as a multibody system evolving a nonlinear configuration manifold to construct an intrinsic global formulation of the dynamics. Based on this, a geometric control system is proposed such that the payload follows an arbitrary desired trajectory. In particular, the effects of the mass distribution of the link connecting the quadrotor and the payload are explicitly considered in the dynamic model and the stability analysis. Furthermore, the control system is designed to eliminate the adverse effects of disturbances. The efficacy of the proposed geometric tracking control system is illustrated by both a numerical example and an indoor flight experiment.

Keywords: Agents-based systems, Autonomous systems, Decentralized control
Abstract: In this paper, we present an original set of flocking rules using an ecologically-inspired paradigm for control of multi-robot systems. We translate these rules into a constraint-driven optimal control problem where the agents minimize energy consumption subject to safety and task constraints. We prove several properties about the feasible space of the optimal control problem and show that velocity consensus is an optimal solution. We also motivate the inclusion of slack variables in constraint-driven problems when the global state is only partially observable by each agent. Finally, we analyze the case where the communication topology is fixed and connected and prove that our proposed flocking rules achieve velocity consensus.

Keywords: Cyber-Physical Security, Estimation, Constrained control
Abstract: Unmanned aerial vehicles (UAVs) suffer from sensor drifts in GPS denied environments, which can lead to potentially dangerous situations. To avoid intolerable sensor drifts in the presence of GPS spoofing attacks, we propose a safety constrained control framework that adapts the UAV at a path re-planning level to support resilient state estimation against GPS spoofing attacks. The attack detector is used to detect GPS spoofing attacks and provides a switching criterion between the robust control mode and emergency control mode. An attacker location tracker (ALT) is developed to track the attacker's location and estimate the spoofing device's output power by the unscented Kalman filter (UKF) with sliding window outputs. Using the estimates from ALT, we design an escape controller (ESC) based on the model predictive controller (MPC) such that the UAV escapes from the effective range of the spoofing device within the escape time.

Keywords: Autonomous vehicles, Automotive systems, Traffic control
Abstract: This paper investigates the impact of disturbances on controlling an autonomous vehicle to smooth mixed traffic flow in a ring road setup. By exploiting the ring structure of this system, it is shown that velocity perturbations impacting any vehicle on the ring enter an uncontrollable and marginally stable mode defined by the sum of relative vehicle spacings. These disturbances are then integrated up by the system and cannot be unwound via controlling the autonomous vehicle. In particular, if the velocity disturbances are zero-mean Gaussians, then the traffic flow on the ring will undergo a random walk with the variance growing indefinitely and independently of the control policy applied. In contrast, the impact of acceleration disturbances is benign as these disturbances do no enter the uncontrollable mode, meaning that they can be easily regulated using the autonomous vehicle. Our results support and complement the existing theoretic analysis and field experiments.

Keywords: Autonomous vehicles, Traffic control, Networked control systems
Abstract: Vehicle-to-vehicle (V2V) communications have a great potential to improve traffic system performance. Most existing work of connected and autonomous vehicles (CAVs) focused on adaptation to downstream traffic conditions, neglecting the impact of CAVs' behaviors on upstream traffic flow. In this paper, we introduce a notion of Leading Cruise Control (LCC) that retains the basic car-following operation and explicitly considers the influence of the CAV's actions on the vehicles behind. We first present a detailed modeling process for LCC. Then, rigorous controllability analysis verifies the feasibility of exploiting the CAV as a leader to actively lead the motion of its following vehicles. Besides, the head-to-tail transfer function is derived for LCC under adequate employment of V2V connectivity. Numerical studies confirm the potential of LCC to strengthen the capability of CAVs in suppressing traffic instabilities and smoothing traffic flow.

Keywords: Traffic control, Autonomous vehicles, Observers for nonlinear systems
Abstract: This article deals with the observation problem in traffic flow theory. The model used is the quasilinear viscous Burgers equation. Instead of using the traditional fixed sensors to estimate the state of the traffic at given points, the measurements here are obtained from Probe Vehicles (PVs). We propose then a moving dynamic boundary observer whose boundaries are defined by the trajectories of the PVs. The main result of this article is the exponential convergence of the observation error, and, in some cases, its finite-time convergence. Finally, numerical simulations show that it is possible to observe the traffic in the congested, free-flow, and mixed regimes provided that the number of PVs is large enough.

Keywords: Traffic control, Autonomous vehicles, Delay systems
Abstract: The problem of controlling traffic using connected automated vehicles is approached by utilizing Lagrangian traffic models. A continuum model with time delay is introduced in the Lagrangian frame in order to capture the open loop dynamics of the traffic behind a vehicle of prescribed motion. The stability of the open loop system is analyzed and compared to that of a benchmark car-following model. Finally, the Lagrangian traffic models are used to propose a longitudinal controller for connected automated vehicles that allows them to respond to connected vehicles behind to stabilize the upstream traffic in a closed loop fashion.

Keywords: Autonomous vehicles, Energy systems, Learning
Abstract: The wide use of infrastructure-to-vehicle communication technologies can enable improved energy efficient autonomous driving. Traditional ecological velocity planning methods have high computational burden, particularly when plug-in hybrid electric vehicles are considered. Consequently, in order to retrieve an optimal velocity profile in real time, it is necessary to rely on significant approximations.

In this paper, the aforementioned issue is addressed by exploiting deep reinforcement learning in order to ``learn'' an eco-driving velocity planner for a plug-in hybrid electric vehicle within a model-free approach. Moreover, we incorporate a state-of-the-art safety controller based on robust model predictive control to guarantee traffic light compliance. Statistical analysis of the simulation results demonstrate that the RL controller outperforms two benchmark controllers, and it generalizes well across a variety of intersection configurations.

Keywords: Transportation networks, Network analysis and control, Traffic control
Abstract: This paper studies the stability of traffic networks when the travelers follow congestion-dependent routing recommendations provided by routing apps. Despite the widespread use of app-based navigation systems, which allow drivers to react in real-time to fluctuations in traffic congestion, a thorough characterization of the benefits and impact of these devices on general and capacitated traffic systems has remained elusive until now. We first propose a dynamical routing model to describe the instantaneous route-update mechanism that is at the core of navigation apps, and then we leverage the theory of passivity for nonlinear dynamical systems to provide a theoretical framework for the analysis of traffic stability. We provide a formal proof of existence of oscillatory trajectories due to the general adoption of routing apps, which demonstrate how drivers continuously switch between highways in the attempt of minimizing their travel time to destination. These findings are used to explain oscillatory behaviors observed in the highway system in Southern California, and inform the design of novel app-based congestion control strategies. Empirical data and illustrative examples demonstrate our theoretical findings.

Keywords: Traffic control, Stochastic optimal control
Abstract: In order to ensure customer satisfaction, ride-hailing providers typically base their routing strategies around minimizing the total traffic delay faced by their users. However, accurately modeling the provider's routing problem is difficult, as realistic traffic networks are non-linear, stochastic, and time-varying. We approach this problem by modelling the dynamic traffic network using the Lighthill, Whitham, and Richards model and use stochastic path integral control to construct the routing strategy of the ride-hailing provider. Differing from previous results, we allow for multiple input and output locations, as well as varying speed limits and maximum traffic densities. Furthermore, we allow the ride-hailing provider to additionally control its traffic flow at splits in the traffic network, which avoids an exponential blow-up in the state space. A numerical example for a representative traffic network is provided to demonstrate the efficacy of the proposed method.

Keywords: Transportation networks, Network analysis and control
Abstract: Through vehicle platooning, autonomous vehicles are capable of maintaining variable longitudinal headway, which can be shorter than the usual headway of human-driven vehicles. Thus, autonomous vehicles are expected to be capable of increasing road capacities. In this work, we consider a scenario where a centralized authority is able to specify the target inter-vehicle headway in autonomous vehicle platoons on the roads and as a consequence, adjust roadways' flow capacities in mixed (human--driven/autonomous) network traffic. We employ a variable, capacity asymmetry degree, which is the ratio between the road capacity when all vehicles are human-driven and the road capacity when all vehicles are autonomous, to characterize and reflect autonomous vehicles? shorter headway compared to human-driven vehicles. We then consider a routing game with inelastic demands on traffic networks with a homogeneous capacity asymmetry degree across the network. We study the impact of the variable capacity asymmetry degree on the overall delay of the network at the Wardrop routing equilibrium. We show that for networks with a single origin-destination pair, we can always decrease the overall or social network delay by decreasing the capacity asymmetry degree (reducing the headway for the autonomous vehicle platoons). Specifically, for series parallel networks with a single origin-destination pair and affine delay functions, we upper bound the improvement on the social delay by reducing the headway for the autonomous vehicle platoons.

Keywords: Identification, Estimation, Optimization
Abstract: We face the factor analysis problem using a particular class of autoregressive processes. We propose an approximate moment matching approach to estimate the number of factors as well as the parameters of the model. This algorithm alternates a step of factor analysis and a step of AR dynamics estimation. Some simulation studies show the effectiveness of the proposed estimator.

Keywords: Identification, Estimation, Optimization
Abstract: We revisit the image compression problem using the framework introduced by Ringh, Karlsson and Lindquist. More precisely, we explore the possibility to consider a family of objective functions and a different way to design the prior in the corresponding multidimensional circulant covariance extension problem. The latter leads to refined compression paradigms.

Keywords: Identification, Estimation, Filtering
Abstract: The problem of noncausal identification of a nonstationary stochastic FIR (finite impulse response) system is reformulated, and solved, as a problem of smoothing of preestimated parameter trajectories. Three approaches to preestimation are critically analyzed and compared. It is shown that optimization of the smoothing operation can be performed adaptively using the parallel estimation technique. The new approach is computationally attractive and yields estimation results that are comparable or better than those provided by the state-of-the-art local basis function approach and the multi-resolution wavelet approach.

Keywords: Identification, Estimation, Subspace methods
Abstract: In this paper structural identifiability of state space models, possibly nonlinear in parameters, is assessed by analyzing the controllability of the output sensitivities. Sensitivity analysis provides a mathematical setting to analyze parameter identifiability from a physically intuitive perspective. Both SISO and MIMO cases are treated; in the former case the output controllability matrix rank directly allows to draw conclusions on the model structural identifiability. In the latter case, the analysis requires special attention due to the ordering induced by the vector derivative. The approach is illustrated on a linear compartmental model

Keywords: Identification
Abstract: In this paper, we study the influence of illconditioned regression matrix on two hyper-parameter estimation methods for the kernel-based regularization method: the empirical Bayes (EB) and the Stein?s unbiased risk estimator (SURE). First, we consider the convergence rate of the cost functions of EB and SURE, and we find that they have the same convergence rate but the influence of the ill-conditioned regression matrix on the scale factor are different: the scale factor for SURE contains one more factor cond(ΦT Φ) than that of EB, where Φ is the regression matrix and cond(?) denotes the condition number of a matrix. This finding indicates that when Φ is ill-conditioned, i.e., cond(ΦT Φ) is large, the cost function of SURE converges slower than that of EB. Then we consider the convergence rate of the optimal hyper-parameters of EB and SURE, and we find that they are both asymptotically normally distributed and have the same convergence rate, but the influence of the ill-conditioned regression matrix on the scale factor are different. In particular, for the ridge regression case, we show that the optimal hyper-parameter of SURE converges slower than that of EB with a factor of 1/n2 , as cond(ΦT Φ) goes to ∞, where n is the FIR model order.

Keywords: Identification, Model Validation, Kalman filtering
Abstract: Finite sample system identification (FSID) methods construct confidence regions for system parameters with non-asymptotic guarantees under minimal assumptions on the noise distribution. This paper deals with constructing non-asymptotic confidence region using Sign Perturbed Sums (SPS) approach for a linear module in a cascade of dynamical systems. In dynamic networks every measurement is usually contaminated with noise. However, the SPS approach was originally devised for systems with no noise on the input. The SPS approach was extended to Errors-In-Variables (EIV) systems where both the input and the output signal are measured in noise under the assumption that the true input signal is an independent sequence. However, in a dynamic network the input of a module is not usually an independent sequence since it is typically the output of another dynamical system. In this paper the SPS approach is extended to EIV systems without making any assumption on the true input signal. Then, the approach is used to construct confidence region for a single module in a simple cascade network by incorporating additional data and taking advantage of the cascade structure. This is done without estimating other modules in the network. The method is illustrated in numerical experiments.

Keywords: Estimation, Identification, Indirect adaptive control
Abstract: The real-time estimation of normal regression-type models with unknown time-varying parameters is considered and discussed from the Bayesian perspective. A novel tracking technique combining the variable regularization approach with the forgetting operation is derived and elaborated into algorithmic details. The regularization of the parameter covariance is accomplished by incorporating soft equality constraints on the regression parameters into the learning procedure. The resultant procedure is designed to smooth the parameter estimate, preventing it from changing too rapidly. Moreover, the form of the constraints guarantees a minimal amount of information about the parameter estimate, which makes the estimator robust with respect to poor system excitation. The forgetting of obsolete information is provided in two different parameterization options and is performed automatically in a way that complies with the degree of the process nonstationarity. The whole concept preserves the self-reproducibility of the statistics of the normal-Wishart distribution.

Keywords: Numerical algorithms, Computational methods, Identification
Abstract: The potential for numerical instabilities of Dynamic Mode Decomposition (DMD), which assumes the completeness of the eigenspace is discussed for cases where the underlying system is defective or nearly defective. A numerically stable approach based on Schur decomposition is presented. The proposed method complements the DMD for cases where eigendecomposition is ill-conditioned. Both mathematical analysis and the results of numerical experiments are presented.

Keywords: Game theory, Agents-based systems, Optimization
Abstract: For the generation of higher-quality solutions to the distributed minimum weighted vertex cover (MWVC) problem, we propose a game theoretic learning algorithm by designing a weighted memory based rule. Being viewed as a rational player, each node in the network stochastically updates its action by following a probability distribution that is determined by neighborhood information including node degrees and weights. Within the framework of game theory, we prove that our method converges with probability 1 to Nash equilibria that correspond to near-optimal vertex cover solutions. Moreover, simulation results show that the memory length provides an additional freedom for solution efficiency improvement such that better system level objectives are more likely to be obtained by using a longer memory length. Comparison experiments with typical distributed algorithms demonstrate the superiority of the presented methodology to the state of the art.

Keywords: Game theory, Optimal control, Stochastic systems
Abstract: We consider a stochastic game with partial, asymmetric and non-classical information. Agents have access only to local information, the information updates are asynchronous and our aim is to obtain relevant equilibrium policies. Our approach is to consider optimal open-loop control until the information update, which allows managing the belief updates in a structured manner. The agents continuously control the rates of their Poisson search clocks to acquire the locks, and they get rewards at every successful acquisition; an acquisition is successful if all the previous stages are successful and the agent is the first one to complete. They also derive a terminal reward, upon successful completion of the project. However, none of them have access to the acquisition status of the other agents, leading to an asymmetric information game. Using standard tools of optimal control theory and Markov decision process (MDP) we solved a two-level control problem; every stage of the dynamic programming equation of the MDP is solved using optimal control tools. We finally reduced the game with an infinite number of states and infinite-dimensional actions to a finite state game with one-dimensional actions. We provided closed-form expressions for Nash Equilibrium in some special cases and derived asymptotic expressions for some more.

Keywords: Game theory, Optimization algorithms, Cooperative control
Abstract: We consider autonomous agents communicating over a random communication network that is subject to failures. Each agent aims to maximize its own utility function that depends on the actions of other agents and an unknown state of the environment. Posing this problem as a game, we study a decentralized fictitious play algorithm with a voluntary communication protocol (DFP-V) for Nash equilibrium (NE) computation. In the voluntary communication protocol, each agent locally manages whom to exchange information with by assessing the novelty of its information and the potential effect of its information on others' assessments of their utility functions. We show convergence of the algorithm to a pure NE in finite time for the class of weakly acyclic games. Numerical experiments demonstrate that the voluntary communication protocol reduces number of communication attempts significantly without hampering performance.

Keywords: Game theory, Stochastic systems
Abstract: A stochastic differential game with two players on an infinite time horizon, a quadratic payoff, and a scalar linear stochastic system that includes the two players' actions and a Rosenblatt process noise is formulated and explicitly solved. The players actions are restricted to the family of linear feedback strategies to allow for explicit solutions. For the infinite time horizon problem, strategies are given explicitly in terms of feedback gains that depend on some values of Gamma functions. Rosenblatt processes are a family of non-Gaussian processes that can have application to physical situations where the noise has been empirically determined to be continuous and to have a long range dependence but not to satisfy a Gaussian distribution. The Rosenblatt processes can be described as suitable double Wiener-It^o integrals with singular kernels. It seems that there are no explicit game solutions when the noise is continuous, long range dependent, and non-Gaussian.

Keywords: Game theory, Stochastic systems, Stochastic optimal control
Abstract: In this paper, we consider a sequential stochastic Stackelberg game with two players, a leader, and a follower. The follower observes the state of the system privately while the leader does not. Players play Stackelberg equilibrium where the follower plays best response to the leader's strategy. In such a scenario, the leader has the advantage of committing to a policy that maximizes its returns given the knowledge that the follower is going to play the best response to its policy. Such a pair of strategies of both the players is defined as Stackelberg equilibrium of the game. Recently, [1] provided a sequential decomposition algorithm to compute the Stackelberg equilibrium for such games which allow for the computation of Markovian equilibrium policies in linear time as opposed to double exponential, as before. In this paper, we extend that idea to the case when the state update dynamics are not known to the players, to propose an reinforcement learning (RL) algorithm based on Expected Sarsa that learns the Stackelberg equilibrium policy by simulating a model of the underlying Markov decision process (MDP). We use particle filters to estimate the belief update for a common agent that computes the optimal policy based on the information which is common to both the players. We present a security game example to illustrate the policy learned by our algorithm.

Keywords: Game theory, Stochastic systems, Cyber-Physical Security
Abstract: We consider a setting where a receiver tries to perfectly recover a source signal privately known to a sender who can send messages via a noisy channel. However, the sender is compromised and may have an incentive to lie about its information. We formulate the problem as a game between the sender and the receiver and show that there is a strategy for the receiver in which it can recover an exponential number of signals. We define a notion of a sender graph and the information extraction capacity of the sender which quantifies the maximum amount of information that can be extracted from the sender. We also show that the rate of information extraction is given by the minimum of the information extraction capacity of the sender and the zero-error capacity of the noisy channel.

Keywords: Game theory, Robust control, Numerical algorithms
Abstract: In this paper, a robust Stackelberg game for a class of uncertain Markov jump delay stochastic systems (UMJDSSs) is investigated. After introducing some definitions and preliminaries, we derive the conditions for the existence of a robust Stackelberg strategy set by means of cross coupled stochastic bilinear matrix inequalities (CCSBMIs), such that the upper bound of each decision maker's cost function is minimized. To overcome difficulties in solving the CCSBMIs, a feasible numerical algorithm based on the Krasnoselskii iteration sequence is proposed, which consists of linear matrix inequalities and cross coupled stochastic matrix equations (CCSMEs). It is shown that the weakly convergence property is attained. Finally, a practical example is solved to demonstrate the effectiveness and efficiency of the proposed scheme.

Keywords: Game theory, Uncertain systems, Robust control
Abstract: Robustness is a key challenge in the integration of learning and control. In machine learning and robotics, two common approaches to promote robustness are adversarial training and domain randomization. Both of these approaches have analogs in control theory: adversarial training relates to H-infinity control and dynamic game theory, while domain randomization relates to theory for systems with stochastic model parameters. We propose a stochastic dynamic game framework that integrates both of these complementary approaches to modeling uncertainty and promoting robustness. We describe policy iteration algorithms in both model-based and model-free settings to compute equilibrium strategies and value functions. We present numerical experiments that illustrate their effectiveness and the value of combining uncertainty representations in our integrated framework. We also provide an open-source implementation of the algorithms to facilitate their wider use.

Keywords: Optimization, Optimization algorithms, Large-scale systems
Abstract: We consider an optimization problem with strongly convex objective and linear inequalities constraints. To be able to deal with a large number of constraints we provide a penalty reformulation of the problem. As penalty functions we use a version of the one-sided Huber losses. The smoothness properties of these functions allow us to choose time-varying penalty parameters in such a way that the incremental procedure with the diminishing step-size converges to the exact solution with the rate O(1/{sqrt k}). To the best of our knowledge, we present the first result on the convergence rate for the penalty-based gradient method, in which the penalty parameters vary with time.

Keywords: Optimization, Optimization algorithms, Networked control systems
Abstract: This paper addresses a class of constrained distributed nonconvex optimization problems considering time-varying directed networks. We propose a novel algorithm named Extended-CPCA (E-CPCA), exploiting the notion of combining Chebyshev polynomial approximation and average consensus. The proposed algorithm has the advantages of being i) able to yield ε globally optimal solutions for any arbitrarily small given tolerance ε; ii) efficient in terms of both oracle complexities and inter-agent communication costs, and iii) distributively terminable when the specified precision requirement is met. The idea of leveraging polynomial proxy and consensus to deal with the mentioned problems over static undirected graphs is first presented in our previous work. The novelties of this work lie in i) the utilization of push-sum average consensus with distributed stopping mechanism to enable agents to acquire a proxy for the global objective over time-varying digraphs without much wastes of extra communications, and ii) the transformation of the optimization of this global proxy to a semidefinite program in order to obtain solutions in a fast and reliable manner. Both the analysis and simulations are provided to illustrate the efficacy of the proposed algorithm.

Keywords: Optimization, Optimization algorithms, Numerical algorithms
Abstract: In semidefinite programming (SDP), a number of pre-processing techniques have been developed, including procedures based on chordal decomposition, which exploit sparsity in the semidefinite program in order to reduce the dimension of individual constraints, and procedures based on facial reduction, which reduces the dimension of the problem by removing redundant rows and columns. So far, these have been studied in isolation. We show that these techniques are, in fact,complementary. In computational experiments, we show that a two-step pre-processing followed by a standard interior-point method outperforms the interior point method, with or without either of the pre-processing techniques, by a considerable margin.

Keywords: Optimization, Computational methods, Sensor networks
Abstract: In this paper, we address the beamforming problem, which asks to choose the best subset of antennas and their corresponding amplitudes and phases to match a given beam pattern. To solve this problem, we propose an optimization formulation that can efficiently solve large scale problems, and is versatile in its ability to express a variety of meaningful subset selection scenarios. Focusing on the case of antennas with fixed positions, without assuming any geometric structure, we show how to cast the beamforming problem as a regularized least squares problem which can be efficiently solved. Drawing inspiration from subset selection problems in submodular optimization, we select meaningful submodular set functions and use their Lovàsz extensions to create convex regularizers promoting antenna selection in a useful manner, as demonstrated in a number of presented scenarios.

Keywords: Optimization, Learning, Estimation
Abstract: Learning sparse models from data is an important task in all those frameworks where relevant information should be identified within a large dataset. This can be achieved by formulating and solving suitable sparsity promoting optimization problems. As to linear regression models, Lasso is the most popular convex approach, based on an l1-norm regularization. In contrast, in this paper, we analyse a concave regularized approach, and we prove that it relaxes the irrepresentable condition, which is sufficient and essentially necessary for Lasso to select the right significant parameters. In practice, this has the benefit of reducing the number of necessary measurements with respect to Lasso. Since the proposed problem is non-convex, we also discuss different algorithms to solve it, and we illustrate the obtained enhancement via numerical experiments.

Keywords: Optimization, Optimization algorithms
Abstract: The shortest path problem is formulated as an l_1-regularized regression problem, known as lasso. Based on this formulation, a connection is established between Dijkstra's shortest path algorithm and the least angle regression (LARS) for the lasso problem. ?Specifically, the solution path of the lasso problem, obtained by varying the regularization parameter from infinity to zero (the regularization path), corresponds to shortest path trees that appear in the bi-directional Dijkstra algorithm.

Keywords: Optimization, Optimization algorithms
Abstract: We present a new class of optimization models, implicit optimization, which include deep learning, nonlinear control, mixed-integer programming as special cases and therefore provide a unified perspective on these different fields, leading to new algorithms and surprising connections. We propose two tractable algorithms to solve such problems based on their implicit equation structure: implicit gradient descent and the Fenchel alternative direction method of multipliers. We illustrate our theory and methods with numerical experiments.

Keywords: Optimization, Quantum information and control, Communication networks
Abstract: In this paper, we explore a new approach to optimization of cost or utility functions defined over a surface, a manifold, or its simplicial decomposition. In the era of ?Big Data,? heterogeneous signal samples sometimes embed with less distortion in a lower dimensional space if the embedding space is a manifold rather than the traditional Euclidean space. If a utility function is defined over the data and if there is a need to identify significant events defined by extreme values of the utility function, we are faced with the problem of identifying the extreme minima/maxima points of the cost/utility function defined over the manifold or its triangulation. The fundamental idea developed here is to observe that at the extreme points the graph of the utility function has extreme curvature. Accordingly, the celebrated Ricci/Yamabe flow for uniformization of the curvature of the graph will show significant ``curvature transport" in the vicinity of the extreme values, hence allowing their rapid identification, obviating the classical sorting. The novel theoretical contribution is to accelerate the process by compounding the Laplace operator.

Keywords: Optimal control, Game theory, Robust control
Abstract: Optimality in a min-max sense for constrained difference equations in the presence of disturbances is studied as a two-player zero-sum game. Sufficient conditions that permit the evaluation and to upper bound the cost of solutions to such systems are presented. Cost evaluation results are also presented for the case in which a control input aims to minimize a cost functional while the objective of disturbance is to maximize it. Sufficient conditions in the form of Hamilton-Jacobi-Isaacs equations are provided to certify closed-loop saddle point optimality. The results are illustrated in an example featuring a linearized and discretized model of an inverted pendulum.

Keywords: Optimal control, Formal Verification/Synthesis, Computational methods
Abstract: The present work deals with quantitative two-phase reach-avoid problems on nonlinear control systems. This class of optimal control problem requires the plant's state to visit two (rather than one) target sets in succession while minimizing a prescribed cost functional. As we illustrate, the naive approach, which subdivides the problem into the two evident classical reach-avoid tasks, usually does not result in an optimal solution. In contrast, we prove that an optimal controller is obtained by consecutively solving two special quantitative reach-avoid problems. In addition, we present a fully-automated method based on Symbolic Optimal Control to practically synthesize for the considered problem class approximately optimal controllers for sampled-data nonlinear plants. Experimental results on parcel delivery and on an aircraft routing mission confirm the practicality of our method.

Keywords: Optimal control, Numerical algorithms, Hybrid systems
Abstract: We present a novel reformulation of nonsmooth differential equations with state jumps which enables their easier simulation and use in optimal control problems without the need of using integer variables.

The main idea is to introduce an auxiliary differential equation to mimic the state jump map. Thereby, also a clock state is introduced which does not evolve during the runtime of the auxiliary system. The pieces of the trajectory that correspond to the parts when the clock state was evolving recover the solution of the original system with jumps. Our reformulation results in nonsmooth ordinary differential equations where the discontinuity is in the first time derivative of the trajectory, rather than in the trajectory itself. This class of systems is easier to handle both theoretically and numerically. We provide numerical examples demonstrating the ease of use of this reformulation in both simulation and optimal control. In the optimal control example a single call of a nonlinear programming (NLP) solver yields the same solution as a multi-stage formulation, without the need for exploring the optimal number of stages by enumeration or heuristics.

Keywords: Optimal control, Numerical algorithms, Nonholonomic systems
Abstract: We consider singular optimal control problems stated for differential-algebraic measure-driven equations. The latter take the form of complementarity systems, which admit jumps of state trajectories produced by open-loop and state-conditioned impulses, acting independently. As an inspiration for the addressed class of models, we turn to mechanical systems driven by time-dependent holonomic (shock impacts, control vibrations) and/or nonholonomic (impactive blocking/releasing system?s degrees of freedom) constraints.

For our problems, we adapt an approach based on singular space-time transformation. This approach involves approximation of discontinuous solutions of the impulsive complementarity system by ordinary, physically meaningful control processes. Since the complexity of the addressed problems is too high for the analytical investigation, we focus on their numerical investigation, and propose a general computational method.

Keywords: Optimal control, Constrained control, Optimization
Abstract: Reach-avoid problems compute control laws under which a dynamic system can reach a desired set of states while avoiding another set. They are used in solving a variety of problems, such as goal-seeking and obstacle avoidance. Hamilton-Jacobi analysis provides a method for solving reach-avoid problems, through a corresponding Hamilton-Jacobi (HJ) equation. Although the HJ equation can be utilized for a general class of problems, computing the solution of the HJ equation by grid-based methods has exponential complexity in the dimension of the continuous state. To alleviate this complexity, this paper presents a generalized version of the Hopf-Lax formula and proves its correctness for solving the reach-avoid problem. The method does not rely on discretization of the state space. A numerical algorithm for the proposed Hopf-Lax formula is presented, and demonstrated using a 12D vehicle example.

Keywords: Optimal control, Numerical algorithms, Differential-algebraic systems
Abstract: Some direct transcription methods can fail to converge, e.g. when there are singular arcs. We recently introduced a convergent direct transcription method for optimal control problems, called the penalty-barrier finite element method (PBF). PBF converges under very weak assumptions on the problem instance. PBF avoids the ringing between collocation points, for example, by avoiding collocation entirely. Instead, equality path constraint residuals are forced to zero everywhere by an integral quadratic penalty term. We highlight conceptual differences between collocation- and penalty-type direct transcription methods. Theoretical convergence results for both types of methods are reviewed and compared. Formulas for implementing PBF are presented, with details on the formulation as a nonlinear program (NLP), sparsity and solution. Numerical experiments compare PBF against several collocation methods with regard to robustness, accuracy, sparsity and computational cost. We show that the computational cost, sparsity and construction of the NLP functions are roughly the same as for orthogonal collocation methods of the same degree and mesh. As an advantage, PBF converges in cases where collocation methods fail. PBF also allows one to trade off computational cost, optimality and violation of differential and other equality equations against each other.

Keywords: Optimal control, Optimization, Constrained control
Abstract: In this article, necessary conditions of optimality in the Gamkrelidze?s form for a state constrained optimal control problem whose dynamics are given by a differential inclusion are addressed. This form differs from the more usual Dubovitskii-Milyutin one in that, now, in the Hamiltonian, the measure multiplier appearing in the latter is replaced by a component related to its integral. Nondegeneracy, and normality conditions are discussed for this dynamic optimization problem under several types of sufficient conditions on the data of the problem.

Keywords: Optimal control, Variational methods, Agents-based systems
Abstract: In this article, we propose a new unifying framework for the investigation of multi-agent control problems in the mean-field setting. Our approach is based on a new definition of differential inclusions for continuity equations in the Wasserstein spaces of optimal transport. The latter allows to recover several known results of the theory of differential inclusions, and to prove an exact correspondence between solutions of differential inclusions and control systems. We show its appropiateness on an example of leader-follower evacuation problem with soft congestions.

Keywords: Formal Verification/Synthesis, Control of networks, Large-scale systems
Abstract: We construct compositional continuous approximations for an interconnection of infinitely many discrete-time switched systems. An approximation (known as abstraction) is itself a continuous-space system, which can be used as a replacement of the original (known as concrete) system in a controller design process. Having synthesized a controller for the abstract system, the controller is refined to a more detailed controller for the concrete system. To quantify the mismatch between the output trajectory of the approximation and of that the original system, we use the notion of so-called simulation functions. In particular, each subsystem in the concrete network and its corresponding one in the abstract network is related through a local simulation function. We show that if the local simulation functions satisfy a certain small-gain type condition developed for a network of infinitely many subsystems, then the aggregation of the individual simulation functions provides an overall simulation function between the overall abstraction and the concrete network. For a network of linear switched systems, we systematically construct local abstractions and local simulation functions, where the required conditions are expressed in terms of linear matrix inequalities and can be efficiently computed. We illustrate the effectiveness of our approach through an application to frequency control in a power gird with a switched (i.e. time-varying) topology.

Keywords: Large-scale systems, Lyapunov methods, Stability of nonlinear systems
Abstract: This paper proposes a methodology for establishing stability of nonlinear dynamical networks through balancing mechanisms that appear in system models explicitly. In addition to global asymptotic stability of equilibria, the notion of strong integral input-to-state stability (Strong iISS) is employed to address robustness of the networks with respect to disturbances and noises in the presence of saturation nonlinearities forming balancing relations. Conservation is covered as the exact balance. Unlike network small-gain criteria known in the literature of iISS, the developed criteria do not lump the balancing relations. The formulation covers positive networks and networks of iISS systems.

Keywords: Resilient Control Systems, Networked control systems, Agents-based systems
Abstract: This paper investigates the multi-agent leader-follower consensus problem under Denial-of-Service (DoS) attacks with data rate constraints. In our analysis, we first derive a lower bound on the data rate for the leader-follower consensus problem, which can guarantee the quantizers of the leader and followers do not overflow at any time, even in the presence of DoS attacks. The main contribution of the paper is the characterization of the trade-off between the tolerable DoS attack levels for leader-follower consensus and the required data rates for the quantizers during the communication attempts among the agents. To mitigate the influence of DoS attacks, we employ dynamic quantization mechanism with scale-up and scale-down capabilities. This paper also reveals the another trade-off between the data rate for the leader state quantization and the one for the follower state quantization. This makes saving the overall communication load and maintaining the resilience possible.

Keywords: Network analysis and control, Large-scale systems, Stability of nonlinear systems
Abstract: This work introduces antagonistic interactions into the so-called biased assimilation model of opinion dynamics, a nonlinear model expressing the bias of the agents towards their own opinions. In this model, opinions exchanged in a signed network are multiplied by a state dependent term having the bias as exponent. For small values of the bias parameters, while for structurally balanced networks polarization always occurs, we show that for structurally unbalanced networks also a state of indecision (corresponding to the centroid of the opinion hypercube) is a local attractor. When instead the biases are all large, the opinions usually polarize. In particular, for large enough biases if all agents take the same initial stance (i.e., are all on the same side of the indecision state), then the opinions polarize all to the same extreme value for both structurally balanced network and structurally unbalanced network.

Keywords: Game theory, Autonomous systems, Networked control systems
Abstract: In this paper, we consider a network of autonomous agents with passive linear-time invariant dynamics involved in a game with coupled constraints. In such networked scenarios, agents have to make decisions compatible with seeking a generalized Nash equilibrium (GNE), while using networked information and satisfying the constraints. Existing methods are developed for multi-integrator agents only and furthermore, ensure the satisfaction of coupled constraints in steady-state only. We propose an inexact-penalty dynamics for passive LTI agents and show that it converges to an epsilon-GNE while ensuring the coupled constraints are met throughout the evolution of the agents? dynamics, not only in steady-state. Our scheme is developed for both the full-decision information setting and the partial-decision one. In the partial-information setting, each agent makes its decision based on a dynamic estimate of the others? states, updated by local communication with its neighbours, which offsets the lack of global information. Applications to optical networks are provided.

Keywords: Decentralized control, Optimization, Game theory
Abstract: This paper addresses the decentralized charging coordination of a fleet of plug-in electric vehicles (PEVs). In particular, we cast the charging coordination task as a constrained multi-objective optimization problem, and we solve it using a novel receding horizon decentralized optimization method based on multi-population games. Our proposed method is able to coordinate the charging process of arbitrary fleets of PEVs, while satisfying hard operational constraints over the system's variables. Our theoretical developments are illustrated through numerical simulations of various PEV-fleets of different sizes.

Keywords: Game theory, Iterative learning control, Stochastic optimal control
Abstract: We study model-based and model-free policy optimization in a class of nonzero-sum stochastic dynamic games called linear quadratic (LQ) deep structured games. In such games, players interact with each other through a set of weighted averages (linear regressions) of the states and actions. In this paper, we focus our attention to homogeneous weights; however, for the special case of infinite population, the obtained results extend to asymptotically vanishing weights wherein the players learn the sequential weighted mean-field equilibrium. Despite the non-convexity of the optimization in policy space and the fact that policy optimization does not generally converge in game setting, we prove that the proposed model-based and model-free policy gradient descent and natural policy gradient descent algorithms globally converge to the sub-game perfect Nash equilibrium. To the best of our knowledge, this is the first result that provides a global convergence proof of policy optimization in a nonzero-sum LQ game. One of the salient features of the proposed algorithms is that their parameter space is independent of the number of players, and when the dimension of state space is significantly larger than that of the action space, they provide a more efficient way of computation compared to those algorithms that plan and learn in the action space. Finally, some simulations are provided to numerically verify the obtained theoretical results.

Keywords: Agents-based systems, Distributed control, Networked control systems
Abstract: In this paper, we develop a hybrid rigidity theory that involves heterogeneous distances (or unsigned angles) and signed constraints for a framework in 2-D space. The new rigidity theory determines a rigid formation shape up to a translation and a rotation by a set of distance and signed angle constraints, or up to a translation, a rotation and, additionally, a scaling factor by a set of unsigned angle and signed angle constraints. In particular, the hybrid rigidity theory provides insights on choosing heterogeneous constraints to address issues associated with flip (or reflection), flex and ordering ambiguities for a target formation. We then apply the rigidity theory to formation shape control with the minimal number of heterogeneous constraints in 2-D space. It is shown that a developed gradient-based control system guarantees a local exponential convergence to a desired formation by relative position measurements between an agent and its neighbor. We provide a simulation example on formation shape control with hybrid constraints to validate the theoretical results.

Keywords: Agents-based systems, Control applications, Flexible structures
Abstract: A consensus problem is proposed for second-order multi-agent systems with heterogeneous mass distribution. The motivation of this work is mainly related to spacecraft attitude coordinated control, in which gyroless configuration is considered, to avoid drift errors and design of estimation filters. The considered spacecraft includes flexible modes and coupling between the rigid and flexible dynamics. Dynamic interaction between the agents is considered. Moreover, the achievement of the consensus and robust stabilization are shown for coordinated heterogeneous multi-agent systems, for undirected and connected graph topology. Finally, the effectiveness of the proposed controller is shown for a precise pointing mission of the Crab Nebula.

Keywords: Agents-based systems, Distributed control, Optimization
Abstract: Submodular maximization problems are a relevant model set for many real-world applications. Since these problems are generally NP-Hard, many methods have been developed to approximate the optimal solution in polynomial time. One such approach uses an agent-based greedy algorithm, where the goal is for each agent to choose an action from its action set such that the union of all actions chosen is as high-valued as possible. Recent work has shown how the performance of the greedy algorithm degrades as the amount of information shared among the agents decreases, whereas this work addresses the scenario where agents are capable of sharing more information than allowed in the greedy algorithm. Specifically, we show how performance guarantees increase as agents are capable of passing messages, which can augment the allowable decision set for each agent. Under these circumstances, we show a near-optimal method for message passing, and how much such an algorithm could increase performance for any given problem instance.

Keywords: Agents-based systems, Networked control systems, Distributed control
Abstract: This paper proposes a novel approach for distributed finite-time event-triggered consensus control of multi-agent systems over directed graphs. In the proposed approach, a potential function is introduced in the control protocol design and a dynamic external variable with a finite-time convergence rate is involved in the construction of triggering thresholds. By using the proposed approach, finite-time consensus can be achieved in a fully distributed manner and the Zeno behavior is ruled out in the framework of finite-time event-triggered consensus. The proposed approach does not need global information and only a directed spanning tree is required for the underlying communication graph. Additionally, the requirement on continuous communication for controller updates or triggering detection is removed. Finally, an example is given to show the feasibility of the proposed approach.

Keywords: Agents-based systems, Smart grid, Optimization algorithms
Abstract: We propose a novel algorithm for solving convex, constrained and distributed optimization problems defined on multi-agent-networks, where each agent has exclusive access to a part of the global objective function. The agents are able to exchange information over a directed, weighted communication graph, which can be represented as a column-stochastic matrix. The algorithm combines an adjusted push-sum consensus protocol for information diffusion and a gradient descent-ascent on the local cost functions, providing convergence to the optimum of their sum. We provide results on a reformulation of the push-sum into single matrix-updates and prove convergence of the proposed algorithm to an optimal solution, given standard assumptions in distributed optimization. The algorithm is applied to a distributed economic dispatch problem, in which the constraints can be expressed in local and global subsets.

Keywords: Agents-based systems, Neural networks, Stability of nonlinear systems
Abstract: This paper proposes an adaptive neural network-based backstepping controller that uses rigid graph theory to address the distance-based formation control problem and target tracking for nonlinear multi-agent systems with bounded time-delay and disturbance. The radial basis function neural network (RBFNN) is used to overcome and compensate for the unknown nonlinearity and disturbance in the system dynamics. The effect of the state time-delay of the agents is alleviated by using an appropriate control signal that is designed based on specific Lyapunov function and Young's inequality. The adaptive neural network (NN) weights tuning law is derived using this Lyapunov function. An upper bound for the singular value of the normalized rigidity matrix is introduced, and uniform ultimate boundedness (UUB) of the formation distance error is rigorously proven based on the Lyapunov stability theory. Finally, the performance and effectiveness of the proposed method are validated through the simulation results on nonlinear multi-agent systems. Comparisons between the proposed distance-based method and an existing, displacement-based method are provided to evaluate the performance of the suggested method.

Keywords: Agents-based systems, Uncertain systems, Optimization
Abstract: This paper focuses on a specific class of convex multi-agent programs, prevalent in many practical applications, where agents cooperate to minimize a common cost, expressed as a function of the aggregate decision and affected by uncertainty. We model uncertainty by means of scenarios and use an epigraphic reformulation to transfer the uncertain part of the cost function to the constraints. Then, by exploiting the structure of the program under study and leveraging on existing results in the scenario approach literature, and in particular using the so called support rank notion, we provide for the optimal solution of the program distributionfree robustness certificates that are agent-independent, i.e., the constructed bound on the probability of constraint violation does not depend on the number of agents, but only on the dimension of the agents? decision. This leads to a significant improvement as it substantially reduces the number of samples required to achieve a certain level of probabilistic robustness as the number of agents increases. Our certificates can be used alongside any convex optimization algorithm centralised, decentralised or distributed, to obtain an optimal solution of the underlying problem. Our theoretical results are accompanied by a numerical example that investigates the electric vehicle charging problem and validates that the obtained robustness certificate is independent of the number of vehicles in the fleet.

Keywords: Boolean control networks and logic networks, Agents-based systems, Computational methods
Abstract: In this paper, we propose distributed algorithms that solve a system of Boolean equations over a network, where each node in the network possesses only one Boolean equation from the system. The Boolean equation assigned at any particular node is a private equation known to this node only, and the nodes aim to compute the exact set of solutions to the system without exchanging their local equations. We show that each private Boolean equation can be locally lifted to a linear algebraic equation under a basis of Boolean vectors, leading to a network linear equation that is distributedly solvable. A number of exact or approximate solutions to the induced linear equation are then computed at each node from different initial values. The solutions to the original Boolean equations are eventually computed locally via a Boolean vector search algorithm. We prove that given solvable Boolean equations, when the initial values of the nodes for the distributed linear equation solving step are i.i.d selected according to a uniform distribution in a high-dimensional cube, our algorithms return the exact solution set of the Boolean equations at each node with high probability.

Keywords: Control of networks, Network analysis and control, Networked control systems
Abstract: We study the strong structural controllability (SSC) of diffusively coupled networks, where the external control inputs are injected to only some nodes, namely the leaders. For such systems, one measure of controllability is the dimension of strong structurally controllable subspace, which is equal to the smallest possible rank of controllability matrix under admissible (positive) coupling weights. In this paper, we compare two tight lower bounds on the dimension of strong structurally controllable subspace: one based on the distances of followers to leaders, and the other based on the graph coloring process known as zero forcing. We show that the distance-based lower bound is usually better than the zero-forcing-based bound when the leaders do not constitute a zero-forcing set. On the other hand, we also show that any set of leaders that can be shown to achieve complete SSC via the distance-based bound is necessarily a zero-forcing set. Furthermore, we present a novel bound based on the combination of these two approaches, which is always at least as good as, and in some cases strictly greater than, the maximum of the two bounds. Finally, we present some numerical results to compare the bounds on various graphs.

Keywords: Learning, Machine learning, Neural networks
Abstract: This paper develops a novel Reinforcement Learning from Demonstration (RLfD) algorithm, called Self-Guided Actor-Critic (SGAC), with the purpose of enhancing exploration of Reinforcement Learning by imitating expert policies from demonstration data. Instead of using human experts or other control algorithms as demonstrations in most of existing methods, SGAC can generate high quality expert data online by continuously querying and adaptively updating an expert that is given by model predictive Deep Deterministic Policy Gradient (MP-DDPG). In this way, the training cost of SGAC is reduced, and distribution mismatch problem leading to unstable learning process is alleviated. In addition, the optimality assumption of the expert in typical RLfD methods is relaxed since the adaptive expert of SGAC can make self-improvement. A non-trivial example of applying SGAC to ship berthing control problem is present. The simulation results show the learning process of SGAC is faster and steadier than typical Reinforcement Learning algorithms and MP-DDPG.

Keywords: Learning, Networked control systems, Optimization algorithms
Abstract: Multi-agent reinforcement learning (MARL) has attracted more and more attention in recent years. It is now widely applied in various fields, including cyber physical systems, smart grid, finance, social network, and among others. The current researches on MARL mainly focus single-time scale, in which the agents have the same decision epoch. While in real applications, it is common that the agents make decisions by different frequencies. In addition, different agents may have separate roles in the system. In this paper, we propose a more general MARL framework by introducing multi-time scale of decision epochs. We assume that agents share information with their neighbors, including state, action, and reward. The global observability of state and action, which is a common assumption, is not required. We propose a decentralized Q-learning algorithm and a modified MADDPG algorithm to solve the problem. The main contributions of this paper are as follows. First, we formulate the multi-time scale multi-agent reinforcement learning (MTMARL) problem. This provides a general framework for the related systems and problems. Second, we provide a networked decentralized multi-time scale multi-agent Q-learning algorithm to solve the problem and prove its convergence. Third, we test the algorithm numerically. The results show that the proposed algorithm performs better than the previous QD-learning and is only slightly worse than the centralized algorithm.

Keywords: Learning, Neural networks
Abstract: The hydraulic clutch actuation path used in heavy duty transmissions often shows a lot of variability due to manufacturing tolerances and ageing effects. Reason for this are in particular varying friction coefficients in the spools and external factors such as compliance with the specified service intervals or the choice of hydraulic fluid. As a direct consequence, the shift quality typically varies from one transmission to the next. To resolve this problem, this paper presents a machine learning algorithm for the feedforward control of the hydraulic clutch actuation path, a model-based and a data-based feedforward approach. The two approaches are evaluated and compared with each other in simulations with a high-fidelity model. As it turns out, the model-based version is to be preferred and therefore used in a real world evaluation on an embedded controller and a 16 tons wheel loader.

Keywords: Learning, Randomized algorithms, Optimization
Abstract: Motivated by the fair allocation of goods in an offline market, we propose and study a new model for online job scheduling on heterogeneous machines. In this model, the goal is to schedule jobs on a set of machines in an online fashion with the overall quality of service as close as possible to an optimal offline benchmark. More precisely, we consider a job scheduling system consisting of a set of machines and indivisible jobs that arrive sequentially over time. When a job arrives, it must be scheduled and processed on a single machine where the utility received for such an assignment depends on the job-machine pair. It is assumed that each machine has a different power/energy budget and its welfare is proportional to the product of its power and its cumulative utilities. The goal is to maximize the total quality of service that is the sum of all the machines' welfare. However, in practice, the power budgets of machines often are not known and must be learned over time. To tackle this issue, we first propose a simple Explore-then-Exploit scheduling algorithm that achieves a sub-linear regret of O(T^{2/3}), where T is the total number of jobs. Here the regret is defined as the expected difference between the total quality of service obtained by the algorithm and its maximum value had we known the power budgets a priori. We then enhance this result by providing an Upper Confidence Bound (UCB) algorithm with only logarithmic regret O(log T). Numerical results are conducted to evaluate the performance of the proposed algorithms for various ranges of parameters.

Keywords: Optimal control, Neural networks, Robotics
Abstract: Optimal state-feedback controllers, capable of changing between different objective functions, are advantageous to systems in which unexpected situations may arise. However, synthesising such controllers, even for a single objective, is a demanding process. In this paper, we present a novel and straightforward approach to synthesising these policies through a combination of trajectory optimisation, homotopy continuation, and imitation learning. We use numerical continuation to efficiently generate optimal demonstrations across several objectives and boundary conditions, and use these to train our policies. Additionally, we demonstrate the ability of our policies to effectively learn families of optimal state-feedback controllers, which can be used to change objective functions online. We illustrate this approach across two trajectory optimisation problems, an inverted pendulum swingup and a spacecraft orbit transfer, and show that the synthesised policies, when evaluated in simulation, produce trajectories that are near-optimal. These results indicate the benefit of trajectory optimisation and homotopy continuation to the synthesis of controllers in dynamic-objective contexts.

Keywords: Learning, Identification, Adaptive systems
Abstract: This paper addresses the problem of online inverse reinforcement learning for systems with limited data and uncertain dynamics. In the developed approach, the state and control trajectories are recorded online by observing an agent perform a task, and reward function estimation is performed in real-time using a novel inverse reinforcement learning approach. Parameter estimation is performed concurrently to help compensate for uncertainties in the agent's dynamics. Data insufficiency is resolved by developing a data-driven update law to estimate the optimal feedback controller. The estimated controller can then be queried to artificially create additional data to drive reward function estimation.

Keywords: Learning, Robust control, Autonomous vehicles
Abstract: Performance and robustness targets have been considered for controller design for decades. However, robust controllers usually suffer from performance limitations due to conservative uncertainty assumptions made a priori to system operation. The increased number of systems (e.g. autonomous vehicles) which require high-performance operation in safety-critical environments is motivating research in novel design methods. Recently, machine learning methods have emerged as a promising way to reduce conservatism, based on data gathered during system operation. We propose a combination of a recursive least squares estimator with a recursive quantile estimator to identify feature-dependent upper and lower uncertainty bounds. We give conditions under which the estimator converges to a robust invariant set, such that the resulting bounds cover a target proportion of the samples up to small error. In contrast to widely applied Gaussian process regression or Bayesian linear regression approaches, we do not imply any assumptions about the probability distribution of the samples. We demonstrate that the estimated bounds achieve the desired data coverage in contrast to state-of-the-art approaches on academic examples, as well as a motion control example for an autonomous race car. Furthermore, the approach exhibits very low computational requirements and is therefore suitable for application on embedded systems.

Keywords: Learning, Identification for control, LMIs
Abstract: There exists a vast amount of literature how dissipativity properties can be exploited to design controllers for stability and performance guarantees for the closed loop. With the rising availability of data, there has therefore been an increasing interest in determining dissipativity properties from data as a means for data-driven systems analysis and control with rigorous guarantees. Most existing approaches, however, consider dissipativity properties that hold only over a finite horizon and mostly only qualitative statements can be made in the presence of noisy data. In this work, we present a novel approach to determine dissipativity of linear time-invariant systems from data where we inherently consider properties that hold over the infinite horizon. Furthermore, we develop a method to verify dissipativity from noisy data with guaranteed robustness.

Keywords: Distributed parameter systems, Kalman filtering, Distributed control
Abstract: This work analyzes the communication requirements of Kalman filters for spatially-invariant diffusion processes with spatially-distributed sensing. In this setting Kalman filters exhibit an inherent degree of spatial localization or decentralization. We address the fundamental question of whether the statistical properties of process and measurement disturbances, namely variance and spatial-autocorrelation length, can further enhance its inherent spatial localization. We show that when disturbances are spatially and temporally uncorrelated, the spatial localization of the filter depends on the ratio of model to measurement error. Building upon this result, we study exponentially-decaying spatially-autocorrelated process and measurement disturbances. We show that certain level of spatial-autocorrelation in the measurement noise reduces the communication burden of the Kalman filter: indeed, the filter is completely decentralized when a matching condition is satisfied. We also show that spatial autocorrelation of the process disturbance has no benefits in terms of communications, as the level of centralization of the filter increases with the autocorrelation length.

Keywords: Distributed parameter systems, Delay systems, Control of networks
Abstract: In this paper, we design a stabilizing controller for a system composed of two sets of linear heterodirectional hyperbolic PDEs, with actuation at one boundary of one of the PDEs, and couplings at the middle boundary with ODEs in a PDE-ODE-PDE configuration. The design approach employs a backstepping transformation to move the undesired system couplings to the proximal boundary (where the actuation is located). We can then express this target system as a time- delay neutral system for which we can design an appropriate control law to obtain an exponentially stable target system.

Keywords: Distributed parameter systems, Lyapunov methods, Robust control
Abstract: This paper presents a computational framework for analyzing stability and performance of uncertain Partial Differential Equations (PDEs) when they are coupled with uncertain Ordinary Differential Equations (ODEs). To analyze the behavior of the interconnected ODE-PDE systems under uncertainty, we introduce a class of multipliers of Partial Integral (PI) operator type and consider various classes of uncertainties by enforcing constraints on these multipliers. Since the ODE-PDE models are equivalent to Partial Integral Equations (PIEs), we show that the robust stability and performance can be formulated as Linear PI Inequalities (LPIs) and LPIs can be solved by LMIs using PIETOOLS. The methods are demonstrated on examples of ODE-PDE systems that are subjected to wide classes of uncertainty.

Keywords: Distributed parameter systems, Lyapunov methods
Abstract: We propose a new design technique for the stabilization of coupled ODE-PDE systems in feedforward form. In particular, we address the stabilization problem of a one-dimensional transport equation driven by a scalar ODE which is controlled via a cone-bounded nonlinearity. The unforced transport equation is conservative but not asymptotically stable. The proposed technique is inspired by the forwarding approach early introduced in the 90's. Well-posedness and asymptotic stability of the closed-loop system are discussed.

Keywords: Distributed parameter systems, Observers for Linear systems, Lyapunov methods
Abstract: The problem of unknown input observer design is considered for coupled PDE/ODE linear systems subject to unknown boundary inputs. Assuming available measurements at the boundary of the distributed domain, the synthesis of the observer is based on geometric conditions and Lyapunov methods. Numerical simulations support and validate the theoretical findings, illustrating the robust estimation performances of the proposed unknown input observer.

Keywords: Distributed parameter systems, Variational methods, Lyapunov methods
Abstract: The general problem of this paper is the analysis of wave propagation in a bounded medium where the uncontrolled boundary obeys a coupled differential equation. More precisely, we study a one-dimensional wave equation with a nonlinear second-order dynamic boundary condition and a Neuman-type boundary control acting on the other extremity. A generic class of nonlinear collocated feedback laws is considered. Hadamard well-posedness is established for the closed-loop system, with initial data lying in the natural energy space of the problem. Moreover, we investigate an example of stabilization through a proportional controller.

Keywords: Estimation, Distributed parameter systems, Adaptive control
Abstract: An adaptive scheme for estimating both boundary reflection coefficients of 2 X 2 linear hyperbolic systems, using only a single boundary measurement, is presented. The design consists first of mapping the system into a target system via a Volterra integral transformation. Using this target system, the output signal is written in regressor form linear in the unknown coefficients, and modified least-squares with forgetting factor is implemented to estimate them. Next, it is demonstrated how a similar approach can be applied to estimate the specific boundary acoustic impedances of a cylindrical metal tube open at both ends. By writing the linearised tube acoustics model as a linear hyperbolic 2 X 2 system, a regressor form with identical parametrization to the former case but with a different signal vector is found. The same adaptive scheme is then applied to estimate the acoustic impedances.

Keywords: Distributed parameter systems, Lyapunov methods, Stability of nonlinear systems
Abstract: In this paper we address the problem of rapid stabilization of a reaction-diffusion equation with distributed disturbance. With the aid of the spectral decomposition of the spatial operator associated to the equation and the sign multivalued operator, which is used to reject the effects of the disturbance, we design a feedback law that exponentially stabilizes, with decay rate as large as desired, the corresponding infinite-dimensional system. The well-posedness of the resulting closed-loop system, which actually is a differential inclusion, is shown with the maximal monotone operator theory.

Keywords: Predictive control for linear systems, Predictive control for nonlinear systems, Stochastic optimal control
Abstract: Stochastic model predictive control (SMPC) approximates the solution to constrained stochastic optimal control problems by solving a simplified problem repeatedly over a reduced prediction horizon. This paper demonstrates and discusses significant open challenges for current SMPC methods in terms of their closed-loop performance and conservatism regarding constraint satisfaction. In particular, we compare two forms of formulating chance constraints in SMPC. First, we consider a direct feedback formulation, which corresponds to the typical implementation of SMPC. Direct feedback formulates chance constraints for the predicted state distribution conditioned on the current measured state at each time step during the receding horizon control. Indirect feedback, in contrast, formulates constraints by introducing a suitable nominal state, which allows to enforce chance constraints on the closed loop. In numerical examples, we demonstrate that direct feedback, i.e. the typical form of SMPC, can result in significant conservatism, allowing almost no constraint violations. This results in significantly reduced performance, which we show can be alleviated with indirect feedback formulations. In addition, we prove that indirect feedback can recover the unconstrained optimal solution given by LQR control whenever it is feasible also for the constrained optimal control problem.

Keywords: Predictive control for nonlinear systems, Constrained control, Optimal control
Abstract: We consider sampled-data Model Predictive Control (MPC) of nonlinear continuous-time control systems. We derive sufficient conditions to guarantee recursive feasibility and asymptotic stability without stabilising costs and/or constraints. Moreover, we present formulas to explicitly estimate the required length of the prediction horizon based on the concept of (local) cost controllability. For the linear-quadratic case, cost controllability can be inferred from standard assumptions. In addition, we extend results on the relationship between the horizon length and the distance of the initial state to the boundary of the viability kernel from the discrete-time to the continuous-time setting.

Keywords: Predictive control for nonlinear systems, Optimal control, Agents-based systems
Abstract: In many applications, resource-aware devices are connected through a network, such as in the Internet of Things and energy hubs. These devices require proper coordination to achieve a high performance without violation of their resource limits. In this paper, we propose an asynchronous resource-aware multi-agent model predictive control to cooperatively coordinate agents to conduct a common task, whose resource concern is handled by a self-triggered control scheme. The consistency and recursive feasibility of the proposed MPC scheme are investigated. A reliable numerical implementation is introduced to deal with non-constant sampling times among agents, which is subsequently validated by a numerical example.

Keywords: Predictive control for nonlinear systems, Numerical algorithms, Autonomous vehicles
Abstract: We investigate tracking tasks for an automatic mobile robot with obstacle avoidance. To this end we apply a linear model-predictive control (LMPC) method to the nonlinear robot model. The LMPC uses a linearized robot model around the reference track and takes into account (fixed or moving) obstacles, which the robot has to avoid. The resulting discretized linear-quadratic optimal control problems are solved numerically by a semismooth Newton method, which turns out to be fast and robust. Furthermore, we propose a structure exploitation strategy to reduce the computational effort of the semismooth Newton method. Simulation results for a two-wheeled robot are presented to validate the control algorithm.

Keywords: Predictive control for nonlinear systems, Adaptive control, Constrained control
Abstract: A centralized model predictive controller (MPC), which is unaware of local uncertainties, for an affine discrete time nonlinear system is presented. The local uncertainties are assumed to be matched, bounded and structured. In order to encounter disturbances and to improve performance, an adaptive control mechanism is employed locally. The proposed approach ensures input-to-state stability of closed-loop states and convergence to the equilibrium point. Moreover, uncertainties are learnt in terms of the given feature basis by using adaptive control mechanism. In addition, hard constraints on state and control are satisfied.

Keywords: Predictive control for nonlinear systems, Lyapunov methods, Stability of nonlinear systems
Abstract: This work considers the problem of stabilizing feedback control design for nonlinear systems. To achieve this, we integrate Koopman operator theory with Lyapunov-based model predictive control (LMPC). A bilinear representation of the nonlinear dynamics is determined using Koopman eigenfunctions. Then, a predictive controller is formulated in the space of Koopman eigenfunctions by using an auxiliary Control Lyapunov Function (CLF) based bounded controller as a constraint which enables the characterization of stability of the Koopman bilinear system. Unlike previous studies, we show ia an inverse mapping - realized by continuously differentiable functions - that the designed controller translates the stability of the Koopman bilinear system to the original closed-loop system. Remarkably, the feedback control design proposed in this work remains completely data-driven and does not require any explicit knowledge of the original system. Moreover, in contrast to standard LMPC, seeking a CLF for the bilinear system is computationally favorable compared to the original nonlinear system. The application of the proposed method is illustrated on a numerical example.

Keywords: Predictive control for nonlinear systems, Robust control, Stability of nonlinear systems
Abstract: In this paper, we propose an asymptotically stabilizing formulation of multi-stage nonlinear model predictive control (NMPC) for plants with state and input dependent uncertainties. We derive time-varying Lyapunov-type sufficient conditions for asymptotic stability. We then propose a novel multi-stage NMPC formulation with time-varying terminal constraints, which guarantees asymptotic stability of the origin based on the derived stability conditions. The time-varying nature of the terminal constraints renders the control law and consequently the controlled system dynamics time-variant. For this situation, the derived time-varying stability conditions provide a suitable framework for stability analysis. We demonstrate the advantages of the proposed scheme over previous formulations of multi-stage NMPC on a cart simulation study.

Keywords: Predictive control for nonlinear systems, Predictive control for linear systems
Abstract: Economic nonlinear model predictive control (NMPC) is a variant of NMPC that directly optimizes an economic performance index instead of a tracking error. Although economic NMPC can achieve excellent closed-loop performance, the associated computational effort as well as the difficulty of guaranteeing stability in practice are its main drawbacks. Motivated by these difficulties, a formal procedure was developed that tunes a tracking (non)linear MPC scheme so that it is first-order equivalent to economic NMPC. This letter introduces TuneMPC, a new open-source software framework that closes the gap between the underlying theory and practical application of this tuning procedure. For user-provided system dynamics, constraints and economic objective, TuneMPC enables automated computation of optimal steady states and periodic trajectories, and returns the corresponding tuned stage cost matrices. To demonstrate the potential of the tool, we apply the technique to the challenging example of an autonomous tethered aircraft flying periodic orbits for airborne wind energy harvesting.

Keywords: Networked control systems, Optimal control, Linear systems
Abstract: In this paper we address the problem of optimal co-design of control and quantization policies for a physically-interconnected system, where each subsystem has a local quantizer. The controllers are assumed to communicate with delay and cooperate in minimizing global quadratic cost. We show that for quantizers that act on the estimation error of the estimator conditioned on common information between controllers, separation holds. In other words, both quantizers can be optimally designed by minimizing a distortion function that is control-independent. Finally, for general class of quantizers we provide structural properties of the optimal control policy.

Keywords: Nonlinear output feedback, Optimal control, Algebraic/geometric methods
Abstract: This paper provides a tractable approximate solution to the classical optimal control problem of a nonlinear control system and its corresponding Hamilton-Jacobi-Bellman (HJB) equation. Our focus is on systems whose right-hand sides are analytic functions, i.e., they admit power series representations. We utilize quadratization techniques to lift the HJB equation, which is a nonlinear partial differential equation, into an operator equation that resembles the well-known Riccati equation. We exploit the structural properties of this operator equation to calculate the block components of its solution using an exact iterative method. The uniqueness of the solution is proven and it is shown that under some technical assumptions the resulting closed-loop system is stable in the sense of Lyapunov. The Small-Gain theorem is applied to show that one only needs to run finitely many iterations to compute a stabilizing near-optimal solution for the optimal control problem. Our theoretical findings are verified using several simulations.

Keywords: Adaptive control, Autonomous systems, Machine learning
Abstract: This paper introduces a framework for learning a minimum-norm stabilizing controller for a system with unknown dynamics using model-free policy optimization methods. The approach begins by first designing a Control Lyapunov Function (CLF) for a (possibly inaccurate) dynamics model for the system, along with a function which specifies a minimum acceptable rate of energy dissipation for the CLF at different points in the state-space. Treating the energy dissipation condition as a constraint on the desired closed-loop behavior of the real-world system, we use penalty methods to formulate an unconstrained optimization problem over the parameters of a learned controller, which can be solved using model-free policy optimization algorithms using data collected from the plant. We discuss when the optimization learns a stabilizing controller for the real world system and derive conditions on the structure of the learned controller which ensure that the optimization is strongly convex, meaning the globally optimal solution can be found reliably. We validate the approach in simulation, first for a double pendulum, and then generalize the framework to learn stable walking controllers for underactuated bipedal robots using the Hybrid Zero Dynamics framework. By encoding a large amount of structure into the learning problem, we are able to learn stabilizing controllers for both systems with only minutes or even seconds of training data.

Keywords: Nonlinear output feedback, Uncertain systems, Observers for nonlinear systems
Abstract: We propose a matrix pencil based approach for design of output-feedback stabilizing controllers for a general class of uncertain nonlinear strict-feedback-like systems. While the dynamic controller structure is based on the dual dynamic high-gain scaling based approach, we cast the design procedure within a general matrix pencil based structure unlike prior results that utilized conservative algebraic bounds of terms arising in Lyapunov inequalities. The proposed approach models the detailed system structure and state dependence structure of uncertain terms. The design freedoms in the dynamic high-gain scaling based controller are extracted in terms of generalized eigenvalues associated with matrix pencils formulated to capture the detailed structures of bounds in the Lyapunov analysis. The proposed matrix pencil approach enables efficient computation of non-conservative bounds with reduced algebraic complexity and significantly enhances feasibility of application of the dual dynamic high-gain scaling based control designs.

Keywords: Stability of nonlinear systems, Lyapunov methods, Model/Controller reduction
Abstract: In this work, we consider event-based implementation of control laws designed for local stabilization of nonlinear systems with center-manifolds. The systems being considered possess linearised models with uncontrollable modes on the imaginary axis. The controller chosen decides both the structure of the center-manifold and its stability. Although the control of systems with center-manifolds is well studied, event-based control of such systems is yet to be probed. This involves the exploration of input-to-state stability (ISS) properties of such systems, with respect to measurement errors. Considering the most general structure for the controller, we prove that a controller that locally asymptotically stabilizes the dynamics on the center-manifold, also renders the overall system locally input-to-state stable (LISS) and find the comparison functions involved in the Lyapunov characterization of ISS. This general characterization required a nonlinear change of variables, involving the center-manifold, which can only be approximately determined in most cases. Because of this, it is found to be unsuitable for designing event-triggered controllers. We then explore an approach that does not resort to this change of variables and present our findings. We discuss the possibility of a simpler relative thresholding mechanism and present simulation results for an illustrative example.

Keywords: Stability of nonlinear systems, Lyapunov methods, Delay systems
Abstract: This paper deals with fixed-time and prescribed- time stabilization of controllable linear-time delay systems using Artstein?s transformation [1]. First, a delayed chain of integrators is stabilized in fixed-time using the desingularization technique originally introduced in [4]. Then, prescribed-time stabilization of controllable linear systems with input delay is obtained through Artstein?s transformation combined with a new prescribed-time stabilizing control design (involving time-varying gains) for LTI systems. Compared to the control design methodology employed in e.g. [10], this new design offers an alternative somehow clearer way for the choice of the time-varying gains. Simulations illustrate the obtained results.

Keywords: Lyapunov methods
Abstract: In this paper we consider the problem of path following control for fully-actuated mechanical systems using the technique of kinetic-potential energy shaping (KPES). By using KPES, damping is injected into the position error coordinates of the closed-loop systems, resulting in exponential convergence to the desired path. It is shown that the technique can be interpreted as a generalised canonical transformation and the results are demonstrated on a simple mechanical system following a unit circle with a non-constant velocity.

Keywords: Electrical machine control, Stability of nonlinear systems
Abstract: We present a new approach for the control design of a two-timescale dynamical system. When using standard singular perturbations, the controller is typically designed to drive the "fast" dynamics rapidly to a steady state that has been chosen in a way that makes the "slow" dynamics behave according to the control objective. However, such a controller requires the time scales to be sufficiently separated and such a separation is sometimes impossible to achieve, resulting in degraded performance. We therefore propose to redesign this controller based on a higher-order singular perturbation analysis to gain in precision and obtain performance that are closer to the expected ones. In particular, we show the benefits of this method on the control of induction motors.

Keywords: Adaptive control, Robust control, Statistical learning
Abstract: Robots operating in real world settings must navigate and maintain safety while interacting with many heterogeneous agents and obstacles. Multi-Agent Control Barrier Functions (CBF) have emerged as a computationally efficient tool to guarantee safety in multi-agent environments, but they assume perfect knowledge of both the robot dynamics and other agents' dynamics. While knowledge of the robot's dynamics might be reasonably well known, the heterogeneity of agents in real-world environments means there will always be considerable uncertainty in our prediction of other agents' dynamics. This work aims to learn high-confidence bounds for these dynamic uncertainties using Matrix-Variate Gaussian Process models, and incorporates them into a robust multi-agent CBF framework. We transform the resulting min-max robust CBF into a quadratic program, which can be efficiently solved in real time. We verify via simulation results that the nominal multi-agent CBF is often violated during agent interactions, whereas our robust formulation maintains safety with a much higher probability and adapts to learned uncertainties.

Keywords: Robotics, Decentralized control, Autonomous vehicles
Abstract: This paper considers the problem of obstacle avoidance for multiple robotic agents moving in an environment with obstacles. A decentralized supervisory controller is synthesized based on control barrier functions that guarantees obstacle avoidance with limited actuation capability. The proposed method is applicable to general nonlinear robot dynamics and is scalable to an arbitrary number of agents. Agent-to-agent communication is not required, yet a simple broadcasting scheme improves the performance of the algorithm. The key idea is based on a control barrier function constructed with a backup controller, and we show that by assuming other agents respecting the same CBF condition, the supervisory control algorithm can be implemented decentrally and guarantees obstacle avoidance for all agents.

Keywords: Autonomous systems, Formal Verification/Synthesis, Robotics
Abstract: The prolific rise in autonomous systems has led to questions regarding their safe instantiation in real-world scenarios. Failures in safety-critical contexts such as human-robot interactions or even autonomous driving can ultimately lead to loss of life. In this context, this paper aims to provide a method by which one can algorithmically test and evaluate an autonomous system. Given a black-box autonomous system with some operational specifications, we construct a minimax problem based on control barrier functions to generate a family of test parameters designed to optimally evaluate whether the system can satisfy the specifications. To illustrate our results, we utilize the Robotarium as a case study for an autonomous system that claims to satisfy waypoint navigation and obstacle avoidance simultaneously. We demonstrate that the proposed test synthesis framework systematically finds those sequences of events (tests) that identify points of system failure.

Keywords: Resilient Control Systems, Stochastic systems, Autonomous systems
Abstract: CPS safety, defined as the system state remaining within a desired safe region, is a critical property in applications including medicine, transportation, and energy. Sensor faults and attacks may cause safety violations by introducing bias into the system state estimation, which in turn leads to erroneous control inputs. In this paper, we propose a class of Fault-Tolerant Control Barrier Functions (FT-CBFs) that provide provable guarantees on safety of stochastic CPS. Our approach is to maintain a set of state estimators, each of which ignores a subset of sensor measurements that are affected by a particular fault pattern. We then introduce a linear constraint for each state estimator that ensures that the estimated state remains outside the unsafe region, and propose an approach to resolving conflicts between the constraints that may arise due to faults. We present sufficient conditions on the geometry of the safe region and the noise characteristics to provide a desired probability of maintaining safety. We then propose a framework for joint safety and stability by integrating FT-CBFs with Control Lyapunov Functions. Our approach is validated through numerical study of a wheeled mobile robot.

Keywords: Sampled-data control, Formal Verification/Synthesis, Robotics
Abstract: This paper considers the general problem of transitioning theoretically safe controllers to hardware. Concretely, we explore the application of control barrier functions (CBFs) to sampled-data systems: systems that evolve continuously but whose control actions are computed in discrete time-steps. While this model formulation is less commonly used than its continuous counterpart, it more accurately models the reality of most control systems in practice, making the safety guarantees more impactful. In this context, we prove robust set invariance with respect to zero-order hold controllers as well as state uncertainty, without the need to explicitly compute any control invariant sets. It is then shown that this formulation can be exploited to address input delays in this system, with the result being CBF constraints that are affine in the input. The results are demonstrated in a high-fidelity simulation of an unstable Segway robotic system in real-time.

Keywords: Hierarchical control, Lyapunov methods, Predictive control for linear systems
Abstract: In this paper we present a multi-rate control architecture for safety critical systems. We consider a high level planner and a low level controller which operate at different frequencies. This multi-rate behavior is described by a piecewise nonlinear model which evolves on a continuous and a discrete level. First, we present sufficient conditions which guarantee recursive constraint satisfaction for the closed-loop system. Afterwards, we propose a control design methodology which leverages Control Barrier Functions (CBFs) for low level control and Model Predictive Control (MPC) policies for high level planning. The control barrier function is designed using the full nonlinear dynamical model and the MPC is based on a simplified planning model. When the nonlinear system is control affine and the high level planning model is linear, the control actions are computed by solving convex optimization problems at each level of the hierarchy. Finally, we show the effectiveness of the proposed strategy on a simulation example, where the low level control action is updated at a higher frequency than the high level command.

Keywords: Formal Verification/Synthesis, Autonomous systems, Constrained control
Abstract: Temporal logic has been widely used to express complex task specifications for cyber-physical systems (CPSs). One way to synthesize a controller for CPS under temporal logic constraints is to first abstract the CPS as a discrete transition system, and then apply formal methods. This approach, however, is computationally demanding and its scalability suffers due to the curse of dimensionality. In this paper, we propose a control barrier function (CBF) approach to abstraction-free control synthesis under a linear temporal logic (LTL) constraint. We first construct the deterministic Rabin automaton of the specification and compute an accepting run. We then compute a sequence of LTL formulae, each of which must be satisfied during a particular time interval, and prove that satisfying the sequence of formulae is sufficient to satisfy the LTL specification. Finally, we compute a control policy for satisfying each formula by constructing an appropriate CBF. We present a quadratic program to compute the controllers, and show the controllers synthesized using the proposed approach guarantees the system to satisfy the LTL specification, provided the quadratic program is feasible at each time step. A numerical case study is presented to demonstrate the proposed approach.

Keywords: Stochastic systems, Automata
Abstract: In this paper, we study formal synthesis of control policies for partially observed jump-diffusion systems against complex logic specifications. Given a state estimator, we utilize a discretization-free approach for formal synthesis of control policies by using a notation of control barrier functions without requiring any knowledge of the estimation accuracy. Our goal is to synthesize a control policy providing (potentially maximizing) a lower bound on the probability that the trajectories of the partially observed jump-diffusion system satisfy some complex specifications expressed by deterministic finite automata. Finally, we illustrate the effectiveness of the proposed results by synthesizing a policy for a jet engine example.

Keywords: Optimization, Optimization algorithms, Embedded systems
Abstract: Control policies that involve the real-time solution of one or more convex optimization problems include model predictive (or receding horizon) control, approximate dynamic programming, and optimization based actuator allocation systems. They have been widely used in applications with slower dynamics, such as chemical process control, supply chain systems, and quantitative trading, and are now starting to appear in systems with faster dynamics. In this talk I will describe a number of advances over the last decade or so that make such policies easier to design, tune, and deploy. We describe solution algorithms that are extremely robust, even in some cases division free, and code generation systems that transform a problem description expressed in a high level domain specific language into source code for a real-time solver suitable for control. The recent development of systems for automatically differentiating through a convex optimization problem can be used to efficiently tune or design control policies that include embedded convex optimization.

Keywords: Stability of nonlinear systems, Control of networks, Biological systems
Abstract: We will review some recent progress on two fronts. First, we discuss a recently developed model of SIS epidemic propagation over hypergraphs. For simplicial and higher-order interactions, we show how a new dynamical behavior domain appears: both a disease-free equilibrium and an endemic equilibrium co-exist and are locally asymptotically stable. Second, we discuss modeling and analysis aspects for the multi-group SIR model. We present analysis results on transient behavior, threshold conditions, stability properties, and asymptotic convergence. In both cases, we pay special attention to monotonicity and contractivity properties of the resulting dynamical models.

Keywords: Healthcare and medical systems, Compartmental and Positive systems, Modeling
Abstract: In the context of an infectious disease outbreak like the global SARS-CoV-2 pandemic, predicting the course of the epidemic is of paramount importance to plan an effective control strategy and to determine its impact. Multiple population-wide non-pharmaceutical interventions are possible, including social distancing, testing and contact tracing. We propose a SIDARTHE epidemic model for the COVID-19 outbreak, which considers eight stages of infection: susceptible (S), infected (I), diagnosed (D), ailing (A), recognized (R), threatened (T), healed (H) and extinct (E). The model distinguishes between infected individuals depending on whether they have been diagnosed and on the severity of their symptoms. The distinction between diagnosed and non-diagnosed infected is important, because diagnosed individuals are isolated, hence less likely to spread the disease, and can explain misperceptions of the case fatality rate and of the epidemic spread. Being able to predict the amount of patients that will develop life-threatening symptoms is important, since the disease frequently requires hospitalisation (and even Intensive Care Unit admission) and challenges the healthcare system capacity. We show how the basic reproduction number can be redefined in the new framework, thus capturing the potential for epidemic containment. Simulation results are compared with real data on the COVID-19 epidemic in Italy; the analysis of different possible scenarios suggests the effectiveness of combined social-distancing measures and widespread testing and contact tracing to control the pandemic.

Keywords: Network analysis and control
Abstract: The ongoing COVID-19 pandemic is caused by a novel coronavirus that was only identified in December 2019. Due to the novelty of the virus and the speed at which it is currently sweeping the world, the questions we have are constantly changing. Is social distancing working? How effectively did we `flatten the curve' back in March? Is airline travel safe? When will this all be over? As we gather data and learn more about the virus, our questions will also continue to evolve. This talk will look at how various tools from Systems and Control Theory and Network Science can be used to model and analyze the spread of the virus across different regions with the questions above in mind. Unlike most existing compartmental models that assume well mixed populations or populations with known degree distributions, we will discuss network models at both the individual (person-to-person contacts) level and the region level (mobility between different cities/states by car/train/plane). In particular we will discuss which models are most suitable for addressing which types of socially relevant questions, and how the models can be used to make projections and policy recommendations.

Keywords: Network analysis and control
Abstract: In traditional compartmental epidemic models, after recovery from an infection, agents return to the susceptible state or they acquire full immunity to reinfection. We present a model for spreading over a network of heterogeneous agents that generalizes existing models to accommodate realistic conditions in which agents may acquire only partial or possibly compromised immunity after exposure to an infection. Reproduction numbers that account for network structure and heterogeneity provide the means to distinguish different behavioral regimes and can be used to design control strategies to mitigate spreading, even when resources are scarce. For example, in the bistable regime, not accounted for in traditional models, if measures are not taken, there can be a rapid resurgent epidemic after what looks like convergence to an infection-free state. To investigate the implications of reactive strategies, such as quarantining and social distancing, we present an active control model in which contact rate is controlled in continuous time as a feedback function of low-pass filtered observations of level of infection in the population. In homogeneous populations, this feedback diminishes spreading. However, in populations with heterogeneity in feedback strategy based on risk profile, we prove conditions that result in sustained oscillations in the infected population level.

Keywords: Stability of nonlinear systems, Lyapunov methods, Power systems
Abstract: This paper presents a new grid-forming converter control strategy, termed hybrid angle control (HAC) that results in almost global closed-loop stability. HAC combines the dcbased matching control with a novel nonlinear angle control that resembles the droop control. The synthesis of HAC is inspired by ideas from direct angle control and nonlinear damping injection technique. The proposed HAC is applied to a detailed nonlinear converter model that is connected to an infinite bus via a dynamic line. We provide insightful parametric conditions for existence, uniqueness, and almost global stability of closed-loop equilibria. Unlike in related stability certificates, our parametric conditions do not demand strong physical damping, but they can be met by appropriate choice of ac and dc converter control gains. Finally, we present an implementation of the HAC and illustrate the behavior of the closed-loop system with a publicly available numerical example.

Keywords: Stability of nonlinear systems, Lyapunov methods, Power systems
Abstract: We investigate local convergence of identical DC/AC converters interconnected via identical resistive and inductive lines towards a synchronous equilibrium manifold. We exploit the symmetry of the resulting vector field and develop a Lyapunov-based framework, in which we measure the distance of the solutions of the nonlinear power system model to the equilibrium manifold by analyzing the evolution of their tangent vectors. We derive sufficient and fully decentralized conditions to characterize the equilibria of interest, and provide an estimate of their region of contraction. We provide ways to satisfy these conditions and illustrate our results based on numerical simulations of a two-converter benchmark.

Keywords: Linear systems, Power electronics, Robust control
Abstract: In this paper, a data-driven method for controller design with constraints on the positive-realness of closed-loop transfer functions over an arbitrary set of frequencies is proposed. The positive-realness of a closed-loop transfer function is represented by a set of convex constraints involving only the frequency response data of the plant model and the parameters of a fixed structure controller. The new convex constraints, are then integrated into a recently developed data-driven robust control framework that can consider other control performance and robustness specifications. The proposed method is applied to the current controller design in traction systems. According to the field standards, the real part of the input admittance of the converters should be positive for a specific range of frequencies. The existing methods in the literature are based on the passivity approaches using a parametric model of the system and usually require a disturbance observer and additional sensors. In the proposed method, only the measurement data is needed for controller design and there is no requirement of additional sensors that reduces the costs and increases the reliability. The effectiveness of the proposed method is validated through numerical simulation including switching converters. The results show that the proposed controller provides required positive-realness as well as good performance in tracking and disturbance rejection.

Keywords: Power electronics, Power generation, Decentralized control
Abstract: This paper focuses on the development of plug-and-play voltage control mechanisms for DC microgrids with flexible structures. To this end, we present a voltage control approach that guarantees stable operation and satisfactory performance of DC microgrids under arbitrary interconnection of distributed generation (DG) units. The local voltage controllers are solutions of loop-shaping-based convex optimization problems. The proposed voltage control framework is applied to a case study of a multiple-DG DC microgrid in MATLAB/Simscape Electrical environment. The simulation results demonstrate the feasibility of the proposed voltage control approach and its viability for DC microgrids.

Keywords: PID control, Robust control, Power electronics
Abstract: In this paper we show that the PI passivity-based control (PBC) for power converters reported in~cite{hernandez2009adaptive} can be properly modified in order to guarantee global asymptotic stability of the closed-loop system even in case of inaccurate knowledge of the equilibrium to be stabilized. The proposed modification consists of the inclusion of a leakage in the integral action that, viewed in suitable incremental coordinates, acts as a damping term that helps to compensate the error induced by the imprecise knowledge of the equilibrium. Interestingly, with an appropriate choice of the parameters, the controller can be interpreted as a modulation index/power droop controller. The theoretical results are validated by simulations on a DC/DC boost converter.

Keywords: Formal Verification/Synthesis, Machine learning, Uncertain systems
Abstract: The deployment of autonomous systems that operate in unstructured environments necessitates algorithms to verify their safety. This can be challenging due to, e.g., black-box components in the control software, or undermodelled dynamics that prevent model-based verification. We present a novel verification framework for an unknown dynamical system from a given set of noisy observations of the dynamics. Using Gaussian processes trained on this data set, the framework abstracts the system as an uncertain Markov process with discrete states defined over the safe set. The transition bounds of the uncertain process are derived from the probabilistic error bounds between the regression and underlying system. An existing approach for verifying safety properties over uncertain Markov processes then generates safety guarantees. We demonstrate the versatility of the framework on several examples, including switched and nonlinear systems.

Keywords: Formal Verification/Synthesis, Computational methods, LMIs
Abstract: In this paper we propose a computational method based on semi-definite programming for synthesizing infinite-time reach-avoid sets in discrete-time polynomial systems. An infinite-time reach-avoid set is a set of initial states making the system eventually, i.e., within finite time enter the target set while remaining inside another specified (safe) set during each time step preceding the target hit. The reach-avoid set is first characterized equivalently as a strictly positive sub- level of a bounded value function, which in turn is shown to be a solution to a system of derived equations. The derived equations are further relaxed into a system of inequalities, which is encoded into semi-definite constraints based on the sum-of-squares decomposition for multivariate polynomials, such that the problem of synthesizing inner-approximations of the reach-avoid set can be addressed via solving a semi-definite programming problem. Two examples demonstrate the proposed approach.

Keywords: Formal Verification/Synthesis, Constrained control, Hybrid systems
Abstract: We consider synthesizing safety controllers for discrete-time dynamical systems with imperfect (i.e., noisy) state measurements. In order to find the actual winning set of a safety game for such systems, one needs to solve a partial information game via power set construction, which, in general, is computationally intractable. In this paper, we propose two conservative but computationally more efficient approaches by computing sets that can be rendered invariant with the noisy measurement. This is achieved by considering a perfect information safety game for the dynamics of an estimated state. The invariant set for this alternative game is shown to be equivalent to a noise-adapted contractive set for the original system. The controllers associated with the two proposed approaches require different knowledge of the initial states: one requires only the initial measurement and the other also requires knowing the initial state exactly. In general, the resulting controlled invariant sets by these two approaches are not comparable, and depending on the problem in hand either one can be preferable. The efficacy of the proposed approach is illustrated with an aircraft taxiing example, where the state estimation task is performed by a perception module.

Keywords: Formal Verification/Synthesis, Control of networks, Delay systems
Abstract: In this paper we construct continuous abstraction for discrete-time time-delay systems via the notion of so-called Razumikhin simulation functions. We show that the existence of such a function guarantees that the mismatch between the output trajectory of the concrete system and that of its abstraction lies within an appropriate bound. By transforming a system with time delay into an interconnected system without time delay, we show that the Razumikhin method is a small-gain type approach for time-delay systems and enables us to effectively manage computational complexity of constructing abstractions. We further extend our approach to compositional construction of large-scale systems containing interconnection and/or local time delays. For linear systems, we provide an algorithmic procedure for compositional construction of abstractions, which is expressed in terms of linear matrix inequalities.

Keywords: Formal Verification/Synthesis, Distributed control
Abstract: This paper addresses a symbolic and abstraction-based approach to distributed control design for interconnected systems. While numerous methods to design distributed control laws already exist, the key feature of the proposed approach is that controller synthesis is based on local distributed sensor information from other subsystems. An effective method is developed for quantification of such partial information in an abstraction in terms of level sets of Lyapunov-like ranking functions. The results are illustrated on an interconnected three-tank system.

Keywords: Robust control, Uncertain systems, Stability of linear systems
Abstract: As the disturbance observer (DOB)-based controller has been widely applied in practice, various aspects of the disturbance observer have been theoretically studied. In particular, robust stability of the linear closed-loop system with single-input single-output (SISO) Q-filter-based DOB has been rigorously analyzed, and finally, a necessary and sufficient condition for robust stability was obtained under the premise that the bandwidth of Q-filter is large. However, even the most recent study about the design of Q-filter-based DOB for robust stability does not offer a practical method for the determination of the Q-filter's bandwidth. In this paper, we present several lemmas regarding the determination of the bandwidth, from which a procedure is developed that can exactly compute the threshold of the bandwidth, so that robust stability (against parametric variations of the plant within a prescribed range) is lost if the bandwidth of the Q-filter becomes lower than that. The proposed procedure is implemented in a MATLAB toolbox named DO-DAT, which is now available at https://do-dat.github.io.

Keywords: Robust control, Uncertain systems, Stability of linear systems
Abstract: The control of fast sampling systems in the presence of safety-critical constraints and uncertainty poses unique computational challenges. This paper presents a novel method for synthesizing low-complexity controllers with strong robustness properties by design, i.e., robust exponential stability and recursive feasibility. The proposed controllers are based on the notion of tubes (sequence of sets of states) that can be parametrized by a tube center and cross-section. By constructing a suitable polyhedral control Lyapunov function from invariant set theory, we show how to design the central tube path using a small-scale optimization problem. Furthermore, this optimization problem can be formulated as multiparametric linear program whose solution can be efficiently computed offline. The advantages of the proposed low-complexity controller are illustrated on a benchmark problem.

Keywords: Robust control, Uncertain systems, Stability of linear systems
Abstract: For the determination of SISO gain and phase margins, the MIMO closed-loop system is broken at one single-loop channel by keeping the remaining loops closed, but these do not consider gain or phase uncertainty simultaneously in all channels. In this case, MIMO gain and phase margins are more suitable robustness measures. Three different MIMO gain and phase margins are known. It is stated in the literature without a formal proof that the MIMO gain and phase margins of the so-called balanced sensitivity transfer function are less conservative than the MIMO gain and phase margins of the complimentary sensitivity function and the sensitivity function. In this paper proofs for conservatism between the different MIMO gain and phase margins are provided.

Keywords: Robust control, Uncertain systems, Stability of linear systems
Abstract: We show that the structured singular value of a real matrix with respect to five full complex uncertainty blocks equals its convex upper bound. This is done by formulating the equality conditions as a feasibility SDP and invoking a result on the existence of a low-rank solution. A counterexample is given for the case of six uncertainty blocks. Known results are also revisited using the proposed approach.

Keywords: Robust control, Uncertain systems, Linear systems
Abstract: A feed-through compensator is a dynamic system designed for a given plant, whose output is added to the output of the plant. It is typically used in order for the zeros of the combination of the plant and the compensator (seen from the new output) to have desired properties. In particular, Isidori and Marconi (2008) utilized a robust feed-through compensator for uncertain non-minimum phase nonlinear systems, which makes the combined system become of minimum phase so that the output feedback stabilization is rather easily achieved. For the design of robust feed-through compensator, they introduced the so-called auxiliary system and showed that, if the auxiliary system admits a robust output feedback stabilizer, then the robust feed-through compensator can be systematically constructed. This paper, while restricted to linear systems, presents a constructive method for designing the robust output feedback stabilizer for the auxiliary system. For this, minimum phaseness of the auxiliary system (not the plant itself) is required. In order to overcome this restriction, we also proposed a kind of nested design for which it is enough for the second auxiliary system of the auxiliary system to have minimum phaseness.

Keywords: Emerging control applications, Mechatronics, Iterative learning control
Abstract: We study in this paper the control of hysteresis-based actuator systems where its remanence behavior (e.g., the remaining memory when the actuation signal is set to zero) must follow a desired reference point. We present a recursive algorithm for the output regulation of the hysteresis remnant behavior described by Preisach operators. Under some mild conditions, we prove that our proposed algorithm guarantees that the output remnant converges to a desired value. Simulation result shows the efficacy of our proposed algorithm.

Keywords: Mechatronics, Control applications, Lyapunov methods
Abstract: In this paper, we study a new current control method for Permanent Magnet Synchronous Motor (PMSM). First, we introduce a modified torque modulation for a velocity control loop. Then, we design a nonlinear current controller based on Lyapunov redesign in the stationary reference (α,β) frame. We show that the nonlinear current control guarantees uniform convergence of current tracking errors in finite-time. Further, uniform convergence of velocity tracking error in finite-time is also proved using the input-to-state stability (ISS). The proposed method is applied to velocity control of PMSM. We present simulation results using co-simulation with AMESim PMSM model and MATLAB/Simulink to validate the performance of the proposed method.

Keywords: MEMs and Nano systems, Model/Controller reduction, Hybrid systems
Abstract: Hysteresis phenomena can significantly deteriorate the performance when performing servo tasks with piezoelectric actuators. The aim of this paper is to model this nonlinear hysteresis effect using a memory element, in particular a MEM-element, and exploit this model to develop a feedforward controller. A one-to-one mapping is established, leading to both a systematic data-driven learning approach of a hybrid-MEM-element capturing the hysteresis phenomena and a unique inverse allowing for an intuitive design of the feedforward controller. The developed approach is experimentally validated on a piezoelectric actuator, revealing a significant performance improvement.

Keywords: Mechatronics
Abstract: In [1] we proposed output-feedback tracking control with an extended high-gain observer-based feedback control for a class of systems that include unknown hysteresis nonlinearities. In this paper, we proposed the control system for a piezomicropositioning tube actuator with uncertain nonlinearities. The proposed control system has a number of features; namely, (i) it can guarantee ultimate boundedness of the tracking error, where the ultimate bound can be made arbitrarily small, for any given initial conditions and for bounded unknown exogenous inputs and modeling parameters, (ii) it provides the possibility of shaping the transient response of the closed-loop system as desired, and (iii) the proposed technique is non-adaptive inversion free technique. In this study, we developed the proposed output-feedback approach for precision motion system, and we applied this approach experimentally to a piezotube micro-positioning actuator. The main contribution of this paper is to show that an extended high-gain observer-based output-feedback control system can stabilize a class of precision motion systems with unknown hysteresis nonlinearties.

Keywords: Optimal control, Mechatronics, Control applications
Abstract: The cascade control of the servo system has an inner control loop and an outer control loop to achieve high-performance robust control. However, additional efforts are required because the inner and outer controllers need to be sequentially tuned. To address this problem, this paper proposes a novel iterative feedback tuning method for cascade control. The proposed method adopts a cost function that is designed taking into consideration the cascade control structure and utilizes it for the IFT algorithm to simultaneously tune the inner and outer control loops. The effectiveness of the proposed method is verified by experiments using a two-inertia system which is subject to vibration. The experimental results verify that the proposed IFT can optimize the cascade controller under various plant conditions.

Keywords: Stability of hybrid systems, Lyapunov methods, Model/Controller reduction
Abstract: This paper extends bipedal trajectory tracking methods to prostheses to enable construction of a class of model-dependent prosthesis controllers using locally available sensor information. The rapidly exponentially stabilizing control Lyapunov functions (RES-CLFs) developed for bipedal robots guarantee stability of the hybrid zero dynamics in the presence of impacts that occur in walking. These methods cannot be directly applied to prostheses because of the unknown human dynamics. We overcome this challenge with two RES-CLFs, one for the prosthesis subsystem and another for the remaining human system. Further, we outline a method to construct these RES-CLFs for this type of separable system by first constructing separable CLFs for partially feedback linearizable systems. This work develops a class of separable subsystem controllers that rely only on local information but provide formal guarantees of stability for the full hybrid system with zero dynamics.

Keywords: Formal Verification/Synthesis, Autonomous robots, Hybrid systems
Abstract: This study proposes an integrated task and motion planning method for dynamic locomotion in partially observable environments with multi-level safety guarantees. This planning framework is composed of a high-level symbolic task planner and a low-level phase-space planner. The low-level consists of a keyframe decision-maker and a motion planner that generates reduced-order locomotion behaviors and proposes safety and tracking criteria for both straight and steering walking. These criteria are characterized by constraints on locomotion keyframe states, and are used to define keyframe transition policies via viability kernels. The high-level task planner, comprised of two-level navigation planners, employs linear temporal logic for reactive game synthesis between the robot and its environment while incorporating safe locomotion keyframe policies into formal task specification design. The synthesized planner commands actions including step length, step height, and heading angle changes, to the underlying keyframe decision-maker, which further determines the robot center-of-mass apex velocity and height. In particular, a belief abstraction method at the task planning level enables belief estimation of dynamic obstacle locations and guarantees safe locomotion with collision avoidance. Simulation results of our Cassie bipedal robot demonstrate locomotion maneuvering in a three-dimensional, partially observable environment consisting of dynamic obstacles and uneven terrain.

Keywords: Optimal control, Networked control systems, Robotics
Abstract: A robotic system can be viewed as a collection of lower-dimensional systems that are coupled via reaction forces (Lagrange multipliers) enforcing holonomic constraints. Inspired by this viewpoint, this paper presents a novel formulation for nonlinear control systems that are subject to coupling constraints via virtual ``coupling'' inputs that abstractly play the role of Lagrange multipliers. The main contribution of this paper is a process---mirroring solving for Lagrange multipliers in robotic systems---wherein we isolate subsystems free of coupling constraints that provably encode the full-order dynamics of the coupled control system from which it was derived. This dimension reduction is leveraged in the formulation of a nonlinear optimization problem for the isolated subsystem that yields periodic orbits for the full-order coupled system. We consider the application of these ideas to robotic systems, which can be decomposed into subsystems. Specifically, we view a quadruped as a coupled control system consisting of two bipedal robots, wherein applying the framework developed allows for gaits (periodic orbits) to be generated for the individual biped yielding a gait for the full-order quadrupedal dynamics. This is demonstrated on a quadrupedal robot through simulation and walking experiments on rough terrains

Keywords: Pattern recognition and classification, Neural networks, Intelligent systems
Abstract: Abnormal gait recognition plays an important role in diagnosis of musculoskeletal disorders. Suspicious walking behaviours should be detected as early as possible, and possibly in real-time, in order to prevent further deterioration of any part of the body. Analysis tools should provide useful and accurate information via convenient setup procedures. In this work, we conduct a system for recognizing gait abnormalities in real-time where an input is an image frame captured by a single RGB camera at any instance. We view abnormal gait recognition as a time-series problem which requires learning long-term dependencies. Hence, the system is presented with variants of Recurrent Neural Networks (RNNs). The proposed deep neural network model involves extracting 135 human body key points using OpenPose prior to performing recognition task which are quantitatively evaluated based on a simple RNN, Long Short-Term Memory Network (LSTM), and Gated Recurrent Unit Network (GRU). Each deep neural network has a model accuracy about 73.4%, 82.8%, and 81.6%, respectively. According to the confusion matrices of different predictive models, LSTM and GRU provide less confusing predictive results than that of a simple RNN. Therefore, we have discovered that deep neural network based on LSTM is, by far, the suitable model to recognize abnormal gaits due to the high model accuracy with less training and inference time.

Keywords: Robotics, Hybrid systems, Optimization algorithms
Abstract: We propose in this paper a motion planning method for legged robot locomotion based on Geometric Heat Flow framework. The task is challenging due to the hybrid nature of dynamics and contact constraints. We encode the hybrid dynamics and constraints into Riemannian inner product, and this inner product is defined so that short curves correspond to admissible motions for the system. We rely on the affine geometric heat flow to deform an arbitrary path connecting the desired initial and final states to this admissible motion. The method is able to automatically find the trajectory of robot?s center of mass, feet contact positions and forces on uneven terrain.

Keywords: Identification, Machine learning
Abstract: This paper discusses the kernel regularization in the frequency domain. In particular, this paper proposes a new kernel which encodes prior knowledge on the rate of high frequency decay. The proposed kernel has a similar structure to the one of the first order spline kernel. By exploiting the known properties of such kernel, the determinant and the inverse of the Gram matrix of the proposed kernel are given in closed form. One of the important advantages of the proposed kernel is the computational burden reduction. In fact, it turns out that the complexity is linear in the dataset size N, while standard methods require O(n^2) memory and O(n^3) flops, where n is the impulse response length usually satisfying N ll n^2 in regularization frameworks.

Keywords: Identification, Estimation, Machine learning
Abstract: We consider the problem of identifying parameters from data for systems with dynamics evolving according to a particular class of Markov chain processes, called Bernoulli Autoregressive (BAR) processes. The structure of any BAR model is encoded by a directed graph with p nodes. The edges of the graph indicate causal influences, or equivalently dynamic dependencies. More explicitly, the incoming edges to a node in the graph indicate that the state of the node at a particular time instant, which corresponds to a Bernoulli random variable, is influenced by the states of the corresponding parental nodes in the previous time instant. The associated edge weights determine the corresponding level of influence from each parental node. In the simplest setup, the Bernoulli parameter of a particular node?s state variable is a convex combination of the parental node states in the previous time instant and an additional Bernoulli noise variable; this convex combination corresponds to the associated parental edge weights and the contribution of the local noise variable. In this paper, we focus on the problem of structure and edge weight identification by relying on well-established statistical principles. We present two consistent estimators of the edge weights, a Maximum Likelihood (ML) estimator and a closedform estimator, and numerically demonstrate that the derived estimators outperform existing algorithms in the literature in terms of sample complexity.

Keywords: Identification, Statistical learning, Adaptive control
Abstract: We present a new finite-time analysis of the estimation error of the Ordinary Least Squares (OLS) estimator for stable linear time-invariant systems. We characterize the number of observed samples (the length of the observed trajectory) sufficient for the OLS estimator to be (ε, δ)-PAC, i.e., to yield an estimation error less than ε with probability at least 1 − δ. We show that this number matches existing sample complexity lower bounds [1], [2] up to universal multiplicative factors (independent of (ε, δ) and of the system). This paper hence establishes the optimality of the OLS estimator for stable systems, a result conjectured in [1]. Our analysis of the performance of the OLS estimator is simpler, sharper, and easier to interpret than existing analyses. It relies on new concentration results for the covariates matrix.

Keywords: Identification, Switched systems, Statistical learning
Abstract: This paper deals with the identification of hybrid dynamical systems that switch arbitrarily between modes. In particular, we focus on the critical issue of estimating the number of modes. A novel method inspired by model selection techniques in statistical learning is proposed. Specifically, the method implements the structural risk minimization principle, which relies on the minimization of an upper bound on the expected prediction error of the model. This so-called generalization error bound is first derived for static switched systems using Rademacher complexities. Then, it is extended to handle non independent observations from a single trajectory of a dynamical system. Finally, it is further tailored to the needs of model selection via a uniformization step. An illustrative example of the behavior of the method and its ability to recover the true number of modes is presented.

Keywords: Identification, Quantum information and control
Abstract: Hamiltonian identification plays a key role in learning the dynamics of closed quantum systems. A novel strategy for Hamiltonian identification is developed in the paper by applying repeated projective measurements with a fixed period. The corresponding measurement outcome sequence forms a time-homogeneous Markov chain, the transition matrix of which is a quadratic function of the Hamiltonian. As a result, the identification method is achieved by two steps: 1) learning the transition matrix of the observed Markov chains; 2) identifying the Hamiltonian using the function relating it to the transition matrix, which is transformed to a manifold optimization problem. A simulation example is provided to show the efficacy of the proposed method.

Keywords: Large-scale systems, Agents-based systems, Learning
Abstract: We propose a family of parametric interaction functions in the general Cucker-Smale model such that the mean-field macroscopic system of equations can be iteratively solved in an optimization scheme aiming to learn the interaction dynamics of the microscopic model from observations of macroscopic quantities. We treat the interaction functions as Green's functions for a semi-linear Poisson differential operator, which allows the transformation of the non-local interaction terms of the macroscopic model into a system of PDEs. The resulting system of hydrodynamic equations is efficiently solved as part of an iterative learning algorithm that estimates the interaction function from particle density evolution data. Finally, we utilize the proposed interaction function model to formulate an efficient learning algorithm based on observations from particle trajectories, and discuss the trade-offs associated with each approach.

Keywords: Mean field games, Agents-based systems, Optimal control
Abstract: A fixed time horizon evacuation model for a large population of confined agents is proposed within the mean field games framework. Unlike previously proposed models relying on simulations or partial differential equation analysis, the proposed model remains linear in the individual agent dynamics and quadratic in the cost functions, ultimately dictating the agents' motion. We use negative definite matrices in the cost function to reflect congestion effect during evacuations. This leads to Riccati equations that generically display a finite escape time. Therefore, we develop an existence theory for the infinite population mean field game based equilibrium dynamics, and establish its Epsilon-Nash property for a large but finite population of agents. Simulation results illustrating the numerical behavior of the model are presented with stress effect and different social interaction scenarios such as congestion and crowd following behavior.

Keywords: Mean field games, Large-scale systems, Stochastic systems
Abstract: In this work a linear quadratic instance of Graphon Mean-Field Games (GMFGs) (see [1], [2]) is analysed. Such games involve an asymptotically infinite population of agents, distributed over a very large scale network which itself is asymptotically infinite. The linear dynamics of each agent together with its quadratic running and terminal cost functions depend upon non-uniform averages (i.e. local and global mean fields) of the states of all other agents in the network system. In the infinite limit of the population and the network, the agent?s dynamics and costs are functions of the family of local mean fields distributed at the nodes of the infinite network. Moreover, the limiting infinite networks are modelled by graphons which are symmetric measurable functions defined on the unit square(see [6]). Specifically, Linear Quadratic Graphon Mean Field Games model the idea of clustering for populations of agents at the nodes of the very large scale network. First, using aprobabilistic approach, we characterize the solutions of the Linear Quadratic Graphon Mean Field Games with solutions to coupled Forward Backward Stochastic Differential Equations(FBSDEs) of McKean-Vlasov type. We next deduce the existence of the so-called Master field, which allows for the decoupling of these FBSDEs. Finally, we derive the infinite dimensional Partial Differential Equation (PDE), so-called Master Equation(see [4]), for which the Master Field is a solution.

Keywords: Mean field games, Stochastic optimal control, Cyber-Physical Security
Abstract: In this paper, we consider a finite horizon, non-stationary, mean field game (MFG) with a large population of homogeneous players, sequentially making strategic decisions, where each player is affected by other players through an aggregate population state termed as mean field state. Each player has a private type that only it can observe, and a mean field population state representing the empirical distribution of other players? types, which is shared among all of them. Recently, authors in [1] provided a sequential decomposition algorithm to compute mean field equilibrium (MFE) for such games which allows for the computation of equilibrium policies for them in linear time than exponential, as before. In this paper, we extend it for the case when state transitions are not known, to propose a reinforcement learning algorithm based on Expected Sarsa with a policy gradient approach that learns the MFE policy by learning the dynamics of the game simultaneously. We illustrate our results using cyber-physical security example.

Keywords: Mean field games, Stochastic optimal control, Learning
Abstract: In this paper, zero-sum mean-field type games (ZSMFTG) with linear dynamics and quadratic cost are studied under infinite-horizon discounted reward function. ZSMFTG are a class of games in which two decision makers whose utilities sum to zero, compete to influence a large population of agents. In particular, the case in which the transition and reward functions depend on the state, the action of the controllers, and the mean of the state and the actions, is investigated. The game is analysed and explicit expressions for the Nash equilibrium strategies are derived. Moreover, two policy optimization methods that rely on policy gradient are proposed for both model-based and sample-based frameworks. In the model-based case, the gradients are computed exactly using the model whereas they are estimated using Monte-Carlo simulations in the sample-based case. Numerical experiments are conducted to show the convergence of the two players' controls as well as the cost function when the two algorithms are used in different scenarios.

Keywords: Optimal control, Stochastic optimal control, Optimization algorithms
Abstract: We consider the problem of finitely parameterized multi-armed bandits where the model of the underlying stochastic environment can be characterized based on a common unknown parameter. The true parameter is unknown to the learning agent. However, the set of possible parameters, which is finite, is known a priori. We propose an algorithm that is simple and easy to implement, which we call Finitely Parameterized Upper Confidence Bound (FP-UCB) algorithm, which uses the information about the underlying parameter set for faster learning. In particular, we show that the FP-UCB algorithm achieves a bounded regret under some structural condition on the underlying parameter set. We also show that, if the underlying parameter set does not satisfy the necessary structural condition, the FP-UCB algorithm achieves a logarithmic regret, but with a smaller preceding constant compared to the standard UCB algorithm. We also validate the superior performance of the FP-UCB algorithm through extensive numerical simulations.

Keywords: Optimization, Stochastic systems, Optimization algorithms
Abstract: In this paper, we propose a consensus-based algorithm for nonconvex optimization on the Stiefel manifold. For a given objective function on the Stiefel manifold, we construct a stochastic interacting particle system for sample points so that all the sample points are expected to asymptotically converge to a single point, which is close enough to a global minimizer. We show the global existence and uniqueness of solutions to our stochastic differential equation (SDE) model for consensus. A predictor-corrector type numerical scheme is then proposed for implementing the SDE model with the guarantee that each sample point stays on the Stiefel manifold. A salient feature of our algorithm is that it is gradient-free, thereby applicable to a wide range of problems. The results of our numerical experiments demonstrate that the proposed method can successfully find a global minimizer even when the objective function is nonconvex.

Keywords: Control applications, Optimization
Abstract: For sequential betting games, Kelly?s theory, aimed at maximization of the logarithmic growth of one?s account value, involves optimization of the so-called betting fraction K. In this talk, we extend the classical formulation to allow for temporal correlation among bets. To demonstrate the potential of this new paradigm, for simplicity of exposition, we mainly address the case of a coin-flipping game with even-money payoff. To this end, we solve a problem with memory depth m. By this, we mean that the outcomes of coin flips are no longer assumed to be i.i.d. random variables. Instead, the probability of heads on flip k depends on previous flips k-1; k-2; ...; k-m. For the simplest case of n flips, with m = 1, we obtain a closed form solution Kn for the optimal betting fraction. This generalizes the classical result for the memoryless case. That is, instead of fraction K* = 2p - 1 which pervades the literature for a coin with probability of heads p >= 1/2, our new fraction K_n depends on both n and the parameters associated with the temporal correlation. Generalizations of these results for m > 1 and numerical simulations are also included. Finally, we indicate how the theory extends to time-varying feedback and alternative payoff distributions.

Keywords: Optimization algorithms, Uncertain systems
Abstract: We study two canonical online optimization problems under capacity/budget constraints, the fractional one-way trading problem (OTP) and the integral online knapsack problem (OKP) under an infinitesimal assumption. Under the competitive analysis framework, it is well-known that both problems have the same optimal competitive ratio. However, these two problems are investigated by distinct approaches under separate contexts in the literature. There is a gap in understanding the connection between these two problems and the nature of their online algorithm design. This paper provides a unified framework for the online algorithm design, analysis and optimality proof for both problems. We find that the infinitesimal assumption of the OKP is the key that connects the OTP in the analysis of online algorithms and the construction of worse-case instances. With this unified understanding, our framework shows its potential for analyzing other extensions of OKP and OTP in a more systematic manner.

Keywords: Optimization, Simulation, Modeling
Abstract: Motivated by the prominence of Conditional Value-at-Risk (CVaR) as a measure for tail risk in settings affected by uncertainty, we develop a new formula for approximating CVaR based optimization objectives and their gradients from limited samples. Unlike the state-of-the-art sample average approximations which require impractically large amounts of data in tail probability regions, the proposed approximation scheme exploits the self-similarity of heavy-tailed distributions to extrapolate data from suitable lower quantiles. The resulting approximations are shown to be statistically consistent and are amenable for optimization by means of conventional gradient descent. The approximation is guided by means of a systematic importance-sampling scheme whose asymptotic variance reduction properties are rigorously examined. Numerical experiments demonstrate the superiority of the proposed approximations and the ease of implementation points to the versatility of settings to which the approximation scheme can be applied.

Keywords: Optimal control, Optimization
Abstract: This paper presents a general Hamilton-Jacobi (HJ) framework for optimal control and two-player zero-sum game problems, both with state constraints. In the optimal control problem, a control signal and terminal time are determined to minimize the given cost and satisfy the state constraints. In the game problem, the two players interact via the system dynamics. Here, a strategy for each player, as well as a terminal time, are determined so that player A minimizes the cost and satisfies the state constraints while player B tries to prevent the success of player A. Dynamics, costs, and state constraints are time-varying. HJ equations are proposed, bridging the viability theory for constrained problems [1] and problems in which the terminal time is specified [2]. A numerical algorithm for computing the solution of the proposed HJ equations is presented and demonstrated with a practical example: vehicle lane-changing while avoiding other vehicles.

Keywords: Optimal control, Robust control, Algebraic/geometric methods
Abstract: This article considers a discrete-time robust optimal control problem on matrix Lie groups. The underlying system is assumed to be perturbed by exogenous unmeasured bounded disturbances, and the control problem is posed as a min-max optimal control wherein the disturbance is the adversary and tries to maximise a cost that the control tries to minimise.

Assuming the existence of a saddle point in the problem, we present a version of the Pontryagin maximum principle (PMP) that encapsulates first-order necessary conditions that the optimal control and disturbance trajectories must satisfy. This PMP features a saddle point condition on the Hamiltonian and a set of backward difference equations for the adjoint dynamics. We also present a special case of our result on Euclidean spaces.

Keywords: Optimal control, Optimization algorithms, Machine learning
Abstract: We consider policy gradient algorithms for the indefinite least squares stationary optimal control, e.g., linear- quadratic-regulator (LQR) with indefinite state and input penalization matrices. Such a setup has important applications in control design with conflicting objectives, such as linear quadratic dynamic games. We show the global convergence of gradient, natural gradient and quasi-Newton policies for this class of indefinite least squares problems.

Keywords: Optimal control
Abstract: This paper introduces a new algorithm for the solution of infinite horizon optimal control problems for input-affine nonlinear systems based on a parameterization of the solutions of an associated state-dependent Riccati equation. This allows the transformation of the problem into an optimisation problem and allows approximate solutions to be derived. The algorithm is illustrated on two nonlinear problems with known solution. If applied to solve the classical Linear Quadratic Regulator problem, the algorithm recovers the solution.

Keywords: Optimal control
Abstract: This note discusses properties of parametric discrete-time Mixed-Integer Optimal Control Problems (MIOCPs) as they often arise in model predictive control with discrete controls. We argue that, in want for a handle on similarity properties of parametric MIOCPs, the turnpike phenomenon known in optimal control is helpful. We provide sufficient turnpike conditions based on a dissipativity notion of MIOCPs, and we prove that the turnpike phenomenon allows specific and accurate guesses for the discrete controls. We also derive an easily checkable sufficient condition for dissipativity of linear-quadratic MIOCPs. Moreover, we show how the turnpike property can be used to derive efficient node-weighted branch-and-bound schemes tailored to parametric MIOCPs. We draw upon numerical examples to illustrate our findings.

Keywords: Network analysis and control, Control of networks, Agents-based systems
Abstract: In complex social networks, the decision-making mechanisms behind human actions and the cognitive processes that shape opinion formation processes are often intertwined, leading to complex and varied collective emergent behavior. In this paper, we propose a mathematical model that captures such a coevolution of actions and opinions. Following a discrete-time process, each individual decides between binary actions, aiming to coordinate with the actions of other members observed on a network of interactions and taking into account their own opinion. At the same time, the opinion of each individual evolves due to the opinions shared by other members, the actions observed on the network, and, possibly, an external influence source. We provide a global convergence result for a special case of the coupled dynamics. Steady state configurations in which all the individuals take the same action are then studied, elucidating the role of the model parameters and the network structure on the collective behavior of the system.

Keywords: Networked control systems, Network analysis and control, Biological systems
Abstract: During the analysis and design of a network system, a frequently encountered situation is when the network structure is known but the node dynamics is practically unknown. Thus, structural analysis problems in which the prior knowledge is mostly about the network structure have received considerable attention in science and engineering fields. Motivated by this line of research, this study focuses on a structural design problem called the structural equilibrium control problem, which is used to find a sign pattern in the control input such that any constant input with the sign pattern drives the system to a desirable steady state in a qualitative sense. This problem is solved by reducing it to a design problem of the so-called sign-solvable equation, which is a linear equation whose qualitative solution can be determined from the prior knowledge of the sign pattern of the coefficients. Further, the result is extended to the case in which additional information about the node dynamics is available. Finally, we apply the proposed framework to a control problem of the biological network of apoptosis. It is demonstrated that our solution is useful if just a qualitative model is available for the biological system.

Keywords: Game theory, Stochastic optimal control, Markov processes
Abstract: This paper deals with the study of adversarial social contagion processes as a two-player game on finite complex networks. In our formulation, a censor affects the information diffusion dynamics on finite complex social networks, modeled as a controlled Markov chain, to minimize the number of infected (information aware) individuals. A stopper chooses when to terminate the information relayed to the network. This makes it a two-player dynamic game between the censor and the stopper, and is named as the Streisand game.

We show that the game has a well-defined upper value, achieved by pure minimax strategies under the feedback information structure, and establish that it is an increasing function of the state. We provide a game-value iteration algorithm to compute the upper value and the minimax strategies.

Keywords: Control of networks, Feedback linearization, Algebraic/geometric methods
Abstract: Feedback linearization allows for the local transformation of a nonlinear system to an equivalent linear one by means of a coordinate transformation and a feedback law. Feedback linearization of large-scale nonlinear network systems is typically difficult, as existing conditions become harder to check as the network size becomes larger. In this paper, we provide novel conditions to test whether a nonlinear network is feedback-linearizable. Specifically, given some dedicated control inputs injected to a set of network nodes, we derive an easy-to-check algebraic condition that can be tested on the Jacobian matrix of the network dynamics evaluated at some desired working point. Furthermore, our requirements are sufficient for (local) controllability, and thus provide a testable condition for controllability of large-scale nonlinear networks. Finally, we validate our findings by enforcing the formation of desired synchronization patterns in networks of coupled oscillators.

Keywords: Network analysis and control, Control of networks, Distributed control
Abstract: In this paper, we propose the use of a distributed discontinuous coupling protocol to achieve convergence and synchronization in networks of non-identical nonlinear dynamical systems. We show that the synchronous dynamics is a solution to the average of the nodes? vector fields, and derive analytical estimates of the critical coupling gains required to achieve convergence.

Keywords: Optimization algorithms, Power systems
Abstract: Inspired by the echolocation behaviors of the bats and the swarm intelligence optimization, bat searching algorithm (BA) was developed to solve unconstrained optimization problems efficiently. However, due to the lack of the gradient term, the accuracy of the BA is not superior, and the enhancement of the algorithm is still of vital importance. The sign gradient descent method (SGD) is a first-order optimization method involving only the sign of the gradient of the function to minimize. Most importantly, the convergence and optimality issues of the SGD have been rigorously studied, which guarantees the competitive performance of SGD method. Therefore, in this paper, a combination of the BA and SGD method is proposed by integrating the SGD term into the update equation of the bats during the searching process. With the social behavior among the bats and the sign gradient descent method, the proposed algorithm shows significant improvement comparing with the original algorithm. Moreover, the convergence issue of the proposed algorithm is studied from system dynamics perspective. The numerical evaluations are provided to demonstrate the improvement of the proposed sign gradient descent method based bat searching algorithm. In the end, the economic load dispatch problem for the power system is studied as an application of the proposed BA algorithms. Based on the numerical results, the proposed BA shows superior performance.

Keywords: Identification, Distributed control, Optimization
Abstract: The identification of structured state-space models with multi-diagonal block Teoplitz system matrices is studied in this paper. Due to the non-convex nature of the identification problem, it is difficult to obtain a global optimal solution. To deal with this problem, the concerned state-space model is casted to an equivalent one with block circulant system matrices and a low-dimension unknown input related term. Then, the identification problem is formulated as a low rank regularized optimization problem which is solved by the sequentially convex programming method. The effectiveness of the proposed identification method is finally verified through numerical simulations.

Keywords: Optimization, Optimization algorithms
Abstract: This paper investigates the robust uncertain two-level cooperative set covering problem (RUTLCSCP). Given two types of facilities, which are called y-facility and z-facility. The problem is to decide which facilities of both types to be selected, in order to cover the demand nodes cooperatively with minimal cost. It combines the concepts of robust, probabilistic, and cooperative covering by introducting "Γ-robust two-level-cooperative α-cover" constraints. Additionally, the constraint relaxed verison of the RUTLCSCP, which is also a linear approximation robust counterpart version of RUTLCSCP (RUTLCSCP-LA-RC), is developed by linear approximation of the constraints, and can be stated as a compact mixed-integer linear programming problem. We show that the solution for RUTLCSCP-LA-RC, ε-under-approximate solution, can also be the solution for RUTLCSCP on some conditions. Computational experiments show that the solutions in 333 instances (10125 instances in total) with 12 types which tinily violate the constraints of RUTLCSCP, can be an efficient under-approximate solutions, while the feasible solutions in other instances are proven to be optimal.

Keywords: Estimation, Kalman filtering, Sensor networks
Abstract: Adaptive algorithms based on in-network processing over networks are useful for online parameter estimation of historical data (e.g., noise covariance) in predictive control and machine learning areas. This paper focuses on the distributed noise covariance matrices estimation problem for multi-sensor linear time-invariant (LTI) systems. Conventional noise covariance estimation approaches, e.g., auto-covariance least squares (ALS) method, suffers from the lack of the sensor's historical measurements and thus produces high variance of the ALS estimate. To solve the problem, the distributed auto-covariance least squares (D-ALS) algorithm is proposed based on the batch covariance intersection (BCI) method by enlarging the innovations from the neighbors. The accuracy analysis of D-ALS algorithm is given to show the decrease of the variance of the D-ALS estimate. The numerical results of cooperative target tracking tasks in static and mobile sensor networks are demonstrated to show the feasibility and superiority of the proposed D-ALS algorithm.

Keywords: Agents-based systems, Intelligent systems, Optimization algorithms
Abstract: Travelling salesman problem (TSP) is a classic combinatorial optimization problem and has become a touchstone of many optimization algorithms. Constructive heuristics with the features of low complexity and using problem knowledge are widely used in online decision-making and can provide high-quality initial solution for iteration algorithms. In this paper, from an agent-based self-organization perspective, the constructive processes from nodes to Hamiltonian graph of a feasible solution are studied. Based on different constructive processes, three novel agent-based constructive heuristic methods (ACHMs) are proposed, including multi-nodes-based ACHM, loop-based ACHM and multi-loop-based ACHM. These constructive heuristic methods build different agent models based on node and loop respectively, and set varied agent actions to make global feasible solutions emerge gradually. Finally, compared with nearest neighbor algorithm and self-organizing mapping, the better performances of these algorithms for TSP are verified by the computational experiments.

Keywords: Statistical learning, Stochastic systems
Abstract: We introduce and study a new class of stochastic bandit problems, referred to as predictive bandits. In each round, the decision maker first decides whether to gather information about the rewards of particular arms (so that their rewards in this round can be predicted). These measurements are costly, and may be corrupted by noise. The decision maker then selects an arm to be actually played in the round. Predictive bandits find applications in many areas; e.g. they can be applied to channel selection problems in radio communication systems, or in recommendation systems. In this paper, we provide the first theoretical results about predictive bandits, and focus on scenarios where the decision maker is allowed to measure at most one arm per round and the rewards are Bernoulli random variables. We derive asymptotic instance-specific regret lower bounds for these problems, and develop algorithms whose regret match these fundamental limits. We illustrate the performance of our algorithms through numerical experiments. In particular, we highlight the gains that can be achieved by using reward predictions, and investigate the impact of the noise in the corresponding measurements.

Keywords: Statistical learning, Stochastic systems, Identification
Abstract: We seek a generalization of regression and prin- ciple component analysis (PCA) in a metric space where data points are distributions metrized by the Wasserstein metric. We recast these analyses as multimarginal optimal transport problems. The particular formulation allows efficient computation, ensures existence of optimal solutions, and admits a probabilistic interpretation over the space of paths (line segments). Application of the theory to the interpolation of empirical distributions, images, power spectra, as well as assessing uncertainty in experimental designs, is envisioned.

Keywords: Statistical learning
Abstract: This paper presents an algorithm and regret analysis for the restless hidden Markov bandit problem with linear rewards. In this problem the reward received by the decision maker is a random linear function which depends on the arm selected and a hidden state. In contrast to previous works on Markovian bandits, we do not assume that the decision maker receives information regarding the state of the system, but can only infer/estimate it based on its actions and the received reward. Additionally, it is assumed that the decision maker knows in advance that the reward is a random linear function which depends on the selected arm, the action, and hidden states. However, the decision maker does not know in advance the probability distributions of these hidden states; thus we call this side information structural side information. Surprisingly, we can still maintain logarithmic regret in the case of polyhedral action set. Furthermore, we show that the structural side information leads to expected regret that does not depend on the number of extreme points in the action space.

Keywords: Nonlinear systems identification, Statistical learning, Estimation
Abstract: We propose a Bayesian probabilistic formulation for system identification of Hamiltonian systems. This approach uses an approximate marginal Markov Chain Monte Carlo algorithm to directly discover a system Hamiltonian from data. Our approach improves upon existing methods in two ways: first we encode the fact that the data generating process is symplectic directly into our learning objective, and second we utilize a learning objective that simultaneously accounts for unknown parameters, model form, and measurement noise. This objective is the log marginal posterior of a probabilistic model that embeds a symplectic and reversible integrator within an uncertain dynamical system. We demonstrate that the resulting learning problem yields dynamical systems that have improved accuracy and reduced predictive uncertainty compared to existing state-of-the-art approaches. Simulation results are shown on the Henon-Heiles Hamiltonian system.