Keywords: Optimal control, Estimation, Healthcare and medical systems
Abstract: A variety of optimal control, estimation, system identification and design problems can be formulated as functional optimization problems with differential equality and inequality constraints. Since these problems are infinite-dimensional and often do not have a known analytical solution, one has to resort to numerical methods to compute an approximate solution. This paper uses a unifying notation to outline some of the techniques used in the transcription step of simultaneous direct methods (which discretize-then-optimize) for solving continuous-time dynamic optimization problems. We focus on collocation, integrated residual and Runge-Kutta schemes. These transcription methods are then applied to a simulation case study to answer a question that arose during the COVID-19 pandemic, namely: If there are not enough ventilators, is it possible to ventilate more than one patient on a single ventilator? The results suggest that it is possible, in principle, to estimate individual patient parameters sufficiently accurately, using a relatively small number of flow rate measurements, without needing to disconnect a patient from the system or needing more than one flow rate sensor. We also show that it is possible to ensure that two different patients can indeed receive their desired tidal volume, by modifying the resistance experienced by the air flow to each patient and controlling the ventilator pressure.

Keywords: Healthcare and medical systems, Biomedical, Human-in-the-loop control
Abstract: Due to concerns at the availability of ventilators to care for the more unwell patients during the COVID-19 pandemic, we consider the question: In an extreme circumstance, where no other options are available, can two patients with different lung physiologies be successfully ventilated on a single ventilator? We present a model that allows one to do system identification, estimation and control of a dual-patient ventilator system. This case study is used throughout other talks in this session.

Keywords: Numerical algorithms, Optimal control, Estimation
Abstract: We give an outline of direct methods for discretizing continuous-time dynamic optimization problems so that one can compute approximate solutions by solving a sequence of finite-dimensional optimization problems. We focus on direct collocation, integrated residual, Runge-Kutta and shooting methods. Error analysis and mesh refinement is discussed as a key part of the solution process. The dual-patient ventilator case study is used for numerical simulations.

Keywords: Optimization algorithms, Optimal control, Optimization
Abstract: The optimization problems arising from transcribing dynamic optimization problems are highly structured. We outline some numerical methods that could be used to efficiently exploit this structure when computing the solution to the optimization problem.The dual-patient ventilator case study is used for numerical simulations.

Keywords: Optimal control, Identification, Estimation
Abstract: The key points discussed in this tutorial session will be summarised. We will also briefly discuss topics not discussed in this session and outline areas for research.

Keywords: Networked control systems, Machine learning, Communication networks
Abstract: Emerging applications in IoT systems employ wireless communication networks to exchange data between spatially distributed components of a control system. As wireless networks are noisy and subject to packet losses - which might impact the operation of the control system - proper distribution of communication resources among components of the wireless control system sharing the communication network is essential to maintain the system operating reliably. Here, in particular, we study settings in which the decision maker must meet additional constraints on the control system while distributing communication resources. The existence of constraints and the infinite dimensionality of the problem make the resource allocation problem challenging. To reduce the dimensionality of the problem, we parameterize the resource allocation policy in terms of neural networks - high capability approximators - and leverage reinforcement learning techniques to design policies that do not require knowledge of plant dynamics or communication models. We further reformulate the resource allocation problem in the dual domain to handle the constraints, which leads naturally to a primal-dual policy gradient algorithm that alternates between updating the policy parameters via reinforcement learning iterations and updating a dual variable that enforces constraint satisfaction. We conclude the paper with numerical simulations that show the strong performance of the learned allocation policies against baseline resource allocation solutions.

Keywords: Control of networks, Decentralized control, Optimal control
Abstract: In this paper, we study the distributed algorithm to solve the optimal resource allocation problem with timevarying cost functions and resources for continuous-time multiagent systems. Subject to a coupled linear equality constraint, all agents aim to minimize the sum of all local cost functions known only to each agent. Since the cost functions and resources are time varying, the optimal solutions are trajectories changing over time rather than some constants. By combining the predictioncorrection method with the finite-time nonsmooth consensus idea, we propose a distributed continuous-time algorithm that ensures that the states of all agents will track the corresponding timevarying optimal trajectories with vanishing error when the cost functions have identical Hessians. Here each agent exchanges only local information through a fixed connected undirected graph in a distributed manner. We perform a simulation for a grid-connected battery energy storage system to illustrate the effectiveness of the proposed distributed continuous-time optimal resource allocation algorithm.

Keywords: Optimization algorithms, Smart grid, Power systems
Abstract: Increased penetration of distributed energy resources, especially solar at a community level, can provide additional resiliency and improve the reliability of the distribution system. By altering the statuses of switches, dynamic microgrids that feature adjustable topologies and boundaries can use distributed generation to serve one or more critical service facilities within them. In this paper, the critical service restoration problem is first formulated in a distributed manner. Moreover, a heuristic distributed optimization approach is proposed, whose optimality gap is shown to be bounded. The case study compares the proposed algorithm with the centralized optimization approach.

Keywords: Game theory, Smart grid, Decentralized control
Abstract: Local energy generation as well as local energy storage represent key opportunities for energy transition. Nevertheless, their massive deployment is being delayed mainly due to cost reasons. Sharing resources at the local level enables not only reducing these costs significantly, but also to further optimize the cost of the energy exchanged with providers external to the local community. A key question that arises while sharing resources is how to distribute the obtained benefits among the various local players that cooperate. In this paper we propose a cooperative game model, where the players are the holders of energy resources (generation and storage); they cooperate in order to reduce their individual electricity costs. We prove that the core of the game is non-empty; i.e., the proposed cooperative game has a stable solution (distribution of the payoffs among the players) for the case where all players participate in a unique community, and no strict subset of players can obtain a better gain by leaving the community.

We propose a formulation of this game, based on the theory of linear production games, which lead us to the two main contributions of this paper. First, we propose an efficient (with linear complexity) centralized algorithm for finding a stable payoff. Second, we provide an efficient distributed algorithm that computes an allocation in the core of the game without any requirement for the players to share any private information. The distributed algorithm requires the exchange of intermediate solutions among players. The topology of the network that enables these exchanges is closely related to the performance of the distributed algorithm. We show, by way of simulations, which are the best topologies for these communication graphs.

Keywords: Game theory, Transportation networks
Abstract: In this paper, we investigate the social resource allocation in an emerging mobility system consisting of connected and automated vehicles (CAVs) using mechanism design. CAVs provide the most intriguing opportunity for enabling travelers to monitor mobility system conditions efficiently and make better decisions. However, this new reality will influence travelers' tendency-of-travel and might give rise to rebound effects, e.g., increased-vehicle-miles traveled. To tackle this phenomenon, we propose a mechanism design formulation that provides an efficient social resource allocation of travel time for all travelers. Our focus is on the socio-technical aspect of the problem, i.e., by designing appropriate socio-economic incentives, we seek to prevent potential rebound effects. In particular, we propose an economically inspired mechanism to influence the impact of the travelers' decision-making on the well-being of an emerging mobility system.

Keywords: Optimization, Optimization algorithms
Abstract: In this paper, we focus on the allocation of goods (or resources) among competitive agents under submodular utility functions. We introduce two quantities, namely stability and fairness, to characterize an allocation. Our definition of stability is motivated by the famous stable marriage problem and measures the incentive that the agents have to deviate from the allocation. Furthermore, we quantify the fairness using the maximin fairness ratio which stems from the maximin fair shares. These shares represent an ideal scenario wherein each agent is given the option to maximize her worst possible outcome. Following that, we design an efficient greedy mbox{round-robin} algorithm for the submodular allocations problem. We prove that our algorithm gives a stable allocation and we extract a curvature-dependant maximin fairness ratio. To demonstrate the empirical performance of our algorithm, we conduct experimental studies and observe that the empirical fairness ratio is higher than 95% of the solution given by a brute-force approach, which outperforms the theoretical guarantee.

Keywords: Networked control systems, Optimization, Control over communications
Abstract: In the design of cyber-physical systems (CPS) where multiple heterogeneous physical systems are coupled via a communication network, a key aspect is to study how network services are distributed among the users. To answer this adequately, we consider the coupling parameters between the control and network layers, and also the time-sensitive limitations and tolerances of the individual physical systems and the network. In this article, we first describe a cross-layer model for CPS wherein multiple stochastic linear processes are coupled via a shared network that provides a diverse range of cost-prone and capacity-limited services with distinct latency characteristics. Service prices are given such that low latency services incur higher communication cost, and prices remain fixed over a constant period of time but will be adjusted by the network for the future time periods. Physical systems decide to use specific services over each time interval depending on the service prices and their own time sensitivity requirements. Considering the service availability, the network coordinates resource allocation such that physical systems are serviced the closest to their preferences. Performance of individual systems are measured by an expected quadratic cost and we formulate a social optimization problem subject to time-sensitive requirements of the physical systems and the network constraints. From the formulated optimization problem, we derive the joint optimal time-sensitive control and service allocation policies.

Keywords: Control applications, Queueing systems, Predictive control for nonlinear systems
Abstract: Data centers are energy-hungry infrastructures that provide cloud computing services. The growing number of data centers in use has led to a drastic increment of the energy consumption associated to these facilities, causing environmental concerns. For that reason, efficient management strategies are needed in order to reduce the energy consumption while the quality of service is kept. This paper presents a unified management approach for the thermal and workload distribution problem in data centers, shaped as a Model Predictive Control problem. The corresponding optimization problem is intractable for conventional solvers because the model is based on multiple queues and the decision variables are a mix of integer and real valued ones. A highly parallelizable particle based optimization algorithm is proposed to solve the optimization problem. Numerical simulations are provided in order to illustrate the effectiveness of the strategy.

Keywords: Fault detection, Fault diagnosis, Linear systems
Abstract: This paper deals with the fault detection and isolation (FDI) problem for linear structured systems in which the system matrices are given by zero/nonzero/arbitrary pattern matrices. In this paper, we follow a geometric approach to verify solvability of the FDI problem for such systems. To do so, we first develop a necessary and sufficient condition under which the FDI problem for a given particular linear time-invariant system is solvable. Next, we establish a necessary condition for solvability of the FDI problem for linear structured systems. In addition, we develop a sufficient algebraic condition for solvability of the FDI problem in terms of a rank test on an associated pattern matrix. To illustrate that this condition is not necessary, we provide a counterexample in which the FDI problem is solvable while the condition is not satisfied. Finally, we develop a graph-theoretic condition for the full rank property of a given pattern matrix, which leads to a graph-theoretic condition for solvability of the FDI problem.

Keywords: Fault diagnosis, Control system architecture, Linear systems
Abstract: A decoupling approach for fault isolation in MIMO systems is presented in this paper. The fault isolation approach is based on a closed-loop concept, where the feedback controller is an integrated part of the set-up. The YJBK-parameterization (after Youla, Jabr, Bongiorno, and Kucera) for controllers is introduced. This allows the feedback controller to be modified by changing the YJBK matrix transfer function without changing the nominal feedback controller.

Modification of the feedback controller via the YJBK transfer matrix causes the output residual response to be changed in the case of parametric faults and unchanged in the fault-free case. This facilitates fault detection. In the isolation case, the controller is modified in a way such that the residual response from the closed-loop system is independent of the modification for one specific parametric fault and else it depends on the parametric faults. This allows for any specific fault and eventually for all faults to be isolated.

Keywords: Fault diagnosis, Linear systems
Abstract: This letter proposes a novel online active fault diagnosis (AFD) method based on invariant sets for linear timeinvariant (LTI) systems with bounded disturbances and noises. In general, the system has the healthy mode and several faulty modes. During system operation, with respect to the healthy and faulty modes, the corresponding healthy and faulty output estimation sets are computed, respectively. Thus, the task of AFD can be converted to separate healthy and faulty output estimation sets, which can consequently distinguish the current system mode and implement AFD. The existing AFD methods implement the objective by designing an input sequence to separate the healthy and faulty output estimation sets. However, it is difficult to use these methods to simultaneously combine AFD and fault-tolerant control. Thus, we propose a new AFD method in this letter to online design inputs step by step, which has potential to overcome some weakness of the existing AFD methods. Particularly, we implement this AFD objective by online designing inputs at each step to maximize the distances of these healthy and faulty output estimation sets. Moreover, this online input design problem can finally be transformed into the resolution of two mixed integer programming (MIP) problems with a similar structure. At the end of this letter, a quadruple-tank benchmark system is used to illustrate the effectiveness of the proposed method by comparing with an existing set-based AFD method.

Keywords: Fault detection, Robotics, Flexible structures
Abstract: This paper presents a unified approach to detection and isolation of both actuator faults and unexpected collisions for a two-link robot with a flexible forearm. The proposed approach is sensorless, i.e., no dedicated exteroceptive sensors are considered, and is based on the design of residuals to be used as monitoring filters. The method has been tested by extensive simulations on the Flexarm robot used as case study. The reported results show the efficacy in detecting and isolating faults in the actuators or collisions on the robot links.

Keywords: Fault diagnosis, Large-scale systems, Control system architecture
Abstract: Structural analysis is a useful tool for fault diagnosability analysis to handle systems that are described by a large set of non-linear differential algebraic equations. Distributed fault diagnosis is an attractive approach for complex systems to reduce computational complexity by partitioning the system into a set of smaller subsystems and perform fault diagnosis of each subsystem. Defining these subsystems requires methods to understand how fault diagnosis properties of each subsystem relates to the properties of the whole system. Another related problem is that large and complex systems are likely to be developed by several companies where each company is developing different subsystems that can be used in different system configurations. In these situations, each subsystem will have limited model information about the other subsystems, which complicates performing structural analysis of the whole system. The main contribution in this work is extending some of the existing results in structural analysis for one system model to a distributed set of connected subsystems. The results show the relationship between structural fault diagnosis properties of the whole system and properties of the set of individual subsystems.

Keywords: Fault tolerant systems, Agents-based systems, Communication networks
Abstract: This paper focuses on the fault-tolerant tracking problem in networks of leader-following heterogeneous linear multi-agent systems. Firstly, a fault estimation observer is constructed to estimate simultaneously the actuator faults and the followers' states. Then, a fault-tolerant tracking controller is proposed. The latter is based on a dynamic internal reference for each agent and exploits the estimated faults. Thus, it is at first shown that the fault-tolerant tracking problem can be indirectly solved through the consensus tracking of the defined internal references. Then, a sufficient condition of consensus tracking for the internal references is proposed. Finally, a sufficient and necessary condition is derived for the fault-tolerant tracking control of linear heterogeneous multi-agent systems. The effectiveness of the proposed approach is illustrated with a numerical simulation.

Keywords: Fault tolerant systems, Fault detection, Agents-based systems
Abstract: In this paper, we approach the problem of a set of network agents reaching resilient consensus in the presence of a subset of attacked nodes. We devise a generalized method, with polynomial time complexity, which receives as input a discrete-time, synchronous-communication consensus algorithm, a dynamic network of agents, and the maximum number of attacked nodes. The distributed algorithm enables each normal node to detect and discard the values of the attacked agents while reaching the consensus of normal agents for the input consensus algorithm. Hence, the proposed method adds an extra layer of resilience to a given discrete-time and synchronous-communication consensus algorithm. Finally, we demonstrate the effectiveness of the method with experimental results, showing some attack circumstances which we can counter, where the state-of-the-art methods fail.

Keywords: Fault tolerant systems, Variable-structure/sliding-mode control
Abstract: In the event of actuator faults and failures the total resources available to the controller can be severely reduced. In such cases it is important for the controller to be reconfigured to avoid saturating and overloading the remaining healthy actuators. Actuator saturation can lead to loss of system performance and in some cases, even instability. This paper proposes a method of reducing the closed-loop performance requirements through a scheduled reference model, with aim to minimise any saturation in the closed-loop system in the event of serious faults. The reference model is then tracked with a robust sliding mode control allocation scheme to ensure asymptotic tracking despite actuator faults/failures and uncertainty in their detection/estimation.

Keywords: Uncertain systems, Statistical learning, Nonlinear systems identification
Abstract: Model uncertainty sets are required in many robust optimization problems, such as robust control and prediction with uncertainty, but there is no definite methodology to generate uncertainty sets for nonlinear dynamical systems. In this paper, we propose a method for model uncertainty set generation via Markov chain Monte Carlo. The proposed method samples from distributions over dynamical systems via metrics over transfer operators and is applicable to general nonlinear systems. We adapt Hamiltonian Monte Carlo for sampling high-dimensional transfer operators in a computationally efficient manner. We present numerical examples to validate the proposed method for uncertainty set generation.

Keywords: Uncertain systems, Computer-aided control design, Computational methods
Abstract: We describe a set-based approach, relying on mean-value extensions, for computing guaranteed under-approximations of ranges (or images) of continuously differentiable vector-valued functions, including what we call robust ranges, i.e. ranges of functions under adversarial uncertainties. Our method is capable of computing efficiently, at a low computational cost, full n-dimensional subsets of the image of a function. As an application, we show how to compute under-approximations of robust reachable sets of non-linear controlled dynamical systems under time-varying uncertainties, which is central to many verification problems in control.

Keywords: Uncertain systems
Abstract: In this paper, we study the effect of non-vanishing disturbances on the stability of fixed-time stable (FxTS) systems. We present a new result on FxTS, which allows a positive term in the time derivative of the Lyapunov function with the aim to model bounded, non-vanishing disturbances in system dynamics. We characterize the neighborhood to which the system trajectories converge, as well as the convergence time. Then, we use the new FxTS result and formulate a quadratic program (QP) that yields control inputs which drive the trajectories of a class of nonlinear, control-affine systems to a goal set in the presence of control input constraints and non-vanishing, bounded disturbances in the system dynamics. We consider an overtaking problem on a highway as a case study, and discuss how to both set up the QP and decide when to start the overtake maneuver in the presence of sensing errors.

Keywords: Uncertain systems, Autonomous systems, Automata
Abstract: Motion planning in an unknown environment demands synthesis of an optimal control policy that balances between exploration and exploitation. In this paper, we present the environment as a labeled graph where the labels of states are initially unknown, and consider a motion planning objective to fulfill a generalized reach-avoid specification given on these labels in minimum time. By describing the record of visited labels as an automaton, we translate our problem to a Canadian traveler problem on an adapted state space. We propose a strategy that enables the agent to perform its task by exploiting possible a priori knowledge about the labels and the environment and incrementally revealing the environment online. Namely, the agent plans, follows, and replans the optimal path by assigning edge weights that balance between exploration and exploitation, given the current knowledge of the environment. We illustrate our strategy on the setting of an agent operating on a two-dimensional grid environment.

Keywords: Uncertain systems, Optimization, Robust control
Abstract: In this paper, we propose a scenario based method for robust optimization over uncertain system identification models. By considering linear regression models with deterministic regressors and assuming that the distribution of the noise acting on the system is known, we give an algorithm for drawing independent samples from the probability distribution which characterizes the uncertainty in the least squares estimate when a finite number of data points are used for parameter estimation. A number of these uncertain model parameters are drawn, and a min-max optimization problem is solved using these models. Under the assumption that the cost function is quasi-convex in the model parameters, we establish probabilistic guarantees on the value of the cost function when it is evaluated at the found scenario solution and the true system parameter. The true system parameter is regarded as a deterministic quantity, and the probabilities are with respect to the observed data from the system and the drawn scenarios. The approach and its theoretical properties are illustrated in a simulation example.

Keywords: Uncertain systems, Robust control, Electrical machine control
Abstract: The majority of currently used active disturbance rejection control (ADRC) techniques is expressed in 2DOF output-based form. For relatively high-order systems, this imposes limitations on the conventional ADRC, like the necessity of having certain higher-order terms available for controller synthesis and the lack of direct compatibility with standard industrial control software, which favors simplicity and intuitiveness of feedback error-based designs. That is why, in this work, a comparative study is conducted between the conventional 2DOF output-based ADRC and the recently proposed 1DOF error-based formulation of ADRC. Through a set of hardware experiments on a converter-fed motor testbed, the error-based ADRC is shown to retain the robustness of the 2DOF design, while having a simplified controller, being free from assumptions on availability of particular signals, and with the entire observer-based control structure being represented in a practically appealing, compact form. The obtained results also show that a major design simplification can be achieved with error-based ADRC without sacrificing the performance of the original output-based form.

Keywords: Uncertain systems, Hybrid systems, Observers for nonlinear systems
Abstract: In this paper, the design of an impulsive observer-based control, for a class of nonlinear systems with time-varying uncertainties, is proposed based on the LMI framework. The nonlinearities under consideration are assumed to satisfy local Lipschitz conditions. The observer uses sampled measurements of the system output. A time-varying Lyapunov function is used to give sufficient conditions for the existence of the impulsive observer-based control. The observer and controller gain can be obtained through feasible solutions of the LMIs proposed.The scheme presented here demonstrates that mathematically and through simulation, the proposed approach properly estimates and stabilizes all states.

Keywords: Uncertain systems, Algebraic/geometric methods, Estimation
Abstract: The problem of computing the reachable set for a given system is a quintessential question in nonlinear control theory. Motivated by prior work on safety-critical online planning, this paper considers an environment where the only available information about system dynamics is that of dynamics at a single point. Limited to such knowledge, we study the problem of describing the set of all states that are guaranteed to be reachable regardless of the unknown true dynamics. We show that such a set can be underapproximated by a reachable set of a related known system whose dynamics at every state depend on the velocity vectors that are available in all control systems consistent with the assumed knowledge. Complementing the theory, we discuss a simple model of an aircraft in distress to verify that such an underapproximation is meaningful in practice.

Keywords: Agents-based systems, Distributed control, Cooperative control
Abstract: In this paper, we study distance-based formation control of single- or double-integrator modeled agents with unknown bounded disturbances. We propose distributed control laws combining a component to eliminate the effects of the disturbances and another to achieve the desired distances in the formation. The stability of the system under the proposed control laws is analyzed based on Lyapunov stability theory of nonsmooth systems and Barbalat's lemma. Further, sufficient conditions are given to ensure the desired formation to be achieved. Finally, simulation results are given to support the analysis.

Keywords: Cooperative control, Agents-based systems
Abstract: In this paper, the formation control problem of multi-agent systems with directed communication graphs is considered. A novel distributed event-triggered approach, which involves complex Laplacian, is proposed to address the problem. The event condition depends on periodic samplings of agent states, which automatically eliminates the possibility of Zeno behavior. Then, it is shown that, under simple verifiable conditions, the proposed control strategy results in the desired formation of agents. Finally, the results are verified by numerical examples.

Keywords: Agents-based systems, Autonomous systems, Cooperative control
Abstract: In this paper a discrete time kinematic model for a swarm of agents moving in a multi-dimensional space is proposed. The model evolves using two main terms accounting for attraction of each swarm agent to a common goal position and inter-agents interactions. This last term can be tuned to obtain desired overall swarm behaviors using an interaction matrix that weighs the displacements among agents. The main model properties will be studied proving that agents aggregate in finite-time in known regions, ensuring swarm centroid convergence to a target position. The entire analysis will be performed assuming that each agent has a finite detection radius. The connections topology evolution will be studied too and it will be proved that under not restrictive assumptions on the detection range size and on the starting connections topology, the swarm reaches, in finite-time, a configuration where each agent can always detect all the other ones. Finally a laboratory experiment is developed to show the applicability of the method.

Keywords: Agents-based systems, Algebraic/geometric methods, Nonlinear systems identification
Abstract: Distance-based formation control of second-order autonomous agents can be seen as a physical system of particles linked by springs, whose evolution may be described by a Lagrangian function. In this letter we present a method based on quadratic programming to approximate the unknown Lagrangian function associated with the distance-based formation control of second-order agents from a set of sample data.

Keywords: Agents-based systems, Cooperative control, Optimal control
Abstract: In this paper, the problem of distance-based formation control over directed graphs is studied. We consider the desired distance-based formation modeled as a directed graph where each edge is assigned to only one of its incident agents to preserve the desired distance. We propose a distributed control solution based on the state-dependent Riccati equation (SDRE) method that guarantees the formation's asymptotic stability. Moreover, we developed a procedure for preventing the formation's flip ambiguity where the signed area constraints are used. We also provide an extension that guarantees collision avoidance besides preventing the flip ambiguity. Simulation results are provided to illustrate the effectiveness of the proposed methods.

Keywords: Adaptive control, Cooperative control, Networked control systems
Abstract: This paper considers the distributed consensus control problem constrained by discrete communications among neighboring agents and discrete control updating. For undi- rected graphs, we progressively propose several event-triggered protocols composed of controllers and event-triggered condi- tions. For directed graphs, we include time-varying coupling gains plus monotonically increasing functions into both the two components of the protocol, in order to avoid the requirement of global information and provide extra freedom for design. It is shown that the presented event-triggered protocols can guarantee that all agents converge to a common state and Zeno behavior is excluded. Compared to the existing papers, the main contribution of this paper is that the protocols here are able to reduce the frequency of communication and control updating, and can be used in directed graphs. Numerical examples are provided to verify the effectiveness of the proposed event- triggered protocols.

Keywords: Cooperative control, Agents-based systems, Distributed control
Abstract: This paper provides a unified solution to a formation control problem with generalized relative measurements. First, we generalize an expression of relative measurements over local frames by introducing a wide class of transformation of the frames. The transformation is provided in an arbitrary subset of a semidirect product, which enables us to describe various types of practical measurements including uncertainty. Second, we derive a strict condition on the agent configuration obtained under the class of relative measurements. Especially, the configuration is described with ambiguity caused by the measurement. Here, we design a distributed controller consisting of clique-wise and agent-wise functions, which respectively work to coordinate the agents and to remove redundant ambiguity from the configuration. As a result, the best performance is guaranteed in terms of providing the smallest ambiguity under appropriate coordination. Finally, the effectiveness of the designed controller is demonstrated through numerical examples.

Keywords: Networked control systems, Agents-based systems, Cooperative control
Abstract: This paper is concerned with a formation shaping problem for point agents in a two-dimensional space, where control avoids the possibility of reflection ambiguities. One solution for this type of problems was given first by considering a potential function which consists of both the distance error and the signed area terms for three or four agents. Then, by exploiting a hierarchical control strategy with such potential functions, the method was extended to any number of agents recently. However, a specific gain on the signed area term must be employed there, and it does not guarantee the global convergence. To overcome this issue, this paper provides a necessary and sufficient condition for the global convergence, subject to the constraint that the desired formation consists of isosceles triangles only. This clarifies the admissible range of the gain on the signed area for this case. In addition, as for formations consisting of general triangles, it is shown when high gain on the signed area is admissible for global convergence.

Keywords: Traffic control, Autonomous vehicles, Stability of nonlinear systems
Abstract: The use of macroscopic information for the controlof a vehicular platoon composed of autonomous vehicles is investigated. A mesoscopic control law is provided, and String Stability is proved by Lyapunov functions and Input-to-State Stability (ISS) concepts. Simulations are implemented in order to validate the controller and to show the efficacy of the proposed approach for mitigating traffic oscillations.

Keywords: Traffic control, Autonomous vehicles
Abstract: In this paper a coupled PDE-ODE model describing the interaction between the bulk traffic flow and a platoon of connected vehicles is adopted to develop a control action aiming at reducing the fuel consumption of the overall traffic flow. The platoon is modeled as a capacity restriction acting on the surrounding traffic. The trajectory of the initial and final points of the platoon are optimized by means of a model predictive control strategy, acting on the speeds of the front-end and back-end of the platoon, thus resulting in controlling both the speed and the length of the platoon. The approach is assessed in simulations.

Keywords: Networked control systems, Autonomous vehicles, Delay systems
Abstract: In platoons under the multiple-predecessor following (MPF) topology, the exchange of information is usually assumed to be via Vehicle-to-Vehicle (V2V) communication links. Communication delays, however, deteriorate internal and string stability and, as a consequence, to guarantee stability of the platoon the time headway between vehicles should be increased. Autonomous vehicles are nowadays equipped with a multitude of sensors, the three primary being radar, lidar and camera. The combination of such sensors allows a vehicle, among others, to detect the distance and speed of nearby objects. In this paper, we incorporate the available sensors of vehicles to anticipate the communication delays vehicles experience with their preceding vehicles, thus improving the stability conditions and subsequently reducing the time headway. More specifically, by incorporating sensors we alleviate the communication delays between neighboring vehicles. We demonstrate that emph{(i)} the system is internally stable irrespective of the size of the communication delays, and emph{(ii)} the time headway can be reduced, by deriving a sufficient condition which provides a lower bound on the time headway that guarantees string stability. With this new lower bound on the time headway, platoons do not need to massively increase the time headway in order to compensate for the effects of communications delays. Simulations demonstrate the effectiveness of the proposed lower bound.

Keywords: Automotive control, Automotive systems, Autonomous vehicles
Abstract: Traffic congestion is one of the major issues for urban traffic networks. The connected and autonomous vehicles (CAV) is an emerging technology that has the potential to address this issue by improving safety, efficiency, and capacity of the transportation system. In this paper, the problem of optimal trajectory planning of battery-electric CAVs in the context of cooperative crossing of an unsignalized intersection is addressed. An optimization-based centralized intersection controller is proposed to find the optimal velocity trajectory of each vehicle so as to minimize electric energy consumption and traffic throughput. Solving the underlying optimization problem for a group of CAVs is not straightforward because of the nonlinear and nonconvex dynamics, especially when the powertrain model is explicitly modeled. In order to ensure a rapid solution search and a unique global optimum, the optimal control problem (OCP) is reformulated via convex modeling techniques. Several simulation case studies are presented to show the effectiveness of the proposed approach and the trade-off between energy consumption and traffic throughput is illustrated.

Keywords: Agents-based systems, Automotive control, Decentralized control
Abstract: This paper presents the design of an integral controller for vehicle platoons with actuator dynamics. The proposed controller ensures string stability with disturbances and simultaneously compensates for constant disturbances through integral action. Sufficient conditions for string stability are satisfied by the use of a suitable state transformation. The proposed controller guarantees disturbance string stability for a prescribed time constant of the actuator dynamics, and we show through simulation that platoons with faster dynamics are also made disturbance string stable.

Keywords: Autonomous vehicles, Automotive control, Feedback linearization
Abstract: In this paper, the problem of coordinated lateral and longitudinal vehicle-following control for connected and automated vehicles (CAVs) is investigated by considering nonlinear dynamics. Based on a nonlinear model of longitudinal and lateral vehicle dynamics and vehicle-to-vehicle (V2V) communication, we propose a triple-step nonlinear control law for a CAV to follow the trajectory of preceding vehicle. The stability regarding the time gap spacing and velocity errors is discussed via Lyapunov theory. Simulation results illustrate the effectiveness of the proposed control for curved lanes.

Keywords: Automotive control, Predictive control for nonlinear systems, Optimization
Abstract: Vehicle platooning has been shown to be quite fruitful in the transportation industry to enhance fuel economy, road throughput, and driving comfort. Model Predictive Control (MPC) is widely used in literature for platoon control to achieve certain objectives, such as safely reducing the distance among consecutive vehicles while following the leader vehicle. In this paper, we propose a Distributed Nonlinear MPC (DNMPC), based upon an existing approach, to control a heterogeneous dynamic platoon with unidirectional topologies, handling possible cut-in/cut-out maneuvers. The introduced method guarantees a collision-free driving experience while tracking the desired speed profile and maintaining a safe desired gap among the vehicles. The time of convergence in the dynamic platooning is derived based on the time of cut-in and/or cut-out maneuvers. In addition, we analyze the level of improvement of driving comfort, fuel economy, and absolute and relative convergence of the method by using distributed metric learning and distributed optimization with Alternating Direction Method of Multipliers (ADMM). Simulation results on a dynamic platoon with cut-in and cut-out maneuvers and with different unidirectional topologies show the effectiveness of the introduced method.

Keywords: Autonomous vehicles, Delay systems, Cooperative control
Abstract: The platoon control problem is considered under a recently proposed leader and predecessor following scheme with a third-order vehicle dynamics. The scheme is investigated with a simple PD-type controller in the face of delayed leader information. It is shown that string stability can be achieved only when the headway parameter is chosen larger than the communication delay. The use of larger headway parameters is also shown to be necessary to avoid the amplification of the acceleration signals backward along the platoon. Guidelines are given for the choice of controller gains and the headway parameter in a way to achieve desirable platoon behavior.

Keywords: Estimation, Algebraic/geometric methods, Numerical algorithms
Abstract: In this paper, an effective algorithm for selecting the design parameters in the modulating functions-based method (MFBM) is introduced. The appropriate selection of these parameters improves the performance of the estimation approach and reduces the computational cost. The effectiveness and robustness of the algorithm are shown through numerical simulations in a noisy environment.

Keywords: Estimation, Statistical learning, Computational methods
Abstract: The problem of estimating the largest gain of an unknown linear and time-invariant filter is studied, known as the H∞-norm estimation problem. The approach presented in this paper is iterative and corresponds to the combination of two state-of-the-art methods: Power Iterations and Weighted Thompson Sampling. The combination is done by means of a well-known technique in communications known as a cyclic prefix, in which the last points of a signal are prepended to it. This allows to take considerably more exact measurements of the frequency response of the system at a set of equispaced frequencies. The discussion is complemented with a simulation study, showing that the proposed algorithm has an increased speed of convergence to the quantity of interest.

Keywords: Estimation, Nonlinear systems identification, Statistical learning
Abstract: In this paper, we propose an estimation method for heteroscedastic multilinear systems. The system consists of a multilinear map of latent functions and an input-dependent noise process. We assume Gaussian-process priors on the unknowns to embed non-parametric models. This leads to a hierarchical model called heteroscedastic multilinear Gaussian processes which do not admit closed-form posterior and predictive distributions. The model is treated in an empirical Bayes fashion where the hyperparameters are estimated by maximizing the marginal distribution. To achieve that, we use a Monte Carlo expectation maximization method based on a Gibbs sampling algorithm. The predictive inference is also introduced where the mean is used as an approximation of the unknown functions. The performance of the proposed method is illustrated in a simulation study.

Keywords: Estimation, Observers for nonlinear systems
Abstract: A state estimation scheme for a class of Persidskii systems is introduced in this paper. Two distinct sets of conditions are formulated for robust stability and convergence of the state estimation error using the theories of input-to-output stability (IOS) and input-to-state stability (ISS). These conditions for the nonlinear error dynamics are established in the form of linear matrix inequalities. Two numerical examples are presented to illustrate the effectiveness of the proposed results.

Keywords: Estimation, Control Systems Privacy, Optimization
Abstract: In this paper, we study differentially private parameter estimation in which a data acquisitor (DA) accesses data (or signals) from multiple privacy-aware data owners (DOs) to estimate some random parameters. To ensure differential privacy, the DOs add Laplace noises to their private signals and only reveal the noisy signals to the DA. Our goal is to add optimal amount of noises (measured by their inverse variances) so that the mean squared error (MSE) of the DA's estimate is minimized while differential privacy is satisfied. In the general case, the optimal private estimation can be formulated as a semidefinite program (SDP), which can be readily solved by off-the-shelf optimization methods. In the special case where different DOs have uncorrelated signals, the optimization problem is decomposed across DOs and can be solved very efficiently in almost closed-form. We observe that, in the optimal solution, the DOs should add more noises to the signal entries that are less useful for estimation. Further, when the DA has DO selection constraint (e.g., due to limited budget), a relaxed SDP is put forth to compute a suboptimal solution with superb empirical performance. Finally, several numerical examples are presented.

Keywords: Estimation, Switched systems, Markov processes
Abstract: Computing the optimal joint maximum a posteriori probability (JMAP) estimate of the state and mode of a Markov jump linear system (MJLS) is known to be a computationally intractable problem. This letter provides a novel approximate method for such a problem that guarantees to be within a pre-specified bound of the optimal estimate. The proposed method builds upon relaxed dynamic programming. Through numerical examples, we show that this method can lead to better estimates with less computations than previous suboptimal methods proposed in the literature.

Keywords: Estimation, Uncertain systems, LMIs
Abstract: This paper investigates the problem of state estimation for linear-time-invariant (LTI) discrete-time systems subject to structured feedback uncertainty and bounded disturbances. The proposed Robust Moving Horizon Estimation (RMHE) scheme computes at each sample time tight bounds on the uncertain states by solving a linear matrix inequality (LMI) optimization problem based on the available noisy input and output data. In comparison with conventional approaches that use offline calculation for the estimation, the suggested scheme achieves an acceptable level of performance with reduced conservativeness, while the online computational time is maintained relatively low. The effectiveness of the proposed estimation method is assessed via a numerical example.

Keywords: Estimation
Abstract: The derivation of tight estimation lower bounds is a key player to design and assess the performance of new estimators. In this contribution, we derive a new compact Cramér-Rao bound (CRB) for the conditional signal model, where the deterministic parameter's vector includes a real positive amplitude and the signal phase. Then, such CRB is particularized to the delay, Doppler, phase and amplitude estimation with band-limited (narrowband) signals, where transmitter and receiver are in relative uniform radial movement. The latter expression is especially easy to use because it only depends on the signal samples. We provide illustrative results for a representative Global Navigation Satellite System positioning example.

Keywords: Predictive control for linear systems, Stochastic systems, Kalman filtering
Abstract: The literature dealing with data-driven analysis and control problems has significantly grown in the recent years. Most of the literature deals with linear time-invariant (LTI) systems in which the uncertainty (if any) is assumed to be deterministic and bounded; relatively little attention has been devoted to stochastic LTI systems. As a first step in this direction, we propose to equip the recently introduced Data-enabled Predictive Control (DeePC) algorithm with a data-based Extended Kalman Filter (EKF) to make use of additional available input-output data for reducing the effect of noise, without increasing the computational load of the optimization procedure.

Keywords: Filtering, Estimation, Kalman filtering
Abstract: The extended and unscented Kalman multiple distribution estimation filters (EKMDEF/UKMDEF) were recently proposed for nonlinear continuous-discrete state-space models and their superior filtering accuracy was shown in the simulation of satellite reentry. The EKMDEF/UKMDEF is based on the multiple distribution estimation (MDE) for the filtered state probability density function (PDF) and this paper provides its alternate derivation with a comparison of the Gaussian sum filter (GSF). This result sheds light on the relationship between the EKMDEF/UKMDEF and the GSF and a new filter that is more stable than the EKMDEF/UKMDEF can be designed. The performance of the proposed filter is examined for nonlinear discrete state-space models using benchmark simulation problems and is compared with those of the representative filters including particle filters (PFs).

Keywords: Kalman filtering, Filtering
Abstract: Standard state estimation techniques, ranging from the linear Kalman filter to nonlinear sigma-point or particle filters, assume a perfectly known system model, that is, process and measurement functions and system noise statistics (both the distribution and its parameters). This is a strong assumption which may not hold in practice, reason why several approaches have been proposed for robust filtering. In the context of linear filtering, a solution to cope with a possible system matrices mismatch is to use linear constraints. In this contribution we further explore the extension and use of recent results on linearly constrained Kalman filtering (LCKF) for robust tracking/localization under measurement model mismatch. We first derive the natural extension of the LCKF to nonlinear systems, and its use to mitigate parametric modelling errors in the nonlinear measurement function. A tracking problem where a set of sensors at possibly mismatched (unknown to a certain extent) positions track a moving object from time of arrival measurements is used to support the discussion.

Keywords: Kalman filtering, Filtering, Uncertain systems
Abstract: We consider a robust filtering problem where the nominal state space model is not reachable and different from the actual one. We propose a robust Kalman filter which solves a dynamic game: one player selects the least-favorable model in a given ambiguity set, while the other player designs the optimum filter for the least-favorable model. It turns out that the robust filter is governed by a low-rank risk sensitive-like Riccati equation. Finally, simulation results show the effectiveness of the proposed filter.

Keywords: Kalman filtering, Uncertain systems, Estimation
Abstract: In this paper, we propose a robust Kalman filtering framework for systems with probabilistic uncertainty in system parameters. We consider two cases, namely discrete time systems, and continuous time systems with discrete measurements. The uncertainty, characterized by mean and variance of the states, is propagated using conditional expectations and polynomial chaos expansion framework. The results obtained using the proposed filter are compared with existing robust filters in the literature. The proposed filter demonstrates better performance in terms of estimation error and rate of convergence.

Keywords: Kalman filtering, Estimation, Computational methods
Abstract: We exploit knowledge of linear substructure in the linear-regression Kalman filters (LRKFs) to simplify the problem of moment matching. The theoretical results yield quantifiable and significant computational speedups at no cost of estimation accuracy, assuming partially linear estimation models. The results apply to any symmetrical LRKF, and reductions in computational complexity are stated as a function of the cubature rule, the number of linear and nonlinear states in the estimation model respectively. The implications for the filtering problem are illustrated by several numerical examples.

Keywords: Kalman filtering
Abstract: In this paper we present a novel approach to continuous-discrete (CD) Kalman filtering. Unlike the EKF or other hybrid UKF-EKF filters, the novel approach does not require direct calculation of system Jacobians and instead uses unscented transforms (UTs) to extract a pair of matrices, each made up of a linear combination of derivatives (with respect to the state), that are used in its place. More specifically, they are used (1) in parts of the filtering process, and (2) in the linearly implicit numerical integration scheme of the filter’s state propagation stage. Extracting these matrices from the filter’s UTs and using them in both the filtering process and model simulation, or what we refer to as coupling the filter to model simulation, avoids having to calculate further function evaluations for standard implicit methods, making the process of state estimation for stiff systems more efficient. Another benefit of the proposed approach is that it offers UKF accuracy levels but improved numerical stability over the UKF. This is because it uses symmetric and positive-definite(PD) representations of the state covariance propagation and measurement update equations.

Keywords: Kalman filtering, Filtering, Aerospace
Abstract: This paper deals with the problem of tracking a maneuvering target using an airborne electronically scanned array radar. Conventional target-tracking filters produce biased target state estimates because the angular motion of a radar platform complicates the cross-correlations between the converted measurements (CMs). As a practical alternative to this issue, the target tracking filter is designed in the predicted line-of-sight Cartesian coordinate system (PLCCS). Provided that the {it a priori} state estimation error is small enough, the statistics of the CMs in PLCCS can be approximated accurately despite using the first and second moments of the radar measurement noises only. The resulting debiasing term leads to a satisfactory target tracking performance of the proposed filter. Furthermore, by exploiting the diagonality of the noise covariance matrix derived in the PLCCS, the proposed tracking filter is decoupled into each axis to induce a suitable real-time implementation. Through simulations, the enhanced performance of the suggested method is demonstrated.

Keywords: Optimization algorithms, Agents-based systems, Control over communications
Abstract: We propose a distributed method to solve a multi-agent optimization problem with strongly convex cost function and equality coupling constraints. The method is based on Nesterov's accelerated gradient approach and works over stochastically time-varying communication networks. We consider the standard assumptions of Nesterov's method and show that the sequence of the expected dual values converge toward the optimal value with the rate of O(1/k²). Furthermore, we provide a simulation study of solving an optimal power flow problem with a well-known benchmark case.

Keywords: Optimization algorithms, Control of networks, Decentralized control
Abstract: We consider a distributed saddle point problem, in which a collection of nodes collaboratively optimize a sum of local component functions through local computations and information exchange with neighbouring nodes. To solve this problem, we propose a decentralized algorithm based on the Extragradient method, whose centralized implementation has been shown to achieve good performance on a wide range of minmax problems. We show that our proposed method achieves linear convergence under suitable assumptions and explicitly characterize how the convergence rate depends on the condition number and the spectral gap of the communication graph. We also present numerical simulations that corroborate our theoretical results.

Keywords: Optimization algorithms, Distributed control, Network analysis and control
Abstract: Some distributed optimization applications require privacy, meaning that the values of certain parameters local to a node should not be revealed to other nodes in the network during the joint optimization process. A special case is the problem of private distributed averaging, in which a network of nodes computes the global average of individual node reference values in a distributed manner while preserving the privacy of each reference. We present simple iterative methods that guarantee accuracy (i.e., the exact asymptotic computation of the global average) and privacy (i.e., no node can estimate another node's reference value). To achieve this, we require that the digraph modeling the communication between nodes satisfy certain topological conditions. Our method is hot-pluggable (meaning no reinitialization of the averaging process is required when the network changes or a node enters or leaves, when there is a communication or computation fault, or when a node's reference value changes); it does not require an initial scrambling phase; it does not inject noise or other masking signals into the distributed computation; it does not require random switching of edge weights; and it does not rely on homomorphic encryption.

Keywords: Optimization algorithms, Distributed control, Control of networks
Abstract: We study the problem of maximizing a submodular function, subject to a cardinality constraint, with a set of agents communicating over a connected graph. We propose a distributed greedy algorithm that allows all the agents to converge to a near-optimal solution to the global maximization problem using only local information and communication with neighbors in the graph. The near-optimal solution approaches the (1-1/e) approximation of the optimal solution to the global maximization problem with an additive factor that depends on the number of communication steps in the algorithm. We then analyze convergence guarantees of the proposed algorithm. This analysis reveals a tradeoff between the number of communication steps and the performance of the algorithm. Finally, we extend our analysis to nonsubmodular settings, using the notion of approximate submodularity.

Keywords: Optimization algorithms, Distributed control, Large-scale systems
Abstract: Trajectory optimization and model predictive control is demanding but challenging for distributed and time-critical system that consists of a large number of dynamic subsystems with sparse physical interactions. The classic dual gradient ascent method suffers from the slow convergence when the system is not well-scaled. This paper proposes a Jacobi-preconditioned dual gradient ascent method that fully exploits the idea behind Differential Dynamic Programming to compute the ascent direction in a distributed and recurrent manner at a linear time-cost with respect to the length of the time horizon. Moreover, we propose a method to compute a fixed step size for the preconditioned dual gradient ascent step that can guarantee global convergence property under certain assumptions. A numerical experiment shows that our proposed algorithm improves performance and robustness to ill-scaled problems over the ordinary non-preconditioned dual ascent algorithm. This algorithm has great potential applications in power grid, chemical plants, and cooperative systems of drones.

Keywords: Optimization algorithms, Machine learning, Distributed control
Abstract: We focus on the problem of minimizing a finite sum f(x) = sum_{i=1}^n f_i(x) of functions of n functions f_i, where f_i are convex and available only locally to an agent i. The n agents are connected in a directed network mathcal{G}(mathbf{V},mathbf{E}), where each agent i can only communicate with agents in its neighborhood determined by mathcal{G}(V,E). In this article, we present the Directed-Distributed Alternating Direction Method of Multiplier (D-DistADMM) Algorithm, which is an Alternating Direction Method of Multiplier (ADMM) based scheme and utilizes a finite-time ``approximate'' consensus method to solve the above optimization problem distributively. At each iteration of the proposed scheme the agents solve their local optimization problem and utilize an approximate consensus protocol to update a local estimate of the global optimization variable. We show that for convex and not-necessarily differentiable objective functions f_i's the proposed textit{D-DistADMM} method converges at a rate O(1/k). We further demonstrate the applicability of our algorithm by solving a distributed least-squares problem.

Keywords: Optimization algorithms, Distributed control, Agents-based systems
Abstract: We address constraint-coupled optimization for a system composed of multiple cooperative agents communicating over a time-varying network. We propose a distributed proximal minimization algorithm that is guaranteed to converge to an optimal solution of the optimization problem, under suitable convexity and connectivity assumptions. The performance of the introduced algorithm is shown on a numerical example of a charging scheduling problem for a fleet of plug-in electric vehicles.

Keywords: Optimization algorithms, Large-scale systems, Control of networks
Abstract: We present an overlapping Schwarz decomposition algorithm for constrained quadratic programs (QPs). Schwarz algorithms have been traditionally used to solve linear algebra systems arising from partial differential equations, but we have recently shown that they are also effective at solving structured optimization problems. In the proposed scheme, we consider QPs whose algebraic structure can be represented by graphs. The graph domain is partitioned into overlapping subdomains (yielding a set of coupled subproblems), solutions for the subproblems are computed in parallel, and convergence is enforced by updating primal-dual information in the overlapping regions. We show that convergence is guaranteed if the overlap is sufficiently large and that the convergence rate improves exponentially with the size of the overlap. Convergence results rely on a key property of graph-structured problems that is known as exponential decay of sensitivity. Here, we establish conditions under which this property holds for constrained QPs (as those found in network optimization and optimal control), thus extending existing work that addresses unconstrained QPs. The numerical behavior of the Schwarz scheme is demonstrated by using a DC optimal power flow problem defined over a network with 9,241 nodes.

Keywords: Stochastic optimal control, Optimization, Discrete event systems
Abstract: For years, there has been interest in approximation methods for solving dynamic programming problems, because of the inherent complexity in computing optimal solutions characterized by Bellman’s principle of optimality. A wide range of approximate dynamic programming (ADP) methods now exists. It is of great interest to guarantee that the performance of an ADP scheme be at least some known fraction, say , of optimal. This paper introduces a general approach to bounding the performance of ADP methods, in this sense, in the stochastic setting. The approach is based on new results for bounding greedy solutions in string optimization problems, where one has to choose a string (ordered set) of actions to maximize an objective function. This bounding technique is inspired by submodularity theory, but submodularity is not required for establishing bounds. Instead, the bounding is based on quantifying certain notions of curvature of string functions; the smaller the curvatures the better the bound. The key insight is that any ADP scheme is a greedy scheme for some surrogate string objective function that coincides in its optimal solution and value with those of the original optimal control problem. The ADP scheme then yields to the bounding technique mentioned above, and the curvatures of the surrogate objective determine the value  of the bound. The surrogate and its curvatures depend on the specific ADP.

Keywords: Stochastic optimal control, Robust adaptive control, Machine learning
Abstract: This study presents a robust dual control method for batch processes under parametric uncertainty. Proximal policy optimization (PPO), a policy gradient reinforcement learning algorithm, is employed to construct an implicit dual controller in a computationally amenable way. The proposed control method can robustly and actively cope with uncertainties seen in a repeated sequence of batch operations by incorporating a penalty term for constraint violation into the reward function and by considering the effect of control inputs on future uncertainty. An application to a bioethanol fermentation process is discussed to demonstrate the effectiveness of the proposed control strategy. It is shown that the proposed robust dual controller has an active learning feature such that the overall performance improves compared to a certainty-equivalence based approach.

Keywords: Stochastic optimal control, Transportation networks, Computational methods
Abstract: We consider the optimal control problem of steering a collection of agents over a network. The group behavior of an ensemble is often modeled by a distribution, and thus the optimal control problem we study can be cast as a distribution steering problem. While most existing works for steering distributions require the agents in the ensemble to be indistinguishable, we consider the setting where agents have specified origin-destination constraints. This control problem also resembles a minimum cost network flow problem with a massive number of commodities. We propose a novel optimal transport based framework for this problem and derive an efficient algorithm for solving it. This framework extends multi-marginal optimal transport theory to settings with capacity and origin-destination constraints. The proposed method is illustrated on a numerical simulation for traffic planning.

Keywords: Stochastic optimal control, Stochastic systems, Game theory
Abstract: We study the linear-quadratic zero-sum stochastic differential game with random coefficients, where the coefficients of the stochastic differential equation (SDE) are random processes and both additive and state multiplicative noise are included in the diffusion term of the corresponding SDE. By applying It^o-Kunita's formula to the quadratic random field, we develop a direct approach, also known as the completion of squares method, to characterize the explicit (feedback) Nash equilibrium and obtain the optimal game value. We show that the corresponding Nash equilibrium is linear in state characterized in terms of the stochastic Riccati differential equation and the linear backward SDE. We also verify the optimality of the Nash equilibrium by characterizing the smooth solution of the stochastic Hamilton-Jacobi-Isaacs equation that is the second-order stochastic partial differential equation obtained from dynamic programming.

Keywords: Stochastic optimal control, Stochastic systems, Optimal control
Abstract: The context of the present paper is stochastic thermodynamics–an approach to nonequilibrium thermodynamics rooted within the broader framework of stochastic control. In contrast to the classical paradigm of Carnot engines, we herein propose to consider thermodynamic processes with periodic continuously varying temperature of a heat bath and study questions of maximal power and efficiency for two idealized cases, overdamped (first-order) and underdamped (secondorder) stochastic models. We highlight properties of optimal periodic control, derive and numerically validate approximate formulae for the optimal performance (power and efficiency)

Keywords: Stochastic optimal control, Stochastic systems, Robust control
Abstract: We propose a new risk-constrained reformulation of the standard Linear Quadratic Regulator (LQR) problem. Our framework is motivated by the fact that the classical (risk-neutral) LQR controller, although optimal in expectation, might be ineffective under relatively infrequent, yet statistically significant events. To effectively trade between average and extreme event performance, we introduce a new risk constraint, which explicitly restricts the total expected predictive variance of the state penalty by a user-prescribed level. We show that, under rather minimal conditions on the process noise~(i.e., finite fourth-order moments), the optimal risk-aware controller can be evaluated explicitly and in closed form. In fact, it is affine relative to the state, and is always internally stable regardless of parameter tuning. Our new risk-aware controller: i) pushes the state away from directions where the noise exhibits heavy tails, by exploiting the third moment~(skewness) of the noise; ii) inflates the state penalty in riskier directions, where both the noise covariance and the state penalty are simultaneously large. The properties of the proposed risk-aware LQR framework are also illustrated via indicative numerical examples.

Keywords: Stochastic optimal control, Stochastic systems, Uncertain systems
Abstract: We consider the stochastic optimal control problem with random coefficients under recursive-type objective functionals captured by backward stochastic differential equations (with random coefficients). The associated Hamilton-Jacobi-Bellman (HJB) equation obtained from the dynamic programming principle is a second-order nonlinear stochastic HJB (SHJB) equation (or stochastic PDE). The solvability of the SHJB equation, together with It^o-Kunita's formula, leads to the verification theorem that is the sufficient condition for optimality. We also show the existence and uniqueness of the (weak) solution to the SHJB equation via the Sobolev space technique. As an application, the linear-quadratic control problem is considered, for which we obtain an explicit optimal solution by applying the verification theorem.

Keywords: Stochastic optimal control, Stability of nonlinear systems, Large-scale systems
Abstract: Control of continuous time dynamics with multiplicative noise is a classic topic in stochastic optimal control. This work addresses the problem of designing infinite horizon optimal controls with stability guarantees for large populations of identical, non-cooperative and non-networked agents with multi-dimensional and nonlinear stochastic dynamics excited by multiplicative noise. We provide constraints on the state and control cost functions which guarantee stability of the closed-loop system under the action of the individual optimal controls, for agent dynamics belonging to the the class of reversible diffusion processes. A condition relating the state-dependent control cost and volatility is introduced to prove the stability of the equilibrium density. This condition is a special case of the constraint required to use the path integral Feynman-Kac formula for computing the control. We investigate the connection between the stabilizing optimal control and the path integral formalism, leading us to a control law formulation expressed exclusively in terms of the desired equilibrium density.

Keywords: Power systems, Decentralized control, Large-scale systems
Abstract: We propose a design procedure for a power-voltage droop controller in structure-preserving DC microgrids under explicit consideration of the electrical network dynamics. Differently from most related literature, the system’s controlled output is taken as the power—not the current—injection at each generation unit, yielding a nonlinear closed-loop system. This makes the output regulation problem non-trivial, yet far more appealing in a practical setting than the usual linear current-voltage droop control. Our approach is inspired by passivity-based control design in the sense that we exploit the natural port-Hamiltonian representation of the system dynamics and its associated shifted Hamiltonian to derive a control law together with sufficient conditions on the tuning gains that guarantee global asymptotic stability. The analysis is illustrated via detailed simulations, where accurate power sharing is manifested among the distributed generation units in the presence of load variations.

Keywords: Sensor networks, Sensor fusion, Decentralized control
Abstract: This paper presents a decentralized Gaussian Process (GP) learning, fusion, and planning (RESIN) formalism for mobile sensor networks to actively learn target motion models. RESIN is characterized by both computational and communication efficiency, and the robustness to rumor propagation in sensor networks. By using the weighted exponential product rule and the Chernoff information, a rumor-robust decentralized GP fusion approach is developed to generate a globally consistent target trajectory prediction from local GP models. A decentralized information-driven path planning approach is then proposed for mobile sensors to generate informative sensing paths. A novel, constant-sized information sharing strategy is developed for sensing path coordination,and an analytical objective function is derived that significantly reduces the computational complexity of the path planning. The effectiveness of RESIN is demonstrated in simulations.

Keywords: Switched systems, Stochastic systems, Large-scale systems
Abstract: In this paper, we propose a compositional framework for the construction of control barrier certificates for large-scale stochastic switched systems accepting multiple control barrier certificates with some dwell-time conditions. The proposed scheme is based on a notion of so-called augmented pseudo-barrier certificates computed for each switched subsystem, using which one can compositionally synthesize state-feedback controllers for interconnected systems enforcing safety specifications over a finite-time horizon. In particular, we first leverage sufficient max-type small-gain conditions to compositionally construct augmented control barrier certificates for interconnected systems. We the quantify upper bounds on exit probabilities - the probability that an interconnected system reaches certain unsafe regions - in a finite-time horizon. We employ a technique based on a counter-example guided inductive synthesis (CEGIS) approach to search for control barrier certificates of each mode while synthesizing safety controllers providing switching signals. We demonstrate our proposed results by applying them to two different case studies.

Keywords: Cooperative control, Agents-based systems
Abstract: This paper studies a distributed multi-armed bandit problem over a network of N agents, each of which can communicate its local observations only with its neighbors, where neighbor relationships are described by a connected graph mathbb G. Each agent makes a sequence of decisions on selecting an arm from M candidates, yet it only have access to local samples of the reward for each action, which is a random variable. A distributed upper confidence bound (UCB) algorithm is proposed for the agents to cooperatively learn the best decision. It is shown that when all the agents share a homogeneous distribution of each arm reward, the algorithm achieves guaranteed weak regret at O([(1+(N-1)rho_{2})^{2}log T]/N) for all N agents, where rho_2 denotes the second largest among the absolute value of all the eigenvalues of the Metropolis matrix of bbb G. A sufficient condition under which the proposed distributed algorithm learns faster than the centralized (single-agent) counterpart is provided.

Keywords: Control of networks, Decentralized control, Large-scale systems
Abstract: This paper studies the distribution of param- eter values in large-scale interconnected systems. Using γ- robustness, a bound on the effect of external disturbances acting on subsystems can be found. However, due to the interconnected nature of the system, the same bound can be reached with different choices of distributed control parameters leading to different local control costs. A notion of fairness and a measurement of unfairness is presented to describe and measure the effects of unevenly distributed parameters in the system. The distribution of bounds on state deviations due to external disturbances is studied and conditions and algorithms are given to ensure such bounds while fairly distributing the required control efforts. Decentralized algorithms to reach fairness or reduce unfairness are presented for various assumptions on systems and constraints. The results are illustrated by examples.

Keywords: Nonlinear output feedback, Robust control, Lyapunov methods
Abstract: In this letter, we consider the problem of regulating the voltage of an islanded Direct Current (DC) network subject to (i) unknown ZIP-loads, i.e., nonlinear loads with the parallel combination of constant impedance (Z), current (I) and power (P) components, and (ii) unknown time-varying disturbances. Using the port-Hamiltonian framework, two decentralized passivity-based control schemes are designed. It is shown that, using the proposed controllers, the desired equilibrium is exponentially stable and local input-to-state stable (LISS) with respect to unknown time-varying disturbances.

Keywords: Networked control systems, Optimization algorithms, LMIs
Abstract: We consider the distributed optimization problem, where a group of agents work together to optimize a common objective by communicating with neighboring agents and performing local computations. For a given algorithm, we use tools from robust control to systematically analyze the performance in the case where the communication network is time-varying. In particular, we assume only that the network is jointly connected over a finite time horizon (commonly referred to as B-connectivity), which does not require connectivity at each time instant. When applied to the distributed algorithm DIGing, our bounds are orders of magnitude tighter than those available in the literature.

Keywords: Network analysis and control, Large-scale systems, Lyapunov methods
Abstract: This paper provides a small-gain theorem for a so-called infinite network, i.e. a network composed of infinitely many finite-dimensional systems. Such a network is mainly motivated by addressing the scalability issue in large-but-finite networks. We develop a so-called relaxed small-gain theorem for input-to-state stability (ISS) with respect to a closed set. It is shown that every exponentially ISS network necessarily satisfies the proposed small-gain condition. Finally, we truncate the infinite network to obtain a large-but-finite network for all stability properties and performance indices obtained for its infinite counterpart are preserved, if each subsystem is individually ISS. The effectiveness of our small-gain theorem is verified by application to an urban traffic network.

Keywords: Linear systems, Networked control systems, Control Systems Privacy
Abstract: This paper studies initial-value privacy problems of linear dynamical systems. We consider a standard linear time-invariant system with random process and measurement noises. For such a system, eavesdroppers having access to system output trajectories may infer the system initial states, leading to initial-value privacy risks. When a finite number of output trajectories are eavesdropped, we consider a requirement that any guess about the initial values can be plausibly denied. When an infinite number of output trajectories are eavesdropped, we consider a requirement that the initial values should not be uniquely recoverable. In view of these two privacy requirements, we define differential initial-value privacy and intrinsic initial-value privacy, respectively, for the system as metrics of privacy risks. First of all, we prove that the intrinsic initial-value privacy is equivalent to unobservability, while the differential initial-value privacy can be achieved for a privacy budget depending on an extended observability matrix of the system and the covariance of the noises. Next, the inherent network nature of the considered linear system is explored, where each individual state corresponds to a node and the state and output matrices induce interaction and sensing graphs, leading to a network system. Under this network system perspective, we allow the initial states at some nodes to be public, and investigate the resulting intrinsic initial-value privacy of each individual node. We establish necessary and sufficient conditions for such individual node initial-value privacy, and also prove that the intrinsic initial-value privacy of individual nodes is generically determined by the network structure.

Keywords: Networked control systems, Linear systems, Distributed control
Abstract: This paper presents three interesting phenomena that occur in the synchronisation of disturbed systems. The first phenomenon describes the necessity that dynamical local controllers have to be used to introduce the model of the disturbances into each of the extended agents. This result extends the Internal-Model Principle from multivariable control to multi-agent systems. A necessary and sufficient synchronisation condition is given. The second phenomenon concerns the fact that the internal disturbance models of all agents modify the synchronous trajectory of the overall system. For specific initial states the synchronous trajectory is completely fixed by the disturbance models and does not reflect the properties of the agents. This fact leads to the third phenomenon that indicates that an undisturbed virtual agent has to be introduced into the overall system to ensure that the synchronous trajectory is generated by the agent dynamics like in undisturbed multi-agent systems. The considered phenomena are illustrated by numerical examples.

Keywords: Networked control systems, Constrained control
Abstract: This paper focuses on an optimal control design problem for a class of perturbed networked control systems where a number of systems, subject to state and input constraints, share a communication network with limited bandwidth. We first formulate an optimal control design problem with a constant feedback gain in order to minimize the communication demand for each system while guaranteeing satisfaction of state and input constraints; we show that this optimization problem is very hard to solve. Then, we formulate the same optimal control design problem with a non-constant feedback gain; we argue that this problem is less difficult and results in a lower, or equal, communication demand in comparison to the design with the constant feedback gain. We illustrate and compare these optimal control designs by a simple example.

Keywords: Networked control systems, Network analysis and control, Fault tolerant systems
Abstract: In this paper, we study the relationship between resilience and accuracy in the resilient distributed multi-dimensional consensus problem. We consider a network of agents, each of which has a state in R^d. Some agents in the network are adversarial and can change their states arbitrarily. The normal (non-adversarial) agents interact locally and update their states to achieve consensus at some point in the convex hull C of their initial states. This objective is achievable if the number of adversaries in the neighborhood of normal agents is less than a specific value, which is a function of the local connectivity and the state dimension d. However, to be resilient against adversaries, especially in the case of large d, the desired local connectivity is large. We discuss that resilience against adversarial agents can be improved if normal agents are allowed to converge in a bounded region B, which means normal agents converge at some point close to but not necessarily inside C in the worst case. The accuracy of resilient consensus can be measured by the Hausdorff distance between B and C. As a result, resilience can be improved at the cost of accuracy. We propose a resilient bounded consensus algorithm that exploits the trade-off between resilience and accuracy by projecting d-dimensional states into lower dimensions and then solving instances of resilient consensus in lower dimensions. We analyze the algorithm, present various resilience and accuracy bounds, and also numerically evaluate our results.

Keywords: Networked control systems, Network analysis and control, Optimization
Abstract: This paper investigates the problem of false data injection attack on the communication channels in a multiagent system executing a consensus protocol. We formulate a non-convex optimisation problem for an attack strategy with minimal one-step attack energy to guarantee instability of the consensus system. We propose an algorithm to solve the problem efficiently. Numerical simulations are provided to illustrate the effectiveness of the attack strategy.

Keywords: Networked control systems, Network analysis and control, Distributed control
Abstract: This work considers the distributed computation of the one-to-one vertex correspondences between two undirected and connected graphs, which is called graph matching, over multi-agent networks. Given two isomorphic and asymmetric graphs, there is a unique permutation matrix that maps the vertices in one graph to the vertices in the other. Based on a convex relaxation of graph matching in Aflalo et al. [1], we propose a distributed computation of graph matching as a distributed convex optimization problem subject to equality constraints and a global set constraint, using a network of multiple agents whose interaction graph is connected. Each agent in the network only knows one column of each of the adjacency matrices of the two graphs, and all agents collaboratively learn the graph matching by exchanging information with their neighbors. The proposed algorithm employs a projected primaldual gradient method to handle equality constraints and a set constraint. Under the proposed algorithm, the agents’ estimates of the permutation matrix converge to the optimal permutation globally and exponentially fast. Finally, simulation results are given to illustrate the effectiveness of the method.

Keywords: Networked control systems, Predictive control for linear systems, Control over communications
Abstract: Rollout control is an MPC-based control method, in which a controller is periodically activated to schedule the transmission of sensor or actuator data. Therein, a preassigned traffic specification acts as a constraint on the scheduled transmissions, ensuring that they are triggered at an admissible rate for the underlying communication network. In this paper, we extend the theory of rollout control by considering bounded disturbances on the controlled plant in the presence of state and input constraints. We use methods from tube MPC, where the error between the nominal (undisturbed) system, used as a prediction model, and the real system is kept in a robust control invariant set. This approach requires satisfaction of a emph{maximum} inter-transmission interval in closed loop, a guarantee that is not straightforward to obtain in rollout control. Our main contribution is to introduce a novel tube MPC scheme adapted for rollout control, which contains an additional constraint on the predicted transmission schedule. We show that by virtue of this schedule constraint, the inter-transmission interval is bounded in closed loop. We also establish recursive feasibility of this novel MPC scheme and show convergence of the system.

Keywords: Networked control systems, Predictive control for linear systems, Switched systems
Abstract: A two-tier architecture for cloud-based MPC is presented consisting of a high rate MPC in the cloud and a low rate MPC on the local device. The system use the cloud MPC as the nominal controller but switches to local MPC in case of an unresponsive network. The two MPCs are designed to be as similar to each other as possible except for the sampling rate. Different alternatives for when to execute the local MPC and how to perform the switching are presented. The approach is evaluated by simulation.

Keywords: Learning, Uncertain systems, Estimation
Abstract: We propose an iterative method to safely learn the unmodeled dynamics of a control-affine nonlinear system using Bayesian Gaussian process (GP) models with polynomial kernel functions. The method maintains safety by ensuring that the system state stays within the region of attraction (ROA) of a stabilizing control policy while collecting data. A quadratic programming based exploration control policy is computed to keep the exploration trajectory inside an inner-approximation of the ROA and to maximize the information gained from the trajectory. A prior GP model, which incorporates prior information about the unknown dynamics, is used to construct an initial stabilizing policy. As the GP model is updated with data, it is used to synthesize a new policy and a larger ROA, which increases the range of safe exploration. The use of polynomial kernels allows us to compute ROA inner-approximations and stabilizing control laws for the model using sum-of-squares programming. We also provide a probabilistic guarantee of safety which ensures that the policy computed using the learned model stabilizes the true dynamics with high confidence.

Keywords: Machine learning
Abstract: Crowdsourcing has emerged as an effective means for performing a number of machine learning tasks such as annotation and labelling of images and other data sets. In most early settings of crowdsourcing, the task involved classification, that is assigning one of a discrete set of labels to each task. Recently, however, more complex tasks have been attempted including asking crowdsource workers to assign continuous labels, or predictions. In essence, this involves the use of crowdsourcing for function estimation. We are motivated by this problem to drive applications such as collaborative prediction, that is, harnessing the wisdom of the crowd to predict quantities more accurately. To do so, we propose a Bayesian approach aimed specifically at alleviating overfitting, a typical impediment to accurate prediction models in practice. In particular, we develop a variational Bayesian technique for two different worker noise models – one that assumes workers’ noises are independent and the other that assumes workers’ noises have a latent low-rank structure. Our evaluations on synthetic and real-world datasets demonstrate that these Bayesian approaches perform significantly better than existing non-Bayesian approaches and are thus potentially useful for this class of crowdsourcing problems.

Keywords: Machine learning, Linear parameter-varying systems, Neural networks
Abstract: This paper presents a variational Bayesian inference Neural Network (BNN) approach to quantify uncertainties in matrix function estimation for the state-space linear parameter-varying (LPV) model identification problem using only inputs/outputs data. The proposed method simultaneously estimates states and posteriors of matrix functions given data. In particular, states are estimated by reaching a consensus between an estimator based on past system trajectory and an estimator by recurrent equations of states; posteriors are approximated by minimizing the Kullback–Leibler (KL) divergence between the parameterized posterior distribution and the true posterior of the LPV model parameters. Furthermore, techniques such as transfer learning are explored in this work to reduce computational cost and prevent convergence failure of Bayesian inference. The proposed data-driven method is validated using experimental data for identification of a control-oriented reactivity controlled compression ignition (RCCI) engine model.

Keywords: Machine learning, Statistical learning, Kalman filtering
Abstract: Artificial neural networks (NNs) have become the de facto standard in machine learning. They allow learning highly nonlinear transformations in a plethora of applications. However, NNs usually only provide point estimates without systematically quantifying corresponding uncertainties. In this paper a novel approach towards fully Bayesian NNs is proposed, where training and predictions of a perceptron are performed within the Bayesian inference framework in closed-form. The weights and the predictions of the perceptron are considered Gaussian random variables. Analytical expressions for predicting the perceptron's output and for learning the weights are provided for commonly used activation functions like sigmoid or ReLU. This approach requires no computationally expensive gradient calculations and further allows sequential learning.

Keywords: Machine learning, Uncertain systems, Formal Verification/Synthesis
Abstract: Gaussian Processes (GPs) are widely employed in control and learning because of their principled treatment of uncertainty. However, tracking uncertainty for iterative, multi-step predictions in general leads to an analytically intractable problem. While approximation methods exist, they do not come with guarantees, making it difficult to estimate their reliability and to trust their predictions. In this work, we derive formal probability error bounds for iterative predictions with GPs. Building on GP properties, we bound the probability that random trajectories lie in specific regions around the predicted values. Namely, given a tolerance epsilon > 0 , we compute regions around the predicted trajectory values, such that GP trajectories are guaranteed to lie inside them with probability at least 1-epsilon. We verify experimentally that our method tracks the predictive uncertainty correctly, even when current approximation techniques fail. Furthermore, we show how the proposed bounds can incorporate a given control law, and effectively bound the trajectories of the closed-loop system.

Keywords: Machine learning, Optimization, Subspace methods
Abstract: Adversarial robustness has become an important research topic given empirical demonstrations on the lack of robustness of deep neural networks. Unfortunately, recent theoretical results suggest that adversarial training induces a strict tradeoff between classification accuracy and adversarial robustness. In this paper, we propose and then study a new regularization for any margin classifier or deep neural network. We motivate this regularization by a novel generalization bound that shows a tradeoff in classifier accuracy between maximizing its margin and average margin. We thus call our approach an average margin (AM) regularization, and it consists of a linear term added to the objective. We theoretically show that for certain distributions AM regularization can both improve classifier accuracy and robustness to adversarial attacks. We conclude by using both synthetic and real data to empirically show that AM regularization can strictly improve both accuracy and robustness for support vector machine's (SVM's), relative to unregularized classifiers and adversarially trained classifiers.

Keywords: Machine learning, Modeling
Abstract: Lately, data-driven control has become a widespread area of research. A few recent big-data based approaches for data-driven control of nonlinear systems try to use classical input-output techniques to design controllers for systems for which only a finite number of (input-output) samples are known. These methods focus on using the given data to compute bounds on the mathcal{L}_2-gain or on the shortage of passivity from finite input-output data, allowing for the application of the small gain theorem or the feedback theorem for passive systems. One question regarding these methods asks about their sample complexity, namely how many input-output samples are needed to get an approximation of the operator norm or of the shortage of passivity. We show that the number of samples needed to estimate the operator norm of a system is roughly the same as the number of samples required to approximate the system in the operator norm.

Keywords: Machine learning, Learning, Nonlinear systems identification
Abstract: We address the problem of learning multiple dynamical systems, which is a kind of multi-task learning (MTL). The existing methods of MTL do not apply to learning dynamical systems in general. In this work, we develop a regularization method to perform MTL for dynamical systems appropriately. The proposed method is based on an operator-theoretic metric on dynamics that is agnostic of model parametrization and applicable even for nonlinear dynamics models. We calculate the proposed MTL-like regularization by estimating the metric from trajectories generated during training. Learning time-varying systems can be regarded as a special case of the usage of the proposed method. The proposed regularizer is versatile as we can straightforwardly incorporate it into off-the-shelf gradient-based optimization methods. We show the results of experiments on synthetic and real-world datasets, which exhibits the validity of the proposed regularizer.

Keywords: Attack Detection, Learning, Sensor networks
Abstract: This paper addresses the detection problem of flip attacks to sensor network systems where the attacker flips the distribution of manipulated sensor measurements of a binary state. The detector decides to continue taking observations or to stop based on the sensor measurements, and the goal is to have the flip attack recognized as fast as possible while trying to avoid terminating the measurements when no attack is present. The detection problem can be modeled as a partially observable Markov decision process (POMDP) by assuming an attack probability, with the dynamics of the hidden states of the POMDP characterized by a stochastic shortest path (SSP) problem. The optimal policy of the SSP solely depends on the transition costs and is independent of the assumed attack possibility. By using a fixed-length window and suitable feature function of the measurements, a Markov decision process (MDP) is used to approximate the behavior of the POMDP. The optimal solution of the approximated MDP can then be solved by any standard reinforcement learning methods. Numerical evaluations are given to demonstrate the effectiveness of the proposed method.

Keywords: Pattern recognition and classification, Subspace methods, Machine learning
Abstract: This paper studies the attack detection problem in a data-driven and model-free setting, for deterministic systems with linear and time-invariant dynamics. Differently from existing studies that leverage knowledge of the system dynamics to derive security bounds and monitoring schemes, we treat the cases where the system dynamics, as well as the attack strategy and attack location, are unknown. We derive fundamental security limitations as a function of only the observed data and without estimating the system dynamics (in fact, no assumption is made on the identifiability of the system). In particular, (i) we derive detection limitations as a function of the informativity and length of the observation window, (ii) provide a data-driven characterization of undetectable attacks, and (iii) construct a data-driven detection monitor. Surprisingly, our results show that while data-driven monitoring requires a larger observation window to attain attack detection capability, once attained it shares the same limitations as model-based monitoring.

Keywords: Attack Detection, Cyber-Physical Security, Control Systems Privacy
Abstract: This paper presents a strategy for detecting anomalies—behavior that is different from nominal—in systems with periodic outputs. Different from classical detectors available in the literature which require the computation of a residual, the detection algorithms proposed here are based on the concept of the Poincaré map and on the notion of cyclic group. The performance of the detectors is illustrated by numerical simulations and validated by experiments on an actuated mass-spring-damper oscillator.

Keywords: Fault tolerant systems, Network analysis and control, Fault detection
Abstract: Secure state-reconstruction is the problem of reconstructing the state of a linear time-invariant system from sensor measurements that have been corrupted by an adversary. Whereas most work focuses on attacks on sensors, we consider the more challenging case where attacks occur on sensors as well as on nodes and links of a network that transports sensor measurements to a receiver. In this paper we provide necessary and sufficient conditions for the secure state-reconstruction problem to be solvable in the presence of attacks on sensors and on the network. Our results unify existing results in the literature that focus either on attacks on sensors, or on the network.

Keywords: Decentralized control, Networked control systems, Resilient Control Systems
Abstract: Decentralized control systems are widely used in a number of situations and applications. In order for these systems to function properly and achieve their desired goals, information must be propagated between agents. However, communication between agents entails connecting to a network, potentially allowing adversaries to infiltrate the system through the network and attack multiple agents. To increase resiliency against these attacks, it is desirable for agents to operate disconnected from the network as much as possible, only communicating with other agents when it is absolutely necessary to achieve their goal or to maintain the safety of the overall system. This in turn decreases communication costs. Previous approaches to decentralized event-triggered control are mainly concerned with minimizing communication costs and therefore assume that every agent is always connected to the network with the ability to receive any information that is sent to it. In this work, we address the issue of maintaining the safety and stability of the overall system when there is no attack but some agents may be disconnected from the network and unable to receive critical information from other agents. We design an event-triggered mechanism for network connection and communication that is a function of only local information and that ensures stability for the overall system in attack-free scenarios. An algorithm describing this communication protocol is provided, and our approach is illustrated via simulation.

Keywords: Resilient Control Systems, Cyber-Physical Security, Attack Detection
Abstract: This study investigates the relationship between resilience of control systems to attacks and the information available to malicious attackers. Specifically, it is shown that control systems are guaranteed to be secure in an asymptotic manner by rendering reactions against potentially harmful actions covert. The behaviors of the attacker and the defender are analyzed through a repeated signaling game with an undisclosed belief under covert reactions. In the typical setting of signaling games, reactions conducted by the defender are supposed to be public information and the measurability enables the attacker to accurately trace transitions of the defender's belief on existence of a malicious attacker. In contrast, the belief in the game considered in this paper is undisclosed and hence common equilibrium concepts can no longer be employed for the analysis. To surmount this difficulty, a novel framework for decision of reasonable strategies of the players in the game is introduced. Based on the presented framework, it is revealed that any reasonable strategy chosen by a rational malicious attacker converges to the benign behavior as long as the reactions performed by the defender are unobservable to the attacker. The result provides an explicit relationship between resilience and information, which indicates the importance of covertness of reactions for designing secure control systems.

Keywords: Computer/Network Security, Linear systems
Abstract: Significant concerns on networked control systems are modifications on the control signals caused by a compromise on the network or the controller, since it can cause a devastating behavior or even entire failure of the system. In this paper, we present a fundamental solution to this problem by proposing a new authenticated computation that checks the matrix-vector multiplications---the main arithmetic of a controller---and verifies the updates on the states of the controller. It enables the plant-side not only to check the computation of the controller with much less computational cost than that required for the computation itself, but also to detect any modifications on the control signals.

Keywords: Human-in-the-loop control, Game theory, Cyber-Physical Security
Abstract: We consider a setting consisting of two sites, and a sequential game between a defender and an attacker who are responsible for securing and attacking the sites, respectively. Each site has a value to the defender, and an associated probability of successful attack, which can be reduced via security investments in that site by the defender. The attacker targets the site that maximizes the expected loss for the defender (after the investments). While prior work has studied the security investments in such scenarios, in this work we consider what happens when the defender exhibits characteristics of bounded-rationality that have been identified by behavioral economics. In particular, humans have been shown to perceive probabilities in a nonlinear manner, typically overweighting low probabilities and underweighting high probabilities. We characterize how such nonlinear probability weighting affects the security investments made by the defender, and bound the inefficiency of the equilibrium investments under behavioral decision-making, compared to a non-behavioral optimal solution.

Keywords: Switched systems, Embedded systems, Optimal control
Abstract: Most modern control systems are switched, meaning they have continuous as well as discrete decision variables. Switched systems often have constraints called dwell-time constraints, (e.g., cycling constraints in a heat pump), on the switching rate. This paper introduces an embedding-based-method to solve optimal control problems that have both discreet and continuous decision variables. unlike existing methods, the developed technique can heuristically incorporate dwell-time constraints while also preserving other state and control constrains of the problem. Simulations are run for a switched optimal control problem with and without the auxiliary cost to showcase the utility of the developed method.

Keywords: Switched systems, Fault detection, Optimization
Abstract: In this paper, fault detection is studied for discrete-time switched systems subject to unknown but bounded disturbances and measurement noise. A fault detection observer with a new structure is investigated based on an L_{infty} criterion to attenuate the effects of uncertainties. The design conditions are given in terms of Linear Matrix Inequalities using Multiple Quadratic Lyapunov Functions, under an Average Dwell Time switching signal. In addition, the FD results are obtained via an ellipsoidal analysis. Finally, a numerical example is performed to illustrate the effectiveness of the proposed approach.

Keywords: Switched systems, Linear systems, Stability of linear systems
Abstract: We demonstrate that a switched affine system is simultaneously stabilisabile i.e. there exists an asymptotically stabilising switching function which is independent of the initial condition if and only if there exists a stable convex combination of the sub-system matrices.

Keywords: Switched systems, Lyapunov methods, Stability of nonlinear systems
Abstract: We use a graph-theory-based argument to propose a novel Lyapunov construction for continuous-time switching systems. Starting with a finite family of continuously differentiable functions, the inequalities involving these functions and the vector fields of the switching system are encoded in a direct and labeled graph. Relaying on the (path-)completeness of this graph, we introduce a signal-dependent Lyapunov function, providing sufficient conditions for stability under fixed-time or dwell-time switching hypothesis. For the case of linear systems, our conditions turn into linear matrix inequalities (LMI), and thus they are compared with previous results, via numerical examples.

Keywords: Switched systems, Lyapunov methods, Stability of nonlinear systems
Abstract: This paper presents sufficient conditions for almost global stability of nonlinear switched systems consisting of both stable and unstable subsystems. Techniques from the stability analysis of switched systems have been combined with the multiple Lyapunov density approach - recently proposed by the authors for the almost global stability of nonlinear switched systems composed of stable subsystems. By using slow switching for stable subsystems and fast switching for unstable subsystems lower and upper bounds for mode-dependent average dwell times are obtained. In addition to that, by allowing each subsystem to perform slow switching and using some restrictions on total operation time of unstable subsystems and stable subsystems, we have obtained a lower bound for an average dwell time.

Keywords: Computational methods, Switched systems, Formal Verification/Synthesis
Abstract: In this paper we consider the problem of robust control invariant set computation for discrete-time switched affine systems. We consider additive disturbances and joint state-input constraints. We provide several constructions of N-step recurrent sets from which we derive an implicit description of invariant sets. This is done with a linear program characterizing set recurrence and backwards reachability with affine feedback controllers. The focus of these methods is scalability to systems with high-dimension. The performance of these methods is evaluated on a system modeling consensus of unmanned aerial vehicles (UAVs) and a system modeling the large-scale control of thermostatically controlled loads (TCLs).

Keywords: Networked control systems, Switched systems, Stability of hybrid systems
Abstract: This paper deals with the design of scheduling policies for networked control systems whose shared networks have limited communication capacity and the controller to plant channels are vulnerable to jamming attacks. We assume that among (N) plants, only (M (< N)) plants can communicate with their controllers at any time instant, and the attack sequences follow an ((m,k))-firm model, i.e., in any (k) consecutive time instants, the control inputs sent to some or all of the plants accessing the communication network, are deactivated at most at (m (< k)) time instants. We devise a new algorithm to allocate the network to the plants periodically such that stability of each plant is preserved under the admissible attack signals. The main apparatus for our analysis is a switched systems representation of the individual plants in an NCS. We rely on matrix commutators (Lie brackets) between the stable and unstable modes of operation of the plants to guarantee stability under our scheduling policies.

Keywords: Stability of nonlinear systems, Robust control, Switched systems
Abstract: Vibrational control stabilizes an unstable nominal system by injecting high-frequency dither signals. However, high-frequency dithers consume much energy, and demand high power and high bandwidth control actuators. To alleviate the energy consumption issue, we propose to switch off the dither injection in a periodic manner - introducing a switched dither signal. By tuning the period of the switching appropriately, system stability can be guaranteed. Moreover, the closed loop system's stability properties are robust with respect to a class of additive disturbances. A simulation of an inverted pendulum system illustrates the theory, and shows that the control energy is significantly reduced with only minimal effect on the overall control performance as compared to classic vibrational control.

Keywords: Constrained control, Adaptive control, Robotics
Abstract: This paper presents a novel, safe tracking control design method that learns the parameters of an uncertain Euler-Lagrange (EL) system online using adaptive learning laws. A barrier function (BF) is first used to transform the full-state constrained EL-dynamics into an equivalent unconstrained dynamics. An adaptive tracking controller is then developed along with the parameter update law in the transformed state space such that the states remain bounded for all time within a prescribed bound. A stability analysis is developed that considers the EL-dynamics' uncertainty, yielding a semi-globally uniformly ultimately bounded (SGUUB) tracking error and the parameter estimation error. The controller design is validated in simulations using a two-link planar manipulator. The results show the proposed method's ability to track the reference trajectory while remaining inside each of the predefined state bounds.

Keywords: Constrained control, Machine learning, Robotics
Abstract: Safety is a critical component in today's autonomous and robotic systems. Many modern controllers endowed with notions of guaranteed safety properties rely on accurate mathematical models of these nonlinear dynamical systems. However, model uncertainty is always a persistent challenge weakening theoretical guarantees and compromising safety. For safety-critical systems, this is an even bigger challenge. Typically, safety is ensured by constraining the system states within a safe constraint set defined a priori by relying on the model of the system. A popular approach is to use Control Barrier Functions (CBFs) that encode safety using a smooth function. However, CBFs fail in the presence of model uncertainties. Moreover, an inaccurate model can either lead to incorrect notions of safety or worse, incur system critical failures. Addressing these drawbacks, we present a novel safety formulation that leverages properties of CBFs and positive definite kernels to design textit{Gaussian CBFs}. The underlying kernels are updated online by learning the unmodeled dynamics using Gaussian Processes (GPs). While CBFs guarantee forward invariance, the hyperparameters estimated using GPs update the kernel online and thereby adjust the relative notion of safety. We demonstrate our proposed technique on a safety-critical quadrotor on SO(3) in the presence of model uncertainty in simulation. With the kernel update performed online, safety is preserved for the system.

Keywords: Constrained control, Lyapunov methods
Abstract: The explicit reference governor (ERG) is an add-on unit that provides constraint handling capabilities to prestabilized systems. The basic idea behind this approach is to translate state and input constraints into an upper-bound on the value of the Lyapunov function, which is then enforced by suitably manipulating the derivative of the auxiliary reference. This paper extends the ERG approach to deal with time-varying linear constraints. In particular, it is shown that under a certain condition on the rate of change of the constraints and the textit{strength} of stability of the prestabilized system (i.e., the speed of the dynamics of the prestabilized system), it is possible to deploy the ERG approach to handle time-varying constraints. To do so, components of the ERG approach are redefined, and constraints satisfaction and convergence properties are proven analytically. The effectiveness of the proposed scheme is demonstrated through a simulation study on filming a movie scene with a drone.

Keywords: Constrained control, Stability of nonlinear systems, Aerospace
Abstract: Constrained attitude maneuvers have numerous applications in robotics and aerospace. In our previous work, a general framework to this problem was proposed with resolution completeness guarantee. However, a smooth reference trajectory and a low-level safety-critical controller were lacking. In this work, we propose a novel construction of a C^2 continuous reference trajectory based on B'ezier curves on SO(3) that evolves within predetermined cells and eliminates previous stop-and-go behavior. Moreover, we propose a novel zeroing control barrier function on SO(3) that provides a safety certificate over a set of overlapping cells on SO(3) while avoiding nonsmooth analysis. The safety certificate is given as a linear constraint on the control input and implemented in real-time. A remedy is proposed to handle the states where the coefficient of the control input in the linear constraint vanishes. Numerical simulations are given to verify the advantages of the proposed method.

Keywords: Constrained control, Neural networks, Identification for control
Abstract: One major issue in industrial control applications is how to handle input constraints due to physical limitations of the actuators. Such constraints introduce nonlinearities in the feedback loop, that are commonly tackled through anti-windup or model predictive control schemes. Since these techniques might result into poor closed-loop performance when an accurate model of the plant is not available, in this work we present an off-line strategy to learn a neural anti-windup control scheme (NAW-NET) from a set of open-loop data collected from an unknown nonlinear process. The proposed scheme, that includes a feedback controller and an anti-windup compensator, is trained to reproduce the desired closed-loop behavior while simultaneously accounting for actuator limits. The effectiveness of the approach is illustrated on a simulation example, involving the control of a Hammerstein Wiener process with saturated inputs.

Keywords: Constrained control, Power systems
Abstract: This paper analyses the controllability properties of microgrids connected to constant power loads subject to input and state constraints. Constraint requirements in Micro- Grids are either inherent, i.e. inverter modulation indices limitations, or imposed, such as current and voltage limitation which are safety critical. In this paper, we propose an analysis of the controllability properties of a microgrid using set theoretic notions; this analysis sheds light on the constraint admissibility properties of a microgrid with constant power loads in terms of constraint satisfaction and robustness to changes in power demands. Lastly, we provide a method of recasting the original nonlinear microgrid control problem into controlling a linear system subject to bounded additive disturbances and output constraints.

Keywords: Constrained control, Predictive control for linear systems, Autonomous systems
Abstract: This paper introduces an add-on, supervisory scheme, referred to as Action Governor (AG), for discrete-time linear systems to enforce exclusion-zone avoidance requirements. It does so by monitoring, and minimally modifying when necessary, the nominal control signal to a constraint-admissible one. The AG operates based on set-theoretic techniques and online optimization. This paper establishes its theoretical foundation, discusses its computational realization, and uses two simulation examples to illustrate its effectiveness.

Keywords: Constrained control, Uncertain systems, Robust adaptive control
Abstract: This paper presents adaptive robust quadratic program (QP) based control using control Lyapunov and barrier functions for nonlinear systems subject to time-varying and state-dependent uncertainties. An adaptive estimation law is proposed to estimate the pointwise value of the uncertainties with pre-computable estimation error bounds. The estimated uncertainty and the error bounds are then used to formulate a robust QP, which ensures that the actual uncertain system will not violate the safety constraints defined by the control barrier function. Additionally, the accuracy of the uncertainty estimation can be systematically improved by reducing the estimation sampling time, leading subsequently to reduced conservatism of the formulated robust QP. The proposed approach is validated in simulations on an adaptive cruise control problem and through comparisons with existing approaches.

Keywords: Hierarchical control, Robust control, Biological systems
Abstract: There are many interesting dynamical systems that can be regarded as hierarchically networked systems in a variety of fields including control. One of the ideas to treat those systems properly is "Glocal (Global/Local) Control," which means that the global purpose is achieved by local actions of measurement and control cooperatively. The key for realization of glocal control is hierarchically networked dynamical systems with multiple resolutions in time and space depending on the layer, which introduce many new theoretical control challenges aiming at practical effectiveness in synthetic biology and engineering. The main issues may include how to achieve synchronization by decentralized control and how to make a compromise of global and local objectives. This talk starts with the background, the idea, and the concept of glocal control based on an understanding of IoT from the control perspective followed by presenting two research topics.

The first topic is on a theoretical framework for hierarchically decentralized control of networked dynamical systems that can take account of the tradeoff between the global and local objectives to achieve the desired harmony under change of the environments. Several new ideas, by exploiting the special structure of the target systems, enable us to develop scalable control design methods based on the powerful theory in classical, modern, and robust control. The effectiveness of the new theoretical foundations on the analysis and synthesis are experimentally confirmed by applications to electric vehicle control.

The second topic is on robust instability analysis for a class of uncertain nonlinear networked systems to guarantee the persistence of nonlinear oscillations in the presence of a dynamic perturbation, which is important in synthetic biology. The problem of robust instability has a very different feature from that of robust stability, and hence a new theoretical setting is needed. The theoretical results are applied to the Repressilator in synthetic biology, and the effectiveness is confirmed by numerical simulations.

Keywords: Machine learning, Autonomous systems, Large-scale systems
Abstract: Machine learning-based techniques have recently revolutionized nearly every aspect of autonomy. In particular, deep reinforcement learning (RL) has rapidly become a powerful alternative to classical model-based approaches to decision-making, planning, and control. Despite the well-publicized successes of deep RL, its adoption in complex and/or safety-critical tasks at scale and in real-world settings is hindered by several key issues, including high sample complexity in large-scale problems, limited transferability, and lack of robustness guarantees. This talk explores our recently developed solutions that address these fundamental challenges for both single and multiagent RL. In addition, this talk highlights the complementary role that classical model-based techniques can play in synergy with data-driven methods in overcoming these issues. Real experiments with ground and aerial robots will be used to illustrate the effectiveness of the proposed techniques. The talk will conclude with an assessment of the state of the art and highlight important avenues for future research.

Keywords: Healthcare and medical systems, Identification, Model Validation
Abstract: We will analyse and discuss how the choice of model complexity impacts the predictive power of some epidemics models. We describe theoretical analysis of parameter identifiability and discuss the discrepancy between different simulation studies and the actual outcome during the COVID-19 pandemic. We also discuss the different information sources available to aid analysis in an ongoing pandemic, and discuss their usefulness, based on our experience from the Swedish health care system. The contribution is a cooperation between modeling experts from the major Swedish universities and Swedish health care experts with long experience, responsible for analysis during the COVID-19 pandemic and tracking the seasonal influenza for many years. We describe how this was used to aid the the planning within the Swedish health care system.

Keywords: Machine learning, Healthcare and medical systems, Pattern recognition and classification
Abstract: The sudden increase in COVID-19 cases is putting high pressure on healthcare services worldwide. At this stage, fast, accurate and early clinical assessment of the disease severity is vital. To support decision making and logistical planning in healthcare systems, this study leverages a database of blood samples from 485 infected patients in the region of Wuhan, China, to identify crucial predictive biomarkers of disease mortality. For this purpose, machine learning tools selected three biomarkers that predict the mortality of individual patients more than 10 days in advance with more than 90% accuracy: lactic dehydrogenase (LDH), lymphocyte and high-sensitivity C-reactive protein (hs-CRP).Overall, this Article suggests a simple and operable decision rule to quickly predict patients at the highest risk, allowing them to be prioritized and potentially reducing the mortality rate. This talk is based on the following paper published in Nature Machine Intelligence: https://www.nature.com/articles/s42256-020-0180-7

Keywords: Nonlinear systems identification, Network analysis and control, Compartmental and Positive systems
Abstract: To better capture the dynamics of the COVID-19 epidemics in Italy, as in other countries with a regional (or federal) administrative structure, it is of utmost importance to capture regional differences in mitigating the spread of the diseases, considering the heterogeneity at the regional level of the effects of national measures and local intervention strategies. Also, it is crucial to model flows of potential infected among the regions at a finer level than aggregate SIR family models allow.In this study we take Italy as a study case and propose a network model where each of the 20 administrative regions in which Italy is split into is considered as a node, edges representing flows of people to and from each region. Both short distance edges and long distance edges are considered in the model. The aim of the study is to capture at a finer level the dynamics of the epidemics both at the regional and the national level highlighting differences between regional dynamics and the aggregate effect of these at the national level. We use the proposed model to analyse the COVID-19 spread across the country, and to explore different feedback control based mitigation strategies to avoid the return of an epidemic in future waves. In so doing, we derive sufficient conditions based on applying appropriate matrix measures to the so-called next generation matrix and explore different optimal control approaches to devise intermittent regional strategies to contain and mitigate future epidemic phenomena.

Keywords: Healthcare and medical systems, Identification, Model Validation
Abstract: The ongoing COVID-19 pandemic has motivated an innumerable number of researchers, companies, and governments to attempt to forecast the spread of the disease. Scouring this vast spectrum of predictions, it is clear that there is no consensus and that capturing the behavior of the outbreak remains elusive. The majority of the data available appears to support the hypothesis that the disease follows some variant of an SIR (susceptible-infected-removed) model. It is well known that the reproduction number of the SIR model (both networked and group models) depends on the number of susceptible individuals in the population. Therefore, forecasts of the spread, and consequently the number of susceptible, are essential for understanding in what stage of the outbreak we are currently operating. Several natural questions arise: - What can be done to improve forecasts of the outbreak/avoid overfitting in order to help design and implement mitigation efforts? - When sickness and deaths are in the balance, as they are now, how should researchers approach the exploitation vs exploration tradeoff? - How can we learn from this outbreak and adapt the findings in preparation for future possible outbreaks?

Keywords: Game theory, Power systems, Distributed control
Abstract: We formulate a Stackelberg game between an attacker and a defender of a power system. The attacker attempts to alter the load setpoints of the power system covertly and intelligently, so that the voltage stability margin of the grid is reduced, driving the entire system towards a voltage collapse. The defender, or the system operator, aims to compensate for this reduction by retuning the reactive power injection to the grid by switching on control devices, such as a bank of shunt capacitors. A modified Backward Induction method is proposed to find a cost-based Stackelberg equilibrium (CBSE) of the game, which saves the players' costs while providing the optimal allocation of both players' investment resources under budget and covertness constraints. We analyze the proposed game extensively for the IEEE 9-bus power system model and present an example of its performance for the IEEE 39-bus power system model. It is demonstrated that the defender is able to maintain system stability unless its security budget is much lower than the attacker's budget.

Keywords: Power systems, Distributed control, Optimal control
Abstract: Power systems exhibit nonlinear dynamics when load stress is high. Conventional linear wide-area controllers will fail in such scenarios, and instead nonlinear controllers will become necessary. This paper develops a state-feedback nonlinear quadratic controller based on perturbed Koopman mode analysis (PKMA) considering a sparse wide-area communication net-work for saving communication cost while guaranteeing closed-loop stability. The sparsity is decided based on the modal residues of the generators on selected nonlinear oscillations modes. The theoretical developments behind this sparse nonlinear control are first established, including the definition of residues for Koopman modes. Bilinear effects in the controller structure arising from quadratic PKMA-based extended models, structural constraints on the quadratic gain matrix design, and second-order phenomena in the closed-loop dynamics are all explained in detail. The design is validated using the IEEE 16-machine, 68-bus power system. The advantages and drawbacks of the proposed approach are illustrated using simulation results.

Keywords: Power systems, Control of networks, Stability of nonlinear systems
Abstract: Many recent works in control of electric power systems have investigated their synchronization through global performance metrics under external disturbances. The approach is motivated by fundamental changes in the operation of power grids, in particular by the substitution of conventional power plants with new renewable sources of electrical energy. This substitution will simultaneously increase fluctuations in power generation and reduce the available mechanical inertia. It is crucial to understand how strongly these two evolutions will impact grid stability. With very few, mostly numerical exceptions, earlier works on performance metrics had to rely on unrealistic assumptions of grid homogeneity. Here we show that a modified spectral decomposition can tackle that issue in inhomogeneous power grids in cases where disturbances occur on time scales that are long compared to the intrinsic time scales of the grid. We find in particular that the magnitude of the transient excursion generated by disturbances with long characteristic times does not depend on inertia. For continental-size, high-voltage power grids, this corresponds to power fluctuations that are correlated on time scales of few seconds or more. We conclude that power fluctuations arising from new renewables will not require per se the deployment of additional rotational inertia. We numerically illustrate our results on the IEEE 118-Bus test case and a model of the synchronous grid of continental Europe.

Keywords: Power systems, Observers for Linear systems, Stability of nonlinear systems
Abstract: The availability of excitation controllers to en-hance transient stability has regained significant relevance in recent years, due to the unprecedented ongoing changes in power systems. Yet, the practical deployment of many reported control schemes is hampered by the fact that their implementation requires the full state vector. Our main contribution is to address this fundamental obstacle by proposing an observer-based excitation controller using modern phasor measurement technology. For this purpose, a linear time-varying observer scheme for the generator frequency and the internal voltage is derived. This observer is then combined with a classical passivity-based excitation controller. Stability of the resulting nonlinear observer-based closed-loop system is shown by deriving an ISS-based separation principle. The performance of the proposed approach is demonstrated via simulation example.

Keywords: Power systems, Stability of nonlinear systems, Smart grid
Abstract: Swing equations are an integral part of a large class of power system dynamical models used in rotor angle stability assessment. Despite intensive studies, some fundamental properties of lossy swing equations are still not fully understood. In this paper, we develop a sufficient condition for certifying the stability of equilibrium points (EPs) of these equations, and illustrate the effects of damping, inertia, and network topology on the stability properties of such EPs. The proposed certificate is suitable for real-time monitoring and fast stability assessment, as it is purely algebraic and can be evaluated in a parallel manner. Moreover, we provide a novel approach to quantitatively measure the degree of stability in power grids using the proposed certificate. Extensive computational experiments are conducted, demonstrating the practicality and effectiveness of the proposal.

Keywords: Machine learning, Healthcare and medical systems, Optimization
Abstract: Artificial Neural Networks (ANN) have been shown to be effective for many predictive tasks, such as system identification and reinforcement learning. However, as they have become more ubiquitous, there have been several examples of models exhibiting anthropomorphic bias (e.g. making predictions correlated with race or gender for unrelated tasks) due to over fitting, amplifying and systematizing bias already inherent in training data. To address this problem, we consider a novel regularization approach for deep learning, inspired by the constrained optimization literature, that directly penalizes unwanted disparities in treatment of populations proportionally to their impact on observed bias. Using this method, we can control bias at training time, as opposed to in a pre- or post-processing step; this results in concurrent out-of-sample improvements in both fairness and accuracy for some data sets. Our methods fit well into existing optimization and training approaches and can be easily generalized across network architectures and notions of fairness. We validate our methods empirically on several real world data sets that contain implicit bias. Namely we consider the impact of race on recidivism prediction, gender on income, and wine color on quality. We also consider fairness in a reinforcement learning setting by controlling the dose of Heparin while being certifiably fair with respect to the patient's insurance provider.

Keywords: Delay systems, Sampled-data control, Control applications
Abstract: This paper addresses the periodic event-triggered stabilization in the sample-and-hold sense of nonlinear time-delay systems by means of a finite-dimensional state feedback law. Differently from most related literature, no continuity assumptions are required on the state feedback available in continuous time, which can hence be designed for instance by universal constructions yielding possible discontinuities, provided that such a feedback depends on a finite number of commensurate time delays. An application to the problem of glucose regulation is then presented, showing the effectiveness of the approach taken.

Keywords: Hybrid systems, Control applications, Systems biology
Abstract: Contract-based design is a compositional approach that enables to satisfy specifications for complex systems by partitioning responsibilities among the component subsystems. A natural application of assume-guarantee reasoning arises in the context of biological systems, which are composed of different modules, each one responsible for some actions. This work presents an application of the assume-guarantee contracts theory to the glucose regulation system, which is a topic of major importance in diabetes treatment, in the context of the so-called Artificial Pancreas. The glucose control problem is defined and solved in a decentralized way by defining suitable contracts to be satisfied by the glucose and insulin subsystems, while the framework naturally takes into account sampling and quantization arising from the digital environment. The results show that the approach is feasible and promising for further investigation.

Keywords: Healthcare and medical systems, Human-in-the-loop control, Control applications
Abstract: Functional electrical stimulation (FES) induced cycling is a common rehabilitative technique applied for those with a movement disorder. An FES cycle system is a nonlinear switched dynamic system that has a potentially destabilizing input delay between stimulation and the resulting muscle force. In this paper, a dual objective control system for a nonlinear, uncertain, switched FES cycle system with an unknown time varying input delay is developed and a Lyapunov-like dwell-time analysis is performed to yield exponential power tracking to an ultimate bound and global exponential cadence tracking. Preliminary experimental results for a single healthy individual are provided and demonstrate average power and cadence tracking errors of -0.05 +/- 0.80 W and -0.05 +/- 1.20 RPM, respectively, for a target power of 10 W and a target cadence of 50 RPM.

Keywords: Delay systems, Stability of nonlinear systems, Systems biology
Abstract: In this paper, the notion of GALES (Global Asymptotic Local Exponential Stability) is extended to nonlinear systems described by Retarded Functional Differential Equations. Necessary and sufficient Lyapunov--Krasovskii conditions ensuring the GALES of nonlinear time--delay systems are provided. The conditions related to the lower bound and to the dissipation rate of the Lyapunov--Krasovskii functional involve only the current value of the solution, making the provided tool easy to use. An example validating the results is presented.

Keywords: Closed-loop identification, Identification, Identification for control
Abstract: One of the primary challenges of system identification is determining how much data is necessary to adequately fit a model. Non-asymptotic characterizations of the performance of system identification methods provide this knowledge. Such characterizations are available for several algorithms performing open-loop identification. Often times, however, data is collected in closed-loop. Application of open-loop identification methods to closed-loop data can result in biased estimates. One method to eliminate these biases involves first fitting a long-horizon autoregressive model and then performing model reduction. The asymptotic behavior of such algorithms is well characterized, but the non-asymptotic behavior is not. This work provides a non-asymptotic characterization of one particular variant of these algorithms. More specifically, we provide non-asymptotic upper bounds on the generalization error of the produced model, as well as high probability bounds on the difference between the produced model and the finite horizon Kalman Filter.

Keywords: Identification, Model/Controller reduction, Linear systems
Abstract: In this paper two online estimation methods are considered for the determining the Loewner matrices of a linear system. Both algorithms employ the gradient descent method and sufficient persistence of excitation conditions guarantee the convergence of estimates to the correct values.

Keywords: Model/Controller reduction, Linear systems, Control applications
Abstract: We introduce a novel structured controller design for the electron beam stabilization problem of the UK's national synchrotron light source. Because changes to the synchrotron will not allow the application of existing control approaches, we develop a novel method to diagonalize the multi-input multi-output (MIMO) system. A generalized singular value decomposition (GSVD) is used to simultaneously diagonalize the actuator response matrices, which is applicable to an arbitrary number of actuator dynamics in a cross-directional setting. The resulting decoupled systems are regulated using mid-ranged control and the controller gains derived as a function of the generalized singular values. In addition, we exploit the inherent block circulant symmetry of the system. The performance of our controller is demonstrated using simulations that involve machine data.

Keywords: Model/Controller reduction, Biomolecular systems, Computational methods
Abstract: The Chemical Master Equation (CME) is commonly used to describe the stochastic behavior of biomolecular systems. However, in general, the CME’s dimension is very large or infinite, so analytical or even numerical solutions may be difficult to achieve. The truncation methods such as the Finite State Projection (FSP) algorithm alleviate this issue to some extent but not completely. To further resolve the computational issue, we propose the Enhanced Finite State Projection (EFSP) algorithm, in which the ubiquitous time-scale separation is utilized to reduce the dimension of the CME. Our approach combines the original FSP algorithm and the model reduction technique that we developed, to approximate an infinite dimensional CME with a finite dimensional CME that contains the slow species only. Unlike other time-scale separation methods, which rely on the fast-species counts’ stationary conditional probability distributions, our model reduction technique relies on only the first few conditional moments of the fast-species counts. In addition, each iteration of the EFSP algorithm relies on the solution of the approximated CME that contains the slow species only, unlike the original FSP algorithm relies on the solution of the full CME. These two properties provide a significant computation advantage. The benefit of our algorithm is illustrated through a protein binding reaction example.

Keywords: Model/Controller reduction, Stability of linear systems
Abstract: We explore a middle ground between data-driven model reduction and data-driven control. In particular, we use snapshots collected from the system to build reduced models that can be expressed in terms of data. We illustrate how the derived family of reduced models can be used for data-driven control of the original system under suitable conditions. Finding a control law that stabilizes certain solutions of the original system as well as the one that reaches any desired state in final time are studied in detail.

Keywords: Machine learning, Optimization, Optimization algorithms
Abstract: In this paper, we propose and analyze SPARQ-SGD, an event-triggered and compressed algorithm for decentralized training of large-scale machine learning models over a graph. Each node can locally compute a condition (event) which triggers a communication where quantized and sparsified local model parameters are sent. In SPARQ-SGD, each node takes at least a fixed number of local gradient steps and then checks if the model parameters have significantly changed compared to its last update; it communicates further compressed model parameters only when there is a significant change, as specified by a (design) criterion. We prove that SPARQ-SGD converges as O(1/nT) and O(1/sqrt(nT)) in the strongly-convex and non-convex settings, respectively, demonstrating that aggressive compression, including event-triggered communication, model sparsification and quantization does not affect the overall convergence rate compared to uncompressed decentralized training; thereby theoretically yielding communication efficiency for `free'. We evaluate SPARQ-SGD over real datasets to demonstrate significant amount of savings in communication over the state-of-the-art.

Keywords: Machine learning, Optimization, Robust control
Abstract: In order to anticipate rare and impactful events, we propose to quantify the worst-case risk under distributional ambiguity using a recent development in kernel methods -- the kernel mean embedding. Specifically, we formulate the generalized moment problem whose ambiguity set (i.e., the moment constraint) is described by constraints in the associated reproducing kernel Hilbert space in a nonparametric manner. We then present the tractable approximation and its theoretical justification. As a concrete application, we numerically test the proposed method in characterizing the worst-case constraint violation probability in the context of a constrained stochastic control system.

Keywords: Machine learning, Optimization, Estimation
Abstract: We address the problem of selecting sensors for the estimation of binary random variables, so as to minimize the probability of error. This problem arises when a large number of sensors are potentially available, but only a few can actually be used for estimation purposes. While sensor selection is a combinatorial problem, we show that the optimization of an upper bound on the probability of error can be formulated as a submodular maximization for which computationally efficient algorithms can provide solutions with guaranteed performance. The submodular optimization that needs to be solved involves the computation of an expected value that generally cannot be computed in closed form, but we show that replacing the expected value by a Monte Carlo empirical mean can result in negligible loss of performance with high probability. We illustrate the use of these results in the context of detecting illegal unreported and unregulated (IUU) fishing.

Keywords: Machine learning, Nonlinear systems identification
Abstract: Although it is known that having accurate Lipschitz estimates is essential for certain models to deliver good predictive performance, refining this constant in practice can be a difficult task especially when the input dimension is high. In this work, we shed light on the consequences of employing loose Lipschitz bounds in the Nonlinear Set Membership (NSM) framework, showing that the model converges to a nearest neighbor regressor (k-NN with k=1). This convergence process is moreover not uniform, and is monotonic in the univariate case. An intuitive geometrical interpretation of the result is then given and its practical implications are discussed.

Keywords: Machine learning, Statistical learning, Learning
Abstract: The present paper addresses the problem of matrix completion with noisy measurements using a deterministic procedure for choosing the sample set. Previous research on the matrix completion problem has been based on probabilistic sampling, with the result that the various theorems hold only with high probability. Also, most available results are for the case where the measurements are noise-free. Authors proposed to generate the sample set as the edge set of a Ramanujan graph, and sufficient condition is provided for the exact and stable recovery of the unknown matrix via nuclear norm minimization, when measurements are noise-free and noisy, respectively.

In the present paper, we use the same sampling approach and perform some numerical simulations for the case of both noise-free and noisy measurements. In the process, we give a comparison between the noisy and noise-free measurements.

Keywords: Automotive systems, Automotive control, Constrained control
Abstract: We present a decision making approach for autonomous driving that concurrently determines the driving mode and the motion plan that achieves the driving mode goal. To do this, we develop two cooperating modules: a mode activator and a motion planner. Based on the current mode in a non-deterministic automaton, the mode activator determines all the feasible next modes, i.e., the modes for which there exists a trajectory that reaches the associated goal. Then, the motion planner generates trajectories achieving the goals of such feasible modes, selects the next mode and trajectory that result in the best performance, and updates the current mode in the automaton. To determine the feasibility, the mode activator uses robust forward and backward reachability that accounts for the discrepancy between the simplified model used in the reachability computation and the more precise model used by the motion planner. We prove that, under normal operation, the mode activator always returns a nonempty set of feasible modes, so that the decision making algorithm is recursively feasible. We validate the algorithm in simulations and experiments using car-like laboratory-scale robots.

Keywords: Automotive systems, Optimization, Stochastic systems
Abstract: Driver assistance systems have become more and more important in recent years due to the increasing degree of automation in road traffic - especially with regard to safety. Often a driver's perception of a situation is prone to inaccuracy, particularly on country roads. Consequently, there is a high mortality rate due to frontal collisions with oncoming traffic (among others). One of the most dangerous maneuvers that leads to such conflicts is overtaking. Overtaking involves other traffic participants and therefore uncertainties exist. We cope with this challenge by introducing a system that evaluates the options a vehicle has during overtaking (completing or aborting the maneuver) using stochastic models of the surrounding traffic. The stochastic models are used to predict the movements of surrounding road users. As a next step, the general feasibility of the possible maneuvers is checked and rated. The results of this analysis can either be directly used for longitudinal control, be forwarded to a low level controller, or serve as a guideline for the decision-making process of a human driver.

Keywords: Stochastic systems, Automotive control
Abstract: The contribution of the paper is to propose an estimator of the combustion stability of spark-ignition engines operating in lean-burn conditions. The stability is here taken in the sense of the Coefficient of Variation of the Indicated Mean Effective Pressure. The proposed estimator of the Coefficient of Variation provides in real-time both an estimation of the expected value and of the confidence interval, based on the Bayesian theoretical framework. The estimator only needs a buffer of few measures, making it suitable for transient control. It could lead to the construction of a robust controller in learn-burn conditions, ensuring a better compromise between the engine efficiency and the prevention of combustion instabilities.

Keywords: Automotive control, Automotive systems, Optimal control
Abstract: Using knowledge of the future route and its topology is known to offer substantial fuel savings, and this is even more true for hybrid electric vehicles, as the battery use can be planned in advance, for instance to take into account coming slopes. However, traffic or other environmental conditions can force to deviate from the initial planning making it no longer optimal.

In this paper, we propose a flexible double layer approach for energy management of hybrid vehicles able to cope with traffic changes. First, before departure, an expected optimal speed and powertrain state reference is computed on a cloud and sent to an on-board controller. Simple, route-specific engine on/off rules are extracted by the controller and used for an on-board fast convex optimization, which can be conducted frequently along the drive, adapting the references to take into account changes of traffic conditions over longer sections of the route as communicated by V2X. Abrupt disturbances are handled by a lower level Model Predictive Control (MPC). If the condition changes are very substantial, so that the empirical on/off rule seems questionable, the cloud can be asked to perform a full optimization again.

Keywords: Energy systems, Optimal control, Electrochemical processes
Abstract: This paper derives provably optimal control trajectories for the Li-ion battery fast charging problem. Conventionally, battery charging protocols must satisfy safety constraints while maximizing the state of charge (SoC) level. In the literature, both computational and experimental studies promote a diversity of algorithms, including pulse charging, multistep constant currents, and more. Although these approaches yield applicable charging algorithms, the literature lacks a rigorous analytical insight into optimal charging trajectories. In this paper, we focus on the Pontryagin’s Minimum Principle for solving optimal control problem for battery fast charging. Specifically, we characterize the optimal control solution with respect to the state constraint bound. The optimal input is analytically derived for a reduced-order electrochemical model. The optimal solutions follow a Bang or Bang-Ride trajectory. Numerical simulations validate the analytical solutions.

Keywords: Game theory, Optimization algorithms, Adaptive control
Abstract: We introduce a novel class of Nash equilibrium seeking dynamics for non-cooperative games with a finite number of players, where the convergence to the Nash equilibrium is bounded by a KL function with a settling time that can be upper bounded by a positive constant that is independent of the initial conditions of the players, and which can be prescribed a priori by the system designer. The dynamics are model-free, in the sense that the mathematical forms of the cost functions of the players are unknown. Instead, in order to update its own action, each player needs to have access only to real-time evaluations of its own cost, as well as to auxiliary states of neighboring players characterized by a communication graph. Stability and convergence properties are established for both potential games and strongly monotone games. Numerical examples are presented to illustrate our theoretical results.

Keywords: Nonlinear output feedback, Lyapunov methods, Stability of nonlinear systems
Abstract: The problem of global finite-time stabilization of lower-triangular nonlinear systems is investigated in this paper. Unlike previous papers that either used the full-state information or assumed that the system nonlinearities satisfy some restrictive conditions, in this paper a novel solution to the global output-feedback finite-time stabilization problem for a larger class of nonlinear systems is proposed under a mild condition. A new analytic method is also developed to prove the global finite-time stability of the closed-loop systems.

Keywords: Adaptive control, Uncertain systems, Robust control
Abstract: In this paper we present a practical prescribed time (PPT) control for a class of uncertain nonlinear systems to achieve pre-assignable precision within a finite time that can be specified in advance irrespective of initial condition or any design parameter. Specifically, by introducing a novel time-varying constraining function, we convert the original finite-time tracking control problem into one with a deferred constraint on tracking error, then by stabilizing of which, we solve the problem with the following appealing features: 1) both settling time and tracking accuracy can be explicitly specified in advance regardless of initial conditions; 2) the proposed control is based upon regular (rather than fractional power) state feedback, thus is continuous and smooth everywhere; and 3) the results are global in that the prescribed time tracking is ensured for states starting from anywhere within physically possible domain.

Keywords: Predictive control for linear systems
Abstract: In this paper, a model predictive control (MPC) approach with finite-time convergence is proposed for constrained discrete-time linear systems. The proposed MPC is constructed based on a criterion of finite-time convergence for discrete-time linear systems, where its Lyapunov function decreases with a rate greater than that of exponential convergence. In the finite-time MPC, the cost function is designed particularly by using the proposed finite-time criterion. Recursive feasibility of optimization and stability of the closed-loop system can be proved within the classical MPC framework.

Keywords: Agents-based systems, Autonomous robots, Lyapunov methods
Abstract: In this paper, the problem of fixed-time leader-follower consensus problem of nonholonomic multi-agent systems is under study. Using the “desingularisation method” introduced in the seminal paper by J.M. Coron [6], new fixed- time controllers/observers for the double integrator system are designed. Following those results, a switching consensus protocol which guarantees the tracking errors stabilization in fixed-time which does not depend on the initial conditions of the multi-agent system is provided. Simulation results on a fleet of wheeled mobile robots show the effectiveness of the proposed scheme.

Keywords: Machine learning, Game theory, Optimization algorithms
Abstract: Learning processes in games explain how players grapple with one another in seeking an equilibrium. We study a natural model of learning based on individual gradients in two-player continuous games. In such games, the arguably natural notion of a local equilibrium is a differential Nash equilibrium. However, the set of locally exponentially stable equilibria of the learning dynamics do not necessarily coincide with the set of differential Nash equilibria of the corresponding game. To characterize this gap, we provide formal guarantees for the stability or instability of such fixed points by leveraging the spectrum of the linearized game dynamics. We provide a comprehensive understanding of scalar games and find that equilibria that are both stable and Nash are robust to variations in learning rates.

Keywords: Machine learning, Game theory, Smart grid
Abstract: Reinforcement Learning Algorithms (RLA) are useful machine learning tools to understand how decision makers react to signals. It is known that RLA converge towards the pure Nash Equilibria (NE) of finite congestion games and more generally, finite potential games. For finite congestion games, only separable cost functions are considered. However, non-separable costs, which depend on the choices of all players instead of only those choosing the same resource, may be relevant in some circumstances, like in smart charging games. In this paper, finite congestion games with non-separable costs are shown to have an ordinal potential function, leading to the existence of an action-dependent continuous potential function. The convergence of a synchronous RLA towards the pure NE is then extended to this more general class of congestion games. Finally, a smart charging game is designed for illustrating convergence of such learning algorithms.

Keywords: Optimization algorithms, Game theory, Variational methods
Abstract: We consider the Nash equilibrium problem in a partial-decision information scenario. Specifically, each agent can only receive information from some neighbors via a communication network, while its cost function depends on the strategies of possibly all agents. In particular, while the existing methods assume undirected or balanced communication, in this paper we allow for non-balanced, directed graphs. We propose a fully-distributed pseudo-gradient scheme, which is guaranteed to converge with linear rate to a Nash equilibrium, under strong monotonicity and Lipschitz continuity of the game mapping. Our algorithm requires global knowledge of the communication structure, namely of the Perron-Frobenius eigenvector of the adjacency matrix and of a certain constant related to the graph connectivity. Therefore, we adapt the procedure to setups where the network is not known in advance, by computing the eigenvector online and by means of vanishing step sizes.

Keywords: Game theory, Machine learning, Agents-based systems
Abstract: In this work, we study the interaction of strategic agents in continuous action Cournot games with limited information feedback. Cournot game is the essential market model for many socio-economic systems where agents learn and compete without the full knowledge of the system or each other. We consider the dynamics of the policy gradient algorithm, which is a widely adopted continuous control reinforcement learning algorithm, in concave Cournot games. We prove the convergence of policy gradient dynamics to the Nash equilibrium when the price function is linear or the number of agents is two. This is the first result (to the best of our knowledge) on the convergence property of learning algorithms with continuous action spaces that do not fall in the no-regret class.

Keywords: Optimization algorithms, Game theory, Optimization
Abstract: We consider an online assortment problem with [n]:={1,2,ldots,n} sellers, each holding exactly one item iin[n] with initial inventory c_iin mathbb{Z}_+, and a sequence of homogeneous buyers arriving over a finite time horizon t=1,2,ldots,m. There is an online platform whose goal is to offer a subset S_tsubseteq [n] of sellers to the arriving buyer at time t to maximize the expected revenue derived over the entire horizon while respecting the inventory constraints. Given an assortment S_t at time t, it is assumed that the buyer will select an item from S_t based on the well-known multinomial logit model, a well-justified choice model from the economic literature. In this model, the revenue obtained from selling an item i at a given time t critically depends on the assortment S_t offered at that time and is given by the Nash equilibrium of a Bertrand game among the sellers in S_t. This imposes a strong dependence/externality among the offered assortments, items revenues, and inventory levels. Despite that challenge, we devise a constant-competitive algorithm for the online assortment problem with homogeneous buyers and evaluate its performance via numerical results.

Keywords: Optimization algorithms, Machine learning, Optimization
Abstract: In this paper, we study the minimax optimization problem in the smooth and strongly convex-strongly concave setting when we have access to noisy estimates of gradients. In particular, we first analyze the stochastic Gradient Descent Ascent (GDA) method with constant stepsize, and show that it converges to a neighborhood of the solution of the minimax problem. We further provide tight bounds on the convergence rate and the size of this neighborhood. Next, we propose a multistage variant of stochastic GDA (M-GDA) that runs in multiple stages with a particular learning rate decay schedule and converges to the exact solution of the minimax problem. We show M-GDA achieves the lower bounds in terms of noise dependence without any assumptions on the knowledge of noise characteristics. We also show that M-GDA obtains a linear decay rate with respect to the error’s dependence on the initial error, although the dependence on condition number is suboptimal. In order to improve this dependence, we apply the multistage machinery to the stochastic Optimistic Gradient Descent Ascent (OGDA) algorithm and propose the M-OGDA algorithm which also achieves the optimal linear decay rate with respect to the initial error. To the best of our knowledge, this method is the first to simultaneously achieve the best dependence on noise characteristic as well as the initial error and condition number.

Keywords: Optimization algorithms
Abstract: Understanding the convergence performance of asynchronous stochastic gradient descent method (Async-SGD) has received increasing attention in recent years due to their foundational role in machine learning. To date, however, most of the existing works are restricted to either bounded gradient delays or convex settings. In this paper, we focus on Async-SGD and its variant Async-SGDI (which uses increasing batch size) for non-convex optimization problems with unbounded gradient delays. We prove o(1/sqrt{k}) convergence rate for Async-SGD and o(1/k) for Async-SGDI. Also, a unifying sufficient assumption for Async-SGD's convergence is proposed, which includes two major gradient delay models in the literature as special cases.

Keywords: Optimization algorithms, Optimization
Abstract: Many science and engineering applications feature non-convex optimization problems where the performance is not explicitly modeled by a cost or reward function, i.e. it is a black box. Examples include most complex design problems where experimental tests are the main method to evaluate performance of chosen values of the decision variables, in fields such as mechanics, fluid-dynamics, electromagnetics and/or magnetohydrodynamics. Solving these problems can be done iteratively: the next value of the decision variables is chosen based on the outcome generated by the previous tests. The time and resource overhead in conducting tests, however, raises the issue of most efficiently choosing the next test point according to previous observations. To approach this issue, a new global optimization strategy based on a Set Membership framework is proposed. Assuming a Lipschitz continuous cost function, the presented algorithm builds an approximation of the latter to decide whether to exploit the best result obtained so far, or to further explore the decision space. The proposed algorithm is presented and some implementation aspects are discussed. Its performance is evaluated on a set of benchmark non-convex problems and compared with those of other global optimization approaches.

Keywords: Optimization algorithms, Uncertain systems, Machine learning
Abstract: This paper presents a safe optimization method for minimizing an unknown cost function subject to unknown inequality and equality constraints. The cost function as well as the constraints evaluation may be corrupted with Gaussian measurement noise with known uncertainty. The focus especially lies in the safe exploration of the cost function, which means that an evaluation of the cost function far away of previous evaluations is not favoured. Also evaluations in regions where the constraints are violated are undesired. This is the case, for example, in technical applications, where systems may become unstable or damaged, if constraints are violated. To this end, a combination of the expected improvement of the cost and its mean is minimized, while bounding the variance of the cost. The inequality and equality constraints are reformulated as constraints for the probability of the constraint violation. The optimization method is evaluated on numerical examples.

Keywords: Optimization algorithms, Computer/Network Security
Abstract: This paper introduces a novel data integrity attack in distributed optimization (DO) problems where all agents cooperatively minimize the sum of their cost functions except a selfish dishonest agent who misleads all agents’ states to converge to its local optimal solution by transmitting falsified information during DO iteration. Different from most existing attacks that require global information, this data integrity attack does not require any extra information and only relies on local information and communication with neighbors. It is shown that even though the dishonest agent shares falsified information, it preserves the linear convergence of the DO algorithm. This means that the attack is implemented locally and difficult to be detected by threshold value method. We provide numerical simulation to verify theoretical results as well as the performance of this attack applied to other DO algorithms. By revealing the vulnerability of DO algorithm to such data integrity attack, this paper conveys the message that besides designing exact and effective DO algorithms, it is equally important to protect the distributed network from potential malicious attacks to avoid performance loss.

Keywords: Markov processes, Adaptive systems, Neural networks
Abstract: In this work we present a multi-armed bandit framework for online expert selection in Markov decision processes and demonstrate its use in high-dimensional settings. Our method takes a set of candidate expert policies and switches between them to rapidly identify the best performing expert using a variant of the classical upper confidence bound algorithm, thus ensuring low regret in the overall performance of the system. This is useful in applications where several expert policies may be available, and one needs to be selected at run-time for the underlying environment.

Keywords: Markov processes, Constrained control, Control applications
Abstract: This paper presents an optimization-based technique for triggering measurements in a hidden Markov model so that the number/rate of measurements is minimized while maintaining accuracy of the state estimates to satisfy a prescribed constraint. Here, the accuracy of a state estimate is characterized by the entropy of the distribution associated with the estimate. After introducing the general approach, this paper also discusses its application to precision agriculture. In particular, we consider site-specific monitoring of soil nitrate levels and show through simulations that the required number of measurements can be reduced with the proposed technique compared to conventional strategies where measurements are taken at a fixed rate.

Keywords: Markov processes, Iterative learning control, Uncertain systems
Abstract: In this paper, we present Online Empirical Value Learning (ONEVaL), an ‘online’ reinforcement learning algo- rithm for continuous MDPs that is ‘quasi-model-free’ (needs a generative/simulation model but not the model per se) that can compute nearly-optimal policies and comes with non- asymptotic performance guarantees including prescriptions on required sample complexity for specified performance bounds. The algorithm relies on use of a ‘fully’ randomized policy that will generate a β-mixing sample trajectory. It also relies on randomized function approximation in an RKHS for arbitrarily small function approximation error, and an ‘empirical’ estimate of value from the next state by several samples of the next state from the generative model. We demonstrate its’ good numerical performance on some benchmark problems. We note that the algorithm requires no hyper-parameter tuning, and is also robust to other concerns that seem to plague Deep RL algorithms.

Keywords: Markov processes, Learning, Stochastic optimal control
Abstract: The importance of safe reinforcement learning (safe RL) is widely recognized for enhancing real world systems. In this study, we construct the censored Markov decision process (CeMDP), a new Markov Decision Process (MDP) framework that describes the interaction of environment, learner and external systems, e.g., human intervention or pre-designed controller for emergency response. We also theoretically analyze the relation of CeMDP to existing frameworks such as the semi-Markov decision process, MDP with Option (OMDP) and standard MDP; the analysis clarifies that CeMDP is a special case of OMDP and can, with environment redefinition, be represented by MDP. This finding allows us to design planning and reinforcement learning algorithms for CeMDP. We confirm the validity of the theory and algorithms by numerical experiments.

Keywords: Markov processes, Linear systems, Stability of linear systems
Abstract: In this paper, we study the stabilization of two interdependent Markov jump linear systems (MJLS) with partial information, where the interdependency arises as the transition of the mode of one system depends on the states of the other system. First, we formulate a framework for the two interdependent MJLSs to capture the interactions between various entities in the system, where the modes of the system cannot be observed directly. Instead, a signal which contains information of the modes can be obtained. Then, depending on the scope of the available system state information (global or local), we design centralized and distributed controllers, respectively, that can stochastically stabilize the overall interdependent MJLS. In addition, the sufficient stabilization conditions for the system under both types of information structure are derived. Finally, we provide a numerical example to illustrate the effectiveness of the designed controllers.

Keywords: Stochastic systems, Optimization algorithms, Distributed control
Abstract: In this paper, we study decentralized solutions for finite-sum minimization problems when the underlying training data is distributed over a network of nodes. In particular, we describe the GT-SAGA algorithm that combines variance reduction and gradient tracking to achieve both robust performance and fast convergence. Variance reduction is implemented locally to asymptotically estimate each local batch gradient at each node, while gradient tracking fuses the local estimated gradients across the nodes. Combining variance reduction and gradient tracking thus enables linear convergence to the optimal solution of strongly-convex problems while keeping a low per-iteration computation complexity at each node. We cast the convergence and behavior of GT-SAGA and related methods in the context of certain practical tradeoffs and further compare their performance with the help of numerical experiments over both convex and non-convex problems.

Keywords: Optimization algorithms, Networked control systems
Abstract: This paper considers the distributed optimization problem of minimizing a global cost function formed by a sum of local smooth cost functions by using local information exchange. Various distributed optimization algorithms have been proposed for solving such a problem. A standard condition for proving the linear convergence for existing distributed algorithms is the strong convexity of the cost functions. However, the strong convexity may not hold for many practical applications, such as least squares and logistic regression. In this paper, we propose a distributed primal-dual gradient descent algorithm and establish its linear convergence under the condition that the global cost function satisfies the Polyak-{L}ojasiewicz condition. This condition is weaker than strong convexity and the global minimizer is not necessarily unique. The theoretical result is illustrated by numerical simulations.

Keywords: Optimization algorithms, Cooperative control, Networked control systems
Abstract: We consider a class of multi-agent optimization problems, where each agent is associated with an action vector and a local cost that depends on the joint actions of all agents, and the goal is to minimize the average of the local costs. Such problems arise in many control applications such as wind farm operation and mobile sensor coverage. In many of these applications, while we have access to (zeroth-order) information about function values, it can be difficult to obtain (first-order) gradient information. In this paper, we propose a zeroth-order feedback optimization (ZFO) algorithm based on two-point gradient estimators for the considered class of problems, and provide the convergence rate to a first-order stationary point for nonconvex problems. We complement our theoretical analysis with numerical simulations on a wind farm power maximization problem.

Keywords: Optimization algorithms, Optimization
Abstract: This paper develops a novel algorithm that tackles a general distributed convex optimization problem with a nonsmooth objective function and a series of coupling nonlinear inequality and linear equality constraints. To address such a problem, we first design a distributed proximal algorithm for problems with coupling equality constraints only, and then extend this algorithm to problems with both inequality and equality constraints by incorporating a virtual-queue-based method. The resulting algorithm, referred to as Proximal Primal-Dual Algorithm (PPDA), is shown to achieve O(1/k) rates of convergence with respect to both optimality and feasibility. Compared to the alternative methods in the literature, PPDA eliminates their assumptions on the smoothness of the objective function and achieves a stronger convergence rate result. In addition, PPDA exhibits faster convergence in a couple of numerical examples.

Keywords: Optimization algorithms, Networked control systems, Machine learning
Abstract: We present a parallelized primal-dual algorithm for solving constrained convex optimization problems. The algorithm is "block-based," in that vectors of primal and dual variables are partitioned into blocks, each of which is updated only by a single processor. We consider four possible forms of asynchrony: in updates to primal variables, updates to dual variables, communications of primal variables, and communications of dual variables. We explicitly construct a family of counterexamples to rule out permitting asynchronous communication of dual variables, though the other forms of asynchrony are permitted, all without requiring bounds on delays. A first-order update law is developed and shown to be robust to asynchrony. We then derive convergence rates to a Lagrangian saddle point in terms of the operations agents execute, without specifying any timing or pattern with which they must be executed. These convergence rates contain a synchronous algorithm as a special case and are used to quantify an "asynchrony penalty." Numerical results illustrate these developments.

Keywords: Agents-based systems, Network analysis and control, Stability of nonlinear systems
Abstract: In social systems, the evolution of interpersonal appraisals and individual opinions are not independent processes but intertwine with each other. Despite extensive studies on both opinion dynamics and appraisal dynamics separately, no previous work has ever combined these two processes together. In this paper, we propose a novel and intuitive model on the interplay between homophily-based appraisal dynamics and influence-based opinion dynamics. We assume that individuals' opinions are updated via the influence network constructed from their interpersonal appraisals, which are in turn updated based on the individual opinions via the homophily mechanism. By theoretical analysis, we characterize the set of equilibria and some transient behavior of our model. Moreover, we establish the equivalence among the convergence of the appraisal network to social balance, the modulus consensus of individual opinions, and the non-vanishing appraisals. Monte Carlo validations further show that the non-vanishing appraisals condition holds for generic initial conditions. Compared with previous works that explain the emergence of social balance via person-to-person homophily mechanism, our model provides an alternative explanation in terms of the person-to-entity homophily mechanism. In addition, our model also describes how individuals' opinions on multiple irrelevant issues become correlated and converge to modulus consensus over time-varying influence networks.

Keywords: Agents-based systems, Networked control systems, Linear systems
Abstract: The aim of this paper is to address consensus in the context of networked agents whose interactions can be modelled by an undirected, signed, weighted, connected and clustered graph. Specifically, we assume that individuals can be split into three groups representing the decision classes on a given specific topic, for instance, the in favour, abstained and opponent agents. Interactions between agents belonging to the same cluster are cooperative, meaning that the link connecting those agents has a non-negative weight, while interactions between agents belonging to different clusters are antagonistic and therefore a non-positive weight is associated to the link connecting them. We will show that under certain regularity assumptions it is possible to devise a simple modification of DeGroot’s algorithm that ensures that the opinions of agents who cooperate converge to consensus, i.e. the opinions of agents belonging to the same cluster converge to the same decision.

Keywords: Network analysis and control, Networked control systems, Control of networks
Abstract: A standard model for opinion formation in networks, which comprises a linear model for opinion-evolution coupled with a non-linear model for self-confidence dynamics, is considered. The main contribution of this study is to characterize the spatial pattern of the model dynamics in response to a local disruption. Specifically, we show that both the linear and nonlinear models exhibit a diffusive structure, such that responses to disruptions decay in amplitude along cutsets away from the disruption source in the network's graph. The spatial characterization is then used to motivate and develop a model reduction technique for the nonlinear self-confidence dynamics. Finally, the formal results and model reduction algorithm are illustrated in an example.

Keywords: Cooperative control, Distributed control, Agents-based systems
Abstract: In this paper, an asynchronous discrete-time opinion dynamics model on a social influence network is considered. At each time instant, a single individual activates and updates two state variables simultaneously. The individual's new private opinion is a weighted average of her current private opinion, the expressed opinions of her neighbors, and a constant prejudice. Meanwhile, the individual's new expressed opinion is equal to her current private opinion, altered due to a pressure to conform to the public opinion as perceived by the individual, being the average expressed opinion among her neighbors. We analyze the system for social networks which are rooted, and show that if no individual holds a prejudice, then a mild assumption on the activation sequence of the individuals guarantees convergence. In particular, the expressed and private opinions of all individuals converge to the same value exponentially fast, with two lower bounds on convergence speeds based on two different assumptions on the network topology. Simulations are provided to illustrate the result, and provide support to the conjecture that the system dynamics may converge even if individuals hold an existing prejudice.

Keywords: Agents-based systems, Stability of hybrid systems, Lyapunov methods
Abstract: Given a social network where the individuals know the identity of the other members, we present a model of opinion dynamics where the connectivity among the individuals depends on both their current and past opinions. Thus, their interactions are not only based on the present states but also on their past relationships. The model is a multi-agent system with active or inactive pairwise interactions depending on auxiliary state variables filtering the instantaneous opinions, thereby taking the past experience into account. When an interaction is (de)activated, a jump occurs, leading to a hybrid model. The proven stability properties ensure that opinions converge to local agreements/clusters as time grows. Simulation results are provided to illustrate the theoretical guarantees.

Keywords: Statistical learning, Formal Verification/Synthesis
Abstract: This paper focuses on the controller synthesis for unknown, nonlinear systems while ensuring safety constraints. Our approach consists of two steps, a learning step that uses Gaussian processes and a controller synthesis step that is based on control barrier functions. In the learning step, we use a data-driven approach utilizing Gaussian processes to learn the unknown control affine nonlinear dynamics together with a statistical bound on the accuracy of the learned model. In the second controller synthesis steps, we develop a systematic approach to compute control barrier functions that explicitly take into consideration the uncertainty of the learned model. The control barrier function not only results in a safe controller by construction but also provides a rigorous lower bound on the probability of satisfaction of the safety specification. Finally, we illustrate the effectiveness of the proposed results by synthesizing a safety controller for a jet engine example.

Keywords: Optimal control, Linear systems, Uncertain systems
Abstract: In this paper, we propose some new convex strategies for robust optimal control. In particular, we treat the problem of designing finite-horizon linear quadratic regulator (LQR) for uncertain discrete-time systems focusing on minimax strategies. A time-invariant linear control law is obtained just solving sequentially two convex optimization problems, hence obtaining a feedback law that takes into account all the available systems samples. In the case of stabilizable systems, we also generalize our approach by including additional constraints on the closed-loop stability in the optimization scheme. Extensions to time-variant control rules are also discussed, leading to novel and intriguing connections between optimal control and multi-task learning.

Keywords: Robust control, LMIs, Machine learning
Abstract: For a partially unknown linear systems, we present a systematic control design approach based on generated data from measurements of closed-loop experiments with suitable test controllers. These experiments are used to improve the achieved performance and to reduce the uncertainty about the unknown parts of the system. This is achieved through a parametrization of auspicious controllers with convex relaxation techniques from robust control, which guarantees that their implementation on the unknown plant is safe. This approach permits to systematically incorporate available prior knowledge about the system by employing the framework of linear fractional representations.

Keywords: Learning, Lyapunov methods, Optimal control
Abstract: Inspired by the success of imitation and inverse reinforcement learning in replicating expert behavior through optimal control, we propose a learning based approach to safe controller synthesis based on control barrier functions (CBFs). We consider the setting of a known non-linear control affine dynamical system and assume that we have access to safe trajectories generated by an expert — a practical example of such a setting would be a kinematic model of a self-driving vehicle with safe trajectories (e.g., trajectories that avoid collisions with obstacles in the environment) generated by a human driver. We then propose and analyze an optimization based approach to learning a CBF that enjoys provable safety guarantees under suitable Lipschitz smoothness assumptions on the underlying dynamical system. A strength of our approach is that it is agnostic to the parameterization used to represent the CBF, assuming only that the Lipschitz constant of such functions can be efficiently bounded. Furthermore, if the CBF parameterization is convex, then under mild assumptions, so is our learning process. We end with extensive numerical evaluations of our results on both planar and realistic examples, using both random feature and deep neural network parameterizations of the CBF. To the best of our knowledge, these are the first results that learn provably safe control barrier functions from data.

Keywords: Learning, Game theory, Robust control
Abstract: We assess the robustness of the equilibria in generalized Nash equilibrium problems in aggregative form subject to linear coupling constraints affected by uncertainty with a possibly unknown probability distribution. Within a data-driven context, we apply the scenario approach paradigm to provide a-posteriori feasibility certificates for the entire set of generalized Nash equilibria of the game. Then, we show that assessing the violation probability of such set merely requires one to enumerate the constraints that ``shape'' it. For the class of aggregative games, this results in solving a feasibility problem on each active facet of the feasibility region, for which we propose a semi-decentralized, structure-preserving algorithm.

Keywords: Iterative learning control, Resilient Control Systems, Robust adaptive control
Abstract: This paper studies the control of safety-critical dynamical systems in the presence of adversarial disturbances. We seek to synthesize state-feedback controllers to minimize a cost incurred due to the disturbance while respecting the safety constraint. The safety constraint is given by a bound on an H-∞ norm, while the cost is specified as an upper bound on an H-2 norm of the system. We consider an online setting where costs at each time are revealed only after the controller at that time is chosen. We propose an iterative approach to the synthesis of the controller by solving a modified discrete-time Riccati equation. Solutions of this equation enforce the safety constraint. We compare the cost of this controller with that of the optimal controller when one has complete knowledge of disturbances and costs in hindsight. We show that the regret function, which is defined as the difference between these costs, called varies logarithmically with the time horizon. We validate our approach on a process control setup that is subject to two kinds of adversarial attacks.

Keywords: Iterative learning control
Abstract: Iterative learning control (ILC) is a high performance control design method for systems working in a repetitive manner by learning from previous experience. Most existing ILC design considers the problem where the desired reference trajectory is defined either fully on the trial duration or on a finite number of intermediate time instants. This paper further expands the applicability of ILC by studying a more general case (named region to region tracking) where there is no desired trajectory defined at all; instead only a region where the system output should reach is given. To solve this problem, a novel ILC algorithm with an norm optimal ILC step and a projection operation is developed. Convergence properties of the algorithm are analysed rigorously. It is also shown that traditional reference trajectory tracking problems can be solved as special cases of the proposed design, resulting in a well-known norm optimal ILC algorithm being recovered and a new point to point ILC algorithm. Numerical simulations are presented to demonstrate the effectiveness of the proposed approach.