Keywords: Learning, Stochastic systems, Simulation
Abstract: The objective in a traditional reinforcement learning (RL) problem is to find a policy that optimizes the expected value of a performance metric such as the infinite-horizon cumulative discounted or long-run average cost/reward. In practice, optimizing the expected value alone may not be satisfactory, in that it may be desirable to incorporate the notion of risk into the optimization problem formulation, either in the objective or as a constraint. In this tutorial, we consider risk-sensitive RL in two settings: one where the goal is to find a policy that optimizes the usual expected value objective while ensuring that a risk constraint is satisfied, and the other where the risk measure is the objective. Various risk measures to be considered include exponential utility, variance, percentile performance, chance constraints, value at risk (quantiles), conditional value-at-risk, prospect theory, and general coherent risk measures. We survey some of the recent work in this area specifically where policy gradient search is the solution approach. In the first risk-sensitive RL setting, we present a template for policy gradient-based risk-sensitive RL algorithms using a Lagrangian formulation. For the setting where risk is incorporated directly into the objective function, we present policy gradient algorithms for an exponential utility formulation, cumulative prospect theory, and coherent risk measures.

Keywords: Identification, Learning, Linear systems
Abstract: Recently, there has been a renewed interest in addressing traditional control problems without the knowledge of the system model. In such cases, the data collected from the system is used to develop the controllers. Although related to the traditional system identification problem, this setting is distinct in the sense that direct methods are used that provide the controller without identifying the system model explicitly. In this tutorial, we will cover the basics of data-driven control design for canonical deterministic control problems like feedback control, stabilization, LQR, LQG etc. Further, we will also look at the stochastic setting with finite sample guarantees for system identification and control design in the presence of random noises in the data.

Keywords: Predictive control for linear systems, Predictive control for nonlinear systems

Keywords: Linear systems

Keywords: Systems biology, Control education, Computational methods
Abstract: Decision-making and memory are widespread behaviours in biomolecular systems. Because the underlying principles are nonlinearity and positive feedback, these topics are directly linked to control engineering. This proposal outlines a tutorial on these topics. First, benchmark models, mathematical and experimental, would be covered, to convey examples of these behaviours. Second, basic methods of Interval Analysis would be covered to provide a natural and rigorous tool to solve associated mathematical equations. Third, advanced topics arising from the study of these behaviours, such as in Algebraic Geometry, Monotone Dynamical Systems, and Chemical Reaction Network Theory would be discussed, to give a sense of open problems. This tutorial would add to the control engineering perspective on decisions and memory.