Keywords: Learning, Stochastic systems, Simulation
Abstract: The objective in a traditional reinforcement learning (RL) problem is to find a policy that optimizes the expected value of a performance metric such as the infinite-horizon cumulative discounted or long-run average cost/reward. In practice, optimizing the expected value alone may not be satisfactory, in that it may be desirable to incorporate the notion of risk into the optimization problem formulation, either in the objective or as a constraint. In this tutorial, we consider risk-sensitive RL in two settings: one where the goal is to find a policy that optimizes the usual expected value objective while ensuring that a risk constraint is satisfied, and the other where the risk measure is the objective. Various risk measures to be considered include exponential utility, variance, percentile performance, chance constraints, value at risk (quantiles), conditional value-at-risk, prospect theory, and general coherent risk measures. We survey some of the recent work in this area specifically where policy gradient search is the solution approach. In the first risk-sensitive RL setting, we present a template for policy gradient-based risk-sensitive RL algorithms using a Lagrangian formulation. For the setting where risk is incorporated directly into the objective function, we present policy gradient algorithms for an exponential utility formulation, cumulative prospect theory, and coherent risk measures.

Keywords: Identification, Learning, Linear systems
Abstract: Recently, there has been a renewed interest in addressing traditional control problems without the knowledge of the system model. In such cases, the data collected from the system is used to develop the controllers. Although related to the traditional system identification problem, this setting is distinct in the sense that direct methods are used that provide the controller without identifying the system model explicitly. In this tutorial, we will cover the basics of data-driven control design for canonical deterministic control problems like feedback control, stabilization, LQR, LQG etc. Further, we will also look at the stochastic setting with finite sample guarantees for system identification and control design in the presence of random noises in the data.

Keywords: Predictive control for linear systems, Predictive control for nonlinear systems
Abstract: Model Predictive Control (MPC) is an advanced control technique that has been used for process control since the 1980s. With the increasing computing power of microprocessors as well as high-speed optimization algorithms, the use of MPC has spread to many real-time embedded applications, often used in automotive, aerospace, industrial automation, and other industries. MPC can handle multi-input multi-output (MIMO) systems with coupled input-output channels as well as constraints on inputs, outputs, and states, which are challenging to handle with classical control methods. This workshop delves into the principles and applications of MPC, providing participants with a comprehensive understanding of the concepts and practical implementations in the simulation environment. During the workshop, we will discuss what is MPC, how it works, various categories of MPC, and their implementation. Leveraging the powerful computational tools of MATLAB and Simulink, participants will gain hands-on experience in designing and simulating MPC controllers for linear and nonlinear applications. Extensive libraries of MATLAB and Simulink's dynamic simulation environment will be utilized to demonstrate real-world applications and facilitate an intuitive understanding of complex control scenarios. By the end of this workshop, participants will be equipped with the skills to implement MPC in various engineering contexts, enhancing their ability to optimize performance and efficiency in control systems.

Keywords: Linear systems
Abstract: As advanced machines become more commonplace in our daily lives, it is becoming increasingly important for these machines to interact seamlessly with the humans that leverage their abilities. Delsys provides cutting-edge EMG solutions that enable engineers to design myoelectrically-controlled Human-Machine Interfaces, simplifying the complexity of human movement into intuitive control commands. This session will highlight the latest advancements offered by Delsys, including API integration and dynamic multi-channel EMG detection, which researchers can leverage to enhance their Human-Machine Interfaces. Mr. Nicholas Ducey will introduce Delsys, provide an overview of Delsys’ history in HMI, review the latest advanced Delsys tools, and discuss the future of EMG in HMI. Ms. Bhawna Shiwani will present on her research, focusing on leveraging real-time neural interfaces and dynamic motor unit identification for enhanced control of upper extremity prosthetics.

Keywords: Systems biology, Control education, Computational methods
Abstract: Decision-making and memory are widespread behaviours in biomolecular systems. Because the underlying principles are nonlinearity and positive feedback, these topics are directly linked to control engineering. This proposal outlines a tutorial on these topics. First, benchmark models, mathematical and experimental, would be covered, to convey examples of these behaviours. Second, basic methods of Interval Analysis would be covered to provide a natural and rigorous tool to solve associated mathematical equations. Third, advanced topics arising from the study of these behaviours, such as in Algebraic Geometry, Monotone Dynamical Systems, and Chemical Reaction Network Theory would be discussed, to give a sense of open problems. This tutorial would add to the control engineering perspective on decisions and memory.