Keywords: Markov processes, Linear systems, Learning
Abstract: Model-based reinforcement learning (RL) approaches are attractive for control applications due to their ability to provide theoretical guarantees on properties such as performance, stability and robustness. In this tutorial, we aim to provide the participants with a detailed understanding of the fundamentals of model-based reinforcement learning. We will present RL algorithms such as those based on the Reward-Biased Method (RB), Upper Confidence Bound (UCB), and Policy Gradient(PG), and discuss their pros and cons. The main focus of the tutorial will be to equip the participants with essential technical toolsets required to design and analyze RL algorithms. Throughout the tutorial, we will spotlight many seminal contributions that have been made over the years and which have been instrumental in laying the foundations of reinforcement learning. We will conclude by presenting some pressing some open problems in the field.

Keywords: Flight control, Observers for nonlinear systems, Control applications
Abstract: A typical drone has several parts to be selected. Improper choice of components leads to a mismatch in the bandwidth of the overall system and hence the tuning of the controller-estimator parameters becomes uncertain and further leads to the change of the complete design with new components. Hence, the importance of bandwidth analysis of each major component of the drone while tuning controller-estimator parameters is discussed. Several testbeds are developed and the characteristics of each black box component are analyzed. A complete systematic approach to the development of a drone is discussed which in turn fulfills the gap between actual hardware working performance and the simulations of flying vehicles in terms of gain tuning of controller and estimator parameters.

Keywords: Optimization algorithms, Optimization, Machine learning
Abstract: In this tutorial, we delve into the realm of algorithmic performance evaluation, particularly focusing on the crucial aspect of convergence rate. Convergence rate plays a pivotal role in assessing the effectiveness of methods across various fields, notably in optimization. Traditionally, accelerating an algorithm to enhance its performance demands an in-depth understanding of the problem's underlying structure, often leading to customized solutions tailored to specific cases. Many accelerated techniques have been developed over the past few decades and have found widespread practical use. However, these methods, while conceptually straightforward, typically rely on algebraic reasoning and may lack intuitive explanations.

Recent efforts have sought to bridge the gap between accelerated algorithms and other scientific domains, such as control theory and differential equations. Nevertheless, these explanations often hinge on intricate arguments and unconventional analytical tools. This tutorial aims to offer an alternative perspective by attempting to elucidate optimization algorithms through the lens of dynamical systems theory, a well-explored field with a robust theoretical foundation. Using conventional reasoning, we will provide an intuitive account of how acceleration phenomena arise.

Furthermore, accelerated methods often introduce additional parameters compared to their slower counterparts, making parameter estimation a challenging task. Additionally, these schemes are typically designed for specific problem settings and lack versatility. In this tutorial, we will explore a novel approach to accelerate optimization algorithms, leveraging generic acceleration principles that can be applied more broadly.

Keywords: Networked control systems
Abstract: The Internet of Things (IoT) connects various sensors to address diverse challenges in agriculture, healthcare, security, and industry. In a typical IoT system, sensors, often referred to as IoT nodes, measure specific parameters. Interface circuits handle signal conditioning and digitization, enabling digital processing and analysis using machine learning algorithms. This, in turn, leads to control and actuation of devices. Standard networking protocols ensure seamless connectivity among IoT nodes and device control.

This talk aims to provide students and research scholars with a fundamental introduction to the industrial revolution and the evolution of IoT-based systems. It covers the various components, architecture, building blocks, and layers of IoT. The lecture delves into the latest advancements, recent trends, and real-world applications of IoT across Precision Agriculture, Healthcare, and Industrial IoT systems, highlighting their significant impacts. The talk weaves these elements together to paint a comprehensive picture of a basic IoT system.

Keywords: Simulation, Nonlinear systems, Control applications
Abstract: Controlling nonlinear systems is a challenging problem that is traditionally addressed by linearizing the system at different operating points, or through advanced methods such as Model Predictive Control (MPC). However, in line with the modern trend of using Machine Learning to design autonomous systems, Reinforcement Learning (RL) has the potential to solve complex control system problems using deep neural networks. In this tutorial, we will walk through the basics of Reinforcement Learning for Controls using MATLAB and Simulink. We will cover the basic concepts, including neural networks, set up the environment models, and define the reward structure to shape the learning process.