Keywords: Networked control systems, Learning, Emerging control theory
Abstract: Recent radical evolution in distributed sensing, computation, communication, and actuation has fostered the emergence of cyber-physical network systems. Regardless of the specific application, one central goal is to shape the network's collective behavior through the design of admissible local decision-making algorithms. This is nontrivial due to various challenges such as local connectivity, system complexity and uncertainty, limited information structure, and the complex intertwined physics and human interactions. In this talk, I will present our recent progress in formally advancing the systematic design of distributed coordination in network systems via harnessing special properties of the underlying problems and systems. In particular, we will present three examples and discuss three types of properties, i) how to exploit network structure to ensure the performance of the local controllers; ii) how to use the information and communication to develop distributed learning rules; iii) how to use domain-specific properties to further improve the efficiency of the distributed control and learning algorithms. We will also discuss challenges and issues arising from these solutions.

Keywords: Computational methods
Abstract: Suppose an N−length signal has known frequency support of size k. Given sample access to this signal, how fast can we compute the DFT? The answer to this question depends on the structure of the frequency support. We start with some example supports to see how (an ideal) O(k log k) complexity can be achieved through existing techniques. We then build on these examples to investigate a family of sets (referred to as homogenous sets) for which an O(k log k) DFT complexity is achievable.

A frequency support J is said to have an additive structure if the size of the sumset |J+J| is not too much larger than |J|. Ie. |J+J|< c|J| for some constant c. By containing sets with additive structure in homogenous sets, we show that DFT computation in O(klog^2k) is possible for frequency supports that have additive structure. The proofs use Freiman’s theorem from additive combinatorics; techniques from multi-coset sampling (Venkataramani & Bresler, 2000), and tiling of integers (Newman 1997).

Keywords: Learning, Machine learning
Abstract: Signal Temporal Logic (STL) is a powerful framework for describing the complex temporal and logical behaviour of the dynamical system. Several works propose a method to find a controller for the satisfaction of STL specification using reinforcement learning but fail to address either the issue of robust satisfaction in continuous state space or ensure the tractability of the approach. In this paper, leveraging the concept of funnel functions, we propose a tractable reinforcement learning algorithm to learn a time-dependent policy for robust satisfaction of STL specification in continuous state space. We demonstrate the utility of our approach on several tasks using a pendulum and mobile robot examples.

Keywords: Machine learning, Statistical learning, Randomized algorithms
Abstract: In this paper, we present a new determin- istic method for constructing binary and bipolar mea- surement matrices for compressed sensing, based on BCH codes. It is shown that the proposed construction always requrese fewer measurements than the proposed in [4] , and in several cases, fewer measurements than the proposed in [5].

Keywords: Neural networks, Machine learning, Optimization algorithms
Abstract: In this paper, we develop a reinforcement learning (RL) based controller capable of stabilizing the swirling pendulum, a novel under-actuated two degrees of freedom system with a number of underlying peculiarities such as non-planar inertial coupling, loss of relative degree, multiple stable and unstable equilibria, etc. These properties make the control and stabilization of the swirling pendulum challenging, especially at the unstable equilibrium points in the upper hemisphere. To the best of our knowledge, the stabilization of the Swirling Pendulum at the upper unstable equilibrium points has not yet been solved using linear, modern, or nonlinear control methods. We present a novel controller that is able to stabilize the swirling pendulum at each of these unstable equilibrium points using the Actor-Critic algorithm. We also present a comparative analysis between reinforcement learning (RL) and traditional control methods for the purpose of stabilizing the swirling pendulum at the unstable equilibrium point in the lower hemisphere. Our results demonstrate that the RL-based approach outperforms the conventional controllers, such as PID and Lead Compensator, while stabilizing the swirling pendulum at these equilibrium points.

Keywords: Learning, Statistical learning, Autonomous systems
Abstract: In this work, we formulate a safe human-in-loop reinforcement problem by allowing human experts to specify safe actions during the training. We propose two algorithms, safe Q-learning and partially safe Q-learning, which use constrained action spaces during training to find a safe optimal policy. In the case of partially safe Q-learning, the concept of safety ratio is introduced to provide the agent an ability to explore while safety is guaranteed. The proposed algorithms are corroborated by performing simulation studies for four different environments of varying complexity. It is shown that a partially safe Q-learning approach outperforms safe Q-learning and Q-learning on various tasks.

Keywords: Neural networks, Machine learning, Biological systems
Abstract: The Baculovirus expression vector system (BEVS) is one of the widely utilized platforms for the development of recombinant proteins, virus-like particles (VLPs), and vaccines. A mathematical model tuned with data generated from such systems is critical for its optimization and control. In this work, a mathematical model driven by physics depicting such a cell behavior has been proposed and validated with experiments conducted and subsequently used to study the physics informed neural networks (PINNs). Since the governing equations are found as a set of stiff ordinary differential equations (ODEs), Stiff-PINN, a variant of PINN that is utilized to solve stiff ODEs, is implemented here. Assuming a quasi-steady state for the oxygen concentration, the equation responsible for stiffness, the results are found to be more accurate compared to regular PINN. Such PINN models, once developed, can act as a replacement of ODE solvers having trouble to handle stiff equations of BEVS.

Keywords: Pattern recognition and classification, Fault detection/accomodation, Statistical learning
Abstract: This work considers the application of a hierarchical process monitoring approach to incipient prediction and prognosis of off-spec events in a dynamic simulation- based diacetone alcohol (DAA) production process. The process encapsulates many complexities that are present in industrial processes, such as the presence of recycle, control loops, nonlinearities, etc., and thus can be used to investigate the monitoring performance of data-driven methods. The hierarchical process monitoring approach is based on principal component analysis (PCA) and slow feature analysis (SFA) which are unsupervised learning methods. The approach has the ability to extract relevant features from high-dimensional datasets while also incorporating dynamic variation of the variables in the analysis. The study consists of several scenarios based on step and pulse disturbances in various units of the process. The simulation results show that the hierarchical process monitoring approach accurately predicts the incipient off-spec events well in advance, and also identifies responsible units with sufficient accuracy.

Keywords: Networked control systems
Abstract: Security and privacy in CPS have received considerable attention in the past decade and there is thrust to develop mechanisms to detect attacks, take remedial actions, build attack-resilient systems, and keep information private. While research on security and privacy have produced a large spectrum of results individually, studies that assess the impact of security on privacy, and vice-versa, are fairly limited. Given that the goals, the information availability, and the mechanisms of the attacker and the eavesdropper are different, one may opine that security and privacy of a system are unrelated. Contrary to this, we show that a fundamental connection and trade-off exists between these two notions. We show this in two settings – stochastic setting where noise is used implement privacy, and deterministic setting where we consider the notion of opacity.

Keywords: Optimization algorithms, Numerical algorithms, Linear systems
Abstract: Opacity is a notion that describes an eavesdropper's inability to estimate a system's 'secret' states by observing the system's outputs. In this paper, we propose algorithms to compute the minimum sparse perturbation to be added to a system to make its initial states opaque. For these perturbations, we consider two sparsity constraints - structured and affine. We develop an algorithm to compute the global minimum-norm perturbation for the structured case. For the affine case, we use the global minimum solution of the structured case as initial point to compute a local minimum. Empirically, this local minimum is very close to the global minimum. We demonstrate our results via a running example.

Keywords: Delay systems, LMIs, Stability of linear systems
Abstract: This paper is on the stability analysis of linear system with time-varying delay. Different from the augmented and multiple integral type Lyapunov-Krasovskii functional, a new delay-product type Lyapunov functionals is proposed by utilizing non-orthogonal polynomial based integral inequality. Using this new functional, a stability criteria is being derived in the form of linear matrix inequality. Effectiveness of the proposed criteria are illustrated through three numerical examples.

Keywords: Delay systems, Linear systems, Algebraic/geometric methods
Abstract: This paper addresses the discretization problem of continuous linear systems in the presence of multiple state delays. A discretization approach is suggested for obtaining the discrete model of multiple time delay systems using linear interpolation. The proposed discretization method employs linear interpolation and zero-order hold (ZOH) to approximate delayed states with different sampling periods. A numerical example illustrates the efficacy of the proposed approach at the end of paper.

Keywords: Switched systems, Stability of linear systems, Stability of hybrid systems
Abstract: This paper deals with stabilization of discrete-time switched linear systems whose switching signals obey pre-specified restrictions on admissible switches between the subsystems and admissible dwell times on the subsystems. We report a restricted switching counterpart of the so-called min switching signals and two sets of sufficient conditions on the subsystems matrices, viz., restricted switching counterparts of the classical Lyapunov-Metzler inequalities and S-procedure characterization, under which these switching signals are stabilizing. We also study algebraic relation between the above two sets of stability conditions. The main apparatuses for our analysis are Lyapunov stability theory, directed graphs and matrix inequalities.

Keywords: Estimation, Kalman filtering, Optimization
Abstract: In this work, we consider Kullback-Leibler divergence (KLD) based sensor placement design problem where Kalman filter is used to estimate the states of linear dynamical systems. Use of KLD enables sensor placement design while directly incorporating the end-user specified estimation accuracy, and is applicable to both Gaussian and non-Gaussian estimates case. These features are absent in the currently used design approaches. In particular, in the current work, we propose the optimal sensor placement to be the one that minimizes the KLD based distance of the estimation error density function (at large time instant) from the user specified reference density function. Application on case studies and comparison with existing design approaches demonstrates the utility of the proposed approach.

Keywords: Predictive control for linear systems, Optimization, Output regulation
Abstract: This study proposes an improved model predictive control (IMPC) for voltage regulation of a practical DC-DC boost converter (DBC). This paper develops the design of discrete-time (DT) DBC subject to parametric, load, and input voltage variations. An improved model-based predictive control is proposed based on a linearized model of DBC around the nominal operating point. The difference between the actual output from the DBC and the output from the mathematical model is utilized to include a new corrective term in the predicted output, which is further used to generate an optimal control sequence while considering constraints. The validation of the proposed control is presented via simulation results.

Keywords: Computational methods
Abstract: Design of a plant and its controller has significant ramifications on the overall system performance. The traditional sequential approach of first optimizing the plant’s physical design and then the controller may lead to sub-optimal solutions. This is due to the interdependence between the physical design and control parameters through the dynamic equations. Recognition of this fact paved the way for investigation into the “Control Co-Design” research theme wherein the overall system’s physical design and control are simultaneously optimized. In this talk I will talk about the origin of the control co-design problem, formulation of the control co-design optimization problems based on different system properties and challenges involved in computing its solution. I will also discuss different solution procedures for solving the control co-design problem along with their assumptions and guarantees.

Keywords: Optimization algorithms, Optimization, Estimation
Abstract: In this paper, we investigate the idea of employing the momentum technique in the iteratively preconditioned gradient-descent (IPG) algorithm with the aim of an improved performance than our previous results. Three formulations are proposed utilizing different momentum terms. A convergence proof is presented for each formulation, providing sufficient conditions for the parameter selections leading to a linear convergence rate. The proposed optimization approaches are applied in the moving horizon estimation (MHE) framework for a unicycle mobile robot location estimation example. The simulation results confirm that the total number of iterations can be reduced when introducing the momentum terms into the original IPG approach.

Keywords: Systems biology, Computational methods, Algebraic/geometric methods
Abstract: Groebner Basis methods offer attractive alternatives to analytically compute solutions of systems of polynomial equations in a variety of contexts including biomolecular folding. However, the increase in size of the Groebner Basis as the number of polynomials increases may be challenging and the growth for specific classes of polynomials is generally unclear. Here, we addressed this growth in a simple model for the folding of a biomolecule based on a chain of N beads linked rigidly and which could interact with each other using spring-like forces. We expressed the conditions for equilibria as systems of polynomial equations. We used the Groebner Basis methods to obtain explicit solutions for N = 3, 4, and 5. We noted that the maximum size of the Groebner Basis as a function of the polynomial system size was like a power law for N ≤ 10. The rigorous solutions obtained compare favorably with numerical solutions. These results should help in understanding biomolecular folding landscapes.

Keywords: Chaotic systems, Information technology (IT) systems, Optimization
Abstract: In this paper, chaotic signal based hashed key generation for various encryption/decryption algorithms is investigated. Due to the merits of chaotic systems in secure communication, this paper proposes (i) chaotic sequence generation using Lorenz system and (ii) hashed chaotic key generation using Secure Hash Algorithm (SHA-256). Further, DES, triple DES, Blowfish and AES algorithms are used to encrypt/decrypt the message signal with the generated hashed chaotic key. To study the performance, simulation is conducted by considering (i) plaintext and (ii) ECG signal as the input signal. The performance of the encryption/decryption algorithms are compared using computational speed, encryption ratio and throughput.

Keywords: Quantum control, Optimization algorithms, Nonlinear systems
Abstract: NMR spectroscopy is all about manipulating the nuclear spin of an atom, and we use radio frequency (RF) pulses for this. This paper introduces the TOPS algorithm, which focuses on phase manipulation in RF pulse design. It addresses the requirement for selective excitation/inversion in MRI and protein resonance experiments and demonstrates the application of the TOPS-2 algorithm in designing phase-modulated frequency selective pulses. The paper presents the algorithm for designing pulses using the TOPS approach to achieve specific objectives such as selective inversion and excitation. Experimental validation of the selective pulse sequence is performed using a sample consisting of 0.5% H_2O in 99.5% D_2O.

Keywords: Optimization, Uncertain systems, Optimization algorithms
Abstract: The uncertain nature of wind reduces the ability to produce rated power from a wind farm on a sustainable basis. Such wind nature variations are generally ignored by the wind farm practitioners while designing the layout thereby leading to overestimation of rated power calculations. These variations in wind are considered in the form of a probability mass function of speed and direction, known as Wind Frequency Maps (WFMs), which have a direct impact on the power production of wind farm. To determine the true capacity of wind farm under uncertainty, several WFMs depicting different wind scenarios need to be considered. Therefore, in this work, a robust optimization formulation has been formulated which can find the layout of wind farm that is immune to uncertain scenarios expressed through different WFMs. The worst-case robust formulation is solved using the Bayesian optimization framework which balances the exploration and exploitation of decision variable space and fuses a best guess with characterization of considered uncertainty. The results have shown the effectiveness of Bayesian optimization to optimize the layout of wind farm for various number of turbines with lower computational costs.

Keywords: Modeling and simulation, Optimization, Control applications
Abstract: Recent years have witnessed evolution of home energy management systems (HEMS) that allow end-use energy consumers to adjust their consumption in response to the needs of the wider energy infrastructure. In the context of electricity networks, this has lead to algorithm development for electricity cost reduction and grid support provision by load shifting or load curtailment or other load control. Such algorithms require models of appliance power profiles; a common approach is to approximate the power profile of an appliance by its rated or average power. However, the actual consumption patterns of any appliance may vary continuously during its operation. This paper presents a model that improves upon the existing state-of-art by introducing the notion of cycle of operation with power consumption varying across cycles. The paper also describes how such models can be incorporated into HEMS algorithms for cost-and-comfort optimization. Results demonstrating the reduction in cost and peak load as well as improvement in the total load profile are presented to demonstrate the usefulness of such representations.

Keywords: Identification, Kalman filtering, Adaptive systems
Abstract: Online estimation of dynamic model parameters using recursive techniques is at the heart of any adaptive control scheme. The majority of the available recursive parameter estimation schemes update model parameters at the same rate as that of the model time update. In many continuously operated processes, however, the system operates at one operating point for a considerable time before setpoints changes are made. Thus, the model parameter changes occur at a significantly slower rate when compared to the model time update. In this work, we develop a shifting window-based block recursive parameter estimation scheme for tracking parameters of a MISO ARMAX model. The variation of the model parameters is modeled as a slow rate random walk process over shifting time windows. Since ARMAX is a nonlinear-in-parameter model, iterated extended Kalman filter (IEKF) algorithm available in the literature for dealing with nonlinear measurement models is adopted to develop a Block IEKF for tracking the ARMAX model parameters. The efficacy of the proposed approach is demonstrated by a simulation study carried out on an artificial system with time-varying model parameters and experimental data obtained from the benchmark quadruple tank system. Analysis of the simulation and experimental results reveals that the proposed Block IEKF scheme, with a judicious tuning of the random walk model, is able to track time-varying parameters of ARMAX models quickly.

Keywords: Sensor fusion, Estimation, Filtering
Abstract: This paper presents a passive-ranging algorithm with a team of aircraft operating with infrared search and track (IRST) systems. The angle-only track output from each aircraft is shared across other aircraft, and the passive ranging algorithm is developed. A nonlinear least squares problem is formulated with the associated angle-only tracks from each aircraft to estimate the target range. A heterogeneous angle-only track-to-track association algorithm is presented and a detailed fusion architecture is presented for multi-target angle-only track-to-track association. Simulations are carried out to check the robustness of the proposed algorithm for passive ranging.

Keywords: Identification, Estimation, Power systems
Abstract: Many critical infrastructures depend on the reliability of electronic circuits, whose health depends on the values of internal circuit elements that degrade over time. It is therefore important to estimate the values of internal circuit elements from external measurements and monitor them for changes. The challenge to doing so is that, typically, the internal voltages and currents are not accessible for measurement. Prior work on internal circuit element monitoring uses data obtained from sensors placed inside the circuit or requires additional circuitry to be added to the given system, both of which are undesirable. We propose a new multi harmonic frequency-domain based system identification method that is non-intrusive. It exploits the latent non-idealities already present in the system, specifically harmonics in the measured input and output signals. We show how sensitivity analysis can be used to both assess the accuracy of the method as well as to optimize the choice of harmonics to use. We use as a motivation and illustrative example the problem of determining the equivalent series resistance and capacitance in a class of dc-link capacitance circuits in Adjustable Speed Drives. We assess the performance of this method using simulations, and establish its validity by experimental results.

Keywords: Estimation, Kalman filtering, Automotive
Abstract: Accurate position estimation, especially in indoor environments, presents a formidable challenge despite significant advancements in estimation methodologies. The integration of additional sensors is often imperative to refine estimates, particularly within indoor settings. The choice of additional sensors depends on the specific applications and operational range. This study focuses on devising a position estimation framework utilizing the internal sensors such as the Inertial Measurement Unit (IMU) and wheel encoders within a scaled-down Electric Vehicle. The proposed approach entails a combination of crafting a digital filter and implementing a Kalman filter to effectively attenuate noise and errors in the IMU data, thereby enhancing the position estimate, and the methodology is validated through experiments. This work contributes to the broader discourse on advancing indoor position estimation while bypassing the need for external positioning systems.

Keywords: Estimation, Learning
Abstract: Off-road vehicles typically have to navigate very rough terrain environments. In the case of military off-road vehicles, terrain environments could be extreme. Accurately estimating terrain is critical for these vehicles' safe and efficient navigation. It is also essential for optimizing energy consumption and minimizing stress on the mechanical components. This paper provides a statistical model approach for terrain profile estimation, i.e., the Gaussian Mixture Model. The approach involves the observation of key data (terrain elevation (height), soil moisture content, stress at tire contact area, and soil particle size) for estimating the terrain profile. It uses the maximum likelihood estimation for mixtures of Gaussian models. We obtain the Gaussian mixture model parameters using the training data, which helps infer the most probable terrain profile from the test data. The simulation results provide the effectiveness and accuracy of the proposed method in the paper.

Keywords: Networked control systems, Stability of linear systems, Stability of hybrid systems
Abstract: This paper deals with the design of scheduling logics for networked control systems (NCSs) whose communication networks have limited capacity and are prone to data losses. Our contributions are twofold. First, we present a probabilistic algorithm to generate a scheduling logic that under certain conditions on the plant and the controller dynamics, the capacity of the network and the probability of data losses, ensures stochastic stability of each plant in the NCS. Second, given the plant dynamics, the capacity of the shared communication network and the probability of data losses, we discuss the design of state-feedback controllers such that our stability conditions are obeyed. A numerical example is presented to demonstrate the results reported in this paper.

Keywords: Control of networks, Cooperative control
Abstract: Achieving the desired rate of convergence (RoC) is an important aspect while designing the stabilizing controllers. Particularly, in multi-agent systems where the control law relies on the relative information among neighboring agents, it is challenging to maintain the desired RoC due to the vulnerability of the communication channels to malicious attacks. This paper addresses this issue for the multi-agent consensus in both leaderless and leader-follower configurations while considering the presence of false data injection (FDI) attack on actuator. The paper proposes a model-free attack detection method, which relies on exponential stability and continuous monitoring of the Lyapunov function. An attack is detected when the system violates the desired decay function, and it is mitigated by leveraging the time-varying nature of the control gain. Different from actuator saturation that might also occur due to an attack, the issue of control saturation is addressed. The results are verified with numerical simulations.

Keywords: Networked control systems, Estimation, Linear systems
Abstract: Networked control systems are control systems with a wireless network back-end that enables seamless exchange of data between the individual subsystems. However, the wireless channel is inherently imperfect in nature and may often experience data losses, yet most literature overlooks the underlying causes of the same. Fading effect, among many other factors, stands out as a prominent cause with its distinct characteristics, including the possibility of correlated packet drops. Traditional methods are inadequate to analyze or solve the problem of state estimation in the presence of these correlated packet drops. In order to overcome the limitation of traditional methods and to enable a comprehensive analysis, we present a dedicated algorithm for a deterministic channel case which is basically derived from a more generalized theorem in [1] and our principal contribution lies in the development of a real-time transceiver system and the creation of a test-bed utilizing software-defined radios (SDRs) to replicate correlated losses induced by fading effect. With this emulator, we have tested the modified estimation algorithm presented and also we have analyzed its effectiveness in dealing with such drops.

Keywords: Networked control systems, Chaotic systems, Control of networks
Abstract: This paper investigates the impulsive control-based leader-follower synchronization problem of a delayed complex dynamical network. This network is assumed to have a leader-follower configuration with the presence of a power leader. The proposed methodology based on contraction theory derives the efficient synchronization condition for the complex delayed dynamical network with a power leader such that all follower systems (delayed or non-delayed) exponentially synchronize their states to the power leader through local interactions. The research demonstrates that synchronization can be achieved effectively by applying control impulses to only a small fraction of nodes within the network. The effectiveness of the proposed control algorithm is demonstrated through simulation.

Keywords: Estimation, Linear systems, Networked control systems
Abstract: Networked control system (NCS) is a composite technology of control system and communication network where each element of the system interconnected by a common network back-end. However, as these systems often depend on unreliable communication channels, it may lead to packet drops during the transmission of measurement or control data. If we identify fading as the cause of packet drops, it implies that these drops may be correlated in time. This phenomenon poses a major challenge for accurate estimation of state variables, which is essential for ensuring the stability and performance of the system. This necessitates the estimation algorithms to be modified to incorporate this additional information to ensure accurate assessment. In this work, a modified state estimation algorithm is developed which can be utilized for the analysis of estimation error covariance in the presence of correlated packet drops. Also, bounds on error covariance in terms of channel condition is derived. The algorithm is validated through simulations and an experimental evaluation has been done using a mobile robot platform.

Keywords: Networked control systems, Stability of nonlinear systems, Stochastic systems
Abstract: This paper proposes dynamic updates in stochastic backstepping control for a networked system. The system is prone to network uncertainties such as packet loss and transmission delay. These uncertainties introduce stochasticity in the system. The dynamic updates and generic triggering conditions for each subsystem are formulated to ensure system stability. The neural networks are used to precisely approximate stabilizing functions and control input because each subsystem dynamics has stochastic functions. The triggering conditions are further used to obtain generic expressions for permissible value of round-trip data packet losses and delay between consecutive triggers for each subsystem. A comprehensive analysis of proposed design is done for trajectory tracking by a networked system prone to uncertainties.

Keywords: Emerging control applications, Identification, Optimization algorithms
Abstract: Effective containment of spreading processes such as epidemics requires accurate knowledge of several key parameters that govern their dynamics. In this work, we first show that the problem of identifying the underlying parameters of epidemiological spreading processes is often ill-conditioned and lacks the persistence of excitation required for the convergence of adaptive learning schemes. To tackle this challenge, we leverage a relaxed property called initial excitation combined with a recursive least squares algorithm to design an online adaptive identifier to learn the parameters of the susceptible- infected-susceptible (SIS) epidemic model from the knowledge of its states. We prove that the iterates generated by the proposed algorithm minimize an auxiliary weighted least squares cost function. We illustrate the convergence of the error of the estimated epidemic parameters via several numerical case studies and compare it with results obtained using conventional approaches.

Keywords: Agents-based systems, Stochastic systems, Optimization algorithms
Abstract: Oberman gave a stochastic control formulation of the problem of estimating the convex envelope of a non-convex function. Based on this, we develop a reinforcement learning scheme to approximate the convex envelope, using a variant of Q-learning for controlled optimal stopping. It shows very promising results on a standard library of test problems.

Keywords: Learning, Iterative learning control, Optimal control
Abstract: We revisit the Reinforce policy gradient algorithm that works with full cost returns obtained over random length episodes. We propose a new Reinforce type algorithm that estimates the policy gradient using a function measurement over a perturbed parameter using a smoothed functional based gradient estimator. We observe that even though we estimate the gradient of the performance objective using sample performance (and not the sample gradient), the algorithm converges to a neighborhood of a local minimum. We further describe the main convergence result

Keywords: Agents-based systems, Optimization algorithms, Networked control systems
Abstract: We develop a novel proof strategy for deriving almost sure convergence of momentum-based distributed stochastic approximation (DSA) schemes. Popular momentum-based schemes such as Polyak's heavy-ball and Nesterov's Accelerated SGD can be analyzed using our template. Our technique enables us to do away with three restrictive assumptions of existing approaches. One, we do not need the communication matrix to be doubly stochastic. Two, we do not need the noise to be uniformly bounded. Lastly, our approach can handle cases where there are multiple or non-point attractors. As an application, we use our technique to derive convergence for momentum-based extensions of the multi-agent TD(0) algorithm, where the above restrictive assumptions do not hold.

Keywords: Estimation, Stochastic systems, Optimization algorithms
Abstract: We consider the problem of sequentially learning to estimate, in the mean squared error (MSE) sense, a Gaussian K-vector of unknown covariance by observing only m < K of its entries in each round. We first establish a concentration bound for MSE estimation We then frame the estimation problem with bandit feedback, and we propose a variant of the successive elimination algorithm. We also derive a minimax lower bound to understand the fundamental limit on the sample complexity of this problem.

Keywords: Optimization algorithms, Randomized algorithms, Numerical algorithms
Abstract: In this paper, we study the well-known "Heavy Ball" method for convex and nonconvex optimization introduced by Polyak in 1964, and establish its convergence under a variety of situations. Traditionally, most algorithms use "full-coordinate update," that is, at each step, every component of the argument is updated. However, when the dimension of the argument is very high, it is more efficient to update some but not all components of the argument at each iteration. We refer to this as "batch updating" in this paper.

When gradient-based algorithms are used together with batch updating, in principle it is sufficient to compute only those components of the gradient for which the argument is to be updated. However, if a method such as backpropagation is used to compute these components, computing only some components of gradient does not offer much savings over computing the entire gradient. Therefore, to achieve a noticeable reduction in CPU usage at each step, one can use first-order differences to approximate the gradient. The resulting estimates are biased, and also have unbounded variance. Thus some delicate analysis is required. In this paper, we establish the almost sure convergence of the iterations to the stationary point(s) of the objective function under suitable conditions when either noisy of approximate gradients are used.

Keywords: Stochastic systems
Abstract: The coming decades will see the large-scale deployment of networked cyber–physical systems to address global needs in energy, water, health care, transportation, etc. However, as recent events have shown, such systems are vulnerable to cyber-attacks. We present a general-purpose technique, called "dynamic watermarking," for detecting malicious activity in networked systems of sensors and actuators. It provides a provable guarantee of detection of any non-zero power attack. This method has been implemented in several systems of interest. We present the results of attacks on an autonomous automobile, a grid-tied photovoltaic system, a process control system, and a tethered helicopter. We also present the results of simulation studies of attacks on larger-scale systems such as the power grid through attacks on its automatic gain control loop, and attacks on the Tennessee-Eastman model, an open-source benchmark that has been developed for the purpose of evaluating process control technology used in industries such as chemical plants and oil refineries.

Keywords: Process control, Identification, PID control
Abstract: The flow through a long pipe (Dead time process) is most widely used for the transport of oil and gas for a long distance through a pipe line and in need of most efficient flow control in industries. Many controllers like Proportional-Integral (PI), Feedback plus feed forward, Fuzzy logic, internal model controller (IMC), internal model based Proportional Integral and Derivative (PID) controller are studied to control the volumetric flow rate through a pneumatic valve or through conical tank or a different geometry. However, there is not much work reported on the flow control through a long pipe. In a dead time process, the outlet flow rate measurement is not known immediately for the necessary control action. Due to this, it may lead to instability in the output control. In the present work, the Identification of flow process is done by open loop step test through experiment. The first order with dead time transfer function is obtained for a flow through a long pipe. The design of the PID controller is carried out by advanced model-based controller. The closed loop experimental results are compared for various set points for flow rate, i.e., at low, medium and high flow rates. It shows that model based PID controllers have resulted in faster response than conventional PID controller. Due to the limitation in built in software of flow rate control, the time constant (τ1) of model-based controller is taken as zero. Better response is expected with non-zero value of τ1.

Keywords: Cooperative control, Autonomous systems, Networked control systems
Abstract: In this experimental work, we implement the control design from existing work (reference [1] in the manuscript) on a swarm of Crazyflie 2.1 quad-copters by deriving the original control in terms of variables that are available to the user in this practical system. A suitable model is developed using the Crazyswarm2 package within ROS2 to facilitate the execution of the control law. We also discuss various components that are part of this experiment and the challenges we encountered during the experimentation. Extensive experimental results, along with the links to the YouTube videos for actual Crazyflie quad-copters, are provided.

Keywords: Cooperative control, Agents-based systems, Control applications
Abstract: In this paper, a collective initial excitation (C-IE) based distributed adaptive coverage control algorithm for a mobile sensor network (MSN) is experimentally verified. The information exchange between agents is leveraged to collectively achieve the C-IE condition for parameter convergence. The C-IE condition is an extension of the previously established IE condition to the multi-agent setting. The C-IE has been shown to be milder than the collective persistence of excitation (C-PE), which is considered the state-of-the-art for parameter convergence in a multi-agent setting. In this work, we present the first-of-its-kind composite control law that guarantees exponential convergence of the combined cost of coverage and parameter estimation. The experimental results involving multiple drones further validate the efficacy of the proposed concept.

Keywords: Estimation, Optimal control, Modeling and simulation
Abstract: In the present study, dynamic modeling, optimal operation, and control of a reactive batch distillation process is illustrated based on experimental and simulation studies using methyl acetate production case study. An equilibrium stage model, incorporating non-ideal VLE and start-up region of heating, is developed based on data generated on an experimental unit by identifying five input parameters representing uncertainties. The optimal control problem is solved using genetic algorithm, considering the objective function identified on the basis of trend analysis using the developed dynamic model, to find the optimal reflux ratio, heat input to the reboiler, and mole ratio of methanol to acetic acid in the initial reaction mixture. Open-loop and closed-loop implementations clearly illustrate the improved performance with respect to the quantity of methyl acetate in the distillate product with a reasonably high conversion and product purity within reasonably short batch duration, illustrating the successful optimal control implementation experimentally.

Keywords: Process control, PID control, Time-varying systems
Abstract: A real time control system is developed for a reactive batch distillation for the esterification of n-butanol with acetic acid catalyzed by poly (o-methylene p-toluene sulfonic acid) (PTSA-POM) to produce n-butyl acetate. Firstly, experiments are conducted to evaluate the effects of manipulated variables, namely reboiler duty and reflux ratio on the performance parameters of acetic acid conversion, purity, yield and time of operation of the run. The top product stream is a ternary azeotrope and its temperature remains constant until the components of the azeotrope are present in the column. The temperature of the reboiler is a variable trajectory and is chosen as the controlled variable and the reboiler duty as the manipulated variable. Based on the results from effect of operating parameters, set point trajectory is generated for the best operating parameters as a function of time, and closed loop operation is performed using a feedback loop with a PI controller for its tracking. The top temperature loop is maintained as open-loop operation at the best reflux ratio. Data acquisition and control is carried out on LABVIEW platform. The column performance is also evaluated in the presence of load disturbance and an alternate set point trajectory and the control system is found to track the trajectories well and also resulted in column performance better than in open-loop operation.

Keywords: Sensor fusion, Pattern recognition and classification, Identification
Abstract: Utility management is essential to a community’s daily operations. Water, electricity, gas and other essential utilities must be measured, monitored and distributed fairly to all end users. This is particularly important in Water Distribution Networks (WDNs). This is clearly visible when we look at the large variation in the per capita water usage between different countries, especially developed and developing countries. The amount of water delivered at various places within a WDN is influenced by a variety of factors such as distance between the end node and the source reservoir, the elevation in relation to the source, the size of the pipeline, the volume loss caused by silt buildup and scaling, leaks, etc. Therefore, proper WDN monitoring is essential for identifying any problems with the distribution of water to each end user fairly. Non-intrusive measurement techniques are one of the most crucial aspects of WDN monitoring. In this work, we describe and report results from a field experiment that uses non-intrusive measuring.

Keywords: Autonomous systems, Control applications
Abstract: This paper presents a comprehensive study of a scaled-down Electric Vehicle designed for investigating vehicle dynamics in connected and automated environments. With an emphasis on achieving autonomy, the paper focuses on assessing the vehicle's manoeuvring capabilities as a foundational step. The experiments include evaluations of 45^circ parking and parallel parking manoeuvres to ascertain steering capabilities and limits at varying speeds and position estimation for vehicle localization. These experiments provide critical insights into the operational boundaries and manoeuvring potential of the scaled-down Electric Vehicle. The findings are instrumental in advancing the vehicle's autonomous capabilities, marking a significant stride towards realizing highly precise and efficient autonomous transportation systems. The results serve as a valuable resource for autonomous vehicle developers, offering a comprehensive understanding of manoeuvrability in real-world scenarios.

Keywords: Autonomous systems, Neural networks, Control applications
Abstract: Behaviour cloning or imitation learning is a popular technique to navigate mobile robots where they try to imitate expert trajectories to move autonomously. However, existing imitation learning methods for autonomous navigation are weak in perceiving the surroundings. Since perception plays an important role in autonomous navigation, current imitation learning methods have shown some limitations. In this paper, We propose a novel state representation learning-based framework for perception and also design a deep neural network-based controller for behaviour cloning that will predict low-level control actions to imitate different path trajectories collected under expert human supervision. We have also validated the framework's efficacy by performing experiments on Turtlebot 3.0 in a real indoor environment.

Keywords: Control education, Control applications, Aerospace
Abstract: Developing a lunar rover prototype of the Indian Space Research Organization, known as ``Pragyan", as a practical demonstrator is an intriguing endeavor. In this paper, the design and construction of the Pragyan prototype, intending to showcase its potential as a proof of concept in environments similar to the lunar surface have been explored. The primary goal is to examine the technical specifications and functionalities of the rover prototype. This paper aspires to understand the engineering principles underpinning its design, focusing on mobility, sensor systems, communication capabilities, and energy sources. By analyzing its capabilities, we aim to gain insights into the challenges of creating a prototype suitable for simulating lunar exploration conditions. Like its lunar counterpart, the rover prototype possesses a robust yet lightweight frame, allowing it to navigate challenging terrains effectively. Additionally, we plan to delve into its autonomous and semi-autonomous capabilities, unraveling how it processes data and makes decisions during simulated missions. Finally, we recognize the importance of scientific outreach and its role in inspiring curiosity for students and plan to explore the educational outreach potential of the rover prototype.

Keywords: Control education, Emerging control applications, Aerospace
Abstract: Drones, popularly known as unmanned aerial vehicles (UAVs), are operated remotely or through pre-programmed instructions. Drones are typically equipped with sensors, cameras, and other technology that allows them to perform various tasks, such as surveillance, aerial photography, surveying, crop monitoring, pesticide spraying, and delivery of goods. The primary goal of this paper is to develop a drone with image recognition capabilities complemented by advanced automation features such as a failsafe mechanism, return-to-home-to-home functionality, auto-landing, and the implementation of planned auto missions through a ground control station. Drones operate using a combination of hardware and software, including GPS systems, cameras, and sophisticated algorithms that allow them to fly autonomously or be controlled remotely. This paper leverages computer vision and facial recognition technologies to create a real-time face recognition system. The system is designed to identify individuals by comparing their faces with a database of reference images. The reference images, each associated with a specific name, create facial encodings. The system captures video frames from a first person view (FPV) camera feed and employs the Face Recognition library to detect faces in the frames. The system calculates facial encodings for each detected face and matches them with the reference encodings. If a match is found, the system labels the face with the corresponding name and draws bounding boxes around it. This paper enables automated and accurate face recognition, making it applicable in various scenarios such as security, attendance tracking, and personalized user experiences.