Keywords: Neural networks, Hybrid systems, Identification and estimation
Abstract: Solving partial differential equations (PDEs) with random input parameters via standard numerical schemes such as finite element methods is computationally expensive, especially when high-dimensional random parameters are involved. Evaluation of the failure probability involves massive repeated solving equations, which would be computationally prohibitive via traditional Monte Carlo methods. Using neural networks as a surrogate model can somewhat alleviate computational complexity. However, constructing a relatively accurate neural network requires a substantial number of labeled data for training. In this paper, we propose a new mixed residual hybrid (MRH) method for failure probability estimation. On the benefits of absorbing equation form into the loss function of neural networks, none of the labeled data is needed in the training phase. Expensive numerical methods shall not be used unless to correct the outputs in suspicious intervals. Compared to the traditional Monte Carlo method requiring millions of computations, numerical experiments demonstrated the efficiency of the MRH method, which only requires a few thousand calculations.

Keywords: Smart grid, Big data, Smart buildings
Abstract: Numerical weather prediction (NWP) and machine learning (ML) methods are popular for weather forecasting. However, NWP models have multiple possible physical parameterizations, which requires site-specific NWP optimization. This is further complicated when regional NWP models are used with global climate models, each with multiple possible parameterizations. In this study, a hybrid numerical-statistical approach is proposed and evaluated for four radiation models. Weather Research and Forecasting (WRF) model is run in both global and regional mode to provide an estimate for solar irradiance. This estimate is then post-processed using ML to provide a final prediction. Normalized root-mean-square error from WRF is reduced by up to 40-50% with this ML error correction model. Results obtained using CAM, GFDL, New Goddard and RRTMG radiation models were comparable after this correction, negating the need for WRF parameterization tuning. Other models incorporating nearby locations and an ensemble set-up are also evaluated, although they produced much smaller improvements.

Keywords: Neural networks, Identification and estimation, Modeling and identification
Abstract: For centuries, the process of formulating new knowledge from observations has driven scientific discoveries. With rapid advancements in machine learning, it is natural to question the possibility of automating knowledge discovery in the scientific field. A benchmark task for automated knowledge discovery is called symbolic regression. The aim of the task is to predict a mathematical equation that best describes the observational data. The advancements in symbolic regression have a major potential to aid research in understanding the dynamics and governing properties of unexplored systems. However, the combinatorial nature of the task makes it an expensive and hard problem to solve efficiently. There are several types of algorithms for symbolic regression, from genetic programming and sparse regression to deep generative models. However, there is no survey that collates these prominent algorithms. Therefore the goal of this paper is to summarize key research works in symbolic regression and perform a comparative study to understand the strength and limitations of each method. Finally, we highlight the challenges in the current methods and future research directions in the application of machine learning in knowledge discovery.

Keywords: Process automation, Image/video analysis, Big data
Abstract: Traffic emissions are known to contribute significantly to air pollution around the world, especially in heavily urbanized cities such as Singapore. It has been previously shown that the particulate pollution along major roadways exhibit strong correlation with increased traffic during peak hours, and that reductions in traffic emissions can lead to better health outcomes. However, in many instances, obtaining proper counts of vehicular traffic remains manual and extremely laborious. This then restricts one’s ability to carry out longitudinal monitoring for extended periods, for example, when trying to understand the efficacy of intervention measures such as new traffic regulations (e.g. car-pooling) or for computational modelling. Hence, in this study, we propose an integrated machine learning pipeline that utilizes traffic images to obtain vehicular counts that can be easily integrated with other measurements to facilitate various studies. We verify the utility and accuracy of this pipeline on an open-source dataset of traffic images obtained for a location in Singapore and compare the obtained vehicular counts with collocated particulate measurement data obtained over a 2-week period in 2022. The roadside particulate emission is observed to correlate well with obtained vehicular counts with a correlation coefficient of 0.93, indicating that this method can indeed serve as a quick and effective correlate of particulate emissions.

Keywords: Data analytics., Neural networks, Intelligent systems
Abstract: Creating an aerodynamic shape, like an airfoil wing, requires many factors to be considered, especially aerodynamic properties such as its lift-to-drag ratio (L/D). Currently, generating feasible airfoil shapes usually requires computationally expensive tools, such as Computational Fluid Dynamics (CFD). In recent years, increasing work has been directed to utilizing machine learning algorithms to synthesize accurate airfoil shapes while reducing the required computational cost. Generative Adversarial Network (GAN) is one of many algorithms to see success in airfoil shape optimization and is shown to generate good airfoils given a small set of training examples. This paper focuses on implementing a conditional GAN (cGAN) based framework with various filters for airfoil inverse design problem. By labelling the training dataset with aerodynamic characteristics separated by pre-defined thresholds to lift-to-drag ratio (L/D) and shape area, the class labels will be able to guide the network to generate different classes of airfoils influenced by these characteristics. Together with layers of Savitzky-Golay (SG) filter and B-Spline Interpolation, the developed model was shown to achieve good performance in generating new airfoils. In addition, we explored the viability of adding Wasserstein loss from Wasserstein GAN into the network architecture, forming a cWGAN-GP. Testing results showed that cWGAN-GP was able to achieve better performance for a specific airfoil class.

Keywords: Neural networks, Intelligent automation, Data analytics.
Abstract: Deep learning methods on remote sensing data are an attractive approach in place of human observation for automating recognition of hundreds of thousands of tree species in nature. However, this approach requires a large amount of training data for each species, while actual data are scarce - only a small subset of tree species data can be acquired, notwithstanding the unknown, new species. To overcome the data scarcity challenge, we propose a knowledge-driven transfer learning framework for tree species profiling, where a base model of multitasking graph neural network is trained on synthetic species data, which are generated from the universal botany domain knowledge and limited field measurement data. This base model is then transferred to a new multitasking graph neural network model to train on real tree data of limited availability. Our proposed species recognition framework was tested for profiling tree species by classifying a few species profile parameters and showed a significant improvement (10%) in the prediction accuracy in comparison with deep learning models trained on just real tree data.

Keywords: Man-machine interactions, Human-computer interaction, Human centered systems
Abstract: Brain-computer interfaces (BCI) have been extensively researched to assist people with motor paralysis in controlling external devices such as a robotic limb. However, most BCI systems required participants to focus on a single task, limiting their ability to generate other mental or physical activities. Therefore, people's performance of the BCI-based robotic control in multitasking was discussed, as eight healthy subjects performed motor-related tasks of motor imagery and two-handed balancing ball movement, while simultaneously performing visuospatial attention to asynchronously trigger "drinking" actions of a humanoid robot arm with accuracies of 90% and 87.5%, respectively. The online results indicate that the BCI-based robot control system developed for multi-task conditions has a high potential for human augmentation.

Keywords: Neural networks, Image/video analysis
Abstract: Over the last decade, convolutional neural networks have been a core element in the recent remarkable advances in machine learning, computer vision, and robotics. Vision transformers have recently demonstrated great success in various computer vision tasks, motivating a tremendously increased interest in their deployment into many real-world vision applications. However, until now, the number of successful applications of transformers in robots is somewhat limited. This work presents an approach to visual place recognition using a vision transformer (ViT). ViT trained from scratch, and two pretrained ViTs in base and large versions have been finetuned on a target dataset. The features extracted by transformers have then been used in place recognition using a k-NN. Finally, contrastive learning has been performed to embed the features and improve recognition performance. The algorithm has been evaluated in a dataset for indoor place recognition comprising images with 6-DOF viewpoint variations. Experimental results demonstrate that considerable gain in recognition accuracy can be obtained by finetuned transformers in comparison to results achieved by CNNs.

Keywords: Neural networks, Data analytics., Hybrid systems
Abstract: Depression is one of the most common mental illnesses that affects millions of people around the world. Most people who are depressed are hesitant to reveal their condition to others. The main purpose of this study is to predict the depression of online users using Natural Language Processing (NLP) tools and deep learning techniques. This paper proposes a sentiment classification approach which is a hybrid of Long Short-Term Memory (LSTM) and Temporal Convolution Network (TCN) models. The proposed LSTM-TCN model addresses the limitations of traditional Convolutional Neural Networks (CNNs), i.e., the inability of CNNs to fully capture text features during feature extraction. Furthermore, a single model cannot properly extract deep text features. A publicly available large-scale dataset comprising Twitter data is utilized for experimentation purposes. A comparison analysis is performed on the Twitter dataset, and the accuracy of the text models, CNN and Recurrent Neural Network (RNN), achieved 91.73% and 90.66%, respectively, illustrating that the proposed LSTM-TCN model outperforms traditional single neural networks.

Keywords: Object recognition, Perception systems, Neural networks
Abstract: Robots have limited perception capabilities when observing a new scene. When the objects on the scene are registered from a single perspective, only partial information about the shape of the objects is registered. Incomplete models of objects influence the performance of grasping methods. In this case, the robot should scan the scene from other perspectives to collect information about the objects or use methods that fill in unknown regions of the scene. The CNN-based method for objects reconstruction from a single view utilize 3D structures like point clouds or 3D grids. In this research, we revisit the problem of scene reconstruction and show that scene reconstruction can be formulated in the 2D image space. We propose a new representation of the scene reconstruction problem for a robot equipped with an RGB-D camera. Then, we present a method that generates a depth image of the object from the pose of the camera that is on the other side of the scene. We show how to train a neural network to obtain accurate depth images of the objects and reconstruct a 3D model of the scene observed from a single viewpoint. Moreover, we show that the obtained model can be applied to improve the success rate of the grasping method.

Keywords: Data analytics., Big data, Neural networks
Abstract: In this paper, we propose a Bi-layer Predictionbased Reduction Branch (BP-RB) framework to speed up the process of finding a high-quality feasible solution for Mixed Integer Programming (MIP) problems. A graph convolutional network (GCN) is employed to predict binary variables’ values. After that, a subset of binary variables is fixed to the predicted value by a greedy method conditioned on the predicted probabilities. By exploring the logical consequences, a learning-based problem reduction method is proposed, significantly reducing the variable and constraint sizes. With the reductive sub-MIP problem, the second layer GCN framework is employed to update the prediction for the remaining binary variables’ values and to determine the selection of variables which are then used for branching to generate the Branch and Bound (B&B) tree. Numerical examples show that our BP-RB framework speeds up the primal heuristic and finds the feasible solution with high quality.

Keywords: Big data, Data analytics., Sensor networks
Abstract: In the present study, a novel time and sensor domain data decomposition and analysis framework is proposed to perform fault diagnosis of cutting tools. The problem of excess computation burden existing in multi-sensor data-driven tool condition monitoring (TCM) system is resolved at the data level by compressing raw signals into a significantly smaller set of time and sensor domain data. The utilisation of the time domain components eliminates the influence of environmental noise on raw signals. Meanwhile, the introduction of the sensor domain components reveals the correlation relationship within multiple sensors. Experimental studies are conducted to verify the effectiveness of the proposed approach and illustrate the advantages of the time and sensor domain features compared with raw signal features.

Keywords: Neural networks, Robot control, Network-based systems
Abstract: Collision-free path planning is challenging for mobile robot navigation tasks. Recently, the deep reinforcement learning method provided a more effective way to derive safe and efficient velocity commands directly from raw sensor information. Twin Delayed Deep Deterministic policy gradient (TD3) is an efficient approach for DRL navigation. However, original TD3 exists issues such as inefficiency learning and slow convergence speed which may influence the model to derive an ideal action for mobile robot navigation. To address these issues, we proposed a novel dueling architecture model, dueling deep deterministic policy gradient (Dueling-TD3), we compose the dueling network architecture into the critic network to increase the Q-value estimate precision. The results demonstrate that our proposed model outperforms the original model in terms of route planning capabilities.

Keywords: Neural networks, Identification and estimation, Nonlinear systems
Abstract: We investigate possibilities of effective modeling the distributed-parameter systems using reservoir computing in such a way as to properly reconstruct the spatio-temporal dynamics defined in a given multidimensional domain. The setting examined here corresponds to situations where the locations of sensors and actuators are fixed allowing to delegate the complex system dynamics to a reservoir-based neural network which is trained based on the available measurement data. The proposed approach consists in imposing a proper partitioning of the spatial domain, then a specific structure of an echo state network is used to form the reservoir capable to follow not only temporal but also spatial dynamics of the system. As a result, an extremely effective approximator of the distributed system is obtained with relatively simple training procedure to deliver readout of the system state. Contrary to the classical numerical methods for solving distributed-parameter systems characterized by a large-scale we are able to reduce the dimesionality and computational-cost useful for state prediction and/or estimation in the real-time. The performance of the proposed approach is evaluated by numerical experiments on the transient displacements of the clamped plate.

Keywords: Vision for robots, Perception systems, Mobile robotics
Abstract: Monocular depth estimation is an important task that can be applied to many robotic applications. Existing methods focus on improving depth estimation accuracy via training increasingly deeper and wider networks, however these suffer from large computational complexity. Recent studies found that edge information are important cues for convolutional neural networks (CNNs) to estimate depth. Inspired by the above observations, we present a novel lightweight Edge Guided Depth Estimation Network (EGD-Net) in this study. In particular, we start out with a lightweight encoder-decoder architecture and embed an edge guidance branch which takes as input image gradients and multi-scale feature maps from the backbone to learn the edge attention features. In order to aggregate the context information and edge attention features, we design a transformer-based feature aggregation module (TRFA). TRFA captures the long-range dependencies between the context information and edge attention features through cross-attention mechanism. We perform extensive experiments on the NYU depth v2 dataset. Experimental results show that the proposed method runs about 96 fps on a Nvidia GTX 1080 GPU whilst achieving the state-of-the-art performance in terms of accuracy.

Keywords: Learning and Statistical methods, Identification and estimation, Image/video analysis
Abstract: Various methods for fitting an unknown functions from the set of noisy measurements are applicable to a wide variety of problems. Among them, the nonparametric algorithms based on the Parzen kernel are willingly used. In the article, we propose a novel and very effective numerical simplification in Parzen approach leading to a significant reduction in computation time. The algorithm is basically developed for multidimensional case. The two-dimensional version of the method is explained in details and analysed. Computational complexity and speed of convergence of the algorithm are studied. Some applications for solving real problems with our algorithms are presented.

Keywords: Visual servoing, Perception systems, Vision for robots
Abstract: In this paper, the authors propose a visual-inertial algorithm to estimate the lateral velocity of a motorcycle traveling at high speed along a single-carriageway road. The approach comprises the following steps. First, a monocular camera captures real-time images of the road ahead. Lane markers present in the image are detected and segmented using image processing techniques. Next, a bird’s eye view transform is applied, and the dashed center lane markers are isolated. The motion of these markers is computed using an image registration algorithm and is expressed in the motorcycle body frame using orientation estimates from an inertial measurement unit. Finally, this measurement is combined with readings from an accelerometer using a Kalman filter to produce a filtered estimate. The approach was validated using data from simulations of two scenarios created in the BikeSim simulation software suite. In the first scenario, the motorcycle performs a double lane-change across both lanes of a straight road. In the second, the motorcycle navigates an s-shaped bend.

Keywords: Activity/behavior recognition, Image/video analysis, Face and Gesture.
Abstract: With the world moving towards online communication, the video datastore has exploded in the past few years. Consequently, it has become crucial to analyse participant’s engagement level in online communication videos. Engagement prediction of people in videos can be useful in many domains, like education, client meetings, dating, etc. Video-level or frame-level prediction of engagement for a user involves development of robust models that can capture facial microemotions efficiently. For the development of an Engagement prediction model, it is necessary to have a widely-accepted standard dataset for engagement analysis. DAiSEE is one of the datasets which consist of in-the-wild data and have a gold standard annotation for engagement prediction. Earlier research done using DAiSEE dataset involved training and testing standard models like CNN based models, but the results were not satisfactory according to industry standards. In this paper, a multi-level classification approach has been introduced to create a more robust model for engagement analysis using DAiSEE dataset. This approach has recorded testing accuracies of 0.638, 0.7728, 0.8195 and 0.866 for predicting boredom level, engagement level, confusion level and frustration level respectively.

Keywords: Vision for robots, Mobile robotics, Perception systems
Abstract: Deep Reinforcement Learning (DRL) is highly efficient for solving complex tasks such as drone obstacle avoidance using cameras. However, these methods are often limited by the camera perception capabilities. In this paper, we demonstrate that point-goal navigation performances can be improved by using cameras with a wider Field-Of-View (FOV). To this end, we present a DRL solution based on equirectangular images and demonstrates its relevance, especially compared to its perspective version. Several visual modalities are compared: ground truth depth, RGB, and depth directly estimated from these 360 degrees RGB images using Deep Learning methods. Next, we propose a spherical adaptation to take into account the spherical distortions of omnidirectional images in the convolutional neural networks (CNNs) used in the actor-critic network and show a significant improvement in navigation performance. Finally, we modify the perspective depth estimation network using this spherical adaptation and demonstrate a further performance improvement.

Keywords: Visual servoing, Dexterous manipulation
Abstract: In this paper, we present a complete pipeline for positioning a feature point of a soft object to a desired 3D position, by acting on a different manipulation point using a robotic manipulator. For that purpose, the analytic relation between the feature point displacement and the robot motion is derived using a coarse mass-spring model (MSM), while taking into consideration the propagation delay introduced by a MSM. From this modeling step, a novel closed-loop controller is designed for performing the positioning task. To get rid of the model approximations, the object is tracked in real-time using a RGB-D sensor, thus allowing to correct on-line any drift between the object and its model. Our model-based and vision-based controller is validated in real experiments for two different soft objects and the results show promising performance in term of accuracy, efficiency and robustness.

Keywords: Robot control, Mobile robotics, Distributed optimization and MPC
Abstract: The information provided to a chasing quadrotor (Chaser) about a moving target’s linear velocity during pursuit offers utility beyond simply knowing how fast the target is moving linearly. During such pursuits for the Chaser, external tracking of its linear velocity is necessary to establish stable close-loop control. The alternative, an open-loop system, relies solely on the linear velocity outputs from its dynamics which usually results in unstable flight performances due to imperfections in the model’s fidelity. Thus, this paper presents a Cascading Velocity Model Predictive Control Framework (CVMPC) for the Chaser to leverage on the real-time linear velocity feedback of the target for stable open-loop linear velocity control. By using this information with Gaussian Processes (GPs), the Chaser’s linear velocity outputs from its dynamics are augmented in real-time and cascaded from one control step into the next within a Model Predictive Controller. Simulations in Gazebo further verify the performance of CVMPC against a MPC with external linear velocity tracking (close-loop system) with RMSE in euclidean linear velocity and distance being less than 1m/s and 1m respectively.

Keywords: Mobile robotics, Feature extraction, grouping and segmentation
Abstract: Navigating delivery robot along the sidewalk safely and robustly in a campus environment is extremely challenging due to the narrow motion space, appearance changes and unstable GPS localization signal under canopies of trees, etc. To that end, we have completed systematic implementation for delivery robot sidewalk navigation, where a robust vision based navigation algorithm has been proposed. And it consists of three main modules: sidewalk segmentation, costmap generation and motion planning. More Specifically, the first module is to find the drivable area of the surrounding environment, where an image-based segmentation neural network has been developed to extract where the robot can traverse. Since it only takes as input immediate and local sensory data, thus releasing the high dependence on a prior map. Then, an inverse perspective mapping follows to generate a bird-eye-view of the drivable area and constructs the local occupancy grid map intuitively. Next, two different motion planners, control-based primitives (Dynamic Window Approach) and state-based primitives (state lattice planner), have been adopted to generate a trajectory candidate for navigating the robot along the sidewalk. Both simulation and real-world sidewalk navigation experiments have been conducted to test and evaluate their performance. The results show that our algorithm can precisely extract the sidewalk area for traversing, and the state-based primitive planner demonstrates superior performance in terms of trajectory length and time cost, achieving 14.3% and 18.7% improvement compared with control-based primitive planner.

Keywords: Scene analysis, Mobile robotics, Image/video analysis
Abstract: Prediction of dynamic environment is crucial to safe navigation of an autonomous vehicle. Urban traffic scenes are particularly challenging to forecast due to complex interactions between various dynamic agents, such as vehicles and vulnerable road users. Previous approaches have used ego-centric occupancy grid maps to represent and predict dynamic environments. However, these predictions suffer from blurriness, loss of scene structure at turns, and vanishing of agents over longer prediction horizon. In this work, we propose a novel framework to make long-term predictions by representing the traffic scene in a fixed frame, referred as allo-centric occupancy grid. This allows for the static scene to remain fixed and to represent motion of the ego-vehicle on the grid like other agents'. We study the allo-centric grid prediction with different video prediction networks and validate the approach on the real-world Nuscenes dataset. The results demonstrate that the allo-centric grid representation significantly improves scene prediction, in comparison to the conventional ego-centric grid approach.

Keywords: Perception systems, Intelligent automation, Activity/behavior recognition
Abstract: The majority of sensing systems in current mining procedures are not optimized to capture the high-fidelity inputs for perception systems. This is due to the lack of accuracy, resolution, update rate, and other shortcomings in the quality and quantity of captured data. High-fidelity input from the multi-modal sensing system is a key component in the development of superior perception technologies. These technologies address critical mining concerns such as performance analysis, progress monitoring, and policy verification for safe operating practices.

This paper proposes the non-trivial procedure of proper sensor selection and appropriate deployment of the sensing system to ensure high fidelity multi-modal sensing to serve the above-ground mining applications. Furthermore, the development of perception systems that incorporate this sensing system and machine learning tools is discussed. The mentioned perception systems are mainly used for in-pit visualization, analysis, and decision support applications. Nonetheless, we hope the proposed techniques could be extended to cover complex control applications too. Additionally, we demonstrate the framework for the incorporation of a representative simulated environment, along with the simulated sensors that were chosen through a precise sensor selection analysis, to produce training data for machine learning models. The models are made to be used for processing the real inputs produced by the physical Mobile Sensing Trailer (MST), which is installed in the mining environment. To the best of our knowledge, such simulation environment and its components did not exist or were not accessible before.

Keywords: Perception systems, Robot sensing and data fusion, Vision for robots
Abstract: Semantic grids are a succinct and convenient approach to represent the environment for mobile robotics and autonomous driving applications. While the use of Lidar sensors is now generalized in robotics, most semantic grid prediction approaches in the literature focus only on RGB data. In this paper, we present an approach for semantic grid prediction that uses a transformer architecture to fuse Lidar sensor data with RGB images from multiple cameras. Our proposed method, TransFuseGrid, first transforms both input streams into top-view embeddings, and then fuses these embeddings at multiple scales with Transformers. Finally, a decoder transforms the fused, top-view feature map into a semantic grid of the vehicle's environment. We evaluate the performance of our approach on the nuScenes dataset for the vehicle, drivable area, lane divider and walkway segmentation tasks. The results show that TransFuseGrid achieves superior performance than competing RGB-only and Lidar-only methods. Additionally, the Transformer feature fusion leads to a significative improvement over naive RGB-Lidar concatenation. In particular, for the segmentation of vehicles, our model outperforms state-of-the-art RGB-only and Lidar-only methods by 24% and 53%, respectively.

Keywords: Localization, navigation and mapping, Mobile robotics, Perception systems
Abstract: In this work, C-TM is presented: a method to build a topo-metric map for delivery robot navigation in large-scale city environments. This system automatically generates a compact map by only saving expensive LIDAR information at key locations. These locations form the nodes of a topological map. Nodes are identified using a Place-Categorization (PC) neural network which output the place category from RGB cameras. Inside nodes, we generate and save high quality LIDAR submaps. Global localization within the map is done with a Visual-Place-Recognition (VPR) neural network. The topo-metric map can be used for navigation on footpath. We deploy C-TM on a four-wheeled autonomous delivery robot and test the effectiveness in two environments, both day and night.

Keywords: Vision for robots, Perception systems, Robot sensing and data fusion
Abstract: Semantic grids are a useful representation of the environment around a robot. They can be used in autonomous vehicles to concisely represent the scene around the car, capturing vital information for downstream tasks like navigation or collision assessment. Information from different sensors can be used to generate these grids. Some methods rely only on RGB images, whereas others choose to incorporate information from other sensors, such as radar or LiDAR. In this paper, we present an architecture that fuses LiDAR and camera information to generate semantic grids. By using the 3D information from a LiDAR point cloud, the LiDAR-Aided Perspective Transform Network (LAPTNet) is able to associate features in the camera plane to the bird's eye view without having to predict any depth information about the scene. Compared to state-of-the-art camera-only methods, LAPTNet achieves an improvement of up to 8.8 points (or 38.13%) over state-of-art competing approaches for the classes proposed in the NuScenes dataset validation split.

Keywords: Intelligent systems, Mobile robotics, Electric vehicles and intelligent transportation.
Abstract: In recent years, following the advances in autonomous driving technology, low-speed autonomous service vehicles such as delivery, patrolling, and road-cleaning vehicles have started to emerge. As a promising future cleaning solution, autonomous cleaning vehicles are expected to address the workforce shortage many countries face in the near future. This paper describes the detailed design of an autonomous research platform for cleaning operations in mixed indoor & outdoor environments. An electric manual vacuum sweeper is retrofitted into an autonomous sweeper equipped with the Drive-by-Wire (DBW) system, computer, sensors, and actuators essential for autonomous driving. A complete autonomous driving software stack is also developed upon this hardware setup to enable the vehicle to navigate itself safely in various challenging operating environments. The system has been extensively tested in different environments on the National University of Singapore (NUS) campus, including private roads, car parks, warehouses, and public plaza areas.

Keywords: Mobile robotics, Robot control, Localization, navigation and mapping
Abstract: Robots are now widely used around humans, in homes and public places like the museums, all due to their many benefits. These autonomous robots are called social or service robots and they always find it difficult to navigate in crowded environments; largely because of the high level of uncertainty in observing and predicting human behaviours in a highly dynamic environment. Uncertainty is propagated during prediction and might grow to levels that renders the whole environment unsafe for the robot leading to the socalled Freezing Robot Problem – FRP. This work presents our proposed approach to proactively account for various uncertainties during the robot’s motion planning using a stochastic Nonlinear Model Predictive Controller (SNMPC). Additionally, using numerical optimization methods enables the planner to compute new control commands in realtime.

Keywords: Feature extraction, grouping and segmentation, Learning and Statistical methods, Scene analysis
Abstract: Deep learning based semantic segmentation models have achieved remarkable results in recent years. However, many deep learning based models encounter the problem of catastrophic forgetting, i.e. when the model is required to learn a new task without labels for old objects, its performance drops significantly for the previous tasks. To solve this problem, an incremental learning method, a Combination of Old Prediction and Modified Label (COPML), is developed in this paper. The proposed method utilizes the prediction results of the old model and the modified labels of the new task to create pseudo labels which are close to the ground truths. By using these pseudo labels for training, the model is expected to preserve the knowledge of old tasks. In addition, knowledge distillation, the replay and parameter freezing strategy are also applied to the proposed method to further assist the model in overcoming catastrophic forgetting. The effectiveness of the proposed method is validated on two semantic segmentation models: Unet and Deeplab3 in Pascal-VOC 2012 dataset and a self-made dataset. The experimental results demonstrate that COPML enables the model to maintain most of the old knowledge while obtaining an excellent performance on a new task.

Keywords: Image/video analysis, Vision for robots
Abstract: Available parking slot detection is the fifirst step for autonomous parking systems. In this paper, we propose a novel parking slot detection method that uses the center information regression and the occupancy classifification of the parking slot. We design a self-calibrated convolutions network (SCCN) to obtain the position, length, occupancy and direction, which can also infer the parking slot type according to the prediction results. The method divides an around view monitor (AVM) image into the 16 × 16 grid cells and performs a SCCN detector for feature extraction. Subsequently, the whole parking slot can be easily inferred via prior geometric information and detection results. We adopt the heatmap, MultiBins, and midline to detect the center keypoint, direction and occupancy, respectively. And we quantitatively evaluate the performance of the proposed method on the public PS2.0 datasets. The experimental results show the outperformance by a precision rate of 99.35%, a recall rate of 99.17%, an occupancy classifification accuracy of 99.12% and all correctly inferred types of parking slots on the datasets.

Keywords: Feature extraction, grouping and segmentation, Image-based modeling, Image/video analysis
Abstract: Deep learning has achieved excellent performance in computer vision tasks, like image recognition, natural language processing, etc. However, in real-world applications, special circumstances brought about by the external world may create domain bias caused by distribution discrepancy between training and testing data, leading to degrading model performance. For example, when auto-driving meets hazy weather, the model performance will drop significantly. In this paper, we explore to solve this problem by utilizing modern Domain Adaptation (DA) methods, which generalizes from the source domain to the target domain by minimizing the distribution difference caused by dataset bias. We firstly propose the cross-domain haze image datasets and benchmark the five classic DA methods. The experiments show that DA methods can mitigate the negative effect of haze and significantly improves the model performance for visual recognition.

Keywords: Image-based modeling
Abstract: Synthetic aperture radar (SAR) ship detection in complex near-shore environment remains a challenging task for traditional deep learning models due to relatively serious missed alarms. To achieve accurate detection of near-shore ships, this paper proposes a two-stage ship target detection algorithm based on Markov random fields and dual-attention network(MRF-DANet). In the first stage, the MRF-DANet proposes to extract the regions of interest (ROI) of ships by fusing multi-segmentations obtained by the MRF and Otsu algorithm, in which the MRF is used to implement the fine segmentation of the near-shore scene while the sea-land being divided by the Otsu. In this way, the MRF-DANet could effectively focus on the possible ships near the coastline and avoid missed alarms. Afterwards, a sequential dual-attention network is designed to extract discriminative deep features in both spatial and channel dimensions for the detection of nearshore ships. Experimental results on the GF-3 AIR-SARShip-1.0 dataset demonstrate that the proposed MRF-DANet performs better than the recent deep learning models in the nearshore ship detection

Keywords: Image/video analysis
Abstract: Due to the significant differences in geometric and nonlinear intensity, multimodal image matching is still a challenging problem. To address this issue, this paper proposes a novel matching method using phase congruency (PC)-based self-similarity structural features for multimodal images. Firstly, the feature points are extracted from the PC maps of the original images by the Harris detector. Then, combined with the theory of the self-similarity, a PC-based self-similarity structural (PCSS) descriptor is designed for multimodal images. Finally, the Euclidean distance is used as the matching measure for the corresponding point recognition. Experimental results conducted on various real multimodal image pairs demonstrate that the proposed method can achieve better matching performance in terms of the number of correct matches and the registration precision in comparison with the traditional methods.

Keywords: Image-based modeling, Feature extraction, grouping and segmentation, Object recognition
Abstract: High-dimensional features extraction and selection is of great significance for synthetic aperture radar (SAR) image change detection. In this paper, a feature selection based on sparse coefficient correlation, abbreviated as SR-PCC, is proposed to realize the local reconstruction of known samples, so as to improve the accuracy of change detection. Firstly, high-dimensional texture features are extracted from real SAR images and then fused by stacking. Secondly, for the known samples, the sparse representation is performed and then the sparse coefficients are obtained. Then, the Pearson correlation coefficient method is used to select sparse coefficients related to the image itself, thus realizing local optimal reconstruction. Finally, the selected features are inputted into the support vector machine (SVM) to realize change detection. Experiments on real SAR images demonstrate the effectiveness of the proposed SR-PCC in high-dimensional feature selection and illustrate that it can provide better change detection maps.

Keywords: Feature extraction, grouping and segmentation, Scene analysis, Image-based modeling
Abstract: Deep learning algorithms have been used on a large scale in high-resolution remote sensing scene classification. However, traditional deep learning models usually suffer from incomplete consideration of spatial features, inadequate extraction of detail and texture features and difficulty in decoding deep features. In order to improve the extraction and generalization ability of convolutional neural networks for detail and texture features, a wavelet attention ResNeXt (WAResNeXt) is designed in this paper. The proposed WAResNeXt firstly extracts the multi-scale detail and texture information of the input feature map by wavelet transform, and then enhances the useful information and suppresses the redundant information by the attention mechanism. Finally, it reconstructs the feature map by the inverse wavelet transform. Experiments on the NWPURESISC45 dataset show that the WAResNeXt can effectively extract the spatial features and the texture features of high-resolution remote sensing images, and can greatly improve the scene classification accuracy.

Keywords: Cyber security in networked control systems, Sensor networks, Control of biological systems
Abstract: First, since time consumption for extracting a will become quite tedious for very large images by pixelwise eigendecomposition, we proposed a fast angle’s solution. Second, a may be not unique that results in the invalidity of H/a decomposition. So we early proposed a sound discriminant to distinguish  is unique or not. Third, since depolarization is serious and  is unstable in high entropy zone, grass and flourishing canopy etc may be misclassified as double bounce scattering. Therefore we proposed a fas algorithm that H/  decomposition is combined with the model-based decomposition, which overcomes the shortcomings of the invalidity of H/ decomposition and the misclassification in high-entropy zone, and its time consumption is much shorter than H/ decomposition.

Keywords: Adaptive control, Nonlinear systems
Abstract: The iterative learning control problem for nonlinear systems without the requirements of Arimoto’s assumptions is investigated in this paper. Typical Arimoto’s assumptions for iterative learning control requires that the initial state error, system parameters, desired trajectory, external disturbance and trial length are all iteration-invariant. When the assumptions are not satisfied, we call the system exists iteration-varying uncertainties. In general, it is a big challenge to design an iterative learning controller for systems with all the iteration-varying uncertainties. In this paper, we transform the iteration-varying uncertainties into an additive-type uncertainty and modify the traditional adaptive law so that a novel adaptive iterative learning controller can be applied to successfully solve the repetitive control problem. Compared with the existing works dealing with a similar issue of iteration-varying uncertainties, this proposed approach can be applied to the more general class of iteration-varying uncertain nonlinear systems and achieve better learning performance.

Keywords: Identification and estimation, Nonlinear systems
Abstract: In this paper, we propose a high order approximation for generalized Caputo fractional derivative of order alpha in (0,1).The approximation order is shown to be (3-alpha) which improves some previous work done to date. We then apply the new approximation to solve a class of generalized time fractional sub-diffusion problem. Some experiments are carried out to demonstrate the accuracy of the proposed methods. The numerical results indicate consistency with the theoretical results and good performance of the methods.

Keywords: Precision motion control, Robust control, Nonlinear systems
Abstract: How well simulation results can be transferred to the real world depends to a large extent on the sim-to-real gap that therefore should be as small as possible. In this work, this gap is reduced exemplarily for a robot with an omni-directional drive, which is challenging to simulate, utilizing machine learning methods. For this purpose, a motion capture system is first used to record a suitable data set of the robot's movements. Then, a model based on physical principles and observations is designed manually, which includes some unknown parameters that are learned based on the training dataset. Since the model is not differentiable, the evolutionary algorithms NSGA-II and -III are applied. Finally, by the presented approach, a significant reduction of the sim-to-real gap can be observed even at higher velocities above 2 m/s. The ablation study also shows that the elements beyond normal simulations, such as the engine simulation, and the machine learning approach, are essential for success.

Keywords: Process control, Precision motion control, Control applications
Abstract: This paper considers the problem of designing of iterative learning control (ILC) laws for linear batch processes. Unlike the majority of existing results about ILC law design for linear batch processes over repetitive/two-dimensional setting where Lyapunov theory is applied, this study is focused on formulating the ILC law design procedures by transforming it into an equivalent problem of (structural) stability analysis for a linear Roesser model for two-dimensional (2D) systems. Then, based on a non-conservative version of stability and stabilization conditions for linear 2D systems, suitable PD-type ILC laws are derived by the application of the linear matrix inequality (LMI) approach. Finally, a numerical example is given to show the validity of the proposed design procedure and some advantages are emphasized when compared to the existing alternatives.

Keywords: Identification and estimation, Nonlinear systems, Hybrid systems
Abstract: A maximum likelihood-based identification algorithm for nonlinear state-dependent switched systems is presented. The data-based modeling of switched systems is in principle more demanding, since assignments of the sampled recordings to their originating subsystems are not given. The resulting identification problem involves latent variables and is therefore solved by an expectation-maximization algorithm. The estimated likelihoods are used to construct the switching condition by a decision tree learning algorithm. The performance of the proposed method is demonstrated by two examples.

Keywords: Control applications, Adaptive control
Abstract: A novel, robust attitude controller for rigid bodies in the presence of time-varying orientation constraints is presented in this paper. Using an error transformation, the dynamic attitude constraints are converted into time-varying quaternion constraints. Subsequently, a robust attitude control law is synthesized using the backstepping philosophy in which barrier Lyapunov functions (BLFs) are used to achieve asymptotic tracking and simultaneously avoid attitude constraint violation. This is accomplished by ensuring the boundedness of BLFs in the closed-loop Lyapunov stability analysis. Besides the nominal scenario, an adaptive control law is also formulated to tackle moment of inertial uncertainties and an unknown, time-varying, bounded disturbance. In this case, the attitude tracking errors are uniformly ultimately bounded, whose bounds can be adjusted by user-defined constants. Note that in both scenarios, the dynamic attitude constraints are not transgressed. Finally, the effectiveness of the proposed controller is demonstrated by carrying out extensive numerical simulations in the presence of parametric uncertainties and disturbance torques.

Keywords: Nonlinear systems, Control applications
Abstract: This study presents optimal control design based on the fractional controller for the Furuta pendulum model with a disturbance on the output. This work solves the task of stabilizing the Furuta pendulum in an upright position. A solution is obtained by applying the two main control systems: integer order, fractional order (different values of order are included in the research). Tuning of the designed control systems is done using genetic algorithm optimization with constraints. The proposed cost is described by a quadratic function based on LQR control design. The comparison of the designed control system is conducted including comparative analysis based on simulations. Overall, the results of simulations reveal that the fractional order controller with the smallest designed order outperforms integer order controller in stabilizing the Furuta pendulum system with the presence of the significant output disturbance.

Keywords: Robust control, Control applications
Abstract: This paper deals with the problem of stability analysis and control scheme synthesis for RLC ladder circuits modeled as two-dimensional (2D) systems, to which class spatially interconnected systems can be associated. This is because in the state-space model more than one indeterminates appear (time - continuous and the coordinates of the node in the structure - discrete). Hence analysis and control synthesis of such systems require developing new results from the systems theory area. As discussed later on due to the finite number of nodes in real systems under consideration, one option (exploited in this paper) for the analysis and synthesis of such systems is embedding temporal and spatial (node to node) dynamics into one 1D, i.e. n=1, model. In this paper, RLC circuits, displayed as a series, i.e. along one spatial (node) indeterminate, constructed with conductances (resistors - R), inductances (L) and capacitors (C), considered are built of electronic elements with values given with some tolerance. This motivates the application of uncertain state-space model with the polytopic type of uncertainty. Then, for such a model, the Linear Matrix Inequalities (LMI) based methodology towards stability analysis and controller design methodology are developed. It actually leads to interesting results from the robust control area. The case study proves the correctness of the proposed methods.