SCHEDULE

Lee, Jongdeog; Lee, Jongkwan

Text-based Voice Codec Algorithm for Tactical Radio Networks in Disconnected, Intermittent, Limited Environment

Abstract

Operations on the battlefield are becoming increasingly dependent on tactical networks as more robots, drones, and mobile devices are deployed. A tactical network is considered a disconnected, intermittent, limited environment, considering adversary attacks on network infrastructure. While tactical radios transmit voice messages over tactical networks, recipients may suffer message delays owing to limited bandwidth. Herein, we propose a new codec algorithm that converts voice to text using a voice recognition algorithm to deliver semantic information with minimal bandwidth consumption. We also provide an adaptive codec selection algorithm that selects the appropriate encoding algorithm according to the network capacity based on the trade-off between the compression rate and sound quality. The algorithm exploits the text-based voice codec algorithm in the minimal network environment; otherwise, it adaptively selects the best audio codec considering the network condition. Experiment results show that the proposed text-based codec can deliver messages under extremely scarce network conditions.

Durant, Dalton; Popov, Andrey A.; Zanetti, Renato

What are You Weighting For? Improved Weights for Gaussian Mixture Filtering

Abstract

Gaussian mixture-type filters have become indispensable tools for modeling intricate and nonlinear systems, offering a departure from traditional Gaussian-centric approaches. This work focuses on the critical aspect of accurate weight computation during the measurement incorporation phase of Gaussian mixture filters. The proposed novel approach computes weights by linearizing the measurement model about each component's posterior estimate rather than the the prior, as traditionally done. This work proves equivalence with traditional methods in linear scenarios and empirically demonstrates improved performance in nonlinear cases. Two illustrative examples, the Avocado and Lorenz '63 models, serve to elucidate the advantages of the new weight computation technique by analyzing filter accuracy and efficiency through varying the number of Gaussian mixture components.

Popov, Andrey A.; Zanetti, Renato

Are Non-Gaussian Kernels Suitable for Ensemble Mixture Model Filtering?

Abstract

In the high-dimensional setting, Gaussian mixture kernel density estimates become increasingly suboptimal. In this work we aim to show that it is practical to instead use the optimal multivariate Epanechnikov kernel. We make use of this optimal Epanechnikov mixture kernel density estimate for the sequential filtering scenario through what we term the ensemble Epanechnikov mixture filter (EnEMF). We provide a practical implementation of the EnEMF that is as cost efficient as the comparable ensemble Gaussian mixture filter. We then showcase that the EnEMF has a significant reduction in error per particle on the 40-variable Lorenz '96 system. We answer the titular question, "are non-Gaussian kernels suitable for ensemble mixture model filtering?" in the affirmative.

Giraldo-Grueso, Felipe; Popov, Andrey A.; Zanetti, Renato

Gaussian Mixture-Based Point Mass Filtering

Abstract

The accuracy of the point mass filter (PMF) relies on the precise placement of grid points. Since the approximated probability distributions are evaluated only at these points, sub-optimal choices in grid placement can result in an inaccurate representation of the posterior distribution. This work addresses this issue by representing the propagated grid points as a Gaussian mixture, enabling a Gaussian sum filter (GSF) update before grid construction. The use of the GSF update enhances the accuracy of the mean and covariance estimates, from which a new grid can be constructed. This approach leads to improved grid placement and reduces the number of points required to achieve satisfactory results. A comparative analysis is conducted between this new approach, the traditional PMF, and a PMF variant that uses an unscented Kalman filter update before grid construction. Using a simple bivariate example, the new variant is shown to approximate the posterior distribution better than the other filters. Furthermore, the new approach is evaluated in two sequential filtering problems: the first involves the Ikeda map, and the second focuses on terrain-relative navigation for Martian exploration. The results show a more accurate, and more consistent filter compared to the other two PMF variants considered.

Servadio, Simone; Lavezzi, Giovanni; Hofmann, Christian; Wu, Di; Linares, Richard

Propagation of Uncertainty with the Koopman Operator

Abstract

This paper proposes a new method to propagate uncertainties undergoing nonlinear dynamics using the Koopman Operator (KO). Probability density functions are propagated directly using the Koopman approximation of the solution flow of the system, where the dynamics have been projected on a well-defined set of basis functions. The prediction technique is derived following both the analytical (Galerkin) and numerical (EDMD) derivation of the KO, and a least square reduction algorithm assures the recursivity of the proposed methodology.

Pérez, Annie-Claude; Jauffret, Claude

What to Do with Biased Measurements?

Abstract

Estimating a parameter with biased measurements is the topic of this paper. To do this, we propose three different models when the available data are corrupted by an additive constant bias. For each model, we detail the computation of the Cramér-Rao lower bound (CRLB). For each model, examples coming the traditional bearing-only target motion analysis (BOTMA) are given, for which we propose an estimator whose empirical performance is compared to the CRLB.

Hubner, Michael; Wohlleben, Kilian; Litzenberger, Martin; Veigl, Stephan; Opitz, Andreas; Grebien, Stefan; Dvorak, Maria-Theresia

A Bayesian Approach - Data fusion for robust detection of vandalism and trespassing related events in the context of railway security

Abstract

In the domain of railway infrastructure, monitoring and securing the operational stability remains a significant problem. Vandalism, trespassing, sabotage and theft are constant threats, endangering the safety and integrity of the entire system. At the same time monitoring of these systems is becoming harder and harder as the systems grow and the amount of data produced by the surveillance equipment scales accordingly. Additionally, since specific sensor modalities can have weaknesses in detecting one kind of threat, it is often necessary to install different sensors to get a better understanding of the situation. In this paper we present a fusion model based on Probabilistic Occupancy Maps (POM) and Bayesian Inference for environmental mapping of critical events such as vandalism and trespassing in the vicinity of railway infrastructure. We show that this approach helps to increase accuracy, while simultaneously decreasing the amount of false alarms generated by a system.

Mamich, Rachel; Michaelson, Kristen; Popov, Andrey A.; Zanetti, Renato

Burnished Flow Filter

Abstract

The Burnished Flow Filter is a particle flow filter constructed from the Kalman filter measurement update equations. The derivation for this filter begins by assuming the classic Kalman Filter measurement update equations are the solution to a stochastic differential equation. By using these well known equations, the derivation of this filter follows naturally to an engineer with a Kalman filtering background. The work presented here shows the derivation, and application of this filter on both linear and nonlinear problems. The Burnished Flow Filter is benchmarked against the widely used Gromov Flow Filter, revealing similar performance in linear problems and demonstrating superior consistency in the nonlinear scenarios under study. Additionally, the Burnished Flow Filter exhibits a smoother flow compared to the Gromov Flow Filter, as evidenced by a smaller state update during the first substep of the measurement update.

Gao, Zhaoqiang; He, Jiazhou; Zhang, Heng

Multi-vector Matching Deformation Measurement Method

Abstract

This paper introduces a novel multi-vector matching deformation measurement method designed to enhance the stability of deformation measurements in static environments by incorporating specific force observations. The approach effectively measures shift deformation and extends the deformation measurement range by integrating with the velocity matching equation. Furthermore, we propose a conversion model that addresses angular deformation and shift deformation, offering a solution for measuring shift deformation in environments where comparison force observations may not be straightforward. Real inertial navigation experiments conducted in a laboratory setting demonstrate the method's capability to measure deformation in static environments while simultaneously capturing shift deformation. The results illustrate the method's effectiveness in practical deformation measurement scenarios.

Wang, Mingxing; Li, Xiao; Li, Xiaolong; Gao, Longji; Chen, Desheng; Cui, Guolong

An Integration Detection Approach for High-Speed Maneuvering Target in Airborne Coherent MIMO Radar

Abstract

This article addresses the multi-channel integration detection issue of high-speed maneuvering weak targets in airborne coherent multi-input multi-output (MIMO) radar. Coherent MIMO radar can significantly improve the detection performance through joint intra-channel and multi-channel fusion processing. Nevertheless, the range migration (RM) and Doppler frequency migration (DFM) are resulted from high-speed motion, and the envelope and phase differences among multi-channels are challenging. To address these limitations, we propose a multi-channel integration approach in generalized Radon-Fourier transform (GRFT) domain. First, the system and signal models are established. GRFT is utilized to accumulate intra-channel energy. Then, we construct a set of coupled equations, and estimate the target's position, speed and acceleration with the Newton-Raphson algorithm and solving linear equations. Based on the estimated outputs, the envelope alignment and phase compensation functions are constructed to eliminate the differences across channels. After that, the multi-channel fusion is realized in GRFT domain. The superiority of the proposed approach is shown via simulations.

Rao, Nageswara S. V.; Ma, Chris Y. T.; He, Fei

ML Classifier Fusion for Three Data Streams with Quality Inversely Proportional to Time Resolution

Abstract

We consider a monitoring scenario of phenomenon using three different streams of measurements whose quality is proportional to their constant inter-arrival times. Each measurement of a stream needs to be binary-classified to reflect the state of interest of the phenomenon. A set of classifiers is separately trained and fused for each stream at its time resolution using measurements collected under known states. We present a machine learning method to fuse the outputs of these fusers to provide a final classification at the finest time resolution. We show that this fused-fusers method provides decisions with likely superior classification probability compared to the best individual classifiers and fused-classifiers. We derive generalization equations that guarantee a superior classification probability of fused-fusers with a confidence probability specified by the classifiers' generalization equations. We apply these results to study a practical problem of classifying Pu/Np target dissolution events at a radiochemical processing facility using gamma spectral measurements of effluent flows.

Weng, Xu; Ling, Keck-Voon; Liu, Haochen; Cao, Kun

Towards End-to-End GPS Localization with Neural Pseudorange Correction

Abstract

The pseudorange error is one of the root causes of localization inaccuracy in GPS. Previous data-driven methods regress and eliminate pseudorange errors using handcrafted intermediate labels. Unlike them, we propose an end-to-end GPS localization framework, E2E-PrNet, to train a neural network for pseudorange correction (PrNet) directly using the final task loss calculated with the ground truth of GPS receiver states. The gradients of the loss with respect to learnable parameters are backpropagated through a Differentiable Nonlinear Least Squares (DNLS) optimizer to PrNet. The feasibility of fusing the data-driven neural network and the model-based DNLS module is verified with GPS data collected by Android phones, showing that E2E-PrNet outperforms the baseline weighted least squares method and the state-of-the-art end-to-end data-driven approach. Finally, we discuss the explainability of E2E-PrNet.

Bogner, Mirjam; Pieper, Fynn; Steger, Christian; Steidel, Matthias; Piotrowski, Janusz A.; Feuerstack, Sebastian

Utilizing 1D FMCW Radar Data for Distance Estimation to Port Infrastructure

Abstract

Assistance systems play an important role in the proceeding transition of surface vessels towards highly automated operations. Particularly when navigating through congested areas like harbors, exact knowledge of distances to nearby obstacles is essential for collision avoidance. This paper applies a combined filtering and clustering approach in order to utilize 1D FMCW radar data for distance estimation to nearby obstacles in the harbor environment. The data processing aims at clearing the raw sensor data from unwanted signals caused by environmental influences like rain or waves and determines a reliable distance from the relevant signals. We evaluate our approach using sea trial data from a research vessel, comparing processed radar distances with a DGPS-based ground truth. The study assesses the performance of three density-based clustering algorithms - DBSCAN, HDBSCAN, and OPTICS - in this context. All of these algorithms show a good performance for processing the 1D FMCW data for our use case, enabling a reliable distance determination to a static obstacle. OPTICS performs slightly better in terms of eliminating disturbing signals than the remaining two algorithms. The processing times of all algorithms were found to be sufficient for online application of the proposed approach.

Chen, Chuang; Sun, Qiankun; Liu, Weifeng; Yan, Junkun

A Smooth and Efficient Trajectory Planning Method for Unmanned Surface Vehicles

Abstract

Existing unmanned surface vehicle (USV) trajectory planning algorithms have problems such as poor feasibility and long planning time, which cannot fully satisfy the requirements of trajectory planning in practical application scenarios. This paper proposes a smooth and efficient trajectory planning method for USV. Firstly, a B-spline trajectory considering only the constraints of the start and end positions is initialized, and the time-parameterized B-spline trajectory is substituted into the three-degree-of-freedom motion model of the USV. Secondly, the point cloud data of the surrounding environment is mapped into a 2D grid map to calculate the distance between the initial trajectory and the obstacles. Finally, the method optimizes the unmanned ship trajectory by smoothing, collision and dynamics feasibility penalty terms which are based on the convex hull nature of the B-spline. It can be proved through simulation experiments that the algorithm in this paper can realize smooth and efficient trajectory planning for USV.

Sun, Qiankun; Chen, Chuang; Liu, Weifeng; Cai, Lei

Bearing-based Multi-ASV Preset-time Oil Spill Surface Encirclement Control Method

Abstract

In the dynamic oil spill surface encirclement task, the disturbance estimation of the oil boom and the setting of the encirclement time are the keys to the success of the encirclement. Therefore, this paper presents a bearing-based multiple Autonomous Surface Vehicles (multi-ASV) preset-time oil spill surface encirclement control method. First, estimating its resistance based on the boom model and modeling the kinematics of ASV under boom interference. Then, combining the spreading of the oil spill surface with the encirclement capacity to determine the encirclement time threshold. Finally, based on the bearing information, the ASV preset time controller is designed to realize the accurate encirclement of dynamic oil spill surface targets. Simulation experiments show the effectiveness of the proposed algorithm.

Harvey, Ryan; Braca, Paolo; Millefiori, Leonardo M.; Willett, Peter

Sequential Hypothesis Testing Based on Machine Learning

Abstract

With the rapid proliferation of Machine-Learning (ML) and Deep Learning (DL) based decision systems, properly characterizing their often unpredictable performance is a key challenge. In this work we introduce the notion of a Sequential Data-Driven Decision Function (S-D3F), as a data-driven analogue to the Sequential Probability Ratio Test (SPRT). Key performance metrics for sequential analysis are shown suitable for use in analyzing the S-D3F’s performance both in terms of error probabilities and average stopping times. The notion of rate function from large deviations theory is extended to this S-D3F test, and it is shown that with a sequential approach the S-D3F can outperform its Fixed Sample-Size (FSS) counterpart in the D3F as the average number of samples needed to make a decision diverges.

Wei, Xinwei; Lin, Yiru; Zhang, Linao; Zou, Zhiyuan; Wei, Jianwei; Yi, Wei

Transformer-based Multi-Target Tracking with Bayesian Perspective

Abstract

The Bayesian inference has a two-step recursion structure, i.e., prediction and updating, which can be viewed as a dynamic reasoning process. Based on this elegant structure, various multi-target tracking (MTT) algorithms have been invented and successfully applied in many areas. On the other hand, Bayesian inference MTT algorithms are model-based methods that rely on models' accuracy and first-order Markov assumption. In recent years, the MTT algorithms based on deep learning have received much attention due to their model-free property and the ability to learn from data, although they have issues such as over-fitting, generalization, etc. In this work, we propose a Transformer-based multi-target tracker whose architecture mimics the Bayesian inference, referred to as the Bayesian inference-based Transformer (BAIT) for MTT. To deal with the model mismatch issues, BAIT uses neural networks instead of the pre-assumed motion and observation models while retaining the excellent architecture of Bayesian inference. BAIT can recursively complete accurate predictions and updates via Transformer by refining the estimation of target states in a Bayesian inference-like manner. Thus, BAIT can be viewed as a combination of model-based and data-based methods. The simulation results show that, because of combining the advantages of Bayesian architecture with intelligent data association structure, BAIT is competitive in simple scenarios and achieves superior performance when the data association task becomes complicated.

Deng, Jiangyun; Sun, Zhi; Chen, Haixu; Li, Xiaolong; Cui, Guolong; Yang, Xiaobo

Passive Localization Method of LFM Signal Transmitter based on Multi-channel Joint Accumulation in FrFT Domain

Abstract

The traditional two-step localization method for transmitter requires to estimate the signal parameters such as angle of arrival (AOA) and time of arrival (TOA), wich confronts the problem of localization error cumulation. While the direct position determination (DPD) can effectively reduce the estimation error and achieve superior localization performance than the two-step localization, which is widely applied in passive radar. Unfortunately, the uncertainty of the transmission signal parameters will causes the localization performance degradation for DPD method in passive localization. To address these issues, this paper considers the LFM signal transmitter localization with passive radar. Firstly, the signal within each receiving channel is accumulated by fractional Fourier transform (FrFT). Then the signal envelope alignment of each channel is performed in the FrFT domain by using the characteristics of FrFT, so as to realize the multi-channel signal accumulation. Finally, the transmitter is accurately localized by the two-dimensional position search. Simulation results show that the proposed method outperforms the two-step method and DPD method in low SNR.

Zhang, Mei; Shen, Xiaojing; Wang, Zhiguo; Varshney, Pramod K.

Robust Primal-Dual Proximal Algorithm for Cooperative Localization in WSNs

Abstract

This paper addresses the localization challenge in cooperative multi-agent wireless sensor networks, specifically focusing on range-based localization. To enhance robustness against outliers in range measurements, we employ the Huber function, leading to the formulation of a robust yet nonconvex optimization problem with coupled agent variables. Confronted with this nonconvex optimization challenge, particularly in largescale networks, we reformulate the problem using Lagrange duality and conjugate theory. This restructuring yields subproblems characterized by smooth strong convexity for dual variables and a simplified form for primal variables, thereby facilitating an efficient solution. Building upon this reformulation, we introduce a novel distributed primal-dual algorithm that employs coordinate descent and proximal minimization techniques within an iterative framework. This approach furnishes closed-form solutions for both primal and dual variables. Theoretically, our method ensures not only the convergence of the sequence of objective function values but also, by leveraging the Kurdyka-Lojasiewicz property, we establish the guaranteed global convergence of the location estimates sequence to a critical point of the original objective function. Notably, our proposed approach exhibits lower computational complexity, communication cost, and storage space compared to existing methods. Numerical experiments underscore the superiority of the proposed method in terms of robustness and localization accuracy when compared to the other methods in the literature.

Golias, Griffin; Nakura-Fan, Masa; Ablavsky, Vitaly

SSP-GNN: Learning to Track via Bilevel Optimization

Abstract

We propose a graph-based tracking formulation for multi-object tracking (MOT) where target detections contain kinematic information and re-identification features (attributes). Our method applies a successive shortest paths (SSP) algorithm to a tracking graph defined over a batch of frames. The edge costs in this tracking graph are computed via a message-passing network, a graph neural network (GNN) variant. The parameters of the GNN, and hence, the tracker, are learned end-to-end on a training set of example ground-truth tracks and detections. Specifically, learning takes the form of bilevel optimization guided by our novel loss function. We evaluate our algorithm on simulated scenarios to understand its sensitivity to scenario aspects and model hyperparameters. Across varied scenario complexities, our method compares favorably to a strong baseline.

Gao, Lin; Battistelli, Giorgio; Chisci, Luigi

Extended object tracking based on superellipses

Abstract

This paper presents an approach for 2-dimensional extended object tracking (EOT). The extended object (EO) is represented as a superellipse characterized by kinematic and shape states, with the latter uniquely specified in terms of four parameters. An approximated measurement model is proposed accounting for the fact that the measurements can be generated from any position inside or on the contour of the EO. Then, EOT is performed by iteratively estimating the kinematic state via a Kalman filter, while the posterior of the shape state is represented and propagated in particle filter form due to the strong nonlinearity of the resulting shape measurement model. Simulation results are provided to assess the performance of the proposed method.

Zou, Qingke; Zhou, Jie

Hyperspectral Super-Resolution Using Nonlinear Unmixing and Nonnegative Tensor Factorization

Abstract

Fusing a hyperspectral image (HSI) and a multispectral image (MSI) to generate a super-resolution image (SRI) with refined spatial and spectral resolution is a popular technique in hyperspectral super-resolution (HSR). Most HSR methods accomplish this task by matrix or tensor decomposition in the framework of linear unmixing. Although these methods are effective to some extent, serious challenges remain. In this work, the linear unmixing is extended to nonlinear unmixing framework and a novel HSR method based on a generalized bilinear unmixing model in tensor format is proposed. Apart from the linear part, it additionally considers bilinear interactions between endmembers. A low-rank prior is incorporated into the abundance maps and nonlinear interaction abundance maps, which can adequately model non-local similarities in images. In addition, the total variance is used to explore the local spatial relationships of the image. The optimization is implemented using the alternating direction method of multipliers (ADMM) algorithm with analytical expressions for each iterative update step, which is difficult to implement even for algorithms that focus on nonlinear unmixing. The proposed method overcomes the inherent linear limitations of the linear unmixing framework and avoids the information loss caused by matrixing the HSI and MSI with 3D-structure. The experimental results of simulations on real hyperspectral datasets demonstrate the superiority of the proposed approach over the compared HSR methods.

Cao, Xi; Tian, Yunlian; Yang, Jiaye; Li, Wujun; Yi, Wei

Trajectory PHD Filter for Extended Traffic Target Tracking with Interaction and Constraint

Abstract

With the increasing demand for traffic situation awareness, extended traffic target (ETT) tracking is a significant yet challenging task especially in tough scenarios with dense and various ETTs. Due to the spatial proximity of ETTs and a noisy sensor, it is challenging for multi-target tracking algorithms to effectively distinguish and track ETTs. To improve the accuracy and robustness of ETT tracking in tough traffic scenarios, we analyze the interaction among ETTs and the lane constraint. Firstly, we develop an interactive motion model for collision avoidance to address trajectory confusion when ETTs are in close proximity. Additionally, we propose a lane constraint method that models lanes as pseudo measurements and constrains the motion of ETTs via pseudo update. Considering the complexity and extendibility, the extended target trajectory probability hypothesis density (ET-TPHD) filter is adopted to achieve a more accurate estimation of ETT trajectories. Specifically, we realize the proposed interactive motion model and lane constraint method based on the ET-TPHD (IC-ET-TPHD) filter. Performance comparisons between our proposed filter and other algorithms are conducted through both simulations and experiments.

Cros, Colin; Amblard, Pierre-Olivier; Prieur, Christophe; da Rocha, Jean-François

Split Covariance Intersection with Correlated Components for Distributed Estimation

Abstract

This paper introduces a new conservative fusion method to exploit the correlated components within the estimation errors. Fusion is the process of combining multiple estimates of a given state to produce a new estimate with a smaller MSE. To perform the optimal linear fusion, the (centralized) covariance associated with the errors of all estimates is required. If it is partially unknown, the optimal fusion cannot be computed. Instead, a solution is to perform a conservative fusion. A conservative fusion provides a gain and a bound on the resulting MSE matrix which guarantees that the error is not underestimated. A well-known conservative fusion is the Covariance Intersection fusion. It has been modified to exploit the uncorrelated components within the errors. In this paper, it is further extended to exploit the correlated components as well. The resulting fusion rule is integrated into standard distributed algorithms where it allows exploiting the process noise observed by all agents. The improvement is confirmed by simulations.

Brouk, James D.; DeMars, Kyle J.

Anonymous, Extent-Informed Navigation for Map-Based Localization using Random Finite Sets

Abstract

This paper presents the anonymous, extent-informed (AEI) update for map-based localization. The presented approach builds upon anonymous feature processing (AFP) by introducing a new prior and landmark likelihood model such that extent-dependencies in the detection process are accounted for directly in the measurement update. The AEI update is applied to a lunar descent scenario, where the simulated vehicle collects optical observations of the lunar surface and compares them to an onboard map. The AEI update is compared to both a Gaussian mixture implementation of AFP and the standard extended Kalman filter (EKF). The results show that the AEI update is able to provide more precise and consistent estimates of the vehicle's position and velocity than the other methods, while also requiring fewer components in the posterior mixture. The AEI update is also shown to be more robust to the presence of clutter and false detection processes.

Hao, Chunyu; Song, Xiaoying; Yang, Fang; Zheng, Wanning; Zhou, Yufeng

Fusion of Individual and Population Graphs in a GNN Brain Disease Network

Abstract

In brain disease research, despite the increasing sophistication of existing graph neural network approaches, it is still a challenge to simultaneously analyse abnormal brain regions associated with diseases and to consider inter-subject relationships to improve the accuracy of disease diagnosis. In this paper, we propose a new graph neural network model for brain disease analysis by fusing information from individual and population graphs. The model designs an individual graph coarsening network to learn the most discriminative features to construct the population graph and identify abnormal brain regions. The individual graph data are fused into the population graph, and then a population hierarchical graph convolutional neural network is constructed to learn the relationship between different subjects and fully integrate the global information. Experimental results show that the proposed method achieves good classification accuracy while identifying the most discriminative brain regions, providing interpretable analyses for the study of brain diseases.

Ristic, Branko; Benavoli, Alessio

Credal Valuation Network for Ongoing Threat Assessment

Abstract

The paper develops a valuation network for sequential assessment of threat under epistemic uncertainty based on theoretical foundations and semantics of imprecise probability theory. The valuations are expressed as credal sets defined by coherent probability intervals on singletons. The combination rule is the generalized Bayes rule introduced by Walley. The model of a single-target threat is based on the classical ``capability-intent'' paradigm in an air surveillance context. Numerical results illustrate the performance of developed credal valuation network (with imprecise probabilities) against the valuation network with precise probabilistic models.

Chen, Zhijin; Ristic, Branko; Kim, Du Yong

Autonomous Area Search in the Framework of Possibility Theory

Abstract

The paper formulates the solution to area search for targets in the framework of possibility theory. The rationale is that the required measurement model parameters, such as the probability of detection and/or the probability of false alarm, are rarely known as precise values. Possibility theory was developed for quantitative modelling of and reasoning with epistemic uncertainty. It provides an elegant Bayesian like solution to target area search. A reward function is proposed as an uncertainty measure which takes into account the epistemic uncertainty. The robustness of the proposed search algorithm is demonstrated by numerical results.

Ristic, Branko; Kim, Du Yong; Rosenberg, Luke

Track-Before-Detect for Airborne Maritime Radar: Application to Real Data

Abstract

Consider the problem of maritime surveillance using a high-resolution airborne radar for the detection and tracking of small surface targets. This is a challenging problem as the sea clutter is spiky with a non-Gaussian amplitude distribution and contains both temporally and spatially varying characteristics. As a possible solution, we have recently proposed a Bayesian track-before-detect algorithm, which assumes a compound K-distributed clutter model with Swerling 1 target fluctuations. The paper considers a suitable modification of this algorithm to work in the range-Doppler domain and evaluates its performance on real datasets collected by the Defence Science and Technology Group's (DSTG) Ingara X-band radar.

Wang, Zhaohui; Zhang, Yue; Zhou, Jin; Cai, Haohao; Liu, Xichun; Wang, Jianji; Zheng, Nanning

CuES: Conditional Uncorrelation-based Characteristic Enhancement and Fusion of Electrical Signals

Abstract

The lifespan of a generator greatly depends on the quality and aging of its stator bar insulation material. Aging of insulation materials can lead to premature equipment failure and significant material loss, resulting in substantial economic losses. However, existing methods for predicting the lifespan of electronic wire bars have several drawbacks, such as slow training speed, the need for a large amount of training data, and a tendency to overfit. To address this issue, we propose a characteristic enhancement algorithm based on conditional uncorrelation. This algorithm leverages characteristic enhancement to generate an extensive dataset and utilizes subset selection to identify relevant electrical parameters for predicting the remaining life span of the stator bar's main insulation configurations. Experimental results demonstrate the advantages of our research compared to deep learning models. Our approach offers a promising solution for accurately predicting the remaining life of stator bar insulation, thereby facilitating effective maintenance planning and minimizing economic losses.

Hu, Zhuohan; Yang, Bo; Lin, Jialiang; Wu, Jiajin; Liu, Wei

MCL4SRec: A Sequential Recommendation Model with Multi-level Contrastive Learning

Abstract

Sequential recommendation (SR) plays an important role across various platforms, aiming to predict users' next items of interest based on their historical interaction sequences. Recent SR studies have employed deep learning techniques, such as Recurrent Neural Networks and Self-Attention (SA) mechanism, demonstrating promising results. Inspired by the emergence of contrastive learning methods, some SR models have utilized contrastive learning to improve the accuracy of recommendations. However, existing SR models employing contrastive learning primarily construct positive and negative sample pairs only from user interaction sequences, i.e., through sequence-level contrastive learning. In our research, we argue that there also exists semantic similarities between items, which can be used to conduct the item-level constructive learning, resulting in better recommendation accuracy. In this paper, we propose MCL4SRec, an SA-based SR model that combines sequence-level and item-level contrastive learning to enhance recommendation accuracy. In our proposed MCL4SRec, the item-level contrastive learning module utilizes items' category information to construct positive and negative sample pairs, capturing semantic similarities and differences between items. Additionally, in MCL4SRec, we propose to use more side information such as category and brand to further improve the accuracy of recommendations. We conduct extensive experiments on three widely-used datasets to evaluate the proposed MCL4SRec. Experimental results indicate that the average improvements compared with the recent well-known baselines range from 7.73% to 16.18% in HR and NDCG, demonstrating the effectiveness of MCL4SRec for SR tasks.

Matoušek, Jakub; Duník, Jindrich; Brandner, Marek

Efficient Spectral Differentiation in Grid-Based Continuous State Estimation

Abstract

This paper deals with the state estimation of stochastic models with continuous dynamics. The aim is to incorporate spectral differentiation methods into the solution to the Fokker-Planck equation in grid-based state estimation routine, while taking into account the specifics of the field, such as probability density function (PDF) features, moving grid, zero boundary conditions, etc. The spectral methods, in general, achieve very fast convergence rate of O(c^N)(O < c < 1) for analytical functions such as the probability density function, where N is the number of grid points. This is significantly better than the standard finite difference method (or midpoint rule used in discrete estimation) typically used in grid-based filter design with convergence rate O(1/N^2 ). As consequence, the proposed spectral method based filter provides better state estimation accuracy with lower number of grid points, and thus, with lower computational complexity.

Pieper, Fynn

Sensor Fusion of 2D-LiDAR and 360-Degree Camera Data for Room Layout Reconstruction

Abstract

Precise floor plans are the foundation for ensuring safety, accessibility, and efficiency within critical infrastructures. To meet this need, we introduce a method that seamlessly merges the accurate depth measurements from a 2D LiDAR with the nuanced data interpretation capabilities of artificial neural networks applied to RGB images. This allows our approach to capture fully furnished rooms with remarkable spatial accuracy and reliability from a single measurement. Central to our methodology is a novel two-stage filtering process. First, we harness geometric analysis to exclude extraneous LiDAR points, then refine this data using semantic segmentation, ensuring only measurements congruent with walls are retained. Walls are inferred through a systematic Hough transform and further validated using an innovative confidence metric. Using RGB data, we augment the wall estimation for room edge detection, allowing us to determine the azimuth position of room corners from the image, even when obscured by furniture. Subsequent data fusion yields layouts with an average 2D IoU of 90.47 % without relying on camera height assumption. By carefully leveraging the unique strengths of each data type, our approach significantly advances the field of room layout reconstruction (RLR). The achieved results hint at a considerable potential for real-world applications.

Goderik, Daniel; Westlund, Albin; Zetterqvist, Gustav; Gustafsson, Fredrik; Hendeby, Gustaf

Seismic Detection of Elephant Footsteps

Abstract

As human settlement expands into the natural habitats of wild animals, the conflicts between humans and wildlife increases. The human-elephant conflict causes a tremendous amount of damage, often to poor villages close to the savannah. In this paper, we continue our earlier reported research on a geophone network aimed for elephant localisation by focusing on the detection challenge. We have now collected larger sets of seismic data with footsteps from both elephants and other big animals including humans. To detect the footsteps, a method is developed that analyses features of the geophone signal, which are then compared to those of an elephant footstep. The method detects 54% of the footsteps and has a classification accuracy of 89%. Subsequently, the detected elephant footstep is used to calculate the direction of arrival (DOA) angle using a delay-and-sum beamformer. The direction to an elephant is estimated with good precision on distances ranging from 8 to 30 meters. This research, not only, showcases a practical solution for mitigating human-elephant conflicts, but also underscores the potential of seismic technology in wildlife management and conservation efforts.

Fan, Zhengyang; Shen, Dan; Bao, Yajie; Pham, Khanh; Blasch, Erik; Chen, Genshe

RNN-UKF: Enhancing Hyperparameter Auto-Tuning in Unscented Kalman Filters through Recurrent Neural Networks

Abstract

The Unscented Kalman Filter (UKF) stands out as a versatile and dynamic algorithm, celebrated for its prowess in estimating the states of nonlinear dynamical systems within uncertain environments. However, the accuracy of UKF state estimations hinges significantly on the thoughtful selection of pivotal hyperparameters, α, β, and κ, which are integral in shaping the distribution of sigma points around the current state estimate. Prevailing methods for tuning these parameters encompass heuristic approaches such as arbitrarily fix α at 0.001, κ at 0, and β at 2 for Gaussian noise, though the efficacy of such rules heavily hinges on the intricacies of the specific problem. Alternatively, the grid search technique seeks to optimize these hyperparameters, but it can become computationally burdensome, particularly when the search space is extensive and intricate. To navigate these hurdles, this paper introduces the RNN-UKF algorithm—a pioneering strategy that leverages recurrent neural networks (RNNs) to autonomously fine-tune UKF hyperparameters. The inherent adaptability of RNNs is harnessed to dynamically adjust the α, β, and κ parameters of the unscented transformation during each state estimation step, all aimed at minimizing the root mean squared error (RMSE). Demonstrated through numerical simulations, we provide compelling evidence that the RNN-UKF approach outperforms both heuristic rule of thumb and grid search techniques in terms of RMSE performance. Moreover, the RNN-UKF methodology showcases its superiority over the conventional extended Kalman filter (EKF) approach, particularly in scenarios characterized by substantial system noise.

Craft, Kyle J.; Demars, Kyle J.

A Variational Approach to Robust Bayesian Filtering

Abstract

A major challenge of applied Bayesian filtering is deriving estimates that are robust to misspecifications in the underlying statistical models, particularly when Bayes' rule does not directly yield analytical posterior probability densities. Variational approaches present a promising alternative to "traditional,"' closed-form Bayesian inference, wherein an approximate posterior is defined by minimizing the statistical dissimilarity to the conventional Bayesian posterior. There are, however, numerous realistic hurdles to defining an ideal dissimilarity measure, from which posteriors are derived using calculus of variations. This work utilizes a recently proposed framework, known as generalized variational inference (GVI), to define robust and tractable approximations of Bayes' rule. The GVI framework is presented and accompanied by a novel sensitivity analysis and gradient-based solution method that is applicable for particle, Gaussian, and Gaussian mixture posterior representations. The proposed GVI filter is applied to a dynamic state estimation scenario with inaccurate measurement modeling and, via Monte Carlo analysis, demonstrates both statistical consistency and improvements over conventional robust filtering methods.

Michaelson, Kristen; Popov, Andrey A.; Zanetti, Renato; Demars, Kyle J.

Particle Flow with a Continuous Formulation of the Nonlinear Measurement Update

Abstract

The incorporation of nonlinear measurement information plays an important role in Bayesian state estimation for real-word systems. While many methods exist for propagating states through continuous-time nonlinear dynamics, a complementary continuous solution for discrete-time nonlinear measurements has so far remained elusive. Building on intuition from our previous work, the Bayesian Recursive Update Filter, we formulate the nonlinear measurement update as an ordinary differential equation (ODE). This formulation naturally extends to particle flow. We define two particle flows: the first is a deterministic flow based on the ODE solution, and the second is stochastic; the numerical integration contains a diffusion term. The proposed particle flows demonstrate excellent performance on a system with deterministic dynamics and a highly accurate nonlinear measurement, a setting known to be challenging for particle filters.

Tang, Hanning; Shen, Xiaojing; Zhao, Hua; Wang, Zhiguo; Varshney, Pramod K.

Bures-Wasserstein Barycentric Coordinates with Application to Diffusion Tensor Image Smoothing

Abstract

This article considers the Wasserstein barycentric coordinates problem for Gaussian distributions which is the inverse problem of the Wasserstein barycenter problem. These coordinates take into account the underlying geometry of the measure space of Gaussian distributions and are thus meaningful for applications such as diffusion analysis and distributed information fusion. When the probability supports are discrete and identical, the theory of Wasserstein barycentric coordinates is well developed. However, for general probability distributions, the computation of Wasserstein barycentric coordinates is intractable since the technical hurdles involve solving a non-convex and non-concave optimization problem. For Gaussian distributions, we derive the closed-form expression of the derivatives for the objective function and propose a projected gradient descent method to solve the problem. Finally, we illustrate its application in diffusion tensor image (DTI) denoising including simulated DTI with different noise levels and DTI of the human brain.

Kang, Jeong Min; Sjanic, Zoran; Hendeby, Gustaf

Visual-Inertial Odometry Using Optical Flow from Deep Learning

Abstract

It is shown how dense optical flow obtained using deep learning can be used to provide high quality visual odometry. The obtained odometric information can be utilized as a component to reduce the inherent drift of inertial navigation systems (INS). This could be a key component to provide autonomous system with robust localization capability in GNSS denied environments. The method leverages the power of estimating optical flow from neural networks, which can provide reliable results even when feature based optical flow fails. Comparison of different methods to decide which points of the dense optical flow that should be used to provide as good visual odometry as possible has been performed. Furthermore, it is exemplified how the methodology can help limit the drift in an INS.

Xiao, Zhuo; Yang, Yi; Zhang, Sixian; Li, Wenbiao; Bao, Pengrong; Han, Deqiang

Foreground Aware Correlation Filter with Adaptive Feature Response Fusion for Real-Time UAV Tracking

Abstract

Background Aware Correlation Filter (BACF) tracker achieves accurate tracking result in visual object tracking by mitigating boundary effects, yet is limited in challenging scenarios especially in viewpoint change and illumination variation, which are frequently encountered in Unmanned Aerial Vehicle (UAV) tracking tasks. To address the shortcomings, we propose a Foreground Aware Correlation Filter with adaptive feature response fusion (FACF). In this paper, we use saliency detection to generate foreground prior knowledge in training phase for suppressing potential noise. Furthermore, recognizing the limitation of BACF, which relies on a single feature, a novel adaptive fusion strategy is designed to fuse multiple feature responses during the detection phase. This strategy aims to enhance the robustness of the tracker. Extensive experiments have been conducted on three challenging benchmarks. The tracking results show that the proposed tracker performs accurate and robust tracking result and satisfies real-time requirement with 48.28fps.

Herrmann, Lukas; Brekke, Edmund Førland; Eide, Egil

Coherent Integration of Optical Flow for Track-Before-Detect Radar Detection

Abstract

The detection of small and dim targets under low signal-to-noise ratio (SNR) circumstances is a commonly encountered yet challenging endeavour in radar signal processing. The standard approach to deal with undesirable background conditions involves coherent processing and integration with subsequent detection directly applied to the radar signals. However, optical flow, a widespread visual tracking method, has rarely been used in this context. In this paper, we address the issue of radar target detection in low SNR scenarios by employing optical flow on radar images. This work focuses on the divergence of the optical flow vector field, utilising a novel approach of coherently integrating consecutive flow fields calculated against a homogeneous reference plane. The proposed methodology allows for more robust target identification and thus a precise initialisation of tracking systems. To validate and demonstrate the benefits of the proposed approach, simulations are conducted and discussed.

Steuernagel, Simon; Thormann, Kolja; Baum, Marcus

Random Matrix-based Tracking of Rectangular Extended Objects with Contour Measurements

Abstract

A widely-used approach for extended object tracking is based on random matrices, where the scattering matrix, i.e., measurement spread, is used to update a symmetric positive definite random matrix representing an elliptic extent. However, for lidar data, a mismatch between the assumed measurement model and observed data hinders the estimation quality of the method. We propose adaptions to the random matrix approach in order to facilitate the application for tracking a rectangular extended object based on contour measurements. Specifically, we derive a suitable scaling factor for the scattering matrix of measurements in this setting. Furthermore, we propose a simple yet effective estimation scheme for the target center, adapting the shape estimate accordingly. The resulting algorithm closely follows the framework of the random matrix approach. A detailed comparison with a variety of state-of-the-art trackers is carried out in a simulation based on real-world lidar parameters, confirming the effectiveness of the approach.

Gommers, Daan; Strik, Dennis; van Leijen, Vincent

Making METOC data portable with video codecs

Abstract

Numerical Weather Predictions and Ocean forecasts (METOC) are typically high quality datasets with a large file size. Some users have limited bandwidth available and require much smaller file size that can be shared by email or satellite communications. Compression of the datasets can be a solution for these use cases. In previous research, we experimented with lossy compression to find the right balance between compression factor, information loss and speed of operations. In this work, we expand on this by evaluating the use of common video codecs to make METOC data portable. Such codes are maintained for a very large user community and offer decompression in near-real time of small compressed datasets while preserving most of the important information.

El Bouch, Sara; Labsir, Samy; Renaux, Alexandre; Vilà-Valls, Jordi; Chaumette, Eric

An Intrinsic Modified Cramér-Rao Bound on Lie Groups

Abstract

The Modified Cramér-Rao Bound (MCRB) proves to be of significant importance in non-standard estimation scenarios, when in addition to unknown deterministic parameters to be estimated, observations also depend on random nuisance parameters. Given the interest of applications that involve estimation on Lie Groups (LGs), as well as the relevance of non-standard estimation problems in many practical scenarios, the main concern in this communication is to derive an intrinsic MCRB on LGs (LG-MCRB). For this purpose, a modified unbiasedness constraint must be defined, yielding a modified Barankin Bound. A closed-form formula of the LG-MCRB is then provided for a LG Gaussian model on SO(2), representing 2D rotation matrices, while considering non-Gaussian random nuisance parameters. The validity of this expression is then assessed through numerical simulations, and compared with the intrinsic CRB on LGs for a simplified illustrative scenario, involving a concentrated Gaussian prior distribution on the random nuisance parameters.

Crouse, David Frederic

Debiasing Nonlinear Transformations Involving Correlated Measurement Components

Abstract

Given a measurement value that has been corrupted with additive multivariate Gaussian noise with nonzero correlation in its covariance matrix, this paper derives two new Taylor series approximations to estimating an unbiased mean and a consistent covariance matrix. The conversion produces mean and covariance estimates that are more consistent than other expansions in the literature when the covariance matrix is not a diagonal matrix.

Park, Hyunwoo; Chung, Hyeonjin; Conti, Andrea; Win, Moe Z.; Kim, Sunwoo

Robust Near-field Beam Tracking via Deep Q-network for THz Communications

Abstract

This paper presents a robust near-field (NF) beam tracking algorithm for terahertz communications based on deep Q-network (DQN). Traditional NF beam tracking methods relying on mobility models are fatal in ultra-massive MIMO systems, where even the slightest error could result in beam tracking failures. Thus, the proposed algorithm aims to maintain a stable beamforming gain by tracking the mobile station through the analysis of received signals without requiring mobile dynamics. By utilizing DQN, the proposed algorithm strengthens its tracking capability from online experiences and updates the combining beam towards positions expected to maximize beamforming gain. Throughout simulations, we compare the proposed algorithm with the Bayesian filter-based NF beam tracking algorithm. The simulation results confirm the robustness of the proposed algorithm for NF beam tracking, especially for abrupt changes in mobile dynamics.

Ye, Shida; Bar-Shalom, Yaakov; Willett, Peter; Zaki, Ahmed

Maximum Likelihood Identification of an Ornstein-Uhlenbeck Model and Its CRLB

Abstract

This paper applies Maximum Likelihood Estimation (MLE) to the identification of a stochastic error model of a gyroscope. The error model used for illustration features an Ornstein-Uhlenbeck process with an unknown time constant driven by a process noise with unknown variance, and a white measurement noise also with unknown variance. As the setup of MLE, the likelihood function (LF) is derived in the steady-state Kalman filter framework and is defined in reference to the parameters of the Kalman filter gain and innovation variance. The resulting log-likelihood function (LLF) is a quadratic function of the measurements, facilitating the evaluation of the Cramer-Rao Lower Bound (CRLB) and makes it possible to confirm the statistical efficiency, i.e., optimality, of the ML estimator presented in this paper.

Carloni Gertosio, Rémi; Gaonach, Gilles; Beyna, Enzo; Martin, Liana; Meyrat, Alexis

Passive Sonar Ranging and Range-Doppler-Bearing Target Motion Analysis

Abstract

In passive sonar, target motion analysis (TMA) provides an estimate of the trajectory of a moving source, generally based on bearing and Doppler measurements. When the source presents a low bearing rate, the problem is ill-conditioned and difficult to solve without observer maneuver. To address this issue, we propose to estimate the instantaneous range of the source from its measured spectrum, using a conventional radiated ship noise model and a sound absorption model in seawater. The introduction of this new measurement in what becomes a range-Doppler-bearing TMA algorithm, ensures the uniqueness of the trajectory. Compared with standard Doppler-bearing TMA, convergence is faster and stability is improved, especially at low bearing rates.

Duminil, Alexandra; Ieng, Sio-Song; Gruyer, Dominique

Assessing fidelity in synthetic datasets: A multi-criteria combination methodology

Abstract

With the development of driving simulators, graphics engines and synthetic-to-real domain adaptation algorithms, synthetic datasets become increasingly more photo-realistic. The advancement of such dataset is crucial for advanced driving systems, particularly for training learning-based methods and validation. An important consideration is around the fidelity of synthetic datasets, particularly regarding their suitability for deep learning applications such as object detection or segmentation. However, quantifying fidelity poses a significant challenges. To address this gap, we propose a set of fidelity scores to quantify the level of fidelity of RGB images from these datasets. Through in-depth examination, we aim to reveal information about the texture patterns and high-frequency components that contribute to the objective perception of data realism in road scenes. Furthermore, a multi-criteria combination using belief theory is performed to merge these scores and give a global score involving the level of fidelity, the level of uncertainty on this decision, and the level of conflict between the scores.

Hangerhagen, Petter; Brekke, Edmund Førland; Eide, Egil; Skjetne, Roger

A Radar Dataset from the Trondheim City Canal

Abstract

In the automotive community, methods for tracking, localization, and situational awareness are routinely tested on well-known open-source datasets from the real world. In many other applications of target tracking, such as maritime radar tracking, there is a lack of such data. In this paper, we present a large dataset consisting of data recorded by a frequency-modulated continuous wave radar overlooking the Trondheim City Canal over several weeks during the summer of 2023. The dataset includes a rich variety of boat traffic, ranging from large ferries to formations of kayaks. All the data have been analyzed by means of classical joint integrated probabilistic data association-based multiple target tracking. We point out several challenges that arise in this dataset, such as merged measurements and multipath. We also demonstrate that the data are sufficient to generate statistical information about traffic patterns in the City Canal.

Thormann, Kolja; Steuernagel, Simon; Baum, Marcus

Indoor Localization based on Short-Range Radar and Rotating Landmarks

Abstract

A novel concept for indoor self-localization based on rotating artificial landmarks with known locations using short-range radar is proposed. First, a processing pipeline for extracting range and angle measurements to the landmarks from a raw radar image is introduced, which consists of a neural network for distance estimation and a basic angle-of-arrival estimator. Second, a particle filter for tracking the pose based on the range and angle measurements is developed. Due to the ability of radar to measure range rate, i.e., the velocity in the direction of a detection, it is possible to robustly detect and localize rotating landmarks with the help of their micro-Doppler pattern. In this way, localization is possible even under difficult conditions (e.g., light changes). Experiments with a wheeled mobile robot and common office fans as landmarks demonstrate the effectiveness of the approach for indoor localization.

Li, Wei; Li, Xiaolong; Yang, Fan; Cui, Guolong; Yang, Xiaobo

A Range Deception Interference Recognition and Target Detection Method Based on Coherent Fusion Processing for Multistatic Radar System

Abstract

When detecting high-speed targets, the strong spoofing of range deception interference (RDI) can lead to failure of true target detection. Fortunately, the fusion processing provides an effective solution to this problem. In this paper, the target echo model under RDI conditions is established based on the range history model for multistatic radar system. On this basis, we propose an effective RDI recognition and target detection method based on coherent fusion processing. Specifically, the method firstly realizes the coherent fusion of single-channel echo by Radon Fourier transform (RFT). Then, topology-based entropy circulation matching (TECM) is used to accomplish the acquisition of the matching positions about target and RDI in different channels. Finally, the matching position is processed by elliptic positioning (EP) to realize the recognition of RDI and target. Simulation experiments verify the effectiveness of the method.

Wodtko, Thomas; Griebel, Thomas; Scheible, Alexander; Buchholz, Michael

Conflict Handling in Time-Dependent Subjective Networks

Abstract

With this work, we contribute novel operators and perspectives to the field of subjective logic. We propose a novel multi-source trust revision approach enabling multi-source fusion, which considers majority tendencies to mitigate occurring conflicts. For this, the degree of conflict is extended for a multi-source use, which allows our definition of so-called conflict shares. Subsequently, combining our and existing trust revision methods, we propose a generalized trust revision approach. Extending trust revision to subjective networks describing time-dependent processes, we propose the use of sub subjective networks and further the transition to recursive subjective networks. Finally, our trust revision approach and the sub subjective network proposal are evaluated and demonstrated based on experiments, which show conflict handling favoring majorities and an efficient evaluation of time-dependent decision processes.

do Nascimento, Vinicius D.; de Farias, Claudio M.; Dutra, Diego L. C.; Alves, Tiago A. O.

Ensemble Learning Approaches for Detecting Fishing Activity in Maritime Surveillance: A Performance Evaluation

Abstract

Detecting fishing trajectories in maritime surveillance is of the utmost importance for identifying illegal fishing activity. In the event of illegal fishing activity, the maritime authority can mobilize resources to engage the vessel; hence, a false flag can be costly. This study investigates the efficacy of ensemble learning techniques for boosting individual model performance and decreasing uncertainty. Employing a range of machine learning models, including logistic regression, decision trees, random forests, neural networks, gradient boosting, and recurrent neural networks, the research evaluates the combination of these using ensemble methods like ensemble mean, weighted ensemble, and stacking approaches to enhance precision and decrease uncertainty. The primary dataset comprises a combination of fishing vessel and cargo vessel trajectories to train and test the models. Methodologically, the paper details the process of data analysis and the application of ensemble learning. A comparative assessment of individual models versus ensemble techniques forms the crux of this study. Results indicate a marked improvement in accuracy and consistency when employing ensemble methods, with weighted and stacking ensembles showing particular promise. These findings suggest that ensemble models outperform their individual counterparts in the context of maritime surveillance. This research makes a notable contribution to the maritime surveillance domain, demonstrating the potential of ensemble learning in enhancing detection capabilities for illegal fishing activities. The implications of these advancements are critical for maritime authorities as they strive to effectively monitor and protect marine ecosystems.

Glover, Timothy J.; Nanavati, Rohit V.; Coombes, Matthew; Liu, Cunjia; Chen, Wen-Hua; Perree, Nicola; Hiscocks, Steven

A Monte Carlo Tree Search Framework for Autonomous Source Term Estimation in Stone Soup

Abstract

Source term estimation of a hazardous release remains a topic of significant interest in the robotics and state estimation communities, with application to many safety critical scenarios including gas or nuclear release, locating suspicious smells or response to emergency incidents. Limited sensing resources and time constraints mean that deciding on how to act in order to improve efficiency of estimation is also of significant interest. This paper has two main focuses: a sequential Monte Carlo technique for performing source term estimation from gas concentration measurements taken on a mobile sensor platform and a Monte Carlo tree search (MCTS) framework to perform sensor motion planning to maximise Kullback-Leibler divergence (KLD). Both algorithms are implemented in the open source tracking and estimation framework: Stone Soup, creating several key contributions to this Python based toolkit. The presented algorithm demonstrates superior performance when compared to a greedy myopic alternative when considering source position estimation error, release rate error and successful rate performance measures.

Jose, Esther; Batta, Rajan; Sudit, Moises

Situational Assessment using Indicator Kriging for Fleet Tracking and Prediction

Abstract

Maritime fleet tracking is a critical piece of naval operations. Leveraging the inherent spatial and temporal autocorrelation of vessels in a fleet, we use spatio-temporal Kriging, an interpolation technique, to estimate the likelihood of finding a vessel at a specific location. This estimation is based solely on the current and/or past locations of other vessels within the fleet. We do this by first fitting covariance models to observed fleet movements. We then use spatio-temporal indicator Kriging to forecast the locations of vessels in a fleet at different times, with or without new information. Our results indicate a notable improvement in accuracy, ranging from 60 to 90% compared to a baseline model. We measure accuracy using ROC AUC values. Furthermore, our study reveals that tracking only a subset of vessels within a fleet significantly enhances understanding of the entire fleet’s movements. However, the number of vessels that needs to be tracked increases as we move further from the last observation of the entire fleet. Future extensions of our work include integrating additional situational information, using other spatio-temporal interpolation techniques, and expanding its application beyond maritime fleets.

Duník, Jindrich; Punčochář, Ivo; Král, Laislav; Straka, Ondrej; Daniel, Ondrej; Prol, Fabricio S.; Liaquat, Muwahida; Bhuiyan, Zahidul

Multi-layer GNSS and LEO-PNT Positioning: Integrity under Constellations’ Correlation

Abstract

This paper deals with the initial integrity eval- uation of the navigation information provided by the multi- layer GNSS and LEO-PNT constellation. Although, the global satellite navigation systems (GNSS) play indispensable role in almost all aspects of today’s society, their signals are prone to intentional or accidental interference. Therefore, low Earth orbit (LEO) constellations aiming at position, navigation, and timing (PNT) solution have recently been introduced as their extension. The LEO-PNT constellations are planned to con- tain hundreds of SVs with better interference resilience and geometric diversity. As a consequence, the multi-layer GNSS and LEO-PNT constellation was shown to offer more accurate PNT solution. In this paper, we focus on another important aspect of the multi-layer navigation information, which is its integrity assessment. In particular, we analyse possible dependencies between GNSS and LEO-PNT constellations and their impact on the integrity evaluated using the solution separation. The analysis is supported by the numerical simu- lations using GPS and LEO-PNT constellations with 32 and 441 satellites, respectively.

Kropfreiter, Thomas; Williams, Jason L.; Meyer, Florian

Multiobject Tracking for Thresholded Cell Measurements

Abstract

In many multiobject tracking applications, including radar and sonar tracking, after prefiltering the received signal, measurement data is typically structured in cells. The cells, e.g., represent different range and bearing values. However, conventional multiobject tracking methods use so-called point measurements. Point measurements are provided by a preprocessing stage that applies a threshold or detector and breaks up the cell’s structure by converting cell indexes into, e.g., range and bearing measurements. We here propose a Bayesian multiobject tracking method that processes measurements that have been thresholded but are still cell-structured. We first derive a likelihood function that systematically incorporates an adjustable detection threshold which makes it possible to control the number of cell measurements. We then propose a Poisson Multi-Bernoulli (PMB) filter based on the likelihood function for cell measurements. Furthermore, we establish a link to the conventional point measurement model by deriving the likelihood function for point measurements with amplitude information (AM) and discuss the PMB filter that uses point measurements with AM. Our numerical results demonstrate the advantages of the proposed method that relies on thresholded cell measurements compared to the conventional multiobject tracking based on point measurements with and without AM.

Yang, Jiaye; Xiong, Yuhuan; Cao, Xi; Peng, Cong; Yi, Wei

Joint Tracking and Classification of Vehicles with the PHD Filter and Gaussian Processes

Abstract

Joint tracking and classification (JTC) of vehicles is a crucial yet challenging task in intelligent transport and automotive systems. The advent of high-resolution modern sensors necessitates treating vehicles as extended targets. Current extended target tracking (ETT) algorithms provide shape estimations for vehicles, making shape size the most intuitive and accessible feature for classification. This paper contributes two key elements to achieve the JTC of vehicles. For one thing, we introduce the rectangular constraints and customize distinguishable measurement models using modified Gaussian processes (GP). For another thing, based on the customized GP models, we strengthen the role of class in the conditional extended target probability hypothesis density (ET-PHD) filter. Subsequently, we propose a class-enhanced JTC-ET-PHD filter and its Gaussian mixture implementation, enabling simultaneous kinematic, shape, and class estimation of vehicles. Finally, numerical results validate the proposed shape estimation and JTC method, affirming their effectiveness in addressing JTC challenges.

Baur, Tim; Hoher, Patrick; Reuter, Johannes; Hanebeck, Uwe D.

Tracking Extended Objects with Basic Parametric Shapes using Deformable Superellipses

Abstract

In extended object tracking, basic parametric shapes such as ellipses and rectangles or non-parametric shape representations such as Fourier series or Gaussian processes can be utilized as shape priors. However, flexible non-parametric shape representations can be disproportionately detailed and computationally intensive for many applications. Therefore, we propose to adopt deformable superellipses for a low-dimensional and flexible representation of basic parametric shapes in this paper. We present a measurement model in 2D space that can cope with boundary and interior measurements simultaneously by recursively estimating an artificial noise variance for interior measurements. We investigate and compare the model in a simulated and real-world maritime scenario with the result that the combination of deformable superellipses and artificial measurement noise estimation performs better than state-of-the-art methods.

Watkins, Luisa; Stinco, Pietro; Tesei, Alessandra; Meyer, Florian

A Probabilistic Focalization Approach for Single Receiver Underwater Localization

Abstract

We introduce a Bayesian estimation approach for the passive localization of an acoustic source in shallow water using a single mobile receiver. The proposed probabilistic focalization method estimates the time-varying source location in the presence of measurement-origin uncertainty. In particular, probabilistic data association is performed to match time-differences-of-arrival (TDOA) observations extracted from the acoustic signal to TDOAs predictions provided by the statistical model. The performance of our approach is evaluated using real acoustic data recorded by a single mobile receiver.

Kumar, Kundan; Särkkä, Simo

Polynomial Chaos Expansion Based Rauch–Tung–Striebel Smoothers

Abstract

This article introduces Gaussian approximation-based smoothing algorithms for nonlinear stochastic state space models using the polynomial chaos expansion (PCE). Initially, we present a smoothing algorithm, where the nonlinear functions of the state space model are approximated using a PCE that is formed using a set of collocation points generated from the filtering distribution. Subsequently, an iterative variant of the proposed smoothing algorithm is also presented. It iteratively forms a PCE approximation to the nonlinear functions by using collocation points generated from the current posterior approximation. The performance of the algorithms is evaluated on pendulum and aircraft tracking problems.

Grimmett, Douglas J

Fused Tracking of Pulsed and Continuous Active Sonar Transmission Modes

Abstract

In recent years, the feasibility of using continuous-active-sonar (CAS) transmissions as an alternative to pulsed-active-sonar (PAS) transmissions has been explored. A CAS system transmits waveforms with high-duty-cycle (HDC) near or equal to 100%. CAS offers potential improvements over short-burst PAS signals by providing an order-of-magnitude or more increase in the number of detection opportunities with a faster revisit rate for improved target localization and holding. Much of the previous work has focused on understanding the performance benefits of CAS as well as the differences between PAS and CAS performance both at the output of the processing chain and at the output of a multitarget tracking algorithm. In this paper we explore a synergistic use of PAS and CAS transmissions occurring in parallel, at the same time. The LCAS’15 experiment provided a data set that included simultaneous transmissions of both PAS and CAS waveforms from a monostatic sonar. Signal and information processing were applied to both the PAS and the CAS received signals. A sonar tracking algorithm was modified to accept detection contacts from both PAS and CAS and provides various PAS-CAS fusion rules/schemes for evaluation. The tracker was applied to the sea trial data for PAS-only and CAS-only which establishes baseline cases, which are compared with other fusion cases that combined PAS and CAS waveform scans. The analysis shows encouraging results with synergistic improvements in tracker metrics by achieving the best performance (and avoiding the worst performance) of either PAS or CAS operating independently. A fusion rule with PAS track initiation /confirmation and CAS track termination provided the longest track holding and fewest false tracks, and provides the lowest uncertainty in target localization error.

Kaplan, Lance M.; Hare, James Z.

Asymptotic Analysis of Uncertain Naïve Bayes via Second-Order Probabilities

Abstract

Likelihood fusion is a special case of Bayesian networks known as naive Bayes. It is well known that as the number of observations goes to infinity with known likelihoods, the aleatoric uncertainty of the queried (or parent) variable goes to zero, and furthermore, the declared values are guaranteed to match the ground truth. This work considers the case that the conditional probabilities are learned with limited training data leading to uncertain likelihoods, and second-order probabilistic reasoning is incorporated to characterize the aleatoric and epistemic uncertainty. Remarkably, it is shown that both the aleatoric and epistemic uncertainty goes to zero despite limited knowledge of the likelihoods. The rate of convergence is dictated by a quasi-divergence value that is related to the Kullback-Liebler (KL) divergence. However, the quasi-divergence can be negative leading to false declarations. This paper investigates when false declarations can emerge and shows how such cases diminish as the amount of training data for the likelihoods increases.

Coraluppi, Stefano; Jenkins, Noah; Lexa, Michael

Simplified Distributed Tracking

Abstract

The purported optimality of single-stage centralized MHT with respect to a MAP data association objective is difficult to achieve in practice. In both single-sensor and multi-sensor domains, distributed MHT offers measurable performance and robustness benefits. Some of these gains are hampered due to complex distributed tracking logic. This paper explores a simplified paradigm for distributed MHT.

Shiri, Fatemeh; Moghimifar, Farhad; Haffari, Reza; Li, Yuan-Fang; Nguyen, Van; Yoo, John

Decompose, Enrich, and Extract! Schema-aware Event Extraction using LLMs

Abstract

Large Language Models demonstrate significant capabilities in processing natural language data, promising efficient knowledge extraction from diverse textual sources to enhance situational awareness and support decision-making. However, concerns arise due to their susceptibility to hallucination, resulting in contextually inaccurate content. This work focuses on harnessing Large Language Models (LLMs) for automated Event Extraction, introducing a new method to address hallucination by decomposing the task into Event Detection and Event Argument Extraction. Moreover, the proposed method integrates dynamic schema-aware augmented retrieval examples into prompts tailored for each specific inquiry, thereby extending and adapting advanced prompting techniques such as Retrieval-Augmented Generation. Evaluation findings on prominent event extraction benchmarks and results from a synthesized benchmark illustrate the method’s superior performance compared to baseline approaches.

Shaheen, Khadija; Chawla, Apoorva; Uilhoorn, Ferdinand Evert; Salvo Rossi, Pierluigi

Partial-Distributed Filtering for Fault Detection, Isolation and Accommodation in Natural-Gas Pipelines

Abstract

This paper explores an innovative method for distributed state estimation aimed at reducing computational complexity while detecting sensor faults in natural gas pipelines. The proposed framework utilizes a partial-distributed ensemble Kalman filter (EnKF), comprising linear local filters and a nonlinear main filter. The main filter handles non-linear computations during the time update, while the simultaneous operation of linear local filters manages linear computations during the measurement update. These local filters generate distinct local state estimates based on their specific sensor measurements, which are then transmitted to an information mixer to compute fault-free state estimates. Moreover, a fault diagnosis strategy is developed using local state variances and residuals. Faulty sensors are identified and isolated by comparing these metrics against a threshold. Additionally, an adaptive thresholding approach is incorporated to enhance effective fault identification. The effectiveness of the proposed technique is demonstrated in systems characterized by high nonlinearity and dimensionality, and featuring simultaneous multiple sensor faults, through extensive simulations and comparative analyses.

Yeo, Kiat Nern; Lau, Yan Ling; Ng, Gee Wah

Data Fusion Pipeline for UAV-Based Real-Time Night Crowd Counting for Public Safety

Abstract

Performing crowd management in large scale outdoor events at night is a challenging yet essential task for public safety and security purposes. Traditional methods of carrying out crowd counting require the deployment of massive manpower and are unable to provide a reliable count for effective resource and manpower planning. In recent years, deep learning based crowd counting methods trained on static images were introduced. However, there still exist real world challenges of variations in crowd density across the scene, illumination, environmental conditions and perspective problems which these methods are unable to fully address. This paper attempts to address the problems of varying crowd densities and illumination through a crowd counting pipeline that fuses illumination enhancement processes with crowd density estimation and crowd localisation techniques to achieve improved accuracy for crowd counts from live UAV video feeds. This data fusion pipeline approach has been demonstrated to provide improved count accuracy on both dataset and real world images compared against standalone state-of-the-art methods.

Camajori Tedeschini, Bernardo; Brambilla, Mattia; Nicoli, Monica; Win, Moe Z.

Cooperative Positioning with Multi-Agent Reinforcement Learning

Abstract

In recent years, cooperative positioning technologies have emerged as promising augmentation systems for providing high-accuracy positioning (HAP) in cooperative intelligent transportation systems (C-ITS). Among the approaches, implicit cooperative positioning (ICP) takes advantage of shared target detections between vehicles to create common reference points for localization refinement. Their performance, however, is limited by reliance on predefined parametric models, low scalability and communication overhead. To address these problems, this paper introduces a deep multi-agent reinforcement learning (MARL) framework modelled as a decentralized-partially observable Markov decision process (Dec-POMDP). We propose an ICP-multi-agent proximal policy optimization (MAPPO) algorithm, where distributed agents (i.e., the connected vehicles) learn their dynamics and those of the surrounding targets by performing belief estimation over dynamic cooperation graphs that are continuously adjusted by de/activating communication links with neighbors agents. A C-ITS scenario is simulated in a CARLA environment accounting for realistic vehicle dynamics and inter-vehicle communications. The findings reveal that our ICP-MAPPO algorithm, leveraging dynamic decentralized execution and centralized training, outperforms ICP in terms of positioning accuracy and communication efficiency.

Adolfsson, Jonatan; Hamrell, Hanna; Gustafsson, David

Tracking of Few-Pixel UAVs in Event Data

Abstract

The event camera is a relatively new type of sensor where each pixel asynchronously reports changes in incident light, resulting in low latency and high energy efficiency. This opens for new possibilities within counter Unmanned Aerial Vehicle (UAV) applications, crucial to tackle the threat of this increasingly widespread technology. However, this calls for new data processing methods. As a contribution to both of these challenges, we investigate the possibility of tracking UAVs that cover only a single or few pixels in event camera data. Scene and background activity noise is suppressed using a novel noise filter. Tracking is performed in the image plane using a standard Multi-Hypothesis Tracker (MHT) with parameters optimised for the data. The method is evaluated on several sequences of UAV data collected in an outdoor setting, exhibiting a variety of motion patterns and target sizes. The GOSPA measure and the fraction of observations within the covariance estimate are used to evaluate the results. Excellent tracking performance is achieved for regularly moving targets with a size of 2-15 pixels. Given the UAV and setup used in this study, this corresponds to distances of about 10-30 m when viewing the UAV from the side, or 20-65 m when viewed from below. Tracking is still possible for irregular motion in the same range, whereas rapid movements at close range and single-pixel targets are more challenging. We show that it is perfectly viable to track UAVs of various pixel sizes in event-based data using an MHT algorithm with an appropriate denoising method.

de Gortari Briseno, Julian; Parać, Roko; Ardon, Leo; Roig Vilamala, Marc; Furelos-Blanco, Daniel; Kaplan, Lance; Mishra, Vinod K.; Cerutti, Federico; Preece, Alun; Russo, Alessandra; Srivastava, Mani

TeamCollab: A Framework for Collaborative Perception-Cognition-Communication-Action

Abstract

Teams of embodied AI-enabled agents are critical for applications in extreme and highly dynamic environments. Developing robust controllers for such agents requires a deep understanding of the challenges encountered when attempting to coordinate and synchronize their individual perception-cognition-communication-action (PCCA) loops for team-wide mission objectives. We introduce a framework to explore the coordination of the PCCA loops across multiple agents in a new simulated physical environment designed to explore collaboration in each PCCA stage. This environment tasks teams of agents with the correct disposal of dangerous objects in an area and forces careful coordination of sensing, communication, movement, and manipulation actions by providing spatially-bounded communication, incorporating situations that require concerted effort by groups of agents, and introducing uncertainty into agents' sensing capabilities. We provide a set of heuristic controllers, an offline oracle model, and an initial exploration of a Reward Machine-based controller that learns its policies from training. Together these approaches serve to provide insights into the complexity of the multi-agent PCCA loop coordination problem. The multiagent PCCA simulation environment, which supports AI and human-controlled agents, and the code for various agent controllers are available at https://github.com/nesl/AI-Collab.

Zhao, Yun; Grayden, David B.; Boley, Mario; Liu, Yueyang; Karoly, Philippa J.; Cook, Mark J.; Kuhlmann, Levin

Inference-based time-resolved chaos analysis of brain models: application to focal epilepsy

Abstract

This paper introduces a new inference-based frame- work for time-resolved chaos analysis of brain models and demonstrates its application to focal epileptic seizures. The intermittent nature of epileptic seizures exhibits an unpredictable behavior that shares some characteristics with chaotic systems. Epilepsy research often uses concepts from chaos theory and non-linear dynamics to better understand the mechanisms of seizure initiation, propagation, and termination. Traditional methods estimate the degree of chaos in brain dynamics directly from time series data. This provides neither an accurate estimate of the chaos nor insights into the key neurophysiological processes driving brain dynamics during epileptic seizures. Therefore, this study proposes a new method to calculate Lyapunov spectra by combining time series data with neurophysiological brain models and a specialised nonlinear Kalman filter. This study thereby provides insights into the temporal evolution of chaos in epileptogenic regions during epileptic seizures and identifies external inputs from adjacent and distant brain regions as major drivers of altered levels of chaoticity. This paper underscores the importance of fusion of neurophysiological computational models and clinical time series data in understanding the dynamic and chaotic aspects of epilepsy to develop more effective diagnostic and treatment strategies.

Liu, Yueyang; Grayden, David B.; Schmidt, Daniel; Soto-Breceda, Artemio; Cook, Mark J.; Kuhlmann, Levin; Karoly, Philippa; Freestone, Dean R.

Forecasting events in multidimensional electroencephalographic brain data: Application to epileptic seizure prediction

Abstract

Forecasting events in multichannel electroencephalographic (EEG) brain recordings remains a formidable task given the noise and complexity in neural systems. Here we compare two dynamical systems motivated approaches to forecasting brain events. The first follows previous state-of-the-art (SOTA) research of time-series features of critical slowing down (autocorrelation, variance) as biomarkers of impending events. The second involves a novel long-term-short-term (LSTM) neural network-based filter to estimate the neurophysiological feature variables of mathematical neural population models of the EEG. Previous critical slowing research presented forecasting results for the best EEG channel, however, in practice the best channel cannot be known a priori. Therefore, here we also consider forecasting by combining the different features across the different EEG channels using logistic regression. One application area where forecasting brain events is important is epileptic seizure prediction. Epileptic seizures are debilitating events and up to 50 million people worldwide with drug-resistant epilepsy could benefit by receiving warnings of impending seizures. Here we apply the above methods to a long-term epileptic seizure prediction dataset from 15 patients. It was found that seizure forecasting with (1) logistic regression and critical slowing features, (2) logistic regression and neurophysiological features, and (3) the best channel using critical slowing features, respectively, achieved median sensitivities of 70, 54 and 67% and median time in low seizure risk of 84, 84, and 81%. This indicates that a multichannel model approach can perform as well as the best channel approach, removing the need to find the best channel. It also suggests neurophysiological features could be used to increase time in low risk. Future work exploring other features, machine learning models and their various combinations could yield further improvements.

Nash, Christian; Nair, Rajesh; Naqvi, Syed Mohsen

Cross-Modal Attention for Multimodal Information Fusion: A Novel Approach to Attention Deficit Hyperactivity Disorder Detection

Abstract

This paper presents a novel method for differentiating Attention Deficit Hyperactivity Disorder subjects from control participants by multimodal data fusion, including video observations and questionnaire responses. By exploiting the well known Video Vision Transformer model, we analyse the video modality to identify the complex spatial-temporal information of ADHD symptoms. Simultaneously, a Multi-Layer Perceptron model is applied to evaluate structured questionnaire data by capturing key cognitive and emotional indicators of the ADHD symptoms. To fuse the two modalities, a cross-modal attention mechanism assigns adaptive weights to each feature based on its classification relevance. The targeted weighting significantly refines the proposed model's decision-making capability by concentrating on the most critical elements of the aggregated information. For training and testing, our novel Multimodal ADHD dataset recorded under the Intelligent Sensing ADHD Trial in collaboration with Cumbria, Northumberland, Tyne and Wear NHS Foundation Trust UK is evaluated. The proposed model, ADViQ-AL achieves a 98.18% classification accuracy, 97.83% sensitivity, and 98.53% specificity in classifying ADHD and control groups.

Dezert, Jean; Shekhovtsov, Andrii; Sałabun, Wojciech; Tchamova, Albena

On Optimal Solution of the Compromise Ranking Problem

Abstract

This paper is about the Compromise Ranking Problem (CRP), a well-known problem in the social choice theory. According to famous Arrow's theorem there is no voting method which is entirely satisfying and fairness. In this paper we formalize the problem as a minimisation problem in a discrete finite search space. We attempt to solve it based on the Least Squares (LS) approach thanks to some appealing metrics to get the optimal solution. Surprisingly, we show that the optimal consensus (compromise) ranking solution disagrees with the commonsense solutions in four simple interesting examples. The search for an optimal solution in agreement with the commonsense appears to be an open very challenging question and our paper warns the users about the impossibility of the main current methods to provide acceptable solutions even for the rather simple examples considered in this work.

Ding, Guanhua; Liu, Jianan; Xia, Yuxuan; Huang, Tao; Zhu, Bing; Sun, Jinping

LiDAR Point Cloud-Based Multiple Vehicle Tracking with Probabilistic Measurement-Region Association

Abstract

Multiple extended target tracking (ETT) has gained increasing attention due to the development of high-precision LiDAR and radar sensors in automotive applications. For LiDAR point cloud-based vehicle tracking, this paper presents a probabilistic measurement-region association (PMRA) ETT model, which can describe the complex measurement distribution by partitioning the target extent into different regions. The PMRA model overcomes the drawbacks of previous data-region association (DRA) models by eliminating the approximation error of constrained estimation and using continuous integrals to more reliably calculate the association probabilities. Furthermore, the PMRA model is integrated with the Poisson multi-Bernoulli mixture (PMBM) filter for tracking multiple vehicles. Simulation results illustrate the superior estimation accuracy of the proposed PMRA-PMBM filter in terms of both the positions and extents of vehicles compared with PMBM filters using the gamma Gaussian inverse Wishart and DRA implementations.

Hao, Yuhang; Fu, Jing; Wang, Zengfu; Pan, Quan

A Deep Reinforcement Learning-Based Whittle Index Policy for Multibeam Allocation

Abstract

In this paper, a non-myopic beam scheduling policy is proposed for multi-target tracking (MTT) in a phased-array radar network, seeking to minimize the discounted sum of tracking error of targets and improve the long-term tracking performance. The Whittle index policy based on the restless multi-armed bandit (RMAB) model can decompose the state space of the underlying optimization problem into independent spaces with reduced sizes. We consider the tracking error covariance (TEC) matrix as the state of each target (arm), which evolves based on the Kalman filter. However, for a real-world MTT, the exact calculation of the Whittle index in multiple dimensions is challenging. The neural network is established to achieve the feature extraction of TEC states and learn the corresponding Whittle index. The deep reinforcement learning (DRL) method is exploited to train the neural network by leveraging the threshold property of the Whittle index policy and engaging in interactions with a single target tracking environment. We propose the DRL-based Whittle index policy, namely DRLWI, aiming to solve the beam allocation problem for MTT with multi-dimensional TEC states. This approach effectively mitigates the exponential computational complexity of classical dynamic programming approaches and the low convergence rate caused by large joint state and action spaces in the simple application of DRL algorithms. Numerical results demonstrate the performance of the proposed DRLWI policy surpasses that of DRL algorithms and myopic policies.

Wang, Jiale; Ng, Gee Wah; Mak, Lee Onn; Cher, Randall; Ryan, Ng Ding Hei; Wang, Davis

QCaption: Video Captioning and Q&A through Fusion of Large Multimodal Models

Abstract

This paper introduces QCaption, a novel video captioning and Q&A pipeline that enhances video analytics by fusing three models: key frame extraction, a Large Multimodal Model (LMM) for image-text analysis, and a Large Language Model (LLM) for text analysis. This approach enables integrated analysis of text, images, and video, achieving performance im- provements over existing video captioning and Q&A models; all while remaining fully self-contained, adept for on-premises deployment. Experimental results using QCaption demonstrated up to 44.2% and 48.9% improvements in video captioning and Q&A tasks, respectively. Ablation studies were also performed to assess the role of LLM on the fusion on the results. Moreover, the paper proposes and evaluates additional video captioning approaches, benchmarking them against QCaption and existing methodologies. QCaption demonstrate the potential of adopting a model fusion approach in advancing video analytics. Index Terms—Large Multimodal Model, Large Language Model, Video Analytics, Model Fusion

Zhang, Guoxin; Liang, Yunfei; Wang, Cong; Yi, Wei; Ngo, Hien Quoc; Matthaiou, Michail; Varshney, Pramod K.

Decentralized Direct Localization Based on Gauss-Newton Method in Multi-Sensor Networks

Abstract

Traditional centralized direct localization methods require the transmission of the complete baseband signal to the fusion center (FC) for target localization. Due to the limited communication bandwidth as well as energy required in transmission, this centralized framework is not suitable for large-scale sensor networks. This paper proposes an information-driven decentralized direct localization framework. Firstly, a maximum-likelihood position estimator, based on the Gauss-Newton method, is derived. Then, a decentralized implementation framework is constructed. At its core, there is no dedicated FC while the sensors transmit information to their neighboring nodes only through single hops, achieving target localization through iterative processes based on the concept of consensus. Simulation results confirm the stability and robustness of the proposed method in different scenarios.

Hellander, Anja; Hendeby, Gustaf

On the feasibility of localization using DVB-T signals and combining TDOA and TWR measurements

Abstract

Due to vulnerabilities of Global Navigation Satellite Systems (GNSS) there is an increased interest in alternative navigation solutions, such as using signals of opportunity (SOPs). We propose a system where a mobile navigator localizes itself using two-way ranging (TWR) measurements to a stationary base station at a known location as well as time difference of arrival (TDOA) measurements from two terrestrial digital television transmitters. We investigate the feasibility of such a system by deriving the Cramér-Rao Lower Bound (CRLB) for varying noise levels and optimizing the placement of the base station, using the real-life positions of transmitters in the area around Linköping, Sweden. We simulate measurements and compute snapshot estimates, verifying that root mean square errors similar to the CRLB can be obtained. The results indicate that for the investigated levels of TWR noise, as long as the TDOA noise is sufficiently low it could be possible to achieve errors of a few tens of meters.

Li, Can; Liu, Zhunga; Pan, Quan; Bai, Xianglong; Zhang, Zuowei; Pan, Kunpeng

Land-Sea Clutter Classification for Over-the-Horizon Radar via Dual Attention Aided Residual Neural Networks

Abstract

Deep learning has been widely used in the field of radar image classification because of its powerful feature extraction capabilities. In the land-sea clutter classification of sky-wave over-the-horizon radar (OTHR), deep learning methods perform poorly due to the radar receiver noise and the ionosphere. Addressing this challenge, a dual attention aided residual neural networks (DAAResNet) is proposed for OTHR land-sea classification. Leveraging prior knowledge that land-sea clutter features predominantly cluster around the 0 Hz frequency, two attention mechanisms are introduced. Firstly, a channel attention module (CAM) is proposed, which directs the network's focus towards critical channels. Secondly, a frequency attention module (FAM) is proposed, which directs attention towards pivotal frequencies. The classification performance of DAAResNet is validated on the original dataset and the scarce dataset. Experimental results show that DAAResNet outperforms state-of-the-art methods.

Hoher, Patrick; Baur, Tim; Reuter, Johannes; Griesser, Dennis; Govaers, Felix; Koch, Wolfgang

3D-Extended Object Tracking and Shape Classification with a Lidar Sensor using Random Matrices and Virtual Measurement Models

Abstract

In extended object tracking, random matrices are commonly used to filter the mean and covariance matrix from measurement data. However, the relation from mean and covariance matrix to the extension parameters can become challenging when a lidar sensor is used. To address this, we propose virtual measurement models to estimate those parameters iteratively by adapting them, until the statistical moments of the measurements they would cause, match the random matrix result. While previous work has focused on 2D shapes, this paper extends the methodology to encompass 3D shapes such as cones, ellipsoids and rectangular cuboids. Additionally, we introduce a classification method based on Chamfer distances for identifying the best-fitting shape when the object’s shape is unknown. Our approach is evaluated through simulation studies and with real lidar data from maritime scenarios. The results indicate that a cone is the best representation for sailing boats, while ellipsoids are optimal for motorboats.

Salhi, Mohammed; Al Hage, Joelle

Zonotopic and Gaussian Information Filter for High Integrity Localization

Abstract

The navigation of intelligent vehicles relies on high integrity localization system capable to bound the estimation errors. This paper introduces a zonotopic and Gaussian Kalman filter in informational form for multi-sensor data fusion and confidence domain computation. By integrating stochastic and set membership uncertainties, the proposed filter ensures accurate localization with a non pessimistic confidence domain, thus addressing the challenges posed by traditional techniques. Taking advantage of the informational form, a fault detection and exclusion step is added to enhance filter robustness. Following a zonotope reduction step, a confidence domain computation, considering both Gaussian and zonotopic uncertainties, is proposed in the context of intelligent vehicles. The accuracy and integrity of the approach are assessed using experimental data, including the fusion of GPS and Galileo pseudoranges with camera measurements after a map matching step. Additionally, a comparative analysis is conducted with the classical Kalman filter.

Vetrekar, Narayan; de Ataide, Marissa; Patel, Krishna; Ramachandra, Raghavendra; Gad, R. S.

Does fusion of complementary spectral bands improves the cross-illumination on the performance of gender prediction?

Abstract

The automatic prediction of gender from the face has been studied extensively because of its potential relevance in numerous applications related to security. Although the problem of gender classification based on the face is substantial, it remains far from being solved under difficult environmental exposure, especially for different illuminations. In this work, we demonstrate the merits and demerits of classifying gender under cross-illumination variants. We present our approach by employing multi-spectral imaging in nine narrow-spectrum bands stemming from the visible to the near-infrared range. The experimental evaluation results were obtained on 78300 sample face images of 145 subjects captured under six different illumination conditions. Further, we present quantitative and qualitative experimental evaluations to determine the average classification accuracy for setting the benchmark results. To demonstrate the goal of this work, we present the results based on three image fusion techniques independently processed using five feature extraction methods for cross-illumination scenarios. This work obtained the highest classification accuracy of 96.32% for cross-illumination conditions, demonstrating the reliability of employing an image fusion approach to combine complementary information from spectral bands in difficult environmental exposure.

Yu, Jingyi; Pychynski, Tim; Barsim, Karim Said; Huber, Marco F.

Causal Knowledge in Data Fusion: Systematic Evaluation on Quality Prediction and Root Cause Analysis

Abstract

Data fusion deals with combining information from multiple sensors to support decision making. In such settings, machine learning methods, that principally only take correlation into account, have been applied widely due to their strong predictive and computational capabilities. In this paper, we investigate potential benefits of introducing causal knowledge in machine learning-based data fusion to address two common downstream tasks, namely, quality prediction and root cause analysis (RCA). To resemble the complex relationships typically associated with sensor data, we create simulation data with explicit modeling of latent confounding. The results of this study indicate that taking into account true causal knowledge significantly improves the performance of RCA, and leads to prediction models that are more robust to severe distribution shifts in the presence of latent confounding. Furthermore, if causal knowledge needs to be inferred from observational data using existing causal discovery methods, we propose a selection criterion to choose the best causal structure. We show that given a sufficient amount of data, the selected causal structure can be used asreliable input to solve the downstream tasks.

Kandel, Deepak; Dera, Dimah

Adaptive Robust Continual Learning based on Bayesian Uncertainty Propagation

Abstract

Robust continual learning (CL) poses fundamental challenges and is essential for developing reliable and adaptable intelligent systems. Learning models must sustain a robust performance as they adapt to dynamically evolving environments through sequential learning, effectively overcoming the catastrophic forgetting problem. This paper proposes a novel, trustworthy CL framework based on the Bayesian variational uncertainty learned during training on each task. We integrate the Bayesian inference and propagate the first two moments of the variational posterior distribution over the probabilistic model's parameters. The variational moments (mean and covariance matrix) are learned simultaneously during training on each task and then used to estimate the predictive distribution. The covariance matrix of the variational posterior distribution captures the variational uncertainty in the learned parameters (particularly critical in the context of sequential learning within dynamic real-world settings). We develop an adaptive evidence lower bound (ELBO) loss function that supports managing the stability-elasticity dilemma. The variational continual optimization minimizes the expected log-likelihood of the data given the model's parameters and the Kullback--Leibler (KL) divergence between the variational distributions learned from the current and previous tasks weighted by an uncertainty-based metric. Moreover, we advance an architecture-based CL technique that masks important network parameters learned from each task based on their variational uncertainty. The proposed Bayesian regularization and architecture-based CL approaches prevent significant changes in the parameters of the learning models to preserve representations of previous tasks. The experiments on benchmark datasets demonstrate the robustness of the proposed framework when learning in continual scenarios compared to the stat-of-the-art CL homologs.

Molhoek, Madelon; van Laanen, Joris

Secure Counterfactual Explanations in a Two-party Setting

Abstract

When multiple parties want to learn from each others' data, but do not want to share this data becuase it is privacy sensitive, using a federated trained Machine Learning (ML) model is a good option. Explanation of the results are essential to use and and therefore trust the outcome of this trained model. However, explanations reveal sensitive information which is not allowed when using privacy sensitive data. In this paper, we introduce a novel approach generating Counterfactual Explanations (CFEs) in a secure way utilising synthetic data. A CFE provides an example data point, that with the smallest change to the original feature values provides a different outcome. Thereby showcasing what needs to change for a different output. In our case two parties owning different features of the same persons jointly train a ML model. In this setting, one party owns one feature and the other party owns multiple features including the target feature, both data must remain confidential to the other party. A CFE is created by first securely generating vertical distributed synthetic data with the aid of a Split Neural Network (Split-NN). We show that the distributed synthetic data maintain characteristics of the original data in the cases where the predictability is high, and do not reveal sensitive information when under an Attribute Inference Attack. Secondly, synthetic Counterfactuals (CFs) are generated and ranked using secure Multi-Party Computation. The ranking is based on the optimization of a selection of distance metrics from the CFE with respect to the original event. The outcome of the CFE can be revealed to both parties. In this way we provide a complete privacy-preserving pipeline to explain a federated trained ML model on vertically partitioned data.

Tienin, Bole Wilfried; Cui, Guolong; Talla Nana, Yannick Abel; Ukwuoma, Chiagoziem Chima; Mba Esidang, Roldan; Senouci, Mohammed Raouf

FedRS-Net: A Federated Learning Approach for Collaborative Multi-Modal Maritime Analytics

Abstract

Ensuring the safety and security of our oceans demands a comprehensive maritime situational awareness (MSA) strategy. However, this task has several challenges, including using multi-modal data, data sharing among agencies, privacy concerns, and bandwidth limitations. To address these challenges, this research introduced FedRS-Net. FedRS-Net is a federated deep learning framework that trains multi-modal remote sensing data without exposing client data. The system employs a communication-efficient federated averaging algorithm and a novel convolutional neural network architecture called redesigned skip connection. It integrates synthetic aperture radar (SAR) and optical satellite imageries to achieve remarkable results. Extensive experiments were conducted on the maritime vessel datasets, resulting in a testing accuracy of 99.8%. Further, applying secure aggregation and momentum-based gradient compression reduced communication costs by 7%. FedRS-Net overcomes privacy concerns and facilitates agency collaboration by enabling collective maritime monitoring through decentralized data. This research provides a robust federated learning solution tailored for multi-modal remote sensing analytics applications.

Yang, Jihao; Nie, Laisen; Deng, Xinyang; Jiang, Wen

A Federated Learning Mechanism with Feature Drift for Feature Distribution Skew

Abstract

Federated learning is a nascent distributed machine learning paradigm that enables multiple clients to collaborate in training a model for a specific task under the coordination of a central server, all while safeguarding the privacy of the user's local data. Nevertheless, the constraint that distributed datasets must remain within local nodes introduces data heterogeneity in federated learning training. In this paper, we focus on how to mitigate the damage caused by the data heterogeneity of feature distribution skew in federated learning models during training. To achieve this goal, we propose a feature drift-corrected federated learning algorithm. We design a feature drift variable derived from the local models of clients and the global model of the server. This variable is incorporated into the client's local loss function to rectify local model parameters. Additionally, we utilize the disparity between the global models before and after to regulate the local model. Validation experiments are conducted on multiple datasets exhibiting feature distribution skew. The implementation results demonstrate the efficacy of our approach in significantly enhancing the model performance of federated learning under feature distribution skew.

Prossel, Dominik; Hanebeck, Uwe D.

Spline-Based Density Estimation Minimizing Fisher Information

Abstract

The construction of a continuous probability density function (pdf) that fits a set of samples is a frequently occurring task in statistics. This is an inherently underdetermined problem, that can only be solved by making some assumptions about the samples or the distribution to be estimated. This paper proposes a density estimation method based on the premise that each sample represents the same amount of probability mass of the underlying density. The estimated pdf is parameterized as the square of a polynomial spline, which makes further processing of the estimated density very efficient. This pdf is inherently non-negative, ensuring a monotone cumulative distribution function, which makes it easy to generate samples from it through inverse transform sampling. Furthermore, it is cheap to evaluate and easy to integrate, making moment calculations fast. To find the coefficients of the polynomials that make up the spline, an optimization problem is derived. The Fisher information is used as a regularizer in this problem to select the solution that contains the least amount of information. The method is shown to work on samples from a variety of different one-dimensional probability distributions.

Walker, Markus; Amirkhanian, Hayk; Huber, Marco F.; Hanebeck, Uwe D.

Trustworthy Bayesian Perceptrons

Abstract

Bayesian Neural Networks (BNNs) offer a sophisticated framework for extending classical neural network point estimates to encompass predictive distributions. Despite the high potential of BNNs, established BNN training methods such as Variational Inference (VI) and Markov Chain Monte Carlo (MCMC) grapple with issues such as scalability and hyperparameter dependence. In addressing these issues, our research focuses on the fundamental elements of BNNs, in particular perceptrons and their predictive capabilities. We introduce a new perspective on the closed-form solution for backward-pass computation for the Bayesian perceptron and prove that the state-of-the-art solution is equivalent to statistical linearization. To assess the efficacy of Bayesian perceptrons and provide insights into their performance in distinct input space regions, a novel methodology utilizing k-d trees as a space partitioning method is introduced to evaluate prediction quality within specific input space regions.

Adas, Akif; Barbieri, Luca; Awasthi, Satyesh; Morri, Pietro; Mentasti, Simone; Arrigoni, Stefano; Sabbioni, Edoardo; Nicoli, Monica

LiDAR-Aided Cooperative Localization and Environmental Perception for CAVs

Abstract

This paper explores the potentialities of deploying vehicular Cooperative Positioning (CP) systems in urban scenarios utilizing real-world data collected via experimental campaigns. We examine the case of two prototype vehicles equipped with LiDAR sensors for perceiving their surrounding environment and with Global Navigation Satellite System (GNSS) receivers for positioning. The considered use case focuses on the cooperative detection of static landmarks, to be used for improving the vehicles’ GNSS positioning. The experimental campaign points out a severe degradation in ego vehicle localization performances due to complex multipath propagation experienced in the urban scenario. To cope with such a problem, we integrate into the CP system a compensation method able to mitigate the position bias originating from the adverse propagating conditions. Experimental results show that integrating the developed compensation into the CP solution enables an accurate detection of the landmark positions, leading to an enhancement of the vehicle localization accuracy.

Zhou, Yujing; Beeson, Ryne

Projected Feedback Particle Filtering for Chaotic Dynamical Systems Using Lyapunov Vectors

Abstract

Particle flow methods are effective in resolving the particle degeneracy issue in the standard particle filtering algorithm. However, flow methods have their own difficulties, such as the necessity to solve a Poisson equation in the feedback particle filtering (FPF) method. This is computationally heavy, and we observe a numerical sensitivity and singularity issue dependent on parameter selection when applying to chaotic dynamical systems with limited particle size and coarse integration step size. In this paper, we address the numerical singularity issue by flowing particles in the unstable subspace (UAS), and we name the novel method the projected FPF. It brings the local dynamical information into the assimilation step by using the finite-time Lyapunov exponents and vectors to project observations and particle states to the UAS, where the error diverges. The projected FPF is tested against the Lorenz 1963 model -- a nonlinear, low-dimensional, chaotic dynamical system.

Deleskog, Viktor; Jonsson, Oskar; Nygårds, Jonas; Hendeby, Gustaf

Poisson Multi-Bernoulli Mixture Filtering with Multistatic Passive Bistatic Radar

Abstract

Passive bistatic radar (PBR) is a cost-effective choice for detection and tracking of aircraft. In this paper we present how the Poisson multi-Bernoulli mixture (PMBM) filter is applied in a multi-target tracking application with multistatic PBR. To handle the PBR measurements, it is proposed that a Gaussian mixture target spatial density is used to represent the target state. A state dependent probability of detection model for PBR is presented and how it is used to design the target birth model. Simulated and experimental data are used to evaluate the performance of the described approaches.

Jia, Haowei; Wang, Gan; Liu, Huajun

Spatial-Temporal Attention Network for Track-Track Association with Biased Data

Abstract

Track-track association (TTA) in complex environ- ment for multi-sensor fusion is a challenging topic due to the uncertainty of measurements, biased data, mismatch caused by different resolution etc. In this work, we proposed an end-to- end deep learning model, named the spatial-temporal attention network (STAN) for TTA tasks in complex scenarios. Three modules in the backbone of STAN for intra-track and inter-track feature representation are based on self-attention mechanism, e.g., the motion mode encoder (MME) module to encode the motion pattern of single moving targets, the spatial structure extraction (SSE) module for capturing the inter-track spatial interaction relation of an individual sensor, and the spatial- temporal fusion (STF) module for intra-track modeling on temporal dimensions, respectively. A relation reasoning head (RRH) is built for track-track relation reasoning based on the encoded track features. Experimental results on different tasks show that our proposed method achieved superior performance for track-track association compared with previous methods.

Liu, Xingchi; Mihaylova, Lyudmila

Active Sensing for Target Tracking: A Bayesian Optimisation Approach

Abstract

Active sensing plays an essential role in searching and tracking a target without initial target state information. This paper studies the active sensing approach for sensor management problems using multiple unmanned aerial vehicles based on the received signal strength measurements of the target. A Bayesian optimisation-based approach is proposed which adopts the Gaussian process method to model the received signal strength in an area over time and then the expected improvement acquisition function is leveraged to decide where to take new measurements considering the uncertainty of the Gaussian process. A unique contribution of this paper consists of the designed spatial-temporal composite kernel function that accounts for the time-varying nature of the signal strength. Numerical results obtained from different measurement noise levels and varying initial Bayesian optimisation settings demonstrate that the proposed approach can efficiently schedule multiple unmanned aerial vehicles to locate the target within a minimum number of initial data. Particularly, it achieves at most 57% lower tracking error and 46% lower lost-track probability as compared to the benchmark approach.

Li, Yichun; Li, Shuanglin; Naqvi, Syed Mohsen

A Novel Audio-Visual Information Fusion System for Mental Disorders Detection

Abstract

Mental disorders are among the foremost contributors to the global healthcare challenge. Research indicates that timely diagnosis and intervention are vital in treating various mental disorders. However, the early somatization symptoms of certain mental disorders may not be immediately evident, often resulting in their oversight and misdiagnosis. Additionally, the traditional diagnosis methods incur high time and cost. Deep learning methods based on fMRI and EEG have improved the efficiency of the mental disorder detection process. However, the cost of the equipment and trained staff are generally huge. Moreover, most systems are only trained for a specific mental disorder and are not general-purpose. Recently, physiological studies have shown that there are some speech and facial-related symptoms in a few mental disorders (e.g., depression and ADHD). In this paper, we focus on the emotional expression features of mental disorders and introduce a multimodal mental disorder diagnosis system based on audio-visual information input. Our proposed system is based on spatial-temporal attention networks and innovative uses a less computationally intensive pre-train audio recognition network to fine-tune the video recognition module for better results. We also apply the unified system for multiple mental disorders (ADHD and depression) for the first time. The proposed system achieves over 80\% accuracy on the real multimodal ADHD dataset and achieves state-of-the-art results on the depression dataset AVEC 2014.

Chance, Zachary

Adaptive Temporal Decorrelation of State Estimates

Abstract

Many commercial and defense applications involve multisensor, multitarget tracking, requiring the fusion of information from a set of sensors. An interesting use case occurs when data available at a central node (due to geometric diversity or retrodiction) allows for the tailoring of state estimation for a target. For instance, if a target is initially tracked with a maneuvering target filter, yet the target is clearly not maneuvering in retrospect, it would be beneficial at the fusion node to refilter that data with a non-maneuvering target filter. If measurements can be shared to the central node, the refiltering process can be accomplished by simply passing source measurements through an updated state estimation process. It is often the case for large, distributed systems, however, that only track information can be passed to a fusion center. In this circumstance, refiltering data becomes less straightforward as track states are linearly dependent across time, and the correlation needs to be properly accounted for before/during refiltering. In this work, a model-based temporal decorrelation process for state estimates with process noise will be studied. A decorrelation procedure will be presented based on a linear algebraic formulation of the problem, and process noise estimates will be created that ensure a conservative system state estimate. Numerical examples will be given to demonstrate the efficacy of the proposed algorithm.

Fan, Mengchen; Geng, Baocheng; Li, Keren; Wang, Xueqian; Varshney, Pramod K.

Interpretable Data Fusion for Distributed Learning: A Representative Approach via Gradient Matching

Abstract

This paper introduces a representative-based approach for distributed learning that transforms multiple raw data points into a virtual representation. Unlike traditional distributed learning methods such as Federated Learning, which do not offer human interpretability, our method makes complex machine learning processes accessible and comprehensible. It achieves this by condensing extensive datasets into digestible formats, thus fostering intuitive human-machine interactions. Additionally, this approach maintains privacy and communication efficiency, and it matches the training performance of models using raw data. Simulation results show that our approach is competitive with or outperforms traditional Federated Learning in accuracy and convergence, especially in scenarios with complex models and a higher number of clients. This framework marks a step forward in integrating human intuition with machine intelligence, which potentially enhances human-machine learning interfaces and collaborative efforts.

Lyu, Chenyi; Liu, Xingchi; Wright, James; Barr, Jordi; Hunter, Alasdair; Mihaylova, Lyudmila

Efficient Centralised and Decentralised Gaussian Process Approaches for Online Tracking within Stone Soup

Abstract

This paper explores the application of centralised and distributed Gaussian process algorithms to real-time target tracking and compares their performance. By embedding the algorithms into the Stone Soup, the focus is on the innovative implementation of Gaussian process methods with learning hyperparameters and implementation with a factorised variance of the Gaussian kernel. The performance of the methods with different kernels was evaluated, not only with the Gaussian kernel. Extensive experiments with various kernel configurations demonstrate their importance in enhancing prediction accuracy and efficiency, especially in real-time tracking. The case studies with manoeuvring targets show significant advancements in tracking capabilities, particularly in wireless sensor networks, using optimised Gaussian process methods. This work advances Stone Soup’s capabilities and lays the groundwork for future investigations into adaptive Gaussian Process applications in tracking and sensor data analysis.

Streit, Roy L.

Bayes Optimal Cardinality Filters for Streaming Count Data

Abstract

A time sequence of counts of the number of sensor detections is the sum of the number of detections of objects and the number of false alarms. We model the count sequence as an insertion-deletion process, and model the time-varying number of objects as a birth-death process. Under these modeling assumptions, we derive the optimal recursive Bayesian posterior distribution for the number of objects conditioned only on the count sequence. The method is potentially applicable in management science to detect changes in demand in decision-independent observed data streams and in social media to estimate the number of users who abuse hashtags. A maximum a posteriori (MAP) algorithm for estimating the parameters of the birth-death and insertion-deletion processes is presented.

Xie, Leiyu; Angelini, Federico; Naqvi, Syed Mohsen

NCL-DASB: GEO-Located Maritime Surveillance Labeled Dataset and Annotation API

Abstract

Due to maritime transportation being the most crucial mode in international trade, maritime traffic safety significantly influences global economic development. Detecting anomalous ship behaviors (DASB) serves as a critical measure to safeguard maritime traffic safety. In recent years, data-driven deep learning technologies have witnessed remarkable advancements, and the introduction of high-quality DASB datasets facilitates the rapid and effective transformation of traditional DASB methods into intelligent ones. In this paper, we initially present a labeled DASB dataset named NCL-DASB, recorded at the Tynemouth port in Newcastle, UK. Subsequently, we propose a standard framework for processing vessel AIS data, enabling the transformation of AIS data into vessel trajectory feature information suitable for deep learning through preprocessing. Finally, we open-source an API for annotating vessel trajectory data in the NCL-DASB dataset, intended for the use of future researchers in their studies.

Li, Liangliang; Gao, Lin; Chisci, Luigi; Wei, Ping; Zhang, Huaguo; Farina, Alfonso

Consensus-based distributed streaming coupled tensor factorization

Abstract

This paper discusses the problem of streaming coupled tensor factorization based on sensor networks, where each sensor observes only some features of the targets, and the measurements from sensors are provided in a streaming tensor fashion. Moreover, the observed features of different sensors might overlap (i.e., coupled tensor), and there is no central processing unit to collect all sensor data. Then, in our work, the canonical polyadic (CP) decomposition is exploited to perform local tensor decomposition based on the measurements of each sensor, and average consensus (AC) for diffusing information throughout the network. The proposed method is verified via simulations

Tian, Yunlian; Cao, Xi; Li, Wujun; Yi, Wei

Incorporating Heading Restrictions for Multilane-Road Target Tracking Using Radar Sensor

Abstract

This paper intends to improve the multilane-road target tracking performance by considering heading restrictions. Most existing tracking algorithms assume that vehicles move independently in an open-field environment. However, the movements of vehicles have to be restricted by the geometry of roads, traffic rules, or preset routes. The effective utilization of prior knowledge regarding such restrictions can help to significantly enhance tracking performance. In this paper, we investigate the problem of state estimation while taking into account the heading restrictions imposed by the road direction. To describe the target longitudinal and lateral maneuvering behavior, we design the heading restrictions within a 2-D road coordinate system. The target state vector is then augmented by the y-intercept of the constraint straight line, and the measurement vector is augmented by constructing two pseudo-measurements. Consequently, the heading restrictions are incorporated into unscented Kalman filter based on two augmented vectors, called HR-UKF. Furthermore, we employ the singular value decomposition method to enhance numerical stability. Finally, the effectiveness of the proposed algorithm is validated through numerical simulations and real-measured data.

Raitoharju, Matti; García-Fernández, Ángel F.; Ali-Löytty, Simo; Särkkä, Simo

Stacked iterated posterior linearization filter

Abstract

The Kalman Filter (KF) is a classical algorithm that was developed for estimating a state that evolves in time based on noisy measurements by assuming linear state transition and measurements models. There exist various KF extensions for non-linear situations, but they are not exact and provide different linearization errors. The Iterated Posterior Linearization Filter (IPLF) does the linearizations iteratively to achieve better linearization. However, it is possible that some measurements cannot be well linearized using the current knowledge, but their linearization may be more successful after more measurements are available. Thus, we propose an algorithm that can store the older state elements and measurements when their linearization error is high. The resulting algorithm Stacked Iterated Posterior Linearization Filter is based on linear dynamic models and uses information from multiple time instances to make the linearisation of the measurement function. Results show that the proposed algorithm outperforms traditional KF extensions when some of the measurements cannot be well linearizad with current knowledge, but better with future knowledge.

Lone, Jaffar Ali; Bhaumik, Shovan; Tomar, Nutan Kumar

Functional Observer-Based Event-Triggered Control for Linear Discrete-time Descriptor Systems

Abstract

This paper is devoted to studying the problem of functional observer-based event-triggered control (ETC) of linear discrete-time descriptor systems. As functional observers directly estimate a linear combination of states without estimating the whole state vector, we demonstrate that utilizing the functional observer's output as the control input to the plant yields enhanced control performance and requires less number of sampling events. An advantage of ETC over conventional time-triggered control lies in its ability to dynamically adjust control actions only when needed, leading to efficient resource usage. In the ETC framework, updates to the observer-based controller occur exclusively during the event-triggered sampling instants defined by some predefined event condition. In this paper, despite a substantial reduction in sampling frequency, we establish a condition for the ultimate boundedness of the closed-loop system, expressed in terms of matrix inequalities. A numerical example is presented to showcase the feasibility and effectiveness of the theoretic results.

Liu, Yang; Liu, Yu; Wang, Xueqian; Zhang, Linping; Jiang, Zhizhuo; Li, Yaowen; Yan, Chenggang; Fu, Ying; Zhang, Tao

A Cross-modal Fusion Method for Multispectral Small Ship Detection

Abstract

The fusion module of RGB and infrared (IR) remote sensing images is the key of multispectral ship detection. Existing works have shown that the cross-attention-based feature fusion can achieve good performance by extracting the complementary information of RGB and IR modalities. However, the existing commonly used cross-attention mechanisms introduce lots of redundancy parameters and mainly focus on global feature interaction of multispectral images, ignoring local detail information that is also important for small ship detection. In this paper, we propose a novel multispectral ship detection approach named LoGFusion. In LoGFusion, we design the cross stage partial module with partial convolution (CSPMPC) to reduce feature redundancy and utilize the local cross-modal fusion module (LoCFM) and global cross-modal fusion module (GCFM) to capture both local and global cross-modal features. Furthermore, we introduce a Multispectral Small Ship Dataset (MSSD) containing over 5k ship targets for small target detection. Experiments on MSSD validate the effectiveness of our method in terms of small ship detection in multispectral images.

Sebbak, Faouzi; Senouci, Mustapha Reda

Optimizing Cardinality-aware Combination Rules in Belief Functions Theory: an Enhanced Framework

Abstract

Belief Functions Theory (BFT), also known as Dempster-Shafer theory, has emerged as a powerful framework for uncertain modeling and reasoning in various domains. At the heart of BFT, lies the concept of combination rules, which govern the fusion process and have a profound impact on the quality and reliability of the results. Currently, there is a noticeable trend in the field, with a growing emphasis on cardinality-aware combination rules, reflecting the need for more nuanced and context-aware approaches to uncertainty management. Despite this, there exists a notable gap in the implementation of cardinality-aware rules within existing frameworks, limiting their applicability in complex decision-making scenarios. This paper addresses this gap by proposing an enhanced MATLAB framework that optimizes computational complexity, thereby enabling efficient implementation of both current and future cardinality-aware combination rules. By providing detailed insights into the framework's structure, key functions, and examples of the implementation of these cardinality-aware rules, this paper aims to bridge the identified gap and enhance the usability of BFT in practical applications. The source code for this enhanced framework has been shared with the community.

Wei, Xinwei; Zhang, Linao; Lin, Yiru; Wei, Jianwei; Zhang, Chenyu; Yi, Wei

Transformer-based Multi-Sensor Hybrid Fusion for Multi-Target Tracking

Abstract

Deep learning (DL) approaches, which do not rely on models and can learn complex relationships within data, garner increasing attention in the model-free multi-target tracking (MTT) domain. However, the study of applying the DL method to multi-sensor fusion-based MTT is relatively less. In this paper, we propose a Transformer-based distributed multi-sensor MTT approach, which adopts a hybrid fusion structure with both feature-level and decision-level fusion. First, for each local sensor, the high-dimensional feature information is extracted from measurements based on a Transformer-based tracking module, which enables continuous tracking of multiple targets and provides outputs of predicted target states and the corresponding uncertainties. Then, the outputs of local sensors are fused using the covariance interception (CI) fusion rule. Finally, to further improve the fusion performance, the decision-level information is fed into the decoder with the feature-level information to obtain the predicted target state and uncertainty after deep fusion. In this way, we realize the deep utilization of different sensor information and achieve a hybrid multi-sensor fusion, namely, feature-level decision-level. Simulation results show that the proposed fusion method outperforms the CI algorithm in various tracking scenarios.

Griebel, Thomas; Müller, Johannes; Buchholz, Michael; Dietmayer, Klaus

Adaptive Kalman Filtering Based on Subjective Logic Self-Assessment

Abstract

Monitoring and self-assessment of tracking algorithms are essential in modern automated driving systems. However, the further use of this self-assessment information is another growing and not thoroughly studied area of research. One option is to adapt the parameters configured in the tracking algorithm online to obtain better and more robust tracking results directly. The paper proposes a novel overall concept and framework for adaptive Kalman filtering using subjective logic. Based on a self-assessment method, we present multiple variants of adaptive strategies to adapt the noise assumptions online for Kalman filtering. This paper focuses mainly on adaptation procedures for multi-sensor Kalman filters. The proposed method is evaluated in various experiments and compared with state-of-the-art adaptive Kalman filters.

Handke, Sebastian Thomas; Broetje, Martina; Steffes, Christian; Koch, Wolfgang

Track Evaluation of GSM based Passive Radar: Model vs. Real-World Results

Abstract

Closing the feedback loop has proven essential for performance improvement and enhancing system reliability in a number of applications. Track evaluation addresses these goals by investigating system influences and analyzing discrepancies between expected and actual operation. Developing a model of a passive radar system which approximates its operational capabilities is therefore not only essential for performance prediction and pre-operational analysis but offers the opportunity for further system improvement. In the following, we assess the fit of a passive radar model to real-world data to investigate if it provides a reasonable representation of the observed results. There, we show that the detection probability model as well as the estimated lower bound on achievable estimation accuracy, where we utilize the Cramér-Rao Lower Bound (CRLB), give a reasonable approximation of real-world results collected by a passive radar system which utilizes Global System for Mobile Communications (GSM) signals.

Liu, Yi; Li, Xi; Yang, Le; Mihaylova, Lyudmila; Li, Ji

On the Gaussian Filtering for Nonlinear Dynamic Systems Using Variational Inference

Abstract

This paper introduces a new variational Gaussian filtering approach for estimating the state of a nonlinear dynamic system. We first assume that the predictive distribution of the state is Gaussian and derive an iterative method for updating the state posterior in the natural parameter space through Kullback-Leibler divergence minimization. The obtained update rule is the same as that of the conjugate-computation variational inference technique in Bayesian learning. The derivation here is simpler and more insightful. We then impose a Wishart prior on the inverse of the state prediction covariance to take into account the impact of approximating the state predictive distribution using a Gaussian density on the state posterior estimation. The prediction covariance is identified jointly with the state using variational inference and the established state posterior update rule to achieve the desired Gaussian filtering. Simulation study examines the performance of the proposed filtering framework in target tracking based on bearing and range measurements.

Gramsch, Christian; Yang, Shishan; Alqaderi, Hosam

A Batch Update Using Multiplicative Noise Modelling for Extended Object Tracking

Abstract

While the tracking of multiple extended targets demands for sophisticated algorithms to handle the high complexity inherent to the task, it also requires low runtime for online execution in real-world scenarios. In this work, we derive a batch update for the recently introduced elliptical-target tracker called MEM-EKF*. The MEM-EKF* is based on the same likelihood as the well-established random matrix approach but is derived from the multiplicative error model (MEM) and uses an extended Kalman filter (EKF) to update the target state sequentially, i.e., measurement-by-measurement. Our batch variant updates the target state in a single step based on straightforward sums over all measurements and the MEM-specific pseudo-measurements. This drastically reduces the scaling constant for typical implementations and indeed we find a speedup of roughly 100x in our numerical experiments. At the same time, the estimation error which we measure using the Gaussian Wasserstein distance stays significantly below that of the random matrix approach in coordinated turn scenarios while being comparable otherwise.

Xia, Yuxuan; Stenborg, Erik; Fu, Junsheng; Hendeby, Gustaf

Bayesian Simultaneous Localization and Multi-Lane Tracking Using Onboard Sensors and a SD Map

Abstract

High-definition map with accurate lane-level information is crucial for autonomous driving, but the creation of these maps is a resource-intensive process. To this end, we present a cost-effective solution to create lane-level roadmaps using only the global navigation satellite system (GNSS) and a camera on customer vehicles. Our proposed solution utilizes a prior standard-definition (SD) map, GNSS measurements, visual odometry, and lane marking edge detection points, to simultaneously estimate the vehicle's 6D pose, its position within a SD map, and also the 3D geometry of traffic lines. This is achieved using a Bayesian simultaneous localization and multi-object tracking filter, where the estimation of traffic lines is formulated as a multiple extended object tracking problem, solved using a trajectory Poisson multi-Bernoulli mixture (TPMBM) filter. In TPMBM filtering, traffic lines are modeled using B-spline trajectories, and each trajectory is parameterized by a sequence of control points. The proposed solution has been evaluated using experimental data collected by a test vehicle driving on highway. Preliminary results show that the traffic line estimates, overlaid on the satellite image, generally align with the lane markings up to some lateral offsets.

Gade, Brita H. Hafskjold; Kloster, Morten; Vooren, Carina N.; Sjøberg, Alexander Meyer

Multi-target tracking within large geographical areas – algorithms for improved accuracy and speed

Abstract

This paper addresses two challenges which are specific for tracking large amounts of vehicles within wide geographical areas. The first problem is related to the Earth's curvature, which may degrade the prediction accuracy over long distances. The second problem is concerning the gating process, which may be slow. This paper proposes a single target tracking algorithm that predicts along the great circle, and thus minimizes the unmodelled prediction error, and a gating algorithm that speeds up the gating process by using a combination of clustering and the n-vector position representation.

Luo, Mingjie; Zhou, Jie; Zou, Qingke

Multisensor Estimation Fusion Based on Kernel Mean Embedding

Abstract

This work deals with the estimation fusion for distributed multisensor systems under the framework of local estimates being taken as probability density functions. The estimation fusion is formulated to an optimization problem that minimizes the sum of squared distances between the fused probability density and each local probability density. The maximum mean discrepancy, which is a distance between two probability density functions, is considered. It is defined by the kernel mean embeddings from the probability density function space to a reproducing kernel Hilbert space. For the quadratic, cubic and Gaussian kernels, either the analytical solutions are derived or the numerical methods are developed for solving the aforementioned optimization problem. Numerical experiments are provided to illustrate the performance of the proposed estimation fusion methods.

Tamir, Ella; Solin, Arno

Learning to Approximate Particle Smoothing Trajectories via Diffusion Generative Models

Abstract

Learning dynamical systems from sparse observations is critical in numerous fields, including biology, finance, and physics. Even if tackling such problems is standard in general information fusion, it remains challenging for contemporary machine learning models, such as diffusion models. We introduce a method that integrates conditional particle filtering with ancestral sampling and diffusion models, enabling the generation of realistic trajectories that align with observed data. Our approach uses a smoother based on iterating a conditional particle filter with ancestral sampling to first generate plausible trajectories matching observed marginals, and learns the corresponding diffusion model. This approach provides both a generative method for high-quality, smoothed trajectories under complex constraints, and an efficient approximation of the particle smoothing distribution for classical tracking problems. We demonstrate the approach in time-series generation and interpolation tasks, including vehicle tracking and single-cell RNA sequencing data.

Laudy, Claire; Museux, Nicolas; Fossier, Simon; Reverdy, Céline; Fougère, Tom; Audouy, Amandine; Lopez, Clara; Chenevier, Florent

HLIF2024: a Competition for High-Level Information Fusion

Abstract

This paper presents the HLIF2024 competition.HLIF2024 is the first competition for High-Level Information Fusion solutions, co-organized with the FUSION2024 conference. The paper presents the challenge use case associated with the competition: information extraction and fusion from notices to air operation data. The aim of the challenge is to automate the process of this data, in order to support pilots with their decision making phases during their flights. To simulate that, the challenge encompasses a fictitious flight during which information is requested by the pilot. To provide the pertinent information to the pilots, the participants are asked to populate a given domain ontology that can then be queried to answer the challenge questions. The participants solutions are evaluated with regards to the correctness, completeness and preciseness of their answers to the questions. The motivation behind the use case is presented, together with the practice data-set that was built and provided to the competition participants. The evaluation methodology and metrics used to rank the participants solutions are detailed. The results of the competition will be presented at the FUSION2024 conference.

Zhang, Donglin; Duan, Zhansheng; Sun, Yiyong; Yin, Feng

LMMSE-Aided WLLS Location Estimators for Source Localization with RSS Measurements

Abstract

Received signal strength (RSS) measurements can be converted to the distance estimates between the emission source and the sensors to construct a system of linear equations, thereby allowing for the use of the weighted linear least squares (WLLS) estimators for location estimation. However, estimating the squared distances from the RSS measurements governed by the log-normal shadowing effect presents a major challenge in such approaches. In this paper, we propose a linear minimum mean square error (LMMSE) estimator of the squared distance between the emission source and the sensor first. Then a LMMSE-aided WLLS (LMMSE-WLLS) location estimator and its unbiased counterpart are presented for source localization. Furthermore, their estimation performance are analyzed in terms of mean square error (MSE) and covariance. It is found that the proposed LMMSE-aided WLLS location estimators have better estimation performance than existing WLLS estimators. Numerical examples also demonstrate the performance superiority of the proposed location estimators for source localization.

D'Afflisio, Enrica; Millefiori, Leonardo M.; Braca, Paolo; Guerriero, Marco

MARITRAC: Maritime trajectory classification using object instance segmentation with model-based generated data augmentation

Abstract

Maritime surveillance, characterized by high-volume data streams, necessitates effective methods for the automatic extraction of meaningful information and accurate classification of vessel patterns. We introduce MARITRAC (maritime trajectory classification), an innovative approach that leverages MASK R-CNN, a state-of-the-art computer vision algorithm, to classify maritime trajectories. The key idea behind MARITRAC is to convert trajectory data into images that capture spatio-temporal patterns. These trajectory images are then used as input to a MASK R-CNN model that is trained on synthetically generated data to classify different types of maritime trajectories. By combining computer vision techniques with trajectory data analysis, MARITRAC provides an effective and automated method for characterizing and distinguishing between different maritime behaviors. To overcome the notable lack of labeled trajectory anomaly datasets, the training is performed with a set of synthetically generated trajectories, created using the piecewise Ornstein-Uhlenbeck dynamic model. The effectiveness of MARITRAC is demonstrated through application and evaluation in two main experiments, involving both synthetic and real-world data. The approach showcases promising performance in classifying maritime trajectories, and the results position MARITRAC as a valuable tool for real-time maritime surveillance.

Forkel, Bianca; Berthold, Philipp; Maehlisch, Mirko

Taking Advantage of Road Users Occluding the Road: Supporting Camera-based Road Tracking in Shared Spaces using Radar Doppler Measurements

Abstract

Conventional road tracking approaches rely on direct measurements of the road. Using camera, LiDAR, or radar sensors, they detect lane markings, textures, road boundaries, curbs, or obstacles. However, all require a direct line of sight to the road. In shared spaces, e.g., a busy campus road, a lot of pedestrian groups are occluding the road for all sensors. Hence, we present a novel measurement concept to support road tracking in presence of other road users. In fact, we are taking advantage of them: Where other road users are moving, there might be the road. Therefore, we probabilistically identify dynamic objects based on radar Doppler measurements. Considering the measurement uncertainties, we then build a probability map of road user paths. This is used to derive probable road areas. Integrating this measurement into an existing camera-based road tracking, we evaluate the accuracy of the resulting road estimate using an HD map.

Chen, Desheng; Li, Xiaolong; Wang, Mingxing; Guan, Lingjie; Cui, Guolong

A Computationally Efficient Multi-Channel Multi-Pulse Coherent Fusion Algorithm for High-Speed Target Detection

Abstract

Compared with the monostatic radar which only performs signal fusion in the multi-pulse dimension, the bistatic multiple-input multiple-output (MIMO) radar can also fuse multi-channel signals to improve the detection ability of high-speed moving targets. However, how to tackle the range migration (RM) caused by high-speed motion and compensate the signal difference between channels are the key problems in bistatic MIMO radar. To solve these problems, this paper proposes a computationally efficient multi-channel multi-pulse fusion algorithm. Firstly, we establish the echo model of bistatic MIMO radar with high-speed moving targets and utilize the Radon Fourier transform (RFT) algorithm to overcome the RM and complete the multi-pulse fusion. Then, the output characteristics of RFT, including peak and phase differences, are analyzed in detail. On this basis, a multi-channel coherent fusion method based on geometric information is proposed, which can eliminate the phase differences among channels and complete coherent fusion. Finally, numerical experiments show that the proposed algorithm can effectively detect the target under a low signal-to-noise ratio (SNR). Compared with the existing methods, the proposed algorithm can achieve a balance between computational complexity and detection performance.

Wu, Qinchen; Sun, Jinping; Yang, Bin

Trajectory Poisson Multi-Bernoulli Filter for Group Target Tracking

Abstract

This paper presents a new trajectory Poisson multi-Bernoulli (TPMB) filter for group target tracking. Due to the collective behavior and dense spatial distribution, exact trajectory estimation is an extremely challenging task in group target tracking. Aiming for improved performance in group trajectory estimation, the virtual leader-follower model is incorporated into the standard TPMB filter in this paper to address coordinated motion within a group. Moreover, the Gaussian implementation and L-scan approximation for the proposed group target trajectory PMB (GTTPMB) filter are also provided. Finally, a simulation scenario with splitting and merging of groups is established to evaluate the proposed GTTPMB filter. The results demonstrate that the proposed filter can effectively estimate the trajectories of group members without introducing additional computational burden.

Cavalcanti, Vinícius M. G. B.; Silva, Felipe O.; de Lima, Danilo A.

Genetic Algorithm-based Tunings for Baro-fused Inertial Navigation Systems

Abstract

Information fusion is of paramount importance for sensitive navigation applications. Inertial Navigation Systems (INS) and barometers are two examples of navigation systems/ sensors that have complementary error characteristics, and which benefit from such integration. This works revisits two well-established fusion algorithms between the aforementioned systems/sensors, namely, one based on closed-loop mechanizations, and the second based on the Extended Kalman Filter (EKF). Historically, performance of such approaches has shown to be sub-optimal due to the excess of empiricism involved in the associated tuning procedures. In this sense, novel Genetic Algorithm (GA)-based tunings for both approaches are investigated, and results from experimentally conducted tests show their superiority for stabilizing the vertical channel error response of the baro-fused INS. As a requirement for the latter, however, one sees the need for employing raw sensor data sets (in the optimization procedure) that correspond to the dynamics and sensor grade of the application at hand.

Fetzer, Toni; Bullmann, Markus; Kastner, Steffen; Deinzer, Frank; Grzegorzek, Marcin

Advancing Smartphone-based Indoor Positioning through Particle Distribution Optimization

Abstract

Smartphone-based indoor positioning and navigation remains a challenging task, as specialized technologies such as ultra-wideband (UWB) or Wi-Fi fine-time measurement are still niche and supported by only a few flagship smartphones. Therefore, standard technologies based on RSSI measurements, mainly Bluetooth Low Energy (BLE) and Wi-Fi, are used to obtain absolute positioning information of pedestrians inside buildings. Sensor fusion methods combine this with relative information from modeling human movement using sensor data provided by the smartphone's IMU. It is also common practice to restrict this movement to the actual accessible areas of the building (e.g. restricting moving through walls), using spatial models based on the building's floor plan. Without further assumptions, this complexity inevitably leads to a non-linear and non-Gaussian state space model. A common tool for (position) estimation in such scenarios is the broad class of particle filters. However, the use of such spatial constraints accelerates the well-known problem of sample impoverishment, which in the worst case can lead to the particle filter completely losing track, getting stuck and never recovering.This work begins with a brief presentation of an award-winning Indoor Positioning System (IPS) derived from previous work. Based on this, we present several approaches using Particle Distribution Optimization (PDO) that attempt to solve the impoverishment problem and ultimately lead to better overall positioning results. In the experiments, we compare them in two different buildings under realistic conditions and discuss the results in detail.

Song, Tieshuai; He, Guidong; Yang, Bin; Dong, Zhao; Wang, Jun; Zhong, Fengjun

mmWave Radar and Image Fusion for Depth Completion: a Two-Stage Fusion Network

Abstract

Pixel-wise depth completion using multi-sensor fusion is crucial in areas such as autonomous driving. While LiDAR and image fusion methods exhibit reliability, it can face challenges in adverse weather conditions, such as rain and fog. In contrast, mmWave radar, emerged in recent years, has stronger anti-interference capability. However, radar point typically features high sparsity. And mmWave radar has lower resolution in the height dimension, leading to increased errors when projected onto the image plane. To solve the problem, this paper proposes a two-stage fusion convolutional neural network. In the first stage, image features are utilized to filter the noisy radar point cloud and learn the mapping of radar points to image regions. In the second stage, we perform multi-scale fusion of the image with the coarse depth map generated in the first stage to predict the missing depth values. Experiment results indicate that our improved strategy reduces the error of depth value estimation. Our network shows a 4.5% improvement in root-mean-square error (RMSE) over the best method.

Lopez, Enzo; Dahia, Karim; Merlinge, Nicolas; Winter-Bonnet, Benedicte; Maschiella, Alain; Musso, Christian

Sequential Markov Chain Monte Carlo methods on Matrix Lie Groups

Abstract

Particle filters on Lie Groups represent a cutting-edge approach in nonlinear filtering and control. By generating randomly distributed particles from proposed densities, while accurately preserving rotation matrices, they offer a promising solution to nonlinear systems. However, particle filters grapple with numerous challenges such as high computational costs due to the large number of particles needed, vulnerability to particle degeneracy as well as the curse of dimensionality. To address these issues, Sequential Markov Chain Monte Carlo (SMCMC) methods aim to mitigate sensitivity to high-dimensional systems by iteratively sampling from the posterior distribution of the system state, gradually refining the estimation. In this paper we extend SMCMC techniques to matrix Lie groups, resulting in the Lie Groups Sequential Markov Chain Monte Carlo filter that circumvent the major drawbacks of particle filters. The proposed approach incorporates enhancements based on the Metropolis-Hastings algorithm, further improving the algorithms efficiency and robustness. To validate the effectiveness of these methods, the algorithms are tested on an Unmanned Aerial Vehicle (UAV) navigation scenario with challenging discrepancies in the noise tuning.

Ketterer, Pascal; Hoher, Patrick; Reuter, Johannes

Runtime Optimization in Interacting Multiple Model Filtering with Down-Sampling and Out-of-Sequence Measurements

Abstract

Interacting multiple model filters are most commonly used in the context of maneuvering targets, as they can represent the different dynamics of a real system by combining the estimates of multiple models. However, the interacting multiple model approach generally requires more computational effort than a single Kalman filter. In this work, down-sampling is used to reduce the computational effort. We propose an adaptive scheme to maintain the accuracy of the estimator to a defined level. To this end, the trace of the innovation covariance matrix is evaluated, and if it lies above a certain threshold, out-of-sequence measurements are iteratively used to improve the estimate until the uncertainty threshold is met. The approach is evaluated by Monte Carlo analysis. The results show that with this approach, the number of measurements to be processed, and thus the computational effort can be dynamically reduced, while the accuracy remains at a desired level.

Jacquemart, Alexandre; Hadj-Bachir, Mokrane; Ieng, Sio-Song; Gruyer, Dominique

A new Method for parametric BBF generation

Abstract

In a large set of applications, belief theory is applied to handle and to manage efficiently uncertainty and conflict. An essential step consists in choosing the appropriate Basic Belief Function (BBF) to generate Basic Belief Assignments (BBA) before the combination stages. In this context, we introduce a novel method that leverages belief theory to generate BBA using a parametric family. This approach offers a structured framework for evaluating objective criteria and selecting the most suitable BBF for a given scenario. The method is designed to accommodate a wide range of applications, from decision-making in uncertain environments to data fusion in complex systems. The key advantage of our method lies in its flexibility and adaptability. By using a parametric family of functions, the method can tailor the BBA generation process to specific requirements, such as the level of noise or discordance in the sources. This allows us to optimize the performance of the fusion architecture and improve decision-making accuracy. Our method was applied to select the most suitable candidate from a set of functions, aiming to minimize the effects of noise and discordance. This involved evaluating the performance of each function against a set of objective criteria, such as robustness and reliability. The results highlighted that our method outperformed existing approaches, demonstrating its effectiveness in generating BBAs that are well-suited to the task at hand. In conclusion, our method offers a new level of adaptability and generality in BBA generation, enabling the customization of hyperparameters to optimize data fusion processes.

Rangel, Pablo; de Abreu Nunes, Vinícius Maravalhas; da Rocha Salazar, Matheus; Simões, Reinaldo Albuquerque; Morgado de Castro Rosa, Luiza; Gomes de Carvalho Júnior, José

A Hybrid Model for Detection and Classification of Fishing Activity: A Context-Based Approach

Abstract

Fishing activity matters to the entire world because it affects the economy, ecosystems, and human sustainability. Detecting and classifying fishing activity is a challenge that has been the focus of some studies over the years, but many of them are limited to using solutions without considering context information. These works show solid classification results but are limited to the classification task only. In general, most of them assume a fishing activity is in progress. Hence, these works have not explored the potential of false fishing detections, leading to misclassifications and forcing the fitting of detections into one type of fishing technique. Our hypothesis is that geographic context information can improve the detection of fishing activity and the classification of different types of fishing. Therefore, individual and collective information were extracted from a public labeled database used in related works. Individual information is the kinematics of each vessel, while collective information is the geographic fishing areas. The model adopts a stacking ensemble strategy, with the first level being a kinematic classifier and the second level a correlation model with geographic context. The solution presented effectively fills the identified gaps and demonstrates robust results.

Ouled Sghaier, Moslem; Hadzagic, Melita; Ye Yu, Jun; Shton, Sofia; Shahbazian, Elisa

Leveraging Generative Deep Learning Models for Enhanced Change Detection in Heterogeneous Remote Sensing Data

Abstract

In this paper, we introduce an innovative approach for Change Detection (CD) in heterogeneous (multimodal) multitemporal remote sensing (RS) images employing deep feature comparison through the utilization of two advanced deep learning models: Generative Adversarial Networks (GANs) and autoencoders. First, Deep Convolutional GANs are implemented to convert the multi-modal image(s) into synthetic image(s) of the same modality. Subsequently, autoencoders are trained and employed to extract compressed representations of both initial images. And finally, a change map is obtained by combining/fusing the original image with its corresponding generated change-free image resulting from the difference between the two learned compressed representations. Our proposed CD technique is general, able to accommodate change detection (CD) algorithms for RS images expressing any type of change. Experimental evaluations on Very High Resolution Synthetic Aperture Radar (VHRSAR) and optical imagery validate the enhanced performance of the proposed method compared to existing state-of-the-art CD techniques in handling heterogeneous RS data.

Hubert, Bastien; Dahia, Karim; Merlinge, Nicolas; Giremus, Audrey

Adaptive Kriging Particle Filter and its Application to Terrain-Aided Navigation

Abstract

In GNSS-denied and poorly-known environments, reliable autonomous navigation is a major challenge, as conventional data fusion algorithms require an extensive knowledge of their surroundings to accurately estimate the vehicle state. To address this issue, we propose to use an adaptive Gaussian process regression to model an approximation of the environment solely based on scarce and noisy samples. This paper takes advantage of the flexibility of Gaussian processes to dynamically model the underlying terrain by adapting the process to the relevant data at each step. To this end, we propose to locally fit the Gaussian process and perform regression by using only a subset of data points selected according to a proximity criterion. The developed method employs a regularised particle filter to effectively estimate the system state using the output of the regression. By integrating Gaussian process-based terrain predictions, the particle filter can effectively compensate for the lack of precise terrain information, thus enhancing navigation performance in GNSS-denied scenarios. To evaluate the effectiveness of the proposed approach, simulations were performed in terrain-aided navigation of an unmanned aerial vehicle. Comparative analysis with existing navigation methods illustrates the superiority of the proposed approach in terms of accuracy and robustness.

Maresca, Salvatore; Malacarne, Antonio; Amir, Malik Muhammad Haris; Ahmad, Fawad; Pandey, Gaurav; Bogoni, Antonella; Scaffardi, Mirco

Genetic Algorithms for Distributed MIMO Radar Antenna Position Optimization

Abstract

In multiple-input multiple-output (MIMO) radars, carrier frequency, signal bandwidth and antenna geometry have a deep impact on the ambiguity function (AF). In particular for systems employing widely separated antennas, sidelobes and azimuth ambiguities may appear in the MIMO-AF, potentially leading to a degradation of the system detection and localization capabilities. The aim of this paper is to optimize antenna positions along the MIMO baseline using genetic algorithms (GAs). Key performance indicators (KPIs), such as the peak-to-maximum and peak-to-average sidelobe ratios, as well as the range and cross-range resolutions are investigated as potential optimization criteria. As a practical study case, a MIMO radar working the in X-band is simulated. The system employs two transmitters (TXs) and four receivers (RXs), with free-located TX-RX antenna pairs. The analysis is conducted for a point-like target at different positions. The optimization is carried out by means of the GA-based function library available in MATLAB©, selecting both single and multiple KPIs as optimization criteria. In this latter case, the advantage is the optimization of more KPIs at the same time, however at the expense of a larger computation time.

Hilmarsen, Henrik; Dalhaug, Nicholas; Anthonsen Nygård, Trym; Brekke, Edmund Førland; Mester, Rudolf; Stahl, Annette

Maritime Tracking-By-Detection with Object Mask Depth Retrieval Through Stereo Vision and Lidar

Abstract

The momentum towards autonomous technology is building up in the maritime domain, as the automotive industry has made big steps towards autonomous driving. The automotive industry has increasingly utilized visual methods for multi-object tracking (MOT), with the help of accessible benchmarking datasets such as KITTI. This paper presents a tracking pipeline that tracks in the world frame by using elements of a well-established visual tracking method that tracks objects in the image frame. The pipeline fuses 3D information from lidar or stereo vision with object masks from a deep learning-based ship detector. To handle occlusions, we implemented a track manager that predicts lost objects' movement until they reappear. Also, we provide a comparison between using lidar and stereo as the depth modality in the tracking pipeline. Results from a real-world experiment indicate that camera-lidar fusion gives consistently precise estimates, while the precision with stereo depends on the range and the type of vessel tracked.

Straka, Ondřej; Havlík, Jindřich

Design of Unitless Normalized Measure of Nonlinearity for State Estimation

Abstract

The paper deals with measures of nonlinearity. In state estimation, they are utilized i) to select a suitable state estimation algorithm by assessing the nonlinearity of a system model, ii) to adapt the estimation algorithm structure or parameters, or iii) to indicate the possible effect of strong nonlinearity that leads to estimate credibility loss. This paper summarizes the state of the art of nonlinearity measures, focusing on the mean-square-error-based measure of nonlinearity. Its weak point is illustrated, and based on this, requirements for the new measure of nonlinearity are formulated. A new nonlinearity measure that is both unitless and normalized is designed. Its properties are demonstrated using numerical tracking experiments.

Ramajo-Ballester, Álvaro; de la Escalera Hueso, Arturo; Armingol Moreno, José María

Towards broader spatial-context 3D object detection for autonomous driving

Abstract

This work presents an exhaustive analysis and a quantitative performance comparison between the use of information from infrastructure and vehicle mounted sensors for 3D object detection in autonomous driving environments. To do so, LiDAR point clouds have been used as the main data input and the most popular and well-established models have been considered for this task: Second, PointPillars and PV-RCNN. They have all been trained on the DAIR-V2X cooperative dataset, since it offers both the infrastructure and vehicle perspective. The broader spatial context and greater field of vision from an elevated point of view demonstrate superior performance by mitigating occlusions and overcoming the inherent limitations of a reduced perception range from onboard a vehicle. However, this comes with its own challenges to avoid losing detection capabilities for smaller objects. The main objective of this work is to provide a like-for-like comparison of the real performance difference, isolating the point of view as the only modified variable.

Gruden, Pina; Nosal, Eva-Marie; Henderson, E. Elizabeth

Automated Acoustic Tracking of a Sperm Whale (Physeter macrocephalus) using a Wide Baseline Array of Sensors

Abstract

Underwater acoustics is a key tool for monitoring marine environments and understanding marine mammal populations. However, extracting meaningful information from passive acoustic recordings poses challenges due to overlapping signals, species-specific vocalization behavior, and missed and false detections. Many methods for marine mammal tracking and localization rely on human operators for signal detection and measurement association, which is a subjective and laborious process. In this paper we demonstrate a fully automated framework for marine mammal tracking and localization using wide-baseline arrays based on a multi-target Bayesian approach. Leveraging a "track-before-localize" strategy and fusing information from multiple sensors and virtual sensors, the framework eliminates the need for detection, classification, or association steps, thereby improving efficiency and objectivity. The feasibility and performance of the proposed framework are demonstrated using real-world data of a clicking sperm whale from the US Navy’s AUTEC test range.

Blair, W. Dale; Bar-Shalom, Yaakov

Design of Two-Model IMM Estimators for Tracking Maneuvering Targets

Abstract

The Interacting Multiple Model (IMM) estimator is well accepted as the best algorithm for tracking maneuvering targets, when the computational cost is considered. The IMM estimator includes a model-conditioned estimator for each kinematic model and the switching between modes or models is assumed to be a finite state Markov chain. The two-model configuration of the IMM estimator is the most commonly used version. The two-model version typically includes either two nearly constant velocity (NCV) motion models or one NCV model and one nearly constant acceleration (NCA) model. In this paper, the design of these two configurations of the IMM estimator is considered. In this case, design refers to the selection of the motion models (i.e.,NCV or NCA) and the corresponding process noise variances. The design methods are first considered for single coordinate tracking with measurements of position, and simulation results are given to illustrate the effectiveness of the design methods. Then, the design methods are applied to radar tracking, and simulation results are given to demonstrate the effectiveness of the design methods.

Jiao, Hao; Zhang, Peng; Yan, Junkun; Dang, Xudong; Jiu, Bo; Liu, Hongwei

Joint Beam Selection and Power Allocation for Multi-target Tracking in C-MIMO Radar Network

Abstract

In this paper, a joint beam selection and power allocation (JBSPA) scheme for multi-target tracking is proposed in a collocated MIMO (C-MIMO) radar network. The goal of this scheme is to achieve better resource utilization efficiency with a given resource budget. Under the condition of sufficient resources, the scheme minimizes the total resource consumption of the C-MIMO radar network. When the sensor resources are insufficient, the scheme maximizes the number of tracked targets that meet the tracking requirements. To evaluate the performance of multi-target tracking, we normalize and utilize the Bayesian Cramér-Rao lower bound (BCRLB) as the performance evaluation criterion. The JBSPA scheme is formulated as a non-convex optimization problem involving integer and continuous variables that are coupled. To address this problem, we propose a fast and effective three-step solution technique. Simulation results demonstrate that the proposed JBSPA scheme can save resources, significantly increase the target capacity, and improve the resource utilization efficiency of the C-MIMO radar network.

Zou, Zhiyuan; Miao, Qing; Wei, Jianwei; Lin, Yiru; Wei, Xinwei; Yi, Wei

Trajectory Generation and Dynamic Continuous Activity Recognition for Radar Swarm Targets

Abstract

The swarm targets have shown great potential for both military and civilian applications, driving a high demand for reliable trajectory generation and accurate activity recognition. In this paper, we propose a trajectory generation method and establish an end-to-end deep learning model for dynamic continuous activity recognition of swarm targets. First, we devise an activity transition model of the drone swarm based on a continuous-time Markov chain (CTMC). Subsequently, the minimum snap trajectory generation algorithm is employed to generate the trajectories. After that, to recognize the dynamic continuous activity of targets, we develop an end-to-end neural network model to extract spatial and temporal features for swarm targets detected by radar across multiple frames. Finally, we demonstrate the effectiveness and robustness of our proposed method through simulation results.

Belfadel, Djedjiga; Haessig, David

Optical Flow for Drone Horizontal Velocity Estimation without GPS

Abstract

Unmanned aerial vehicles (UAVs) are widely used in military and civilian applications, such as surveillance, disaster monitoring, and rescue missions. They rely heavily on global positioning systems (GPS) for navigation. However, the accuracy and availability of GPS positioning, navigation, and timing (PNT) data can suffer losses due to jamming and spoofing. To address overreliance on GPS infrastructure, this paper presents a vision-aided system to enable UAVs to estimate velocity and continue operation when GPS signal is intermittently lost. The system fuses data from onboard sensors including the inertial measurement unit (IMU), barometer, and optical flow camera using an extended Kalman filter (EKF). The efficiency of this approach was validated through simulation experiments involving a UAV navigating in a circular and square motions. Statistical analysis confirmed the proposed system can provide state estimation and smooth trajectory control for UAVs during GPS signal outages.

Li, Siyuan; Han, Deqiang; Dezert, Jean; Yang, Yi

Learning-Based BBA Modeling Approach with Multi-Method Fusion

Abstract

Dempster-Shafer evidence theory (DST) is a theoretical framework for uncertainty modeling and reasoning, with modeling the basic belief assignment (BBA) as one of its most crucial and challenging tasks. The prevailing BBA determination methods have their own pros and cons, and the joint use of them is expected to provide a better BBA. To realize an end-to-end BBA modeling without explicitly using various prevailing BBA modeling methods, a learning-based BBA modeling approach with multi-method fusion (LBMMF) is proposed in this paper. Deep learning is used to train a deep network which learns the mapping from the training samples to the comprehensive BBAs obtained by jointly using the prevailing BBA modeling methods as the generalized training labels. Given a test sample, the corresponding BBA can be obtained in an end-to-end manner, which is the output of the trained deep neural network. Experimental results show that to use the BBA obtained by our method can achieve better classification performance.

Yang, Zhuo; Han, Deqiang; Yang, Yi; Dezert, Jean

A Dual-threshold Based Evidential Openmax Approach for Open Set Recognition

Abstract

Traditional pattern recognition systems, tasked with categorizing inputs into known classes, often struggle when they encounter samples they haven’t been trained to recognize. This introduces the need for the open set recognition—enhancing models to reject unidentified samples effectively. The Openmax method represents a significant breakthrough in this field by leveraging deep learning to spot and handle these new, unseen classes, broadening the traditional Softmax layer to accommodate an “unknown” class and employing a single threshold to separate the known from the unknown. However, the reliance of the original Openmax method on a single threshold may result in incorrect classifications if the parameters are not selected appropriately. To address this, we introduce a dual-threshold fusion mechanism based on Dempster-Shafer evidence theory in this paper. This approach releases the difficulty of finding a precise threshold in the complex and dynamic real-world environments. By integrating deep networks with a novel evidence-based system, the refined approach can bolster the robustness of rejecting undefined classes.

Li, Wei; Han, Deqiang; Dezert, Jean; Yang, Yi

Multimodal Coordinated Representation Learning Based on Evidence Theory

Abstract

In multimodal learning, multimodal coordinated representation is an important yet challenging issue, which establishes the interaction between different modalities to describe multimodal data more effectively. Existing coordinated representation methods are implemented in the deep feature space (or encoding space) of each modality. In this paper, based on the framework of evidence theory, we propose a novel coordinated representation method, where multimodal data is described as the basic belief assignment (BBA), and coordinated learning is implemented in the evidential space (i.e., the BBA based space). That is, the information interaction between different modalities is implemented at the level of evidence modeling (or uncertainty modeling). To use the intra-class and inter-class difference information of multimodal data, we design an evidential coordinated constraint. Furthermore, to represent each modality clearly, we introduce an ambiguity constraint.Experimental results of multimodal classification show that our proposed method is rational and effective.

Zhang, Yuanhang; Lin, Zhidi; Sun, Yiyong; Yin, Feng; Fritsche, Carsten

Regularization-Based Efficient Continual Learning in Deep State-Space Models

Abstract

Deep state-space models (DSSMs) have gained popularity in recent years due to their potent modeling capacity for dynamic systems. However, existing DSSM works are limited to single-task modeling, which requires retraining with historical task data upon revisiting a forepassed task. To address this limitation, we propose continual learning DSSMs (CLDSSMs), which are capable of adapting to evolving tasks without catastrophic forgetting. Our proposed CLDSSMs integrate mainstream regularization-based continual learning (CL) methods, ensuring efficient updates with constant computational and memory costs for modeling multiple dynamic systems. We also conduct a comprehensive cost analysis of each CL method applied to the respective CLDSSMs, and demonstrate the efficacy of CLDSSMs through experiments on real-world datasets. The results corroborate that while various competing CL methods exhibit different merits, the proposed CLDSSMs consistently outperform traditional DSSMs in terms of effectively addressing catastrophic forgetting, enabling swift and accurate parameter transfer to new tasks.

Li, Xiang; Deng, Xinyang; Jiang, Wen; Geng, Jie

KN-RUE: Key Nodes based Resampling Uncertainty Estimation

Abstract

With the continuous development and advancement of neural networks, in the application of neural networks, users not only require neural networks to be able to complete a given task but also want to know when they can trust the network's prediction results and when they need to be cautious about the prediction results. In response to the need for uncertainty estimation of neural networks, many researchers have invested in the study of uncertainty estimation. Existing uncertainty evaluation methods are difficult to apply to deep neural networks with large parameter scales, complex internal structures, and mappings between inputs and outputs that are hard to express. This paper proposes a key nodes based resampling uncertainty estimation method ((KN-RUE), which achieves uncertainty estimation of prediction results for arbitrarily given large-scale neural networks. In this method, the first step involves analyzing the differences in feature space between adversarial and clean samples, identifying the main nodes affected by adversarial samples, and determining the critical nodes within the network. Next, by resampling the parameters of key nodes, the model is extended while ensuring model performance as much as possible, thus completing the measurement of uncertainty in prediction results. Through experiments, the effectiveness of the extended model and the superiority of uncertainty estimation performance in KN-RUE have been verified.

Mazzi, Ludovico; Brambilla, Mattia; Guardiani, Michele; Arpaio, Maximilian James; Nicoli, Monica

Aircraft Localization by Interacting Multiple Model Filtering in Wide Area Multilateration

Abstract

Global air traffic has been steadily growing since the beginning of the new century, increasing the need for accurate and reliable positioning in real-time tracking of multiple aircrafts. This paper presents an Interacting Multiple Model (IMM) tracking solution and an assessment of a real Wide Area Multilateration (WAM) aircraft tracking scenario, where measurements from distributed Ground Stations (GSs) are gathered by a Central Processing Station (CPS) running the tracker. The assessment considers a main European airport, where a network of 44 GSs is used to monitor a congested area of 300 × 250 km. Tracking measurements refer to time differences of arrival (TDOAs) computed starting from the time of arrival (TOA) measured over downlink signals. Specifically, this work considers messages sent over the aviation transponder interrogation mode S. We present the results on IMM-based WAM tracking on airborne maneuvering targets, showcasing the improvements with respect to the conventional Automatic Dependent Surveillance - Broadcast (ADS-B) solution based on global navigation satellite systems (GNSSs).

Kawakami, Rikuto; Igeta, Yorito; Furukawa, Hidemitsu; Kakegawa, Moe; Suzuki, Yuto; Inagawa, Gianluca; Ogawa, Jun

Physical Reservoir Computing for Interactive Estimation of Weight from Food Texture in 3D-Printed Soft Matter in Picking Operations

Abstract

This paper aims to improve the ”Gel Biter,” a device that can simultaneously acquire chewing texture data from three different parts (the upper jaw, tongue, and lower jaw) composed of oral mimic end effectors with different softness (Young’s modulus), created using 3D printer technology, for use as a picking device. The Gel Biter utilizes physical reservoir computing, exploiting the deformation of the soft material in the oral model during chewing, to classify the texture of food materials with high accuracy. To apply this principle as a picking system, we examine whether it is possible to determine whether fried foods and fish roe have been cooked correctly and the extent of the weight being gripped based on texture. The results of the gripping object estimation experiments demonstrated that it is possible to distinguish between overcooked fried chicken and properly cooked fried chicken with 94.7% accuracy, and to identify the weight difference of artificial salmon roe in 5g, 10g, and 20g increments with 94.5% accuracy. These results suggest that the Gel Biter’s piezoelectric sensors through a physical reservoir computing system can determine information on weight and cooking status based on texture information.

Duník, Jindřich; Matoušek, Jakub; Straka, Ondřej; Blasch, Erik; Hiles, John; Niu, Ruixin

Stochastic Integration Based Estimator: Robust Design and Stone Soup Implementation

Abstract

This paper deals with state estimation of nonlinear stochastic dynamic models. In particular, the stochastic integration rule, which provides asymptotically unbiased estimates of the moments of nonlinearly transformed Gaussian random variables, is reviewed together with the recently introduced stochastic integration filter (SIF). Using SIF, the respective multi-step prediction and smoothing algorithms are developed in full and efficient square-root form. The stochastic-integration-rule-based algorithms are implemented in Python (within the Stone Soup framework) and in MATLAB and are numerically evaluated and compared with the well-known unscented and extended Kalman filters using the Stone Soup defined tracking scenario.

Scheible, Alexander; Griebel, Thomas; Buchholz, Michael

Self-Monitored Clutter Rate Estimation for the Labeled Multi-Bernoulli Filter

Abstract

Decision making in automated vehicles is based on the environment model, which is typically computed by a tracking module from information gathered by sensors. Thus, for safe and robust operation of the vehicle, the assessment of the current quality of the tracking module is crucial. This work makes a step towards this goal by providing a clutter rate estimation method with a self-monitored quality assessment for the labeled multi-Bernoulli filter. The significance of the proposed quality index is demonstrated by comparing it with the actual estimation error calculated with ground truth data. The simulation results show that the developed quality index is a meaningful value that can be computed online without the need for ground truth data. Moreover, it is competitive and closely related to the estimation error.

Krejčí, Jan; Kost, Oliver; Straka, Ondřej; Duník, Jindřich

Pedestrian Tracking with Monocular Camera using Unconstrained 3D Motion Model

Abstract

A first-principle single-object model is proposed for pedestrian tracking. It is assumed that the extent of the moving object can be described via known statistics in 3D, such as a pedestrian height. The proposed model thus need not constrain the object motion in 3D to a common ground plane, which is usual in 3D visual tracking applications. A nonlinear filter for this model is implemented using the unscented Kalman filter (UKF) and tested using the publicly available MOT-17 dataset. The proposed solution yields promising results in 3D while maintaining perfect results when projected into the 2D image. Moreover, the estimation error covariance matches the true one. Unlike conventional methods, the introduced model parameters have convenient meaning and can be adjusted for a problem at hand.

Mignet, Franck C.; Slijkhuis, Filip; Abouhafc, Abdelhaq; Pavlin, Gregor; Laskey, Kathryn B.

A Qualitative Causal Approach to Determining Adequate Training Data Quantity for Machine Learning

Abstract

This paper proposes an improved analysis of the Qualitative Models of Data Generating Processes (QM-DGP). The approach supports (i) determination of the complexity of a Machine Learning problem and (ii) a coarse determination of the quantities of training data that are needed to train good quality models. Compared to the previously published approach to the QM-DGP analysis, this paper introduces a more thorough and theoretically sound treatment of the learning complexity. Firstly, the approach provides more rigorous determination of the complexity of the data generating processes (DGP). Secondly, the determination of the learning complexity and the required training data volumes is based on sound statistical principles for the estimation of the distributions over categorical variables. The effectiveness of the proposed method was experimentally confirmed in controlled settings. Different ground truth models were used to sample test and training data. The approach correctly predicts the size of the training data sets for which machine learning yields models supporting classification close to Bayes Error. While the majority of the experiments were carried out on probabilistic graphical models (PGM), the experiments with Neural Networks confirmed that the QM-DGP approach is not limited to PGMs.

Shin, Changkyo; Dagan, Ofer; Ahmed, Nisar; Choi, Han-Lim

Fault-tolerant Bayesian Decentralized Data Fusion Using Reliability Variables and Mixture Models

Abstract

In uncertain and dynamic environments, decentralized data fusion (DDF) techniques have been widely used to estimate the states and the uncertainty levels over large mission spaces in a robust and scalable way. In data fusion frameworks using distributed sensor networks, undetected sensor failures can degrade the quality of fusion results of the entire system. Therefore, DDF methods which are robust to inconsistent data are needed. In this paper, a fault-tolerant Bayesian DDF method using Gaussian mixture models is developed. The probability of agent reliability states, which represent consistency of local estimates that agents share with their neighbors, are modeled as weights of mixture models and estimated together with the target process. The target process and reliability states are updated in a decentralized Bayesian way, exploiting the properties of Gaussian mixture models. To prevent the hypothesis explosion problem of Gaussian mixture models, a mixture compression method considering the physical meaning of mixture weights is utilized. A numerical simulation on a 2D dynamic target tracking problem is presented to verify performance of the suggested algorithm and compared with existing DDF methods. It is shown that the suggested algorithm gives more compact fusion results compared to existing fault-tolerant DDF method.

Wielandner, Lukas; Venus, Alexander; Wilding, Thomas; Witrisal, Klaus; Leitinger, Erik

MIMO Multipath-based SLAM for Non-Ideal Reflective Surfaces

Abstract

Multipath-based simultaneous localization and mapping (MP-SLAM) is a well established approach to obtain position information of transmitters and receivers as well as information regarding the propagation environments in future MIMO communication systems. Conventional methods for MP-SLAM consider specular reflections of the radio signals occurring at smooth, flat surfaces, which are modeled by virtual anchors (VAs) that are mirror images of the physical anchors (PAs), with each VA generating a single multipath component (MPC).. However, non-ideal reflective surfaces (such as walls covered by shelves or cupboards) cause dispersion effects that violate the VA model and lead to multiple MPCs that are associated to a single VA. In this paper, we introduce a Bayesian particle-based sum-product algorithm (SPA) forMP-SLAM in MIMO communications systems. Our method considers non-ideal reflective surfaces by jointly estimating the parameters of individual dispersion models for each detected surface in delay and angle domain leveraging multiple-measurement-to-feature data association. We demonstrate that the proposed SLAM method can robustly and jointly estimate the positions and dispersion extents of ideal and non-ideal reflective surfaces using numerical simulation.

Yurdakul, Ogŭl Can; Çetinkaya, Mehmet; Çelebi, Enescan; Özkan, Emre

A Rao-Blackwellized Particle Filter for Superelliptical Extended Target Tracking

Abstract

In this work, we propose a new method to track extended targets of different shapes such as ellipses, rectangles and rhombi. We provide an analytical framework to express these shapes as superelliptical contours and propose a Bayesian filtering scheme that can handle measurements from the contour of the object. The method utilizes the Rao-Blackwellized particle filtering algorithm with novel sensor-object geometry constraints. The success of the algorithm is demonstrated using both simulations and real-data experiments, and the algorithm has been demonstrated to be of high performance in various challenging scenarios.

Schumacher, Max-Lion; Huber, Marco F.

Probabilistic Global Robustness Verification of Arbitrary Supervised Machine Learning Models

Abstract

Many works have been devoted to evaluating the robustness of a classifier in the neighborhood of single points of input data. Recently, in particular, probabilistic settings have been considered, where robustness is defined in terms of random perturbations of input data. In this paper, we consider robustness on the entire input domain as opposed to single points of input. For the first time, we provide formal guarantees on the probability of robustness, given a random input and a random perturbation, based only on sampling or in combination with existing pointwise methods. We prove that the error becomes arbitrarily small for enough input data. This is applicable to any classification or regression model and any random input perturbation. We then illustrate the resulting bounds and compare them against the state of the art for models trained on the MNIST, California Housing, and ImageNet datasets.

Landzaat, Tom; Driessen, Hans; van Hintum, Hans

TDOA based ADS-B validation using a Particle Filter and Statistical Hypothesis testing

Abstract

ADS-B is a widely used protocol that transmits aircraft’s position, velocity among other data. The protocol is not encrypted leading to the need of validation. A validation algorithm is proposed that makes use of Time Difference of Arrival localization to validate the position and velocity of ADS-B transmitting targets. Nowadays, Air navigation service providers (ANSP) commonly have at least one TDOA localization system in operation, allowing for cost effective implementation. Validation is achieved by using a Particle Filter and hypothesis tests. A novel method is used where the initial density is generated effectively based on the first set of TDOA measurements. Validation is possible when two or more ground stations receive the same ADS-B transmission, therefore the Particle Filter is designed to process such measurements. The algorithm is tested on data provided by Air Traffic Control The Netherlands’ North sea surveillance system. Results show that the validation works and that the algorithm is able to detect spoofing. Based on spoofed ADS-B messages and true TDOA measurements, the real and fake target can be detected when the distance is roughly 750 to 1000 meters (depending on the situation and the various tuning parameters). In addition, validation based on two or more ground stations per measurements has the effect that the validation area is increased, when compared to traditional filters that require 4 ground stations for tracking.

Sjøberg, Alexander M.; Gade, Brita H. H.; Vooren, Carina; Kloster, Morten

Association of SAR Measurements in Coastal Regions using Existing Tracks of Marine Vessels

Abstract

This paper considers a measurement-to-track association (MTA) problem of marine surface vessels in coastal regions. In particular, we consider a case where a set of non-cooperative measurements is associated with a set of existing tracks. We perform a case study involving marine vessels situated in a coastal environment dominated by islands, fjords and peninsulas. Euclidean distances, or more generally, Mahalanobis distances, between measurements and predicted positions may be an attractive approach due to its simplicity of implementation and low computational complexity. However, such metrics may not be feasible for association in coastal regions, as opposed to open waters. We propose an algorithm where numerical methods are used to produce a probability density map for evaluation of a set of non-cooperative measurements, obtained at a time between two positions belonging to a particular track. The proposed associator provides a list of hypotheses intended for a multiple hypothesis tracker (MHT), where each hypothesis is evaluated according to the deviation of expected arrival time.

Iacob, David-Octavian; Mikus, Philipp; Ospel, Matthias; Still, Luisa; Blonde-Weinmann, Cyril; Oispuu, Marc

Optimizing Sensor Placement in Urban Environments for Time Difference of Arrival Shooter Localization and Event Classification

Abstract

This study addresses the optimization of the placement of multiple acoustic sensors for shooter localization and event classification in urban environments. Using ray casting solutions of the Eikonal equation, the shortest propagation paths in the urban model are computed for all source-receiver pairings. Subsequently, the expected Times of Arrival (TOAs) from virtual sources are used to evaluate the localization performance of a given sensor setup using a Monte Carlo approach. Similarly, the modelled signal paths are used to estimate the signal-to-noise ratio (SNR) of the source at the sensor level in order to predict the expected classification performance. Subsequently, a genetic algorithm solves the underlying optimization problem based on these performance metrics and identifies optimal sensor network configurations for shooter localization and event classification within the urban environment. The method is experimentally validated using audio data of propane gas cannon shots recorded at the French-German Research Institute of Saint-Louis.

Venturino, Antonello; D'Afflisio, Enrica; Forti, Nicola; Braca, Paolo; Willett, Peter; Win, Moe Z.

Adaptive Resilience in Navigation: Multi-Spoofing Attacks Defence with Statistical Hypothesis Testing and Directional Receivers

Abstract

This paper explores filtering methods to protect range-based localization systems from spoofing attacks on vehicles with directional receivers. It focuses on scenarios where multiple spoofers, potentially from unmanned vehicles, disrupt vehicle localization by strategically positioning themselves between the target and the transmitter. The paper introduces an Adaptive Resilience Navigation Filter (ARNF) that detects ongoing attacks, identifies compromised signals, and mitigates their effects using statistical hypothesis testing. Simulations demonstrate the ARNF's effectiveness under realistic Global Navigation Satellite System conditions, comparing it with the 2-Stage Extended Kalman Filter and an ideal Clairvoyant Extended Kalman Filter.

Broghammer, Fabio; Wiedemann, Thomas; Zhang, Siwei; Dammann, Armin; Gentner, Christian; Djurić, Petar M.

Localization and Sensing on Vulcano Island – A Glimpse into Future Space Exploration with Swarms

Abstract

Robotic swarms or portable sensor networks are emerging technologies for sensing physical processes that are spatially distributed- and temporally dynamic, both on Earth and in future Moon/Mars exploration missions. We develop a portable network composed of a multitude of self-organized ''sensor eggs''. These eggs are equipped with ultra-wideband (UWB) transceivers, providing precise time and position information without additional infrastructures like Global Navigation Satellite Systems (GNSSs). Each egg is additionally equipped with environmental sensors, for example, a Sulfur dioxide gas sensor to explore volcanic activity. We use a real time decentralized particle filter (DPF) to estimate the a-posteriori probability density functions (PDFs) of the egg positions. These PDFs are then used in a static state binary Bayes filter for estimating the gas sources with potentially complex structures such as cracks on the volcano surface. The proposed sensor network is verified with an in-field experiment at La Fossa volcano on the island of Vulcano, Italy, in 2023.

Kornfeld, Nils; Leich, Andreas; Roth, Michael

Kalman filtering aspects in camera and deep learning based tracking for traffic monitoring

Abstract

In multiple object tracking applications for traffic monitoring the underlying algorithms often use rectangular, axis-aligned bounding boxes from deep-learning based object detection systems as a measurement input. Often the association of the measurements to trajectories is performed in the image domain, where after for every bounding box an already associated pseudo-measurement in a world coordinate system is estimated, which is finally used as a measurement input to a Kalman filter. In contrast to this approach this article examines a multiple object tracking system with a measurement model which maps the estimated state of objects in world coordinates to the aforementioned rectangular bounding boxes in an image coordinate system. In addition the choice of the state vector elements modeled to represent the vehicles is shown and discussed. The approach presented in this article allows for association founded in physical reality, the estimation of the spacial dimensions of tracked objects and avoids shortcomings of a two-staged approach with association in the image coordinate frame.

Giurgi, Dǎnuț-Vasile; Dezert, Jean; Josso-Laurain, Thomas; Devanne, Maxime; Lauffenburger, Jean-Philippe

Fusion of Semantic Segmentation Models for Vehicle Perception Tasks

Abstract

In self-navigation problems for autonomous vehicles, the variability of environmental conditions, complex scenes with vehicles and pedestrians, and the high-dimensional or real-time nature of tasks make segmentation challenging. Sensor fusion can representatively improve performances. Thus, this work highlights a late fusion concept used for semantic segmentation tasks in such perception systems. It is based on two approaches for merging information coming from two neural networks, one trained for camera data and one for LiDAR frames. The first approach involves fusing probabilities along with calculating partial conflicts and redistributing data. The second technique focuses on making individual decisions based on sources and fusing them later with weighted Shannon entropies. The two segmentation models are trained and evaluated on a particular KITTI semantic dataset. In the realm of multi-class segmentation tasks, the two fusion techniques are compared and evaluated with illustrative examples. Intersection over union metric and quality of decision are computed to assess the performance of each methodology.

Zhu, Xinchao; Yang, Chaoqun; Zhou, Chengwei; Shi, Zhiguo

CBMeMBer Filter based Resolvable Group Target Tracking via Graph Theory and Leader-Follower Model

Abstract

Resolvable group target tracking is of great challenge due to the complex motion interaction between group targets, which leads to tracking performance degradation. To solve this problem, a cardinality-balanced multi-target multi-Bernoulli filter based on the graph theory and leader-follower model is proposed. In the proposed filter, firstly, the group targets are divided into leaders and followers by mean of the leader-follower model. Furthermore, the graph theory is used to establish the state transition equations between those divided group targets. Lastly, the process of state prediction is given, and its corresponding implementation is derived by Gaussian mixture approximations. Simulation experiments verify the superiority and effectiveness of the proposed filter.

Bernabeu, Joan M.; Ortega, Lorenzo; Blais, Antoine; Grégoire, Yoan; Chaumette, Eric

On Time-Delay Estimation Accuracy Limit Under Phase Uncertainty

Abstract

Accurately determining signal time-delay is crucial across various domains, such as localization and communication systems. Understanding the achievable optimal estimation performance of such technologies, especially during design phases, is essential for benchmarking purposes. One common approach is to derive bounds like the Cramér-Rao Bound (CRB), which directly reflects the minimum achievable estimation error for unbiased estimators. Different studies vary in their approach to dealing with the degree of misalignment in the global phase originating from both the transmitter and the receiver in a single input, single output (SISO) link during time-delay estimation assessment. While some treat this phase term as unknown, others assume ideal calibration and compensation. As an alternative to these two opposing approaches, this study adopts a more balanced approach by considering that such a phase can be estimated with a defined uncertainty, a measure that could be implemented in many practical applications. The primary contribution provided lies in the derivation of a closed-form CRB expression for this alternative signal model, which, as observed, exhibits an asymptotic behavior transitioning between the results observed in previous studies, influenced by the uncertainty assumed for the mentioned phase term.

Gehlen, Joshua; Ulmke, Martin; Springer, Jannik; Govaers, Felix; Koch, Wolfgang

Tensor Decomposition based Bearing-Only Target Tracking - an Analysis based on Real Data

Abstract

This paper presents the application of a novel target tracking technique employing tensor decompositions for discretizing the target state space. The time evolution of the conditional probability density is realized by a Fokker-Planck equation solver and the measurement update, as usual, by applying Bayes’ rule. The method is applicable to non-Gaussian and non-linear system equations and enables the treatment of complex non-Gaussian target state densities. In addition, the efficient tensor decomposition scheme, in principle, allows for high-dimensional target states. The new tracking filter is applied to the problem of tracking an agile air target using bearing measurements from distributed acoustic and electromagnetic array sensors based on real data. It is shown that the new filter is able to initiate and maintain the target track with localization errors comparable to those of a standard particle filter.

Alemaw, Abrham Shiferaw; Zontone, Pamela; Marcenaro, Lucio; Marin, Pablo; Martin Gomez, David; Regazzoni, Carlo

Integrated Learning and Decision Making for Autonomous Agents through Energy based Bayesian Models

Abstract

Generalizability and interpretability are common terminologies that can be found in today's machine learning algorithm design. Generalizability requires a clear understanding of one's own action (self-awareness) and a robust interaction with the environment (situation awareness). Many current studies are devoted in developing an algorithm that is more robust in generalizing unseen situations while explaining self-action. However, such algorithms are complex and are not yet fully developed to be used in production. Intelligent transportation systems like self-driving cars are one of the emerging technologies that need generalizability and explainability in anomalous conditions. We propose to enhance generalizability and interpretability of a self-driving car model by introducing a novel methodology that fuses multi-sensorial data from proprioceptive and exteroceptive sensors of an agent, coupled in a Hierarchical Dynamic Bayesian Network model, in an Active Inference framework. The developed model has three stages: 1) a lower dimensional unsupervised learning stage, considering odometry and action modalities, carried out by first applying Null Force Filtering and then by applying modified GNG clustering algorithms; 2) a self-supervised higher-dimensional video modality learning stage assisted by the learned odometry vocabularies; and 3) an online model-based active learning in continuous and discrete state spaces, and action spaces, in the Active Inference framework. The developed system is tested using the CARLA simulator environment for localizing interacting agents, and exhibits low error compared to state-of-the-art methods.

Wei, Yuan; Lan, Jian; Zhang, Le

Multiple Extended Object Tracking Using PMHT with Extension-Dependent Measurement Numbers

Abstract

For multiple extended object tracking (MEOT), data association and object extension estimation are key problems, and the number of measurements generated by each object plays a key role in both problems. For data association, probabilistic multiple hypothesis tracking (PMHT) naturally assumes multiple measurements can be assigned to a single object and has a linear computation complexity, and thus it fits MEOT well. In existing PMHT approaches to MEOT, the measurement number is usually used for data association only, not for direct extension estimation. Since the measurement number contains the extension information, e.g., an object with a bigger extension tends to generate more measurements given the sensor resolution, utilizing the measurement number for extension estimation is expected to improve the tracking performance. This paper proposes a PMHT approach combined with a random-matrix model using extension-dependent measurement numbers. The proposed approach derives a new auxiliary function of which the likelihood function part reflects the extension information contained in the measurement number. Then, using Expectation-Maximization, analytical forms of iteration formulae for kinematic states and extensions of multiple objects can be obtained approximately. Since more information is considered, the tracking performance of MEOT is improved, especially in extension estimation. Simulation results demonstrate the effectiveness of the proposed approach.

Bavirisetti, Durga Prasad; Kiss, Gabriel Hanssen; Lindseth, Frank

A Pole Detection and Geospatial Localization Framework using LiDAR-GNSS Data Fusion

Abstract

The integration of Light Detection and Ranging (LiDAR) and Global Navigation Satellite System (GNSS) technologies marks a significant advancement in the fields of autonomous driving and intelligent transportation systems. This research introduces a methodology for geolocalizing road objects, specifically poles, by leveraging the detailed spatial data from LiDAR combined with the location capabilities of GNSS, while carefully accounting for these sensor offsets. Our approach takes advantage of the synergy between LiDAR’s exceptional spatial resolution and GNSS’s global positioning capability. This precision is crucial for the navigation systems of autonomous vehicles. By processing LiDAR data to detect objects and calculate their positions relative to the sensor, and then transforming these positions into global coordinates using inverse geodesic calculations, we present a methodology that can perform object geolocation in various environments. This paper details the development of the methodology, the challenges encountered, and the solutions devised, showcasing the approach’s performance through experimental results and suggesting future directions for further research.

Kumru, Murat; Özkan, Emre

Tracking Arbitrarily Shaped Extended Objects Using Gaussian Processes

Abstract

In this paper, we consider the problem of tracking dynamic objects with unknown shapes using point cloud measurements, which are generated by sensors such as lidars and radars. Specifically, our objective is to extend the Gaussian process-based extended object tracking (GPEOT) framework to encompass a broader class of objects. The derivation of the existing GPEOT algorithms is based on the assumption that the object of interest is star-convex. This assumption enables the modeling of the object's extent through a radial distance function, which is described by a Gaussian process (GP). To enhance the flexibility of the resulting trackers, we propose the utilization of a potential function to indicate the unknown object extent. This approach enables the representation of objects with arbitrary shapes, including those that are non-convex and composed of disconnected components. Closely following the original formulation of GPEOT, the potential function is then modeled by a GP, which systematically accounts for the intrinsic spatial correlation of the extent. Furthermore, we develop a state-space model that incorporates both kinematic variables and an approximate description of the underlying GP model. The state vector can be estimated via a standard Bayesian technique, which leads to an EOT algorithm. Through simulation experiments, we demonstrate the suggested method can satisfactorily estimate the kinematic variables of the objects while simultaneously learning their complex shapes.

Yang, Xiaowei; Liu, Haiqi; Zhao, Hua; Meng, Fanqin; Shen, Xiaojing

Set-Valued Modeling for Drop-Point Constrained Dynamic Systems

Abstract

This paper addresses the problem of purpose-constrained state modeling within a set-valued framework, where only drop-point information of state trajectories is available. Due to the lack of complete trajectory information, existing estimation methods with linear equality constraints struggle to achieve effective state prediction. This paper primarily investigates the integration of drop-point constraint information into the entire system state evolution process via convex optimization projection in the set-valued framework, aiming to reconstruct a linear dynamic system model. Subsequently, by employing multiple affine transformations of ellipsoids and designing a weight matrix, the smoothness of state trajectories is enhanced to better align with real motion patterns. Finally, through simulation experiments, we validate the significant advantages of the reconstructed system model over traditional unconstrained system models under the set-valued framework. Additionally, we demonstrate the impact of drop-point constraints on state trajectory evolution under different initial point conditions with the same drop point.

Siebler, Benjamin; Lehner, Andreas; Sand, Stephan; Hanebeck, Uwe D.

Magnetic Field Mapping of Railway Lines with Graph SLAM

Abstract

The earth’s magnetic field along railway tracks is strongly distorted by magnetic material in the vicinity, e.g., steel in rails and reinforced concrete. The resulting magnetic distortions are persistent in time and characteristic for a certain part of the track. Thus, these distortions can be seen as fingerprints that enable localization when a map of the magnetic field is available. This is particularity interesting for areas where global navigation satellite system (GNSS) signals are not available, such as tunnels. Unfortunately, creating the magnetic map in a GNSS-denied area requires a position reference system that is most likely not available. This paper addresses this problem with a graph-based simultaneous localization and mapping (SLAM) algorithm that uses only odometer and magnetometer measurements. The key idea of the proposed algorithm is to use the magnetic field to detect loop closures and to calculate the relative transformation between different nodes in the pose-graph. The algorithm is evaluated based on a data set recorded with the advanced TrainLab of the Deutsche Bahn traveling on a track in Berlin. Results show that the graph SLAM algorithm together with the magnetic loop closure detection reduces and bounds the position error of the odometry.

Strand, Leah; Honer, Jens; Knoll, Alois

Joint Vehicle Pose and Extent Estimation in the Context of Multi-Camera Traffic Surveillance

Abstract

In this paper, we introduce a novel method for the estimation of vehicle pose and extent in traffic surveillance scenarios based on camera data. The state estimation is performed in a common world frame, enabling the seamless integration of the image data from different viewpoints. Our approach incorporates the non-linear transformation between the measurements and the states directly into the framework of an Unscented Kalman filter. Two measurement models are proposed: one designed for bounding boxes and another for discretized object contours extracted from segmentation masks. The method is evaluated using data from a real-world traffic surveillance system, demonstrating the high effectiveness and good feasibility of our approach for localizing passing cars.

Govaers, Felix

A Quantum Algorithm for the Prediction Step of a Bayesian Recursion

Abstract

The prediction step is a crucial element of the Bayesian recursion for target tracking and state estimation in general. Discrete representations of the probability density function (pdf) can deal with non-linear models and non-Gaussian noise, however, the prediction step is challenging to solve on classical computers. In this paper, a novel concept of quantum simulation to solve the application of a continuous noise motion model to a pdf is presented. The pdf is prepared as the squared amplitudes for the basis states spanned by the number of used qubits. The number of required qubits grows linear in the number of time steps to simulate in a prediction phase. One step performs a single Brownian motion, which is used to generate the diffusion of the Wiener increments. A drift function can be implemented based on a separate register, which holds the pdf on the velocity information for an adequate discretization. The approach is visualized in terms of quantum circuits and evaluated based on a quantum simulator.

Broetje, Martina

Detection and tracking of airplanes on runways with passive radar data

Abstract

This paper discusses the use of a passive radar experimental system based on mobile communication signals for detecting and tracking airplanes on the airport runways. We employ a two-stage strategy, taking advantage of the fact that the estimation region is limited to the runways. In the first stage, we focus the detection of moving targets in the passive radar data. In the second stage, we classify the runway targets and determine their location on the runway. This also involves a classification task, where the detections are assigned to specific runways or other moving objects such as flying airplanes. The approach is evaluated with data from a measurement campaign conducted at Frankfurt airport.

Valeyrie, Nicolas; Houssineau, Jérémie; Strode, Christopher; Pailhas, Yan

On the Estimation of the Size of a Target Population

Abstract

This paper is concerned with the Bayesian estimation with possibility theory of the number of targets in an area of interest when no prior information is available. The estimation relies solely on the number of detections that is reported every time the area is surveyed. It is assumed that detections can be true and false positives. Possibility theory is capable of quantifying the complete absence of prior information on the number of targets with the introduction of a prior possibility function that is equal to one everywhere on the set of natural numbers. This has no equivalent in probability theory. The prior possibility function is turned into a posterior possibility function from which a point estimate of the number of targets can be derived. Besides the point estimate, a meaningful notion of confidence interval can be derived from the posterior possibility and necessity associated with subsets of the natural numbers.

Zelioli, Luca; Farahnakian, Fahimeh; Farahnakian, Farshad; Middleton, Maarit; Heikkonen, Jukka

Enhancing Peatland Classification using Sentinel-1 and Sentinel-2 Fusion with Encoder-Decoder Architecture

Abstract

Peatland classification provides valuable information for greenhouse gas inventory and biodiversity protection. In this paper, we proposed an encoder-decoder-based architecture for peatland classification that fuses two open-source satellite data, Sentinel-1 and Sentinel-2. We show the effect of fusion by comparing the multi-modal fusion architecture with uni-modals which are trained only based on one input data source. We also investigate the influence of skip connections as the main component of the encoder-decoder to recover fine-grained details that are lost during the downsampling process. The experimental results are acquired on a study area in Finland which covers a variety minerotrophic aapa mire peatlands. The results demonstrate that multi-modal architecture consistently outperforms uni-modal architectures for peatland classification. In addition, the fusion architecture with one skip connection achieved a total accuracy of 57.44%. This shows 8.51% accuracy improvement compared with the model without skip connections.

Dalhaug, Nicholas; Stahl, Annette; Mester, Rudolf; Brekke, Edmund Førland

Combining Short and Wide Baseline Stereo Cameras for Improved Maritime Target Tracking

Abstract

Target tracking is essential for autonomous vehicles to avoid collisions. Using a stereo camera for the target tracking gives a dense representation of the targets, contrary to the the sparser data on typical radars and lidars. With a wider baseline stereo camera the depth measurements are more accurate, but the stereo matching challenge is greater, especially in the maritime domain with reflections on the water. Earlier classical methods of tracking using stereo cameras have often tracked targets by first doing water surface estimation and then finding objects perturbing the plane. The challenge is then to get a good estimate of the water surface plane while still having precise measurements to the targets. We propose both a short baseline method and a multi-baseline method for target detection. The multi-baseline method uses a short baseline stereo camera to find the water plane and uses a wider baseline stereo camera to get accurate target measurements. The targets are consistently being tracked when using data collected during the summer of 2023 from an autonomous ferry prototype compared to ground truth GNSS tracks. The short baseline method achieves minimal error for a day cruiser boat 40 m away using a camera baseline of only 12 cm. The multi-baseline method further improves the accuracy of boat measurements, especially for a far-away small kayak.

Ma, Wen; Zhu, Hongyan

Source Localization Using TDOA with Sensor Position Errors Based on Constrained Total Least Squares and ADMM

Abstract

Source localization for the nonlinear measurement model based on time difference of arrival (TDOA) measurements remains a vital research area and has been intensively studied for the past few decades. However, the localization accuracy decreases significantly as the random measurement noise becomes large. In addition, when sensors are mounted on moving platforms like vehicles or aircrafts, inevitable sensor position errors might pose more severe challenges on source localization accuracy. This paper proposes to construct a pseudo-linear measurement model that introduces both the TDOA measurement noise and the sensor position error firstly. Next, the constrained total least squares (CTLS) formulation is presented, and the iterative alternating direction method of multipliers (ADMM) is employed to solve the resulting optimization model. Simulation results show that the proposed method can approach Cramer Rao lower bound (CRLB) better and outperforms several existing methods when considering sensor position uncertainties and large TDOA measurement errors.

Cominelli, Marco; Gringoli, Francesco; Kaplan, Lance M.; Srivastava, Mani B.; Bihl, Trevor; Blasch, Erik P.; Iyer, Nandini; Cerutti, Federico

Neuro-Symbolic Fusion of Wi-Fi Sensing Data for Passive Radar with Inter-Modal Knowledge Transfer

Abstract

Wi-Fi devices, akin to passive radars, can discern human activities within indoor settings due to the human body's interaction with electromagnetic signals. Current Wi-Fi sensing applications predominantly employ data-driven learning techniques to associate the fluctuations in the physical properties of the communication channel with the human activity causing them. However, these techniques often lack the desired flexibility and transparency. This paper introduces DeepProbHAR, a neuro-symbolic architecture for Wi-Fi sensing, providing initial evidence that Wi-Fi signals can differentiate between simple movements, such as leg or arm movements, which are integral to human activities like running or walking. The neuro-symbolic approach affords gathering such evidence without needing additional specialised data collection or labelling. The training of DeepProbHAR is facilitated by declarative domain knowledge obtained from a camera feed and by fusing signals from various antennas of the Wi-Fi receivers. DeepProbHAR achieves results comparable to the state-of-the-art in human activity recognition. Moreover, as a by-product of the learning process, DeepProbHAR generates specialised classifiers for simple movements that match the accuracy of models trained on finely labelled datasets, which would be particularly costly.

Harvey, Ryan; Pattipati, Krishna; Willett, Peter

A CRLB for Passive Only TDOA Localization From a Three-Dimensional Hydrophone Array

Abstract

This paper presents a mechanism for evaluating the Root Mean Square Error (RMSE) of a Minimum Variance Unbiased Estimator (MVUE) of a target state in 3D space using acoustic measurements. The target state is represented by (θ, φ, r) and it is estimated using Time Difference of Arrival measurements at the sensors and we assume that the sound-speed c is unknown. We then examine the interaction between azimuth angle θ on range RMSE, and the impacts of measurement noise variance on RMSE of (θ, φ, r, c) estimates. These results and analytical formulations can be used as a baseline to evaluate proper 3D array geometry design, as well as inform the potential RMSE improvements when using a biased minimum mean square error (MMSE) estimator over an unbiased (MVUE) one for the same set of measurements.

Fontana, Marco; Hayder, Thomas; Freilinger, William; García-Fernández, Ángel F.; Maskell, Simon

A Poisson Multi-Bernoulli Mixture approach to tracking trains using Distributed Acoustic Sensing

Abstract

This paper presents an extended target tracking method to track trains using Distributed Acoustic Sensing (DAS) data. The problem is approached using a measurement likelihood based on a Set of Points on a Rigid Body (SPRB) model applied to a clustered version of the Poisson Multi-Bernoulli Mixture filter. The method efficiently handles asymmetric noise within the set of measurements returned by each train, and proposes a solution to merged measurements appearing at crossings. We use experimental data obtained from trains to show that the proposed algorithm has lower localisation and false target error, leading to better performance in terms generalized optimal sub-pattern assignment (GOSPA) metric.

Miao, Qing; Zou, Zhiyuan; Li, Wujun; Yi, Wei

Track-Before-Detect for Automotive Multi-Radar Systems with Time-Varying Fields of View

Abstract

Track-before-detect (TBD) and multi-sensor fusion are two popular methods of weak target detection which can improve the performance by increasing the number of measurements. In this paper, we combine these two methods, proposing a novel multi-sensor track-before-detect (MS-TBD) method for automotive platforms. It can utilize the information from both the spatial and temporal dimensions of the target by jointly processing the measurement from different radars. In particular, the traditional TBD method is often based on an implicit assumption: the presence of targets is unchanged in the sliding window. However, this assumption may not be applicable for automotive multi-sensor systems due to the time-varying fields of view (FOV). To solve the problems mentioned above, we first present an energy accumulation strategy for automotive multi-radar systems and then propose a multiple-hypothesis detection method with the adaptive threshold (AT). It is demonstrated by simulations that the proposed methods show superior performance.

Ioannou, Giorgio; Gaglione, Domenico; Millefiori, Leonardo M.; Renga, Alfredo; Braca, Paolo; Willett, Peter

Dark-VADER: Detection of Anomalous AIS Message Delays for Maritime Situational Awareness

Abstract

Maritime situational awareness (MSA) refers to the effective understanding of activities related to the maritime environment. Central to MSA, particularly concerning non-military vessels, is the automatic identification system (AIS), which provides real-time data on vessel movements. However, anomalies such as intentional AIS transponder disablement pose significant challenges to MSA, potentially indicating illicit activities. This paper introduces the Dark-VADER (dark vessel AIS delay event recognition) algorithm, designed to detect AIS switch-offs by comparing the frequency of message reception from a vessel under examination with that of neighboring vessels. Leveraging a statistical hypothesis testing procedure based on a Bernoulli process, the algorithm distinguishes between normal and anomalous behavior. Validation using real-world AIS data confirms the fitness of the selected distribution model for times between message arrivals, essential for the algorithm’s operation. Overall, this preliminary work provides a foundational framework for improving maritime AIS anomaly detection, with avenues for future development towards more robust and dynamic approaches.

Funk, Christopher; Noack, Benjamin

Conservative Compression of Information Matrices using Event-Triggering and Robust Optimization

Abstract

Distributed sensor fusion requires the transmission of intermediate fusion results, consisting of point estimates and associated error covariance or information matrices. Bandwidth constraints necessitate data compression techniques for error covariance and information matrices, which typically dominate data volume. To ensure the safe use of the fusion results for decision-making, these techniques must be conservative, i.e., not lead to the compressed error covariance or information matrices underestimating the true estimate error. This work introduces a novel approach for the conservative compressed transmission of information matrices, that builds on a previous event-based method for covariance matrices. The proposed method allows the entire sensor fusion pipeline to operate in 'information space', facilitating efficient fusion operations without the need to compute corresponding covariance matrices. Contributions include an event-trigger for information matrices and a robust-optimization-based bounding mechanism ensuring conservativeness. The proposed approach is evaluated in the context of transmitting error information matrices generated by extended information filter SLAM to a receiver for further processing.

Schmitt, Eva Julia; Noack, Benjamin

Event-based Multisensor Fusion with Correlated Estimates

Abstract

Many automation tasks require to fuse information that is acquired by distributed sensors and passed through a wireless network across multiple nodes. The growing number of connected sensors and agents increases the burden on the communications network and the energy consumption. Further challenges in information fusion arise from correlated data shared between nodes. To mitigate the negative effects, an efficient multi-sensor fusion approach is presented in this paper. A system design that uses stochastic event-based instead of periodic transmissions is proposed based on two different algorithms, the augmented state approach and fast covariance intersection. Furthermore, two different network topologies are investigated and a methodology to handle correlations among both finite impulse response and recursive estimates is developed. Together, the results represent a wide range of network topologies and possible correlation structures and give insights into the estimation performance and network utilization.

Fernandez-Matellan, Raul; Puertas-Ramirez, David; Martin Gomez, David; Boticario, Jesus G.

Fusion of Physiological Signals for Modeling Driver Awareness Levels in Conditional Autonomous Vehicles using Semi-Supervised Learning

Abstract

The evolution of autonomous vehicles (AVs) requires a paradigm shift towards the integration of human factors to improve safety and efficiency at levels 2, 3 and 4 of automation. This paper presents a comparison of three different fusion technologies (Low-Level fusion, Medium-Level fusion, and a hybrid fusion), highlighting the critical role of multimodal data integration and semi-supervised learning in predicting and adapting to levels of driver awareness. Our approach uses semi-supervised learning to deal with the data labelling problem, using unlabelled data to train an autoencoder and sparsely labelled data to train a 4-state classifier. Our model facilitates the fusion of data from different physiological signals, including skin electrodermal activity, heart rate, body temperature and acceleration. Using real driving data, the Medium-Level fusion approach gives the best performance, achieving 84% accuracy in predicting situations where the user may not be aware enough to take control of the vehicle. This research highlights the essential nature of fusion technologies to create adaptive and user-centred AV systems.

Guo, Honggang; Liao, Zhikun; Liang, Zhonghong; Mu, Pengcheng; Yuan, Jie; Wang, Lin

Kalman Filter State Transformation Application in INS/GNSS Integrated Navigation for Polar Navigation

Abstract

Kalman filter is an important technology to realize information fusion between Inertial Navigation System (INS) and Global Navigation Satellite System (GNSS). One of the most important variables in the Kalman filter structure is the state variable, which is the basis of maintaining system stability. However, due to the inherent singularity in polar regions, the INS/GNSS integrated navigation system will not work properly in polar regions. Different coordinate systems are often used in trans-polar navigation to solve the problems in the polar regions, therefore the Kalman filter state variable transformation is inevitable in the process of entering or leaving the polar regions, which will bring instability and even failure to the system. Here, this paper proposes a Kalman filter state transformation algorithm for INS/GNSS polar integrated navigation based on Psi-angle error model to ensure the numerical stability of the Kalman filter during its state transformation. Key to this success is to establish the transformation relationship between different filter state variables as well as their covariance matrices in different navigation coordinate systems, then the state variables and the covariance matrices are transformed simultaneously. After presenting the state variable transformation algorithm and the system description, a numerical evaluation is carried out to assess the presented algorithm with regard to stability and accuracy.

Löffler, Wendi; Bengtsson, Mats

Train Localization During GNSS Outages: A Minimalist Approach Using Track Geometry And IMU Sensor Data

Abstract

Train localization during Global Navigation Satellite Systems (GNSS) outages presents challenges for ensuring failsafe and accurate positioning in railway networks. This paper proposes a minimalist approach exploiting track geometry and Inertial Measurement Unit (IMU) sensor data. By integrating a discrete track map as a Look-Up Table (LUT) into a Particle Filter (PF) based solution, accurate train positioning is achieved with only an IMU sensor and track map data. The approach is tested on an open railway positioning data set, showing that accurate positioning (absolute errors below 10 m) can be maintained during GNSS outages up to 30 s in the given data. We simulate outages on different track segments and show that accurate positioning is reached during track curves and curvy railway lines. The approach can be used as a redundant complement to established positioning solutions to increase the position estimate’s reliability and robustness.

Mu, Pengcheng; Jin, Shilong; Liao, Zhikun; Liang, Zhonghong; Wang, Yuanhan; Wang, Lin

A Self-calibration Kalman Filter Algorithm for Dual-axis RINS Based on the Transverse Ellipsoidal Earth Model

Abstract

The Kalman filter method plays a crucial role in enhancing the navigation accuracy of the dual-axis rotational inertial navigation system (RINS) through periodic estimation and compensation of device errors. Due to the particularity of polar geography, the traditional RINS mechanism in the local-level geographic frame loses efficacy in the polar region. This paper proposes a self-calibration Kalman filter algorithm based on the transverse ellipsoidal earth model to solve the self-calibration problem of RINS in polar region. This method firstly transforms the state of the local-level geographic frame to the transverse frame, and then constructs the prediction model and observation model of the Kalman filter based on the carrier state and error parameters at the transverse frame. In the self-calibration stage, a suitable rotation strategy is employed to stimulate the errors of RINS, and the proposed algorithm is utilized to estimate and compensate for the resulting errors. In addition, the traditional spherical earth model is improved to ellipsoidal earth model in this algorithm to avoid additional errors in the polar region. Since the transverse frame is equally applicable at middle latitudes, the Monte Carlo simulation and experimental verification are conducted in the mid-latitude region. The results demonstrate that the proposed method enables precise calibration of all error parameters, which aligns consistently with results obtained within the local-level geographic frame.

Ajgl, Jiří; Straka, Ondřej

On fusion of probability density functions using tensor train decomposition

Abstract

Non-linear filters consider probability density functions in various non-parametric representations. They often suffer from the curse of dimensionality. Computation of weights over a grid of points becomes infeasible even for low dimensions. Filters processing data produced in different sensor nodes provide their own probability densities. Combination of such densities is desired. A favourite paradigm is to construct a fused density as a weighted arithmetic or geometric mean of the individual densities. This paper prospects the fusion for tensor train representation of densities produced by point-mass filters. In this representation, the weights are neither evaluated for a whole grid nor fully stored in the memory of the filters. Aspects of tensor-train-based fusion are discussed, such as computation of auxiliary characteristics and experience with numerical examples.

Geletu, Mihreteab Negash; Lauffenburger, Jean-Philippe; Josso-Laurain, Thomas; Devanne, Maxime; Wogari, Mengesha Mamo

Evidential Deep Learning For Sensor Fusion

Abstract

Perception in autonomous vehicles (AVs) is a challenging task. On the one hand, the driving environment is cluttered and the weather conditions vary. On the other hand, perception sensors have their own inherent shortcomings. To overcome these problems, deep learning-based methods often relying on probabilities are used. In this paper, deep learning-based multi-modal fusion architectures are implemented with the evidence theory. The evidential implementation defines the pieces of evidence using distances to prototypes of feature vectors obtained thanks to the neural networks. These belief functions are combined by Dempster’s rule. Experimental analysis is done on the KITTI dataset and the analysis shows that the evidential models have better performance than the probabilistic baseline.

Liang, Zhonghong; Liao, Zhikun; Luo, Hui; Wang, Yuanhan; Mu, Pengcheng; Wang, Lin

Collaborative Calibration Algorithm in Redundant Dual-axis RINS Configuration

Abstract

Dual-axis rotational inertial navigation system (DRINS) can achieve self-calibration of error parameters to improve the navigation performance. However, the traditional self-calibration methods rely on the external reference information. This paper focuses on the dual DRINSs configuration and proposes a collaborative calibration algorithm. Considering the error parameters of redundant DRINS, the 60-D Kalman filter is established, in which the geometric constraint between systems are deployed to establish the observation equation without external reference information. A novel collaborative calibration scheme is designed based on the asynchronous rotation to make all the error states observable. Monte Carlo simulations and real system experiments are conducted to verify the effectiveness of the proposed algorithm. The result shows the algorithm works well.

Dill, Sebastian; Rohr, Maurice; Güney, Gökhan; Antink, Christoph Hoog

Evaluation of Accuracy and Angle Dependency of 3D Pose Estimation through Stereo Camera Information Fusion with MediaPipe Pose

Abstract

In recent years, significant research has been conducted on video-based human pose estimation (HPE). While monocular 2D HPE has been shown to achieve high performance, monocular 3D HPE is more challenging. Fusing the advantages of high accuracy in 2D HPE with the increased usability of 3D coordinates, we propose a method based on MediaPipe Pose 2D HPE on stereo cameras, epipolar geometry and direct triangulation to reconstruct 3D poses. We use the CMU Panoptic database, which provides recordings of humans from 31 different HD views and 3D ground truth data, to research which accuracy can be achieved from fusing only two cameras without prior stereo calibration. We also research which camera perspectives to employ, analyzing the angle dependency of our approach.

Musso, Christian; Lefebvre, Sidonie; Thetas, Sophie

Moving target’s detection performances in a sequence of infrared multispectral images

Abstract

The paper deals with the detection performance of a moving target in multispectral IR image sequences with low SNR. In this context, track-before-detect (TBD) is the generally used method, which consists in accumulating raw images over time to track before detect. For each target hypothesis (position, velocity, amplitude), the signal is integrated over time. In this way, the potential target with the best spatio-temporal correlation will be a candidate for a detection test. In this article, we develop a minimum detection bound independent of the TBD method used. Specifically, this bound gives the minimum average number of multispectral images required to detect the target.

Fierro, Nicolás; Adams, Martin; Cament, Leonardo

Extended Target Tracking with 3D-INSEG and its Benefits in Dense Scenarios

Abstract

In Multiple Extended Object Tracking (MEOT), it is assumed that a solitary target can produce multiple measurements. The quality of these measurements is paramount for obtaining accurate estimates of tracks over time. To test state-of-the-art MEOT algorithms, both simulated and real laser data, recorded in open spaces, have been used. MEOT algorithms work well in these scenarios, but when applied in more cluttered or restricted spaces, they often fail to produce good results, because close proximity target measurements are considered as measurements with the same origin. To address these cases, this article applies the 3D INstance SEGmentation (3D-INSEG) algorithm to MEOT to process stereo image sequences, extracting 3D information corresponding to each detected target using cameras. The algorithm selects pixels from each detected target and calculates the disparity map from stereo pairs, projecting them into 3D space using this disparity map. Subsequently, these measurements undergo processing by an extended target Poisson multi-Bernoulli mixture (PMBM) filter with a gamma Gaussian inverse-Wishart (GGIW) implementation. The advantages of MEOT with the 3D-INSEG-generated data are demonstrated in this article via a comparison with MEOT based on Velodyne LiDAR data points recorded from the same scenario processed by the same MEOT algorithm.

Chong, Zhen Yuen; Pritchett, Henry; Li, Qing; Gan, Runze; Kındap, Yaman; Godsill, Simon

Implementation of Non-Gaussian Motion Models Within Stone Soup

Abstract

In recent years, state-space models for highly manoeuvrable objects have been proposed based on non-Gaussian continuous time jump-based Lévy processes, the so-called Lévy state-space model. In these models, the standard Brownian motion driving process for continuous time processes is replaced with a heavy-tailed non-Gaussian alternative. This retains all the flexibility of its Gaussian counterpart in terms of possible dynamical model structures and operations with irregular time stamps or heterogeneous data sources. These models aim to operate in areas such as surveillance of irregularly moving drones or people, and tracking wildlife or biological data. Implementation is relatively straightforward since the Kalman filters of the Brownian motion case can be replaced in the non-Gaussian case by mixtures of Kalman filters within a marginalised particle filtering framework. While the Stone Soup tracking software environment includes both Kalman filtering and generic particle filtering, it does not currently allow the combination of these tasks within a marginalised particle filtering framework. We discuss the significant challenges involved in incorporating these models and algorithms into Stone Soup, and present initial simulation results for the new software.

Memon, Saleemullah; Krayani, Ali; Zontone, Pamela; Marcenaro, Lucio; Martin Gomez, David; Regazzoni, Carlo

Learning 3D LiDAR Perception Models for Self-Aware Autonomous Systems

Abstract

Intelligent transportation systems (ITSs) provide a paradigm change in perceiving and interacting with transportation networks, leading to enhanced levels of safety, sustainability, and efficiency. Vehicular-to-everything (V2X) communication is the core component in the ITSs. The proprioceptive and exteroceptive sensors allow these vehicles to be aware of the surrounding environment and respond to emergencies by utilizing their abilities to reach a high level of self-awareness. In this paper, we propose a self-awareness approach to learn a generative dynamic Bayesian network (G-DBN) from the real-time LiDAR perception. Without reducing the dimensionality, we perform offline training and online testing phases on the three-dimensional (3D) point clouds. In the offline training phase, initially, the raw point clouds are preprocessed using a joint probabilistic data association filter (JPDAF) to obtain the 3D tracks of the multiple vehicles in space. Then, we perform an unsupervised clustering on all the generalized states (GSs) containing positions and velocities (a 6D vector) by considering the growing neural gas (GNG) technique, thus achieving a trained model from the 3D LiDAR point clouds. In the online testing phase, the high-dimensional Markov jump particle filter (HD-MJPF) utilizes the G-DBN’s probabilistic information to predict the positions of multiple vehicles and to detect the abnormalities at the discrete and continuous levels in normal and abnormal scenarios. Our proposed approach is useful for learning high-dimensional generative models and provides a way to meet the current curse of dimensionality challenges, that machine learning models are suffering.

Baerveldt, Martin; Shuai, Jiangtao; Brekke, Edmund Førland

Improved Fusion of AIS Data for Multiple Extended Object Tracking

Abstract

In maritime situational awareness, the Automatic Identification System (AIS) is a vital source of information. Recent work has explored the fusion of AIS information and exteroceptive measurements to improve maritime target tracking performance, also for extended object tracking. However, in extended object tracking, the discrepancy between the center of the ship and the position reported by the AIS system is no longer negligible and is a source of systemic bias, which can degrade tracking performance. In this paper, we introduce a method for estimating this discrepancy based on AIS information and the estimation provided by the Gaussian process target model from the exteroceptive sensor data. We use this method combined with an extended object Poisson multi-Bernoulli mixture (PMBM) filter to perform multiple extended object tracking. We also introduce a specific method for initialization of targets using AIS measurements in this filter. We validate the proposed method with LiDAR and AIS data, collected from an inland waterway in Belgium. The results show that compensating for the bias in this manner results in better tracking performance, primarily due to better initialization of new targets.

Vouch, Oliviero; Nardin, Andrea; Minetto, Alex; Zocca, Simone; Dovis, Fabio; Konitzer, Lauren; Parker, Joel J. K.; Ashman, Benjamin; Bernardi, Fabio; Tedesco, Simone; Fantinato, Samuele

Bayesian Integration for Deep-Space Navigation with GNSS Signals

Abstract

Recent advancements in spaceborne receiver technology have extended the application of Global Navigation Satellite System (GNSS)-based navigation systems to space missions. However, the actual availability and usability of GNSS signals in deep-space is still questionable, lacking experimental evidence. The Lunar GNSS Receiver Experiment (LuGRE) is a joint NASA-Italian Space Agency (ASI) payload aiming to showcase GNSS-based Positioning, Navigation and Timing (PNT) during its transfer orbit to the Moon. Operating without direct interface with on-board Guidance, Navigation & Control (GNC) subsystems, the LuGRE receiver requires alternative means of aiding to pursue precise Orbit Determination (OD) in the challenging space environment. This paper investigates a custom Trajectory-Aware EKF (TA-EKF) architecture that integrates aiding observations in the form of a pre-mission design of the LuGRE trajectory. Two alternative designs are presented, integrating aiding observations in the observation-domain and state-domain, respectively. The proposed architectures are evaluated by post-processing raw GNSS observables collected in a real-time Hardware-in-the-Loop (HIL) simulation with GNSS Radio Frequency (RF) signals. A comprehensive assessment leveraging Monte Carlo (MC) analyses characterizes the OD performance under aiding observation errors and mismodeling, comparing the TA-EKF models against a standalone Extended Kalman Filter (EKF) solution.

Dreo, Johann; Laudy, Claire; Lobentanzer, Sebastian; Baric, Marko; Gaydukova, Ekaterina; Schwikowski, Benno

Reproducible Mapping of Tabular Data into Semantic Knowledge Graphs with OntoWeaver and BioCypher

Abstract

Large-scale high-level information fusion and data integration is a pressing need in several scientific domains. Recently, the biomedical community established BioCypher, a tool to help create large Semantic Knowledge Graphs (SKGs) in a simple and reproducible way. In this article, we introduce OntoWeaver, a companion tool to BioCypher that allows to easily extract tabular data into SKGs by using a simple declarative mapping. OntoWeaver allows implementing reproducible mappings from tabular data to SKGs with a simple declarative configuration. The use of OntoWeaver and BioCypher is demonstrated in two different use cases: cancer database integration and invasive species monitoring. We believe that OntoWeaver and BioCypher, both free and open-source software, can help several scientific communities working on SKGs and high-level information fusion problems.

Zubasti Recalde, Pablo; Saiz Fernández, Mario; García Herrero, Jesús; Molina López, José Manuel

Computer Vision-based road surveillance system using autonomous drones and sensor fusion

Abstract

This paper shows an innovative approach to road monitoring by integrating autonomous drones and sensor fusion within a computer vision-based system. By employing different sets of algorithms, drones equipped with cameras, GPS, and ultrasonic distance sensors can efficiently detect and geolocate road damages, providing crucial data for maintenance and infrastructure management. The system's key components include automated planning, autonomous flight capabilities, object detection, and sensor fusion techniques, enhancing scalability and adaptability. The main objective is to use context information to compute a flying plan where the subsequent detection of defects allows us to expand and enhance GIS data. The implementation of such a system holds significant potential for improving road safety and optimizing maintenance costs, marking a notable advancement in the convergence of autonomous technologies, sensor fusion, and computer vision for effective road surveillance.

Shahsafi, Soroush; Naderkhani, Farnoosh

Enhancing Stock Trading Performance with Deep Q-Learning by Addressing Noisy Data through Advanced Denoising Techniques

Abstract

This study presents a comparative analysis of different reinforcement learning configurations for the stock trading problem, with IBM as a case study. To address the challenge of noisy data, we explore the effectiveness of various denoising methods, including Wavelet Transform, Temporal Attention Network (TAN), and Fourier Transform, in improving model performance. We employ two different architectures, Multilayer Perceptron (MLP) and Long Short-Term Memory (LSTM), to calculate Q-values for each possible action, resulting in six distinct configurations. Evaluation is based on key metrics such as yearly returns, Sharpe ratios, and maximum drawdowns over a specified timeframe. We compare the performance of our models against benchmark strategies including Relative Strength Index (RSI), Moving Average Convergence Divergence (MACD), and Bollinger Bands. Our results demonstrate that DQN-based trading outperforms benchmark methods. Furthermore, configurations utilizing TAN, whether in conjunction with MLP or LSTM, consistently exhibit superior performance. These findings suggest that TAN-based denoising methods combined with DQN offer promising solutions for enhancing stock trading strategies using reinforcement learning techniques.

Somero, Michele; Urli, Federico; Snidaro, Lauro; Liani, Alessandro

Defect detection MultiHeadAttention Fusion model on images acquired with different light sources

Abstract

In this paper, we discuss and try a multi-image fusion approach with a multibranch Convolutional Neural Network (CNN) that implies a MultiHeadAttention (MHA) technique in the fusion center. This work studies the employment of the architecture on actual data containing different images of the same USB device. The images differ in the direction of the light at the moment of the acquisition. We observed that instead of employing a simple concatenation fusion of the outputs, the network architecture could employ a multibranch classification featurewise, which utilizes a multi-head attention mechanism instead of a channel attention one.

Ouattara, Koffi Ismael; Petrovska, Ana; Hermann, Artur; Trkulja, Nataša; Dimitrakos, Theo; Kargl, Frank

On Subjective Logic Trust Discount for Referral Paths

Abstract

Subjective Logic (SL) enriches probabilistic logic by incorporating uncertainty and subjective belief ownership, enabling the expression of uncertainty about subjective beliefs. Unlike traditional probabilistic logics, SL 1) accommodates situations where different agents express beliefs about the same proposition, integrating the subjective nature and ownership of beliefs; and 2) addresses existing limitations in Dempster-Shafer Theory of evidence (DST), particularly in modelling trust transitivity. In modern computer systems, trust assessment extends beyond direct relationships to complex networks, necessitating the consideration of referral and direct trust relationships. This paper introduces a novel trust discount operator for referral edges in complex networks, addressing challenges in discounting trust across two and multiple referral edges. Through our empirical analysis, we demonstrate the effectiveness of the proposed operator and establish a relationship between path length and trustworthiness.

Urli, Federico; Somero, Michele; Snidaro, Lauro; Johnson, Chad; Vallisa, Tiziano; Visentini, Ingrid

FeU-Net: overcomplete representations with large kernels for edge detection

Abstract

In recent years, segmentation algorithms utilizing deep learning have achieved outstanding performance in medical image segmentation. However, accurately delineating small anatomical structures continues to be a challenging task, even for the most advanced methods that produce impressive results. This challenge might arise from the use of small kernels and downsampling operations, which often emphasize complex high-level features at the expense of low-level details like edges. Inspired by recent research highlighting this challenge, we developed a novel architecture that combines the standard U-Net with an additional branch harnessing the potential of large convolutional kernels. These large kernels are utilized in a decreasing-increasing manner over image features of the same size, guiding the network to focus on smaller parts. The proposed method demonstrated strong potential in segmenting small anatomical structures, surpassing our baseline and matching the performance of a robust state-of-the-art network across various datasets and domains, all while maintaining a relatively small number of parameters.

Frisch, Daniel; Hanebeck, Uwe D.

Gaussian Mixture Particle Filter Step based on Method of Moments

Abstract

We propose a novel update step of a Gaussian mixture particle filter for nonlinear state estimation. The update procedure works as follows: First, unweighted samples are drawn in an optimal deterministic sense from a prior Gaussian mixture. These samples are then assigned weights from the likelihood function, and we compute higher-order moments from this sample-based posterior. These moment approximations converge with $L^{-1}$ instead of $L^{-1/2}$ as our samples are optimal deterministic. Finally, the continuous posterior approximation is determined as the Gaussian mixture that has minimal Fisher information under the constraint of having the aforementioned moments. To achieve this, we employ a closed-form solution of the Fisher information that involves Gaussian root mixture densities.

Grini, Jon Torgeir; Mester, Rudolf; Anthonsen Nygård, Trym; Dalhaug, Nicholas; Brekke, Edmund Førland; Stahl, Annette

FusedWSS: Water Surface Segmentation Fusing Machine Learning and Geometric Cues

Abstract

Navigating unmanned surface vehicles (USVs) in urban waterways presents unique challenges due to irregular waterlines, obstacles, and reflections in the water. Determining the collision-free navigable area is crucial to enable safe USV operation. This paper introduces Fused Water Surface Segmentation (FusedWSS), a novel approach to water surface segmentation that aims to enhance navigation capabilities for USVs in complex harbor environments using a stereo camera. The method locates the water plane by performing plane fitting with outlier rejection and plane validation on the reconstructed 3D point cloud. From the plane parameters, the virtual horizon line is inferred and used for point cloud and image cropping. The water surface mask and virtual horizon line are fused with a deep learning-based semantic segmentation method to produce accurate and reliable water masks for each image frame. Additional refinement of the water mask is performed using detected obstacle masks. Validation was carried out using data from the MilliAmpere 2 autonomous ferry prototype in Trondheim, Norway, and a publicly available maritime dataset, demonstrating the efficacy of the methods.

Daniyan, Abdullahi; Inchingolo, Alessio V.; McAinsh, Andrew; Burroughs, Nigel

Enhanced Kinetochore Detection During Mitotic Human Cell Division using CFAR

Abstract

In this paper, we present an innovative application of the Constant False Alarm Rate (CFAR) algorithm, traditionally utilized in radar signal processing, to enhance the accuracy and reliability of kinetochore (KT) tracking in live-cell lattice light-sheet microscopy (LLSM) imaging of human cells during the mitotic phase of cell division. Fluorescently labelled KTs appear as spots in diffraction-limited light microscopy. Traditional KT detection methods, such as adaptive thresholding, often struggle with cells' dynamic and noisy backgrounds, leading to less efficient KT identification. By adapting the CFAR algorithm to the specific challenges of KT detection in 3D, we present a method that offers improved precision and stability in detecting KTs across different stages of mitosis. The performance of the CFAR-KT method was rigorously compared to the adaptive thresholding approach across a cohort of 31 cells, with results highlighting CFAR-KT's enhanced detection efficiency. Despite a slightly lower mean detection count compared to the adaptive method, the CFAR-KT method achieved lower false positives and a higher success rate in tracking KTs over the duration of the cell division process, underscoring its effectiveness in capturing the dynamics of KTs.

Ting, Albert; Shapero, Samuel

Scaling Sparse Approximation with a Two-Layer Spiking Locally Competitive Algorithm

Abstract

Many applications, such as radio channel estimation, require solving for unknowns in overcomplete bases. Nonlinear solvers like Basis Pursuit Denoising (BPDN) can leverage sparse statistics to improve accuracy relative to linear solvers. The Locally Competitive Algorithm (LCA) is a nonlinear dynamical system that converges on the solution to BPDN, and can be implemented via a Spiking Neural Network that is orders of magnitude faster and more power efficient than CPU-based BPDN solutions. However, the Spiking LCA scales quadratically with the dimensionality of the state estimate, which can quickly make physical implementation impractical. In this work, we introduce a multi-layered complex-valued LCA architecture, which -- by taking advantage of hierarchical sparsity in the channel estimation problem -- allows sub-quadratic scaling of computational resource requirements, reducing resource requirements for a 357 complex channel estimation by 7.6x relative to a single layer solution. We implemented a 1470 complex channel, 2-layer Spiking LCA on Intel's Loihi chip, and demonstrated a 10x reduction in temporal smear relative to a linear solver.

Bondarchuk, Jennifer; Trezza, Anthony; Bucci, Donald J.

Efficient Implementation of Multi-sensor Adaptive Birth Samplers for Labeled Random Finite Set Tracking

Abstract

Adaptive track initiation remains a crucial component of many modern multi-target tracking systems. For labeled random finite sets multi-object filters, prior work has been established to construct a labeled multi-object birth density using measurements from multiple sensors. A naive construction of this adaptive birth set density results in an exponential number of newborn components in the number of sensors. A truncation procedure was provided that leverages a Gibbs sampler to truncate the birth density, reducing the complexity to quadratic in the number of sensors. However, only a limited discussion has been provided on additional algorithmic techniques that can be employed to substantially reduce the complexity in practical tracking applications. In this paper, we propose five efficiency enhancements for the labeled random finite sets multi-sensor adaptive birth procedure. Simulation results are provided to demonstrate their computational benefits and show that they result in a negligible change to the multi-target tracking performance.

Hou, Elizabeth; Greenwood, Ross; Kumar, Piyush

Machine Learning Models for Improved Tracking from Range-Doppler Map Images

Abstract

Statistical tracking filters depend on accurate target measurements and uncertainty estimates for good tracking performance. In this work, we propose novel machine learning models for target detection and uncertainty estimation in range-Doppler map (RDM) images for Ground Moving Target Indicator (GMTI) radars. We show that by using the outputs of these models, we can significantly improve the performance of a multiple hypothesis tracker for complex multi-target air-to-ground tracking scenarios.

Sætran, Ole Halvard; Rolfsjord, Sigmund

Enhancing Predicted Distributions for Constant Acceleration and Turn Rate Motion Models: A Deep Learning Approach

Abstract

Gating and association are key components of target tracking, most commonly using the predicted distribution of the track to evaluate new measurements. While a Gaussian predicted distribution is widely used, it is not optimal for some groups of targets, such as fixed-wing aircraft and surface vessels. This article introduces the Constant Acceleration and Turn Rate - Neural Network (CAT-NN) method, which uses a neural network to calculate the predicted distribution for such targets. Our work can be seen as an expansion of the CAT distribution by correcting for track uncertainty, thereby unlocking CAT distributions for a much wider range of tracking applications. Simulations show that the CAT-NN predicted distribution is a better match for the true predicted distribution than both the CAT and Gaussian distributions for a fixed-wing aircraft. It also outperforms the Gaussian distribution in a simulation tracking scenario with a single fixed-wing aircraft in heavy clutter. The CAT-NN model is runtime efficient and implemented as a custom hypothesiser component for the Stone Soup tracking framework.

Sidheekh, Sahil; Tenali, Pranuthi; Mathur, Saurabh; Blasch, Erik; Natarajan, Sriraam

On the Robustness and Reliability of Late Multi-Modal Fusion using Probabilistic Circuits

Abstract

Multimodal fusion is important for building intelligent systems that exploit patterns across diverse data sources for improved decision-making. However, the reliability and robustness of these systems in safety-critical domains are often compromised by the inherent noise and incompleteness of data. Probabilistic Circuits (PCs) have recently emerged as a promising approach for late (or decision) fusion. Their strength lies in being both expressive and capable of inferring source credibility due to their ability to tractably perform exact probabilistic inference. However, their ability to handle missing data and their reliability in practical scenarios remains underexplored. This work investigates the robustness of PCs as fusion functions in scenarios with missing and noisy data; particularly by examining their impact on the calibration and reliability of the resulting classifiers. Our findings show that PCs not only enable the modeling of complex correlations across modalities but also lead to calibrated and reliable classifiers, highlighting their potential as a robust fusion mechanism in multimodal systems.

Yoon, Han Jun; Matsumoto, Shou; Costa, Paulo; Cho, Jin-Hee

Towards an Efficient Simulation-Based Anytime Inference in Subjective Bayesian Networks

Abstract

Subjective Bayesian networks (SBN) integrate Bayesian Networks (BN) with Subjective Logic, enabling the representation of second-order uncertainty, denoting the uncertainty surrounding the probability distribution of an event. Although prior research predominantly centers on exact inference within the SBN framework, there is a notable dearth of exploration into the realm of approximate inference in SBN. Our work is specifically geared towards addressing this gap, focusing on the application of diverse sampling methodologies (i.e., forward and Gibbs sampling) for approximate inference in SBN. The primary contribution of this work lies not only in the introduction of approximate inference in SBN but also in the formulation of an ``anytime'' SBN inference algorithm. This implies that a best inference estimate can be obtained at any given moment, given trade-offs in the precision. Moreover, the allocation of computational resources is a customizable and potentially optimizable process. Through a rigorous series of experiments, we empirically demonstrate that the number of iterations to convergence decreases as we provide more samples for both forward and Gibbs sampling. Furthermore, we discover the difference between approximate and exact inference in belief and uncertainty mass of subjective opinion becomes more unpredictable as the error gets large in BN probability. Lastly, in our experiments, we demonstrate the number of BN samples has a greater impact on belief than the number of SBN iterations. These findings indicate that the family of greedy algorithms (based on local graded changes -- such as gradients) can be a promising approach for finding optimal allocations of computational resources in this framework. The software assets produced and used in this work will be made available as an open source Python library.

Peng, Bohua; Chen, Bin; He, Wei; Thorne, William; Kadirkamanathan, Visakan

Efficient Token Sparsification Through the Lens of Infused Knowledge

Abstract

Leveraging large language models (LLMs) to fuse heterogeneous knowledge is an exciting emerging field. However, with billions of parameters, these pretrained language models are prohibitively computationally expensive at inference time. Token sparsification methods can proactively accelerate inference by selecting important features from the sequence but often require task-dependent retraining. To address this, we propose Bilevel Token prUniNg wiTh Infused kNowledGe (Bunting), an interpretable token pruning method that leverages task-level knowledge encoded in prefixes to guide token sparsification, eliminating the need for task-specific retraining. Bunting performs Bayesian Token Sparsification, where the inner loop learns a joint representation to perform the task, and the outer loop learns adaptive attention masks for sparse representations, thus pruning redundant tokens layer-by-layer without compromising the pretrained abilities of LLMs. Additionally, we introduce an innovative antiphrasis evaluation protocol to test model adaptivity on rhetorical relations. Furthermore, we demonstrate that precomputed prefixes can effectively guide token sparsification in different knowledge-intensive tasks, maintaining task-level knowledge to identify important tokens and reduce the finetuning burden. Experimental results demonstrate that our method achieves over 0.3x wall-clock speed-up with only 0.14x learnable parameters in knowledge-intensive tasks. Our findings suggest that token pruning can improve out-of-distribution detection, with sarcasm being more challenging to detect than immorality.

Thompson, Fletcher; Hansen, Peter Nicholas; Galeazzi, Roberto; Palma, Marco; Brock, Andreas Libonati; Mariani, Patrizio

Autonomous Inspection and Data Fusion for Maritime Critical Infrastructures

Abstract

Automation and robotics are essential for the effective monitoring and inspection of maritime critical infrastructures and marine environments. We demonstrate the use of an unmanned surface vehicle, integrated with acoustic and optical sensors, to perform fast and accurate inspections of maritime infrastructures in confined areas (i.e., presence of buildings and multiple obstacles). High resolution maps are obtained fusing LiDAR and 2D acoustic multi-beam forward looking camera point clouds. The technological pipeline is demonstrated through a survey in Copenhagen harbour. The data acquisition leverages on methods to improve path planning and localization to correct failures in the RTK GNSS due to shadowing of buildings and other obstacles. The entire data flow is streamlined to produce fast delivery time and data access, optimizing data acquisition and processing, providing results into a dedicated web service tailored to users’ needs for knowledge and information extraction.

López, Michael Ernesto; Vasstein, Kjetil; Brekke, Edmund; Mester, Rudolf; Stahl, Annette

A General Low-Parameter 3D Ship Hull Extent Model for Object Tracking

Abstract

In autonomous vehicle systems, it is paramount to detect other objects in the vicinity and track their movement. Extended Object Tracking (EOT) provides a convenient framework for tracking objects using high-resolution sensor data by defining models for the object's spatial dimensions (a.k.a. extent). In maritime applications, the objects of interest are mainly other maritime vessels, and these vary greatly in shape and size. This diversity proves to be a challenge for defining general extent models that both give accurate representations for most vessels and that do not depend on a large number of parameters. In this paper, a general three-dimensional low-parameter ship hull model designed for EOT is presented. The presented extent model is constructed by intertwining a polynomial representation along the vertical direction with a frequency representation along the horizontal plane. However, to reduce the dimension of the parameter space without compromising its accuracy, the horizontal frequency representation is modified by performing a Principal Component Analysis (PCA). In particular, this extent representation does not require an underlying discretization grid, which makes the model scalable and therefore well-suited for modeling objects that vary greatly in size.

Bencivenga, Pasquale; Isoletta, Giorgio; Opromolla, Roberto; Fasano, Giancarmine

Attitude Motion Characterization of Resident Space Objects via Fusion of Ground-based and Space-based Light Curves

Abstract

Due to the growing number of fragmentation-related debris and the launch of mega constellations of satellites, the characterization of Resident Space Objects has been assuming a growing importance in the context of Space Situational Awareness programs to enable accurate orbit propagations and related functionalities such as collision avoidance. Based on the analysis of light curves, photometric characterization can provide useful information concerning the objects’ surface material, shape, and attitude motion. In this context, this paper proposes an attitude motion classifier of unknown space objects using light curves. In particular, the focus is the combination of data from multiple sensors, either ground or space-based, in order to get a more reliable classification than the one arising from a single photometric measurement. Each light curve is classified using a spectral analysis method based on the Lomb-Scargle Periodogram and the Phase Dispersion Minimization approaches. The classifier’s outputs are then fused first at sensor level and then across multiple sensors to derive a unique classification for the observed space object. The performance of the presented architecture is assessed in a numerical environment able to reproduce synthetic light curves accounting for complex object geometries, and a realistic evolution of orbital and rotational dynamics. A correct classification has been produced for all the considered test cases preliminary proving the effectiveness of the proposed approach.

Brady, John-Joseph; Luo, Yuhui; Wang, Wenwu; Elvira, Víctor; Li, Yunpeng

Regime Learning for Differentiable Particle Filters

Abstract

Differentiable particle filters are an emerging class of models that combine sequential Monte Carlo techniques with the flexibility of neural networks to perform state space inference. This paper concerns the case where the system may switch between a finite set of state-space models, i.e. regimes. No prior approaches effectively learn both the individual regimes and the switching process simultaneously. In this paper, we propose the neural network based regime learning differentiable particle filter (RLPF) to address this problem. We further design a training procedure for the RLPF and other related algorithms. We demonstrate competitive performance compared to the previous state-of-the-art algorithms on a pair of numerical experiments.

Leroy, Idyano; Saucan, Augustin A.; Petetin, Yohan; Clark, Daniel

An Analysis of the Mutual Information Upper Bound for Sensor-Subset Selection

Abstract

The ability to rapidly select an optimal subset of sensors is of critical importance in massive multi-sensor target tracking. Various information metrics exist for selecting the subset of sensors that is most informative with respect to the target being tracked. Moreover, information bounds were proposed as approximate metrics in order to speed up the selection algorithms. In this paper, we provide an analysis on the information loss and its impact on the subset selection problem when employing an information upper bound instead of the exact mutual information metric. We design several greedy sensor-selection algorithms that sequentially evaluate the exact mutual information between a set of sensors and the target. Subsequently, we compare these algorithms with a sensor-selection method that employs an information upper bound and highlight situations where the latter finds sub-optimal solutions.

Wright, James S.; Sun, Mengwei; Davies, Mike E.; Proudler, Ian K.; Hopgood, James R.

Implementation of AKKF-based Multi-Sensor Fusion Methods in Stone Soup

Abstract

This paper explores the increasing demand for accurate and resilient multi-sensor fusion techniques, particularly within 3D tracking systems enhanced by drone technology. Employing the adaptive kernel Kalman filter (AKKF) methodology within the Stone Soup framework, our research seeks to develop robust fusion approaches capable of seamlessly amalgamating data from a multi-sensor arrangement with fixed ground sensors and dynamic sensors mounted on drones. By capitalising on the adaptive nature of the AKKF, we aim to refine the precision and dependability of 3D object tracking in intricate scenarios. Through comprehensive empirical evaluations, we illustrate the effectiveness of our proposed AKKF-based fusion strategies in enhancing tracking performance within the Stone Soup framework, thus contributing to the advancement of multi-sensor fusion methodologies within this framework.

Dassori, Ignacio; Adams, Martin; Vasquez, Jorge

Four-Legged Gait Control via the Fusion of Computer Vision and Reinforcement Learning

Abstract

This article explores the integration of fully autonomous legged robots in obstacle filled environments, simultaneously addressing the challenges of navigation and control. Despite the potential of legged robots for dynamic tasks, their deployment in complex environments has been hindered by the difficulty of developing effective autonomous control systems. In particular, the motion planning problem is addressed in this article, by formulating it as a Partially Observable Markov Decision Process (POMDP) and applying Proximal Policy Optimization (PPO), a model-free Deep Reinforcement Learning (DRL) algorithm. To improve sample efficiency and real-world applicability, the proposed method incorporates a Central Pattern Generator (CPG) for motion planning and a Variational Autoencoder (VAE) for terrain representation, reducing the complexity of action and observation spaces. Referred to as the VAE-CPG architecture, its performance is demonstrated using the Unitree Laikago robot within the PyBullet simulation environment, aiming to show its effectiveness in simulated construction sites. Our findings indicate that by reducing the legged action space to periodic gait patterns and optimizing the gait based on sensory feedback, we achieve enhanced adaptability and efficiency. This work presents a viable means towards the deployment of autonomous legged robots and their improved efficiency in real applications.

Wolf, Laura M.; Baum, Marcus

Track-to-track Association based on Deterministic Sampling using Herding

Abstract

Multi-sensor multi-object tracking in a track-to-track fusion framework involves the grouping of tracks (from different sensors) that belong to the same perceived object. In particular for collective perception scenarios in large-scale traffic systems the number of sensors and objects can be huge, as a large number of vehicles can be equipped with multiple sensors. In order to cope with the intractable number of possible associations, recently a stochastic optimization approach for track-to-track association was proposed. The key idea is to successively improve an initial association by means of performing random modifications, i.e., actions, on the current association. In this work, we develop a novel deterministic version of the algorithm, which employs herding in order to deterministically choose the next action. Simulations demonstrate that the deterministic version of stochastic optimization provides comparable results to the stochastic version with a significantly lower variance.

Schuster, Sonja; Wetzel, Johannes; Zeitvogel, Samuel; Laubenheimer, Astrid

Automatic Extrinsic Multi-Sensor Network Calibration based on Time Series Matching

Abstract

We propose an automatic calibration approach to determine the extrinsic (inter-sensor) calibration of a multisensor network for people tracking. A plan-view approach is used and pairwise overlapping detection areas of the distributed sensors are assumed. By exploiting intra-sensor tracks of an unknown number of tracking targets, we solve the referencing problem of the sensor fields of view by a matching of time series, avoiding any manual effort for the extrinsic calibration. We realize the automatic calibration exclusively based on intra-sensor tracking information by combining a trackwise w-RANSAC with a rotation-invariant distance measure, and an effective prefilter method based on the walking speed of topological and temporal matched track pairs. Our automatic calibration routine is evaluated on a multi-sensor network, consisting of five depth sensors with a top-down view on an indoor scene, in which five people are randomly walking for approximately one minute. The track mapping accuracy of our automatic calibration method is compared to a calibration based on a manual selection of homologous image points. Therefore, we propose an evaluation method regarding the global track mapping accuracy. By excluding known track matches of our dataset from the calibration process, we derive an assumption about the global tracking performance of the calibrated multi-sensor network.

Mari, Marco; Snidaro, Lauro

Ensemble of KalmanNets for Maneuvering Target Tracking

Abstract

Tracking a maneuvering target requires the modeling of the target's movements by multiple pre-defined mathematical models. However, the uncertainty in the target's dynamics can lead traditional Model Based (MB) tracking algorithms to a significant performance degradation when model mismatch occurs. To tackle this problem, we propose the use of a RNN for the purpose of learning complex target dynamics. Following the recent advances in state estimation provided by KalmanNet, a neural network-aided Kalman Filter, the proposed approach aims to exploit its tracking performance in a multiple model schema to compensate for model mismatch across maneuvers, leading to a more prompt response to motion switches. The results over a simulated set of maneuvering target trajectories demonstrate the potential of the proposed approach over the MB solution.

Ravier, Robert J.; Garagić, Denis; Galoppo, Travis; Jameson, Rex; Rhodes, Bradley J.; Zulch, Peter

Constrain, Correspond, Correct: Distributed Game-Theoretic Data Association for Assignment Games on Multimodal Sensing Grids

Abstract

Assignment games are a promising framework for autonomous management of a multimodal Tactical Sensing Grid (TSG). They provide theoretical guarantees and exhibit excellent empirical performance in maintaining custody of all observed targets in a scene. However, TSG sub-grid initiation of an assignment game requires multimodal data association between playing nodes (the sub-grid). Playing nodes must achieve consensus on target labels and identities before game play proceeds. Using the locally centralized communication network assumed by an assignment game, we propose the Triple-C distributed method for solving this association problem. The method we propose is suitable for fairly general scenarios, requiring that each node have an intrinsic notion of what constitutes an outlier and what constitutes similarity. At the core of the Triple-C method is a collection of parallelizable assignment games played between two nodes. Triple-C yields theoretical guarantees on output association consistency. We evaluate the performance of the Triple-C method on multiple simulations, showing that it computes associations quickly and accurately, thus enabling a TSG to maintain full situational awareness in a two-vehicle scenario.

Incitti, Francesca; Salfinger, Andrea; Snidaro, Lauro; Challapalli, Sri

Leveraging LLMs for Knowledge Engineering from Technical Manuals: A Case Study in the Medical Prosthesis Manufacturing Domain

Abstract

Ontologies are nowadays widely used to organize information across specific domains, being effective due to their hierarchical structure and the ability to explicitly represent relationships between concepts. Knowledge engineering, like compiling companies’ vast bodies of knowledge into these structures, however, still represents a time-consuming, largely manually performed process, esp. with significant amounts of knowledge often only recorded within unstructured text documents. Since the recently introduced Large Language Models (LLMs) excel on text summarization, this raises the question whether these could be exploited within dedicated knowledge fusion architectures to assist human knowledge engineers by automatically suggesting relevant classes, instances and relations extracted from textual corpora. We therefore propose a novel approach that leverages the taxonomic structure of a partially defined ontology to prompt LLMs for hierarchical knowledge organization. Unlike conventional methods that rely solely on static ontologies, our methodology dynamically generates prompts based on the ontology’s existing class taxonomy, prompting the LLM to generate responses that extract supplementary information from unstructured documents. It thus introduces the concept of using ontologies as scaffolds for guiding LLMs, in order to realize a mutual interplay between structured ontological knowledge and the soft fusion capabilities of LLMs. We evaluate our proposed algorithm on a real-world case study, performing a knowledge fusion task on heterogeneous technical documentation from a medical prosthesis manufacturer

Liu, Zixin; Tiller, Zachary; Godsill, Simon

Inference for Non-Gaussian Dynamical Models with Time-varying Skew

Abstract

In this paper we introduce tracking models based on non-Gaussian continuous time stochastic processes with time-varying skewness. The idea behind this is that the skewness of the dynamical model may be able to model a propensity for an object to undergo manoeuvres of a particular type, for example velocities tending in a particular direction, but that these may change over time. This process is constructed based on a random series representation of conditionally Gaussian Levy processes, which enables straightforward simulation of the models. We demonstrate the specific example of alpha-stable processes and find that such processes can capture abrupt changes owing to their heavy-tailed behaviour, and demonstrate the random changes in direction caused by the time-changing skewness of the distribution. We propose methods for joint tracking of both states and skewness for such processes, based on a marginalised particle filter, which are demonstrated to perform well even with limited numbers of particles.

Zeng, Jing; Mannari, Prabhanjan; Acharya, Aalok; Tharmarasa, Ratnasingham

Radar Data Clustering and Bounding Box Estimation with Doppler Measurements

Abstract

High-resolution automotive radars, which are widely used nowadays, yield multiple measurements per frame from a single target. Clustering these measurements accurately and finding the tight bounding boxes are two challenging problems. In this work, the shape is estimated using a rectangular bounding box using the position and range rate measurements from the radar. While the Doppler (or range rate) measurements provide extra information about the target velocity, the presence of micro-Doppler (for example, returns from tires of a car) can significantly degrade the clustering, bounding box and heading estimates. It is necessary to cluster the measurements corresponding to different targets, as well as those that occur due to micro-Doppler. A clustering method is developed that can effectively use the Doppler information to differentiate closely spaced targets while avoiding the drawbacks of micro-Doppler. The bounding box estimate is refined by using only the measurements corresponding to the target bulk and, in turn, further aids in clustering iteratively. The effectiveness of the proposed approach is verified using simulations for different scenarios.

Wei, Shaoxiu; Liang, Mingchao; Meyer, Florian

A New Architecture for Neural Enhanced Multiobject Tracking

Abstract

Multiobject tracking (MOT) is an important task in robotics, autonomous driving, and maritime surveillance. Traditional work on MOT is model-based and aims to establish algorithms in the ramework of sequential Bayesian estimation. More recent methods are fully data-driven and rely on the train- ing of neural networks. The two approaches have demonstrated advantages in certain scenarios. In particular, in problems where plenty of labeled data for the training of neural networks is available, data-driven MOT tends to have advantages compared to traditional methods. A natural thought is whether a general and efficient framework can integrate the two approaches. This paper advances a recently introduced hybrid model-based and data-driven method called neural-enhanced belief propagation (NEBP). Compared to existing work on NEBP for MOT, it introduces a novel neural architecture that can improve data association and new object initialization, two critical aspects of MOT. The proposed tracking method is leading the nuScenes LiDAR-only tracking challenge at the time of submission of this paper.

Pravong, Vivien; Condomines, Jean-Philippe; Öman Lundin, Gustav; Puechmorel, Stéphane

Unscented Kalman Filter using Optimal Quantization

Abstract

This paper presents a novel approach to deal with nonlinear filtering by augmenting an Unscented Kalman Filter (UKF) with an Optimal quantization algorithm, named OQ-UKF. The Unscented Kalman Filter uses a sigma-point based method to approximate the distribution of an unknown random variable onto which is applied a nonlinear transformation, providing a cloud of evolving points. However, the generation of these socalled sigma-points is done by a deterministic algorithm which needs tuning in order to accurately capture the distribution of the estimate. This tuning is often problem-dependent due to nonlinearities and sometimes not optimal. We propose to fuse an UKF with Optimal quantization whose objective is to find the best approximation of the density of a random variable. The designed OQ-UKF is described in this paper, and its performance is evaluated for some relevant practical problems, such as pose estimation of a two-dimensional mobile robot.

Li, Qing; Gan, Runze; Godsill, Simon

Decentralised Gradient-based Variational Inference for Multi-sensor Fusion and Tracking in Clutter

Abstract

This paper investigates the task of tracking multiple objects in clutter under a distributed multi-sensor network with time-varying connectivity. Designed with the same objective as the centralised variational multi-object tracker, the proposed method achieves optimal decentralised fusion in performance with local processing and communication with only neighboring sensors. A key innovation is the decentralised construction of a locally maximised evidence lower bound, which greatly reduces the information required for communication. Our decentralised natural gradient descent variational multi-object tracker, enhanced with the gradient tracking strategy and natural gradients that adjusts the direction of traditional gradients to the steepest, shows rapid convergence. Our results verify that the proposed method is empirically equivalent to the centralised fusion in tracking accuracy, surpasses suboptimal fusion techniques with comparable costs, and achieves much lower communication overhead than the consensus-based variational multi-object tracker.

Seyedmohammadi, S. Jamal; Atapour, S. Kawa; Abouei, Jamshid; Mohammadi, Arash

KnFu: Effective Knowledge Fusion

Abstract

Federated Learning (FL) is a decentralized approach that allows for collaborative training of Machine Learning (ML) models across multiple local nodes, ensuring data privacy and security while leveraging diverse datasets. Conventional FL, however, is susceptible to gradient inversion attacks, restrictively enforces a uniform architecture on local models, and suffers from model heterogeneity (model drift) due to non-IID local datasets. To mitigate some of these challenges, the new paradigm of Federated Knowledge Distillation (FKD) has emerged. FKD is developed based on the concept of Knowledge Distillation (KD), which involves extraction and transfer of a large and well-trained teacher model's knowledge to lightweight student models. FKD, however, still faces the model drift issue. Intuitively speaking, not all knowledge is universally beneficial due to the inherent diversity of data among local nodes. This calls for innovative mechanisms to evaluate the relevance and effectiveness of each client's knowledge for others, to prevent propagation of adverse knowledge. In this context, the paper proposes Effective Knowledge Fusion (KnFu) algorithm that evaluates knowledge of local models to only fuse semantic neighbors' effective knowledge for each client. The KnFu is a personalized effective knowledge fusion scheme for each client, that analyzes effectiveness of different local models' knowledge prior to the aggregation phase. In this context, closeness of clients' knowledge is measured by estimating the class distributions of local datasets based on the transmitted localized knowledge. Comprehensive experiments were performed on MNIST and CIFAR10 datasets. KnFu outperforms its FL-based, FKD-based, and local training baselines in scenarios where the clients have small datasets with intermediate degree of heterogeneity. In scenarios with large and highly heterogeneous local datasets, however, local training seems to be preferable to knowledge fusion-based.

Wakefield, Joshua J.; Neal, Adam; Haslinger, Stewart; Ralph, Jason F.

Sonar Path Planning Using Reinforcement Learning

Abstract

Passive towed array sonar systems play an essential role in submarine situational awareness. However, the detection and localisation of sound-emitting objects is a more challenging task compared to their active counterparts due to a lack of immediate range information. By making manoeuvres and changing the bearings at multiple positions, a passive sonar can localise and track the source of the sound. Reinforcement learning is the process of learning an optimal strategy to guide an agent's actions towards optimising its cumulative reward for a given task. This work evaluates an agent's ability to control a passive towed array sonar system for optimal source localisation and tracking in the underwater environment, using collision avoidance as a practical example application.

Zhang, Wenyu; Khojasteh, Mohammad Javad; Meyer, Florian

Particle Flows for Source Localization in 3-D Using TDOA Measurements

Abstract

Abstract—Localization using time-difference of arrival (TDOA) has myriad applications, such as passive surveillance systems and marine mammal research. We aim to localize an unknown number of static sources in 3-D based on TDOA measurements. The proposed localization algorithm based on particle flow (PFL) can overcome the challenges related to the highly non-linear TDOA measurement model, the data association uncertainty, and the uncertainty in the number of sources to be localized. Different PFL strategies are compared in a challenging multisensor source localization scenario. In particular, we consider PFL- based computations based on the exact Daum and Huang (EDH) and the localized exact Daum and Huang (LEDH) processing strategies with one and multiple Gaussian kernels. Our numerical results demonstrate that the number of sources can be determined correctly, and accurate location estimates can be obtained. An EDH implementation with 100 Gaussian achieves the best accuracy-complexity tradeoff.

Yang, Feng; Niu, Jingru; Shi, Lihong; Zheng, Litao

Identification and Tracking of Multi-group Targets in Circular Formation under Multi-sensor Networks

Abstract

To address the challenges posed by structure identification, data transmission and information fusion in distributed group target tracking, this paper proposes a novel distributed structure identification and tracking algorithm for resolvable group targets with circular formations. The proposed algorithm combines the Joint Probabilistic Data Association algorithm and the K-medoids clustering method within each sensor to estimate all target states and partition them into different subgroups. Then, the circular formation of each subgroup is identified based on its geometric features. In addition, the Consensus on Information is introduced to fuse each local information after matching the states of group targets across multi-sensor networks. Simulation results demonstrate the effectiveness of the proposed algorithm.

Waxman, Daniel; Djurić, Petar M.

A Gaussian Process-based Streaming Algorithm for Prediction of Time Series With Regimes and Outliers

Abstract

Online prediction of time series under regime switching is a widely studied problem in the literature, with many celebrated approaches. Using the non-parametric flexibility of Gaussian processes, the recently proposed INTEL algorithm provides a product of experts approach to online prediction of time series under possible regime switching, including the special case of outliers. This is achieved by adaptively combining several candidate models, each reporting their predictive distribution at time t. However, the INTEL algorithm uses a finite context window approximation to the predictive distribution, the computation of which scales cubically with the maximum lag, or otherwise scales quartically with exact predictive distributions. We introduce LINTEL, which uses the exact filtering distribution at time t with constant-time updates, making the time complexity of the streaming algorithm optimal. We additionally note that the weighting mechanism of INTEL is better suited to a mixture of experts approach, and propose a fusion policy based on arithmetic averaging for LINTEL. We show experimentally that our proposed approach is over five times faster than INTEL under reasonable settings with better quality predictions.

Hare, James Z.; Liang, Yuchen; Kaplan, Lance M.; Veeravalli, Venugopal V.

On Network Quickest Change Detection with Uncertain Models: An Experimental Study

Abstract

We study the problem of Quickest Change Detection (QCD) in a complex networked system consisting of a set of heterogeneous agents that sequentially feed information to a central fusion center. At any unknown deterministic time, a persistent anomaly occurs, causing the distribution of observations from an unknown distinguishable subset of agents to simultaneously change from a nominal (pre-change) distribution to an anomalous (post-change) distribution, and the goal of the fusion center is to detect the change as quickly as possible subject to a false alarm constraint. Traditionally, various fusion rules have been proposed that assume that the distributions at each agent are either completely known or unknown and are locally solved using the Cumulative Sum (CuSum) and Generalized Likelihood Ratio (GLR) statistics, respectively. When an agent has access to training data, the Uncertain Likelihood Ratio (ULR) test generalizes distributional assumptions using uncertain distributions. However, the ULR has not been implemented for network change detection. This paper empirically studies incorporating the ULR statistics into the existing fusion rules for QCD and compares the average detection delay. Our results show that the ULR test can improve the average detection delay over the GLR tests using certain fusion techniques, while approaching the detection delay of the CuSum tests as the training data increases. Our results provide insights into future theoretical analysis to improve network QCD with imprecise knowledge of the distributions.

Jin, Yuchuan; Stenhammar, Theodor; Bejmer, David; Beauvisage, Axel; Xia, Yuxuan; Fu, Junsheng

Towards Accurate Ego-lane Identification with Early Time Series Classification

Abstract

Accurate and timely determination of a vehicle's current lane within a map is a critical task in autonomous driving systems. This paper utilizes an Early Time Series Classification (ETSC) method to achieve precise and rapid ego-lane identification in real-world driving data. The method begins by assessing the similarities between map and lane markings perceived by the vehicle's camera using measurement model quality metrics. These metrics are then fed into a selected ETSC method, comprising a probabilistic classifier and a tailored trigger function, optimized via multi-objective optimization to strike a balance between early prediction and accuracy. Our solution has been evaluated on a comprehensive dataset consisting of 114 hours of real-world traffic data, collected across 5 different countries by our test vehicles. Results show that by leveraging road lane-marking geometry and lane-marking type derived solely from a camera, our solution achieves an impressive accuracy of 99.6%, with an average prediction time of only 0.84 seconds.

Ferry, James P.; Ahmed, Adam S.

A Bayesian Decision Theory Paradigm for Test and Evaluation

Abstract

Traditional methods for the Test and Evaluation of military systems are based on a combination of ensuring that requirements are met and optimizing the information gained during testing. However, neither approach can address the following fundamental question: how much is a test worth? There are obvious practical benefits to being able to answer this question. Furthermore, the inability of traditional methods to address it suggest that they are not capturing the essence of what the Test and Evaluation process is. This paper presents a new approach to Test and Evaluation based on Bayesian Decision Theory. It maintains the current knowledge one has about the parameters that govern a system's behavior, updating this knowledge using a Bayesian filter whenever new data arrive. It couples this Bayesian filter with a utility function that is based on the system's operational utility: i.e., the value of the deployed system to its various stakeholders. In this paradigm, the value of the system to its stakeholders and the cost of testing can be expressed in a common currency of dollars. Therefore it answers the question of how much a test is worth as an organic by-product of how it functions. This paradigm, called Dynamo, is explored in a simple scenario that demonstrates the insights it offers into the nature of Test and Evaluation. It demonstrates that an arbitrary utility function can be decomposed into an intrinsic component that is concerned only with producing the correct decision, and a cost of imprecision which values information about a system for its own sake. Some testing protocols implicitly optimize utility functions that have a negative cost of imprecision, leading to the pathological behavior of declining to test even when testing is free. The paper concludes with a discussion of how Dynamo was implemented to perform a case study on test data from a specific radar system.

Bucco, Thomas John; Koliander, Günther; Kreidl, Bernd; Hlawatsch, Franz

Online Learning of Model Parameters and Object Classes in Extended Multiobject Tracking

Abstract

Most multiobject tracking methods rely on a statistical model that involves unknown parameters. Here, we propose a Bayesian method for class-aided online learning of model parameters within extended multiobject tracking. We address the case where the extended objects belong to unknown object classes defined by unknown values of the model parameters. The proposed method learns the number of object classes, the class parameters, and the objects’ class affiliations simultaneously with the tracking process, and the learned class and parameter information is leveraged for improved tracking. This is enabled by a parameter-dependent state-space model for extended multiobject tracking that incorporates a Dirichlet process prior, and by a related Gibbs sampler for online learning. Our simulation results demonstrate substantial gains in tracking performance due to class-aided online parameter learning.

Ajirak, Marzieh; Liu, Yuhao; Djurić, Petar M.

Filtering of High-Dimensional Data for Sequential Classification

Abstract

In many science and engineering problems, we observe high-dimensional data acquired sequentially. At each time instant, these data correspond to one of a predefined number of classes. The sequence of classes follows a certain pattern, with the transition probabilities of the classes being unknown. Our hypothesized generative model of the observed data involves two latent processes. The first is a root process representing the sequence of classes, while the second is a low-dimensional process generated as a Markovian process, depending on the current class and the previous value of the low-dimensional process. The observed high-dimensional process is generated from the low-dimensional state process. Our objective is to infer the posterior distributions of the classes as they evolve over time based on the observed data and the adopted model. To achieve this, we propose a method for estimating the latent processes. We demonstrate the effectiveness of our approach on synthesized data.

Beeson, Ryne

Generalized Bernoulli Gauss von Mises Distribution for Uncertainty Realism on Saddle-Center Spaces

Abstract

Most aspects of space situational awareness (SSA) rely on accurate and efficient uncertainty realism, propagation, and nonlinear filtering. A new frontier for SSA is the application to the cislunar realm, which lacks a global orbital element coordinate set. The dynamics of a representative model, the circular restricted three-body problem (CR3BP), for the cislunar domain provides the opportunity to define local orbital elements using dynamical systems techniques such as normal form theory. Motivated the structure of the CR3BP SSA problem, we construct a generalized Bernoulli Gauss von Mises distribution, that is defined on local orbit element coordinates generated from normal form theory at a saddle-center-center equilibrium point, and show its ability to capture what may be a common deformation mode of the CR3BP.

Sun, Wei; Tang, Xuning; Chang, Kuo-Chu

Regression Model Bias Evaluation by Estimating Conditional Densities with Gaussian Mixtures

Abstract

The exploration of AI fairness has emerged as a crucial area of research receiving growing attention in recent years. Various metrics have been proposed to assess group fairness, which examines whether the model outcomes correlate with sensitive attributes such as gender, ethnicity, age, and others. These fairness metrics primarily assess the statistical independence and conditional independence between the model prediction and the true target variable concerning the sen- sitive attributes. In the fair AI literature, these relationships can generally be categorized into three criteria: Independence, Separation, and Sufficiency, each intuitively defined based on different conditioning variables accordingly. Calculating fairness metrics for classification models is relatively straightforward using confusion matrices. However, it becomes more challenging for regression models due to the continuous nature of the dependent variable and the involvement of probability density function. Previous works on algorithmic fairness often simplify or use less-than-ideal versions of fairness criteria in regression settings. In this paper, we propose a novel approach to calculate Independence, Separation, and Sufficiency scores directly on density level for regression models. We achieve this by estimating the relevant conditional densities with Gaussian mixtures and directly applying them to group fairness approximation. This approach offers greater accuracy when dealing with continuous outputs compared to transformation methods. We validate our approach through empirical studies using both simulated and public datasets. Comparative performance analysis against the most recent existing method demonstrates the effectiveness of our algorithm.

Michaelis, Martin; Berthold, Philipp; Luettel, Thorsten; Maehlisch, Mirko

Multimodal Odometry Estimation With Automated Sensor Selection

Abstract

For autonomous driving applications, knowledge of the ego position, orientation, and velocity is a necessary prerequisite for recognizing landmarks and moving targets. We use radar sensors for the determination of these quantities in a radar odometry system. Radar odometry uses the advantage of a direct measurement of the radial speed using radar sensors. Radar sensors are less susceptible to bad weather and lighting conditions than camera and lidar sensors. In addition, radar data is not susceptible to wheel slippage or blocked wheels compared with wheel speed measurements. However, radar data is still susceptible to clutter. In order to achieve a combination of good precision under optimal conditions and good precision under adverse weather conditions, we fuse measurements from radar sensors, wheel speed sensors and the gyrometer. We do not simply combine these measurements according to assumed covariances. Instead, we check the plausibility of the measurements based on their likelihood. Subsequently, we weight the results of the sensor combinations accordingly. The decision about sensor weighting is carried out in a principled, probabilistic manner and adaptively with regard to environmental influences. We validate our approach using real data. Our approach is more precise under adverse conditions than using wheel speed sensors and gyrometers alone. On the other hand, it is more precise under good conditions than using only radar measurements.

Monteiro Junior, Almir Antônio; Brandão, Diego; da Rocha Henriques, Felipe; de Faria, Claudio M.; González, Pedro Henrique

Optimizing Wireless Sensor Network Planning: Integrating Biased Random-Key Genetic Algorithm and Local Branching for Scalable Solutions

Abstract

This study addresses the Wireless Sensor Network Planning Problem with Multiple Sources/Destinations (WSNP-MSD), an optimization challenge focused on reducing the sensor count within a network topology for a specified area, considering numerous sources and destinations. We introduce a hybrid strategy for tackling WSNP-MSD, particularly effective for large-scale scenarios, combining a Biased Random-key Genetic Algorithm with a Local Branching Technique. This methodology is justified by the limitations exact methods may encounter when the number of variables increases. Through computational experiments, we demonstrate the superiority of our proposed method over conventional exact methods in managing large instances of the WSNP-MSD.

Cozens, James M.; Godsill, Simon J.

Bimodal Multi-Object Localisation, Siteswap Inference, and Analysis for Competitive Juggling

Abstract

This paper presents an adaptive approach to real-time multi-object localisation in addition to siteswap inference, and performance evaluation metrics for juggling routines, employing a proposed bimodal machine learning-enhanced state-space model implementation. Considering the complex multi-modal characteristics exhibited by objects during performances, the paper introduces a bespoke Interacting Multiple Model (IMM) component for increased siteswap beat detection accuracy and gravitational acceleration inference, and a scheme for causal siteswap inference derived through machine learning-enhanced IMM model outputs. The algorithm effectively models the transitory behaviour of the system, enabling rapid and smooth transitions between the two discrete tracking cases (airborne, and caught) and accurate siteswap inference under a variety of camera and environmental conditions. The employment of beat tracking algorithms that exploit optimal compromises in time domain onset detection functions and Tempograms, enables effective error correction of siteswap detections, in addition to providing performance analysis and visualisation utilities. Experimentally, the algorithm is capable of object tracking and siteswap inference with up to 11 objects for a variety of challenging siteswaps and conditions, serving as a versatile performance analysis, evaluation, and visualisation utility.

Choi, Eunjee; Kim, Jong-Kook

TT-BLIP: Enhancing Fake News Detection Using BLIP and Tri-Transformer

Abstract

Detecting fake news has received a lot of atten- tion. Many previous methods concatenate independently encoded unimodal data, ignoring the benefits of integrated multimodal information. Also, the absence of specialized feature extraction for text and images further limits these methods. This paper introduces an end-to-end model called TT-BLIP that applies the bootstrapping language-image pretraining for unified vision- language understanding and generation (BLIP) for three types of information: BERT and BLIPTxt for text, ResNet and BLIPImg for images, and bidirectional BLIP encoders for multimodal information. The Multimodal Tri-Transformer fuses tri-modal features using three types of multi-head attention mechanisms, ensuring integrated modalities for enhanced representations and improved multimodal data analysis. The experiments are per- formed using two fake news datasets, Weibo and Gossipcop. The results indicate TT-BLIP outperforms the state-of-the-art models.

Kurz, Marcel; Hoffmann, Folker; Brandenburger, Andre; Charlish, Alexander

Investigating the effect of variable UAV altitude control on emitter localization

Abstract

This research paper investigates the anticipated improvement in emitter localization time simulating a UAV (unmanned aerial vehicle) sensor platform that allows for variable flight altitudes, contrary to maintaining a fixed flight altitude. The study aims to quantify efficiency gain and evaluates whether these gains justify the additional hardware and software complexities involved with variable flight control. The considered UAV sensor platform carries a radio-frequency (RF) direction-finder system. The sensor platform is maneuvered by a controller maximizing the Fisher information to minimize the required mission time until emitter localization. Additionally, a benchmark control strategy further introduced as Loitering is considered, which steers the platform in a circular maneuver around the emitter at constant radius. Simulations are conducted involving various parameters to thoroughly compare the altitude control modes and quantifying the improvements of enabling variable altitude control. The comparison reveals only minor improvements in the scenarios, which initialize the sensor platform at altitudes lower than 50 [m]. The effort required to fulfil the requirements for variable height control is discussed, with the conclusion that the effort does not outweigh the effect for UAVs with initial altitudes below 50 [m].

Wang, Lili; Legrand, Keith A.; Sundaram, Shreyas

Distributed Information Bayesian Recursive Update Filter

Abstract

This paper addresses consensus-based networked estimation of the state of a nonlinear dynamical system. This paper first presents an information form of the recently proposed Bayesian recursive update filter (BRUF), a Kalman filter that uses a recursive update to incorporate information from nonlinear measurement systems. Under the assumptions that the system is collectively observable and the network is strongly connected, a distributed information Bayesian recursive update filter (DIBRUF), a distributed form of the information Bayesian recursive update filter (IBRUF), is proposed, which exploits consensus on information vectors and matrices. Compared to the distributed extended Kalman filter (DEKF), the DIBRUF reduces the linearization error of the extended Kalman filter (EKF) by dividing the measurement update into N steps. Unlike the BRUF and IBRUF, which require local observability, the DIBRUF requires only the network to be collectively observable, as the sensors can share information among the network. Simulation experiments demonstrate the validity of the proposed approach.

Semeraro, Simone; Legrand, Keith A.

Gaussian Mixture Based Progressive Chernoff Fusion

Abstract

Probabilistic decentralized data fusion is the process of combining probabilistic beliefs from multiple sensors to reduce uncertainty and is broadly applicable to problems in aerospace, robotics, and wireless sensor networks. Fusing statistical information into a fused density is especially useful in distributed sensor networks, where each node or agent possesses only limited computation and sensing capabilities. The Chernoff fusion rule prevents information double-counting and produces a fused density that is equidistant from the input densities in an information-theoretic sense. This paper presents a novel approach to Gaussian mixture Chernoff fusion based on the progressive Bayes framework, where the optimal fused mixture is obtained through homotopy continuation. An advantage of the new approach is that it does not require fitting of Gaussian mixtures to lossy samples. A new and simple strategy for determining the optimal Chernoff weighting parameter is also presented and shown to outperform more complicated methods.

Svenson, Pontus; Holst, Anders; Wallberg, Anders; Nevalainen, Paavo; Farahnakian, Farshad; Álamo, Alfonso; Germinara, Vincenzo; Schweizer, Daniel; Leicht, Matthis; Anneken, Mathias; Hoppe, Adrian H.; Karalis, Aristeidis; Labib, Ashraf; Beltrán, Maria Eugenia; Hernández, Liss; Partanen, Petteri; Markkanen, Minna

AI-ARC Baltic Demo: Detecting Illegal Activities at Sea

Abstract

We describe the AI-ARC (Artificial Intelligence-based Virtual Control Room for the Arctic) system, which aims to enhance maritime domain awareness and surveillance. The system is micro-service based and fuses data from various sources, utilizing AI-driven micro-services and an advanced visualization platform to increase the situation awareness of maritime surveillance operators. The results of the Baltic sea demonstration, aiding in the detection of illegal activities, environmental protection, are presented. The system was evaluated using historical data from real criminal incidents. The results show that the AI-ARC approach could help increase the situation awareness of law enforcement operators.

Zheng, Litao; Cai, Yunze; Yang, Feng; Shi, Lihong

A Novel Distributed Bernoulli Filter with Adaptive Event-Triggered Communication

Abstract

This paper addresses communication bandwidth reduction and energy efficiency enhancement of a peer-to-peer sensor network for distributed target detection and tracking. A distributed Bernoulli filter with event-triggered communication is developed where each node broadcasts only local posteriors that achieve significant information gain. Specifically, for the cases where the Bernoulli density is no-target or single-target, the corresponding event-triggered strategies are constructed, respectively, in which the information discrepancy is measured via the Jeffreys divergence, and the triggering threshold is determined by the local information confidence coefficient. In addition, the presented method is combined with flooding protocol for internode communication, and weighted conservative fusion approaches are used to fuse the target existence probabilities and spatial distributions. Finally, simulation results demonstrate the effectiveness and superiority of the proposed approach.

Lee, Sung-Joo; Jung, Boyoung; Park, Seung-Jin; Ra, Won-Sang

Re-entry Target Identification with RCS Measurements Considering Multi-radar Geometry

Abstract

This paper addresses the problem of re-entry target identification using radar cross section (RCS) measurements obtained from distributed radars. Considering that the RCS pattern is subordinate to the target class and the aspect angle, the target identification problem is formulated as maximum a posteriori estimation associated with the multiple hypothesis of these two unknowns. Once each hypothesis is propagated through the approximate aspect angle dynamics, the corresponding hypothesis probability is evaluated by using the available RCS measurements and the pre-trained RCS distribution for the corresponding hypothesis. To improve the target identification performance, the RCS measurement likelihood is calculated by exploiting the geometric relation between the target and the multi-radar. Computer simulations are carried out to demonstrate the superiority and reliability of the proposed method over the existing machine-learning based algorithm.

Nagao, Hiromichi; Ito, Shin-Ichi; Matsumura, Mitsuru

Dominant Mode Extraction Based on the Four-Dimensional Variational Method

Abstract

Understanding the formation mechanism of magnetic domain patterns is important to improve the performance of magnetic materials. The magnetic domain patterns depend on the parameters of the time-dependent Ginzburg-Landau (TDGL) equation and the sweep rate of the external magnetic field. Although conventional analytical approaches can predict the patterns formed in the case of a high sweep rate, more versatile methods are required to understand the formation mechanism of complicated patterns for any temporal variation in the external field. This study proposes a method that extracts dominant modes from a posterior distribution based on the four-dimensional variational method (4DVar). The method decomposes the magnetic domain patterns into eigenvectors of the Hessian matrix of the cost function, defined as the difference between the observed and simulated magnetic domain patterns. The eigenvectors are extracted using a second-order adjoint method (SOA) and power iteration. The patterns are reconstructed by superimposing the extracted eigenvectors, and their time evolution can be obtained from the weights of the eigenvectors. Experiments demonstrate that the patterns are sufficiently reconstructed using a small number of the eigenvectors. This enables us to understand the pattern evolution as the change of the dominant eigenvectors.

Jung, Boyoung; Lee, Chan-Seok; Ra, Won-Sang

Ballistic Target Tracking Using Range Spread Measurements of a Wideband Radar Seeker

Abstract

This paper deals with the ballistic target tracking problem using range spread measurements. Based on the fact that the range spread measurement distribution can be successfully approximated by the Gaussian mixture model, the problem is formulated in the framework of Gaussian sum filtering. A probabilistic mode merging algorithm is developed to ensure both the computational efficiency and the suboptimal performance. The proposed filter also guarantees the operational reliability of the tracking algorithm, since it does not suffer from the degeneracy problem often encountered with the particle filter, which causes target tracking failure. The simulations for a typical ballistic target interception scenario demonstrate the effectiveness and the superior performance of the suggested method over the existing nonlinear filters.

Houssineau, Jeremie; Xue, Chenbao; Cai, Han; Uney, Murat; Delande, Emmanuel

Decentralised multi-sensor target tracking with limited field of view via possibility theory

Abstract

Quantifying negative information in an efficient way is a challenging task, especially when this information has to be communicated on a network. In this article we leverage the unique properties offered by possibility theory to quantify and approximate the negative information arising in the context of tracking a target with a sensor that has a limited field of view. We also verify experimentally that the corresponding target tracking methodology can be applied in a decentralised manner to a sensor network, while maintaining a performance close to the idealised case where the initial location of the target is better-known.

Donandt, Kathrin; Söffker, Dirk

Incorporating Navigation Context into Inland Vessel Trajectory Prediction: A Gaussian Mixture Model and Transformer Approach

Abstract

Abstract—Using data sources beyond the Automatic Identifi- cation System (AIS) to represent the context in which a vessel is navigating and consequently improve situation awareness is still rare in machine learning approaches to vessel trajectory prediction (VTP). In inland shipping, where vessel movement is constrained within fairways, supplementary navigational context information is indispensable. In this contribution targeting inland VTP, Gaussian Mixture Models are applied, on a fused dataset of AIS and discharge measurements, to generate multi-modal distribution curves, capturing typical lateral vessel positioning in the fairway and dislocation speeds along the waterway. By subsequently sampling the probability density curves of the GMMs, feature vectors are derived which are used, together with spatio-temporal vessel features and fairway geometries, as input to a VTP transformer model. The incorporation of these distribution features of both the current and forthcoming navigation context improves prediction accuracy. The superiority of the model over a previously proposed navigation context- sensitive transformer model for inland VTP is shown. The novelty lies in the provision of preprocessed, statistics-based features representing the conditioned spatial context, rather than relying on the model to extract relevant features for the VTP task from contextual data. Oversimplification of the complexity of inland navigation patterns by assuming a single typical route or selecting specific clusters prior to model application is avoided by giving the model access to the entire distribution information. The methodology’s generalizability is demonstrated through the usage of data of three distinct river sections. It can be integrated into an interaction-aware prediction framework, where insights into the positioning of the actual vessel behavior in the overall distribution at the current location and discharge can further enhance trajectory prediction accuracy.

Di, Kuangyu; Li, Tiancheng; Li, Guchong; Song, Yan; Dang, Xudong

Label Matching: It Is Complicated

Abstract

This paper addresses the intractable track matching problem involved in multi-sensor multi-target tracking using the labeled multi-Bernoulli filters. Unlike the unlabeled density defined in the common state space, the labeled multi-target density is defined in the joint state and label space, where the label contains time-series/history information of the underlying track. To measure the similarity between labeled densities (individual tracks) that is required for inter-sensor track matching and fusion, one has to account for the divergences in both state and label spaces. The challenge, however, arises from the lack of a proper metric to measure the label difference. It requires considering the entire trajectory of the track, encompassing the whole-life information from the birth of the track to the present. In this paper, we provide a solution of comparing and matching labels based on the whole-life time-series state distributions of the labels/tracks, by extending the common divergences like the Cauchy-Schwarz and Kullback-Leibler from distributions at a single time-instant to those over time-series. Representative scenarios are considered for illustration.

Jousselme, Anne-Laure; Pannetier, Benjamin

From tactical picture to situation assessment evaluation: A CUAS illustration

Abstract

Evaluating information fusion algorithms and systems is instrumental to the proper prediction of error, to the rational improvement of solutions and \textit{in fine} to the acceptance of solutions by end-users. Performance criteria define general semantics for a desirable behavior of the systems, while corresponding metrics implement that semantics for computable quality. Evaluation of the first levels of the JDL model of data fusion (detection and individual object assessment) is classically measured with objective and more or less standardized metrics. Evaluation of higher levels of processing such as situation assessment is less formalized as the evaluation criteria seat somewhere between the tactical picture quality and the decision-maker situation awareness. In this paper, we propose a formalization of the situation assessment problem, which bridges level 1 and 2 of the JDL model. Secondly, we define a global measure of quality encompassing the criteria of completeness, accuracy, clarity, which can be applied to both level 1 and level 2. We illustrate the metrics on a Counter-Unmanned-Aerial System (CUAS) scenario, comparing two uncertainty handling methods for a threat assessment solution through a Dynamic Bayesian Network. We finally conclude and sketch ideas for future steps of this research.

Jousselme, Anne-Laure; Costa, Paulo C.; Akli, Aurélie; Arcieri, Gianfranco

A model for an imperfect knowledge base for high-level information fusion experiments

Abstract

Evaluation of fusion algorithms constitutes a pivotal phase in the life-cycle of designing solutions for fusion problems. Benchmark datasets together with standardized evaluation criteria and metrics are thus needed to run proper experiments. When it comes to higher-level tasks such as situation assessment, datasets are conspicuously absent. Among the reasons are the lack of formal characterization of situation assessment, and the lack of structure to guide the collection of data at an appropriate level of semantics and granularity. In this paper, we propose a model for an imperfect knowledge base (IKB) to support high-level information fusion experiments. The model includes characterization of imperfect information from partially reliable sources. An Infon is an elementary piece of information which can bear on both concrete and abstract objects. The corresponding knowledge base encodes infons about physical objects of interest in a situation (\eg, a vessel), about sources providing infons, and about other infons. The imperfection dimensions of information are captured and connected to reliability dimensions of the source. An implementation within a graph database is proposed, exemplified with data of a maritime scenario. An example of comparison of artificial and human agents executing a fusion task is shown. Future work are finally briefly discussed.

Sel, Artun; Hayek, Samer; Kassas, Zaher M.

Robust Position Estimation using Range Measurements from Transmitters with Inaccurate Positions

Abstract

The problem of position estimation using range measurements from transmitters with inaccurately known po- sitions is considered. The true position of each transmitter is assumed to lie within a disk of a known radius, centered at the inaccurate position. A robust estimation framework is proposed, formulating a min-max optimization problem and presenting a tractable solution approach. A sensitivity map is constructed to quantify the positioning error in different regions due to inaccuracies in the transmitters’ positions. Numerical simulations are presented demonstrating the construction and application of the sensitivity map for estimating the position of a mobile receiver. It is shown that the sensitivity map yields invaluable insights to the expected positioning error in various regions within the environment. Experimental results are presented of a vehicle navigating in a real-world GPS-jammed environment using pseudorange measurements from 7 cellular transmitters whose positions are inaccurately known. The vehicle’s positioning error is justified utilizing the offline-generated sensitivity map.

Hoyt, Shaun J.; Blair, W. Dale; Lanterman, Aaron D.

Non-Linear Bias Mitigation in Multi-Sensor Multi-Track Fusion

Abstract

When performing track correlation and fusion in conjunction with bias estimation for sensor registration, the pattern match bias estimation is usually performed by modeling the biases as additive constants to the tracks in Cartesian space. Since sensor biases actually occur in sensor polar or spherical coordinates, the bias model of adding constants to the tracks can only be applied to a group of somewhat closely-spaced tracks before the linear assumption of the biases in Cartesian coordinates breaks down. A methodology to estimate sensor biases in the native coordinate frame in which they occur is presented, along with simulation results that illustrate its performance. Modeling the biases in sensor coordinates allow for tracks throughout the field of view to be used for sensor bias estimation, producing better sensor registration and track picture. In this research, sensor tracks are transmitted to a fusion center, where track correlation, bias estimation, and fusion are performed. Murty’s K-best hypotheses algorithm is utilized to generated the top K hypotheses for track-to-track correlation. Each hypothesis produces an estimate of the sensor biases. The correlation hypotheses are corrected for their sensor bias estimates and new correlation scores are computed, and the biascorrected correlation hypotheses are ranked to find the best. The best hypothesis is selected as the most recent system track picture. The system tracks produced by the best hypothesis are correlated against the previous system track picture to maintain system track continuity. The performance of the bias estimation is assessed against the root mean squared error and normalized estimation error squared errors of the estimated biases versus the true biases. A scenario with four tracks and two sensors is used to demonstrate the observability of these biases. The results show that the biases as applied to the remote sensor are observable and mitigated, allowing for a more accurate track picture.