Low Latency Anomaly Detection with Imperfect Models

Abstract

The problem of anomaly detection deals with detecting abrupt changes/anomalies in the distribution of sequentially observed data in a stochastic system. This problem applies to many applications, such as signal processing, intrusion detection, quality control, medical diagnosis, etc. A low latency anomaly detection algorithm, which is based on the framework of quickest change detection (QCD), aims at minimizing the detection delay of anomalies in the sequentially observed data while ensuring satisfactory detection accuracy. Moreover, in many practical applications, complete knowledge of the post-change distribution model might not be available due to the unexpected nature of the change. Hence, the objective of this dissertation is to study low latency anomaly detection or QCD algorithms for systems with imperfect models such that any type of abnormality in the system can be detected as quickly as possible for reliable and secured system operations. This dissertation includes the theoretical foundations behind these low latency anomaly detection algorithms along with real-world applications. First, QCD algorithms are designed for detecting changes in systems with multiple post-change models under both Bayesian and non-Bayesian settings. Next, a QCD algorithm is studied for real-time detection of false data injection attacks in smart grids with dynamic models. Finally, a QCD algorithm for detecting wind turbine bearing faults is developed by analyzing the statistical behaviors of stator currents generated by the turbines. For all the proposed algorithms, analytical bounds of the system performance metrics are derived using asymptotic analysis and the simulation results show that the proposed algorithms outperform existing algorithms