Kalman: Another Anomaly Detection System

My previous post dealt with ASTUTE, a traffic anomaly detection system that uses the queue volume equilibrium property to detect anomalies of a distributed nature. This post deals with a different anomaly detection system known as KALMAN. The developers of this approach presented their system in the 2005 IMC. It makes use of the Kalman filter to extract normal traffic traces from a network, leaving the rest (known as the residual) for anomaly analysis.

The Kalman filter is a technique that allows for predicting future values based on past measurements that may be mingled with noise, or random variations in measurement. This filter is useful for other things, such as predicting the position of a target, given previous measurements that may be slightly inaccurate.

In the anomaly detection system KALMAN, the Kalman filter compares predictions of traffic state with the actual traffic state (with the more recent measurement data). Then, ROC curves are used to decide the accuracy of the prediction. In short, there is a traffic matrix (showing traffic at each intermediate node) and a path matrix from which origin-destination flows can be constructed. The system identifies the individual flows that are anomalous so that they can be analyzed for what type of anomaly it has.