Tracking probabilistic correlation of monitoring data for fault detection in complex systems

Document Type

Conference Proceeding

Publication Date

12-22-2006

Abstract

Due to their growing complexity, it becomes extremely difficult to detect and isolate faults in complex systems. While large amount of monitoring data can be collected from such systems for fault analysis, one challenge is how to correlate the data effectively across distributed systems and observation time. Much of the internal monitoring data reacts to the volume of user requests accordingly when user requests flow through distributed systems. In this paper, we use Gaussian mixture models to characterize probabilistic correlation between flow-intensities measured at multiple points. A novel algorithm derived from Expectation-Maximization (EM) algorithm is proposed to learn the "likely" boundary of normal data relationship, which is further used as an oracle in anomaly detection. Our recursive algorithm can adoptively estimate the boundary of dynamic data relationship and detect faults in real time. Our approach is tested in a real system with injected faults and the results demonstrate its feasibility. © 2006 IEEE.

Identifier

33845564157 (Scopus)

ISBN

[0769526071, 9780769526072]

Publication Title

Proceedings of the International Conference on Dependable Systems and Networks

External Full Text Location

https://doi.org/10.1109/DSN.2006.70

First Page

259

Last Page

268

Volume

2006

This document is currently not available here.

Share

COinS