Generalized anomaly detection model for windows-based malicious program behavior

Document Type

Article

Publication Date

12-1-2008

Abstract

In this paper we demonstrate that it is possible in general to detect Windows-based malicious program behavior. Since S. Forrest et al. used the N-grams method to classify system call trace data, dynamic learning has become a promising research area. However, most research works have been done in the UNIX environment and have limited scope. In Forrest's original model, "Self" is defined based on a normal process whereas "Non-Self" corresponds to one or two malicious processes. We extend this technique into the Windows environment. In our model, "Self" is defined to represent the general pattern of hundreds ofWindows program behaviors; "Non-Self" is defined to represent all program behaviors that fall out of norm. Because of the difficulty in collecting program behavior, insufficient research results are available. We collected around 1000 system call traces of various normal and malicious programs in the Windows OS. A normal profile was built using a Hidden Markov Model (HMM). The evaluation was based on the entire trace. Our classification results are promising.

Identifier

84872866358 (Scopus)

Publication Title

International Journal of Network Security

e-ISSN

18163548

ISSN

1816353X

First Page

428

Last Page

435

Issue

3

Volume

7

This document is currently not available here.

Share

COinS