NJIT eTD: The New Jersey Institute of Technology's electronic Theses & Dissertations
Classifying malicious windows executables using anomaly based detection
Sutaria, Ronak
Document Type:
Department of Computer Science
Master of Science
Computer Science
Advisory Committee:
Manikopoulos, Constantine N.
Statica, Robert
Hu, Jie
Borcea, Cristian
Thesis Date:
2006, January
Malicious executables
Malware detection

A malicious executable is broadly defined as any program or piece of code designed to cause damage to a system or the information it contains, or to prevent the system from being used in a normal manner. A generic term used to describe any kind of malicious software is Maiware, which includes Viruses, Worms, Trojans, Backdoors, Root-kits, Spyware and Exploits. Anomaly detection is technique which builds a statistical profile of the normal and malicious data and classifies unseen data based on these two profiles.

A detection system is presented here which is anomaly based and focuses on the Windows® platform. Several file infection techniques were studied to understand what particular features in the executable binary are more susceptible to being used for the malicious code propagation. A framework is presented for collecting data for both static (non-execution based) as well as dynamic (execution based) analysis of the malicious executables. Two specific features are extracted using static analysis, Windows API (from the Import Address Table of the Portable Executable Header) and the hex byte frequency count (collected using Hexdump utility) which have been explained in detail. Dynamic analysis features which were extracted are briefly mentioned and the major challenges faced using this data is explained. Classification results using Support Vector Machines for anomaly detection is shown for the two static analysis features. Experimental results have provided classification results with up to 94% accuracy for new, previously unseen executables.

Complete Thesis:
njit-etd2006-016 (57 pages ~ 4,032 KB pdf)
Please complete this Feedback Form to inform us about your experience using this website. It will assist us in better serving your information needs in the future. Thank You!
Created September 8, 2008
To view these documents you will need the Acrobat Reader Plug-in. If you do not have it you can download it free from