Malware Detection using Statistical Analysis

of Byte-Level File Content

S. Momina Tabish, M. Zubair Shafiq, Muddassar Farooq

Next Generation Intelligent Networks Research Center (nexGIN RC)
National University of Computer & Emerging Sciences (FAST-NUCES)
Islamabad, 44000, Pakistan
{momina.tabish,zubair.shafiq,muddassar.farooq}@nexginrc.org

ABSTRACT 1. INTRODUCTION
Commercial anti-virus software are unable to provide pro- Sophisticated malware is becoming a major threat to the
tection against newly launched (a.k.a “zero-day”) malware. usability, security and privacy of computer systems and net-
In this paper, we propose a novel malware detection tech- works worldwide [1], [2]. A wide range of host-based so-
nique which is based on the analysis of byte-level file con- lutions have been proposed by researchers and a number
tent. The novelty of our approach, compared with existing of commercial anti-virus (AV) software are also available in
content based mining schemes, is that it does not memo- the market [5]–[21]. These techniques can broadly be clas-
rize specific byte-sequences or strings appearing in the ac- sified into two types: (1) static, and (2) dynamic. Static
tual file content. Our technique is non-signature based and techniques mostly operate on machine-level code and dis-
therefore has the potential to detect previously unknown and assembled instructions. In comparison, dynamic techniques
zero-day malware. We compute a wide range of statistical mostly monitor the behavior of a program with the help of
and information-theoretic features in a block-wise manner an API call sequence generated at run-time. The applica-
to quantify the byte-level file content. We leverage standard tion of dynamic techniques in AV products is of limited use
data mining algorithms to classify the file content of every because of the large processing overheads incurred during
block as normal or potentially malicious. Finally, we corre- run-time monitoring of API calls; as a result, the perfor-
late the block-wise classification results of a given file to cat- mance of computer systems significantly degrades. In com-
egorize it as benign or malware. Since the proposed scheme parison, the processing overhead is not a serious concern for
operates at the byte-level file content; therefore, it does not static techniques because the scanning activity can be sched-
require any a priori information about the filetype. We have uled offline in an idle time. Moreover, static techniques can
tested our proposed technique using a benign dataset com- also be deployed as an in-cloud network service that moves
prising of six different filetypes — DOC, EXE, JPG, MP3, PDF complexity from an end-point to the network cloud [28].
and ZIP and a malware dataset comprising of six different Almost all static malware detection techniques including
malware types — backdoor, trojan, virus, worm, construc- commercial AV software — either signature-, or heuristic-,
tor and miscellaneous. We also perform a comparison with or anomaly-based — use specific content signatures such as
existing data mining based malware detection techniques. byte sequences and strings. A major problem with the con-
The results of our experiments show that the proposed non- tent signatures is that they can easily be defeated by packing
signature based technique surpasses the existing techniques and basic code obfuscation techniques [3]. In fact, the ma-
and achieves more than 90% detection accuracy. jority of malware that appears today is a simple repacked
version of old malware [4]. As a result, it effectively evades
Categories and Subject Descriptors the content signatures of old malware stored in the database
of commercial AV products. To conclude, existing commer-
D.4.6 [Security and Protection]: Invasive Software cial AV products cannot even detect a simple repacked ver-
sion of previously detected malware.
General Terms The security community has expanded significant effort in
Experimentation, Security application of data mining techniques to discover patterns
in the malware content, which are not easily evaded by code
obfuscation techniques. Two most well-known data mining
Keywords based malware detection techniques are ‘strings’ (proposed
Computer Malware, Data Mining, Forensics by Schultz et al [7]) and ‘KM’ (proposed by Kolter et al [8]).
We take these techniques as a benchmark for comparative
study of our proposed scheme.
The novelty of our proposed technique — in contrast to
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are the existing data mining based technique — is its purely
not made or distributed for profit or commercial advantage and that copies non-signature paradigm: it does not remember exact file
bear this notice and the full citation on the first page. To copy otherwise, to content/contents for malware detection. It is a static mal-
republish, to post on servers or to redistribute to lists, requires prior specific ware detection technique which should be, intuitively speak-
permission and/or a fee. ing, robust to the most commonly used evasion techniques.
CSI-KDD’09, June 28, 2009, Paris, France.
The proposed technique computes a diverse set of statistical
Copyright 2009 ACM 978-1-60558-669-4 ...$5.00.
and information-theoretic features in a block-wise manner on a dataset that consists of 1, 001 benign and 3, 265 mali-
on the byte-level file content. The generated feature vec- cious executables. These executables have 206 benign and
tor of every block is then given as an input to standard 38 malicious samples in the portable executable (PE) file
data mining algorithms (J48 decision trees) which classify format. They have collected most of the benign executa-
the block as normal (n) or potentially malicious (pm). Fi- bles from Windows 98 systems. They use three different
nally, the classification results of all blocks are correlated approaches to statically extract features from executables.
to categorize the given file as benign (B) or malware (M). If The first approach extracts DLL information inside PE
a file is split into k equally-sized blocks (b1 , b2 , b3 , · · · , bk ) executables. Further, the DLL information is extracted us-
and n statistical features are computed for every k-th block ing three types of feature vectors: (1) the list of DLLs (30
(fk,1 , fk,2 , fk,3 , · · · , fk,n ), then mathematically our scheme boolean values), (2) the list of DLL function calls (2, 229
can be represented as: boolean values), and (3) the number of different function

0b 1 0 1 0 n/pm 1
calls within each DLL (30 integer values). RIPPER — an
inductive rule-learning algorithm — is used on top of every
f1,1 , f1,2 · · · f1,n
B CC BB C B n/pm C
1

B C B C
feature vector for classification. These schemes based on
b f2,1 , f2,2 · · · f2,n
B CA B@ C B
@ ... C
2 F D C DLL information provides an overall detection accuracy of
@ ... ⇒ ..
. A ⇒ A ⇒
B/M,
83.62%, 88.36% and 89.07% respectively. Enough details
bk fk,1 , fk,2 · · · fk,n n/pm about the DLLs are not provided, so we could not imple-
ment this scheme in our study.
where (F) is a suitable feature set, (D) is a data mining The second feature extraction approach extracts strings
algorithm for classification of individual blocks. The file from the executables using GNU strings program. Naı̈ve
is eventually categorized as benign (B) or malware (M) by Bayes classifier is used on top of extracted strings for mal-
the correlation module (C). Once a suitable feature set (F) ware detection. This scheme provides an overall detection
and a data mining algorithm (D) are selected, we test the accuracy of 97.11%. This scheme is reported to give the
accuracy of the solution using a benign dataset consisting of best results amongst all, and we have implemented it for
six filetypes: DOC, EXE, JPG, MP3, PDF and ZIP; and a malware our comparative study.
dataset comprising of six different malware types: backdoor, The third feature extraction approach uses byte sequences
trojan, virus, worm, constructor and miscellaneous. The (n-grams) using hexdump. The authors do not explicitly
results of our experiments show that our scheme is able to specify the value of n used in their study. However, from
provide more than 90% detection accuracy1 for detecting an example provided in the paper, we deduce it to be 2
malware, which is an encouraging outcome. To the best (bi-grams). The Multi-Naı̈ve Bayes algorithm is used for
of our knowledge, this is the first pure non-signature based classification. This algorithm uses voting by a collection of
data mining malware detection technique using statistical individual Naı̈ve Bayes instances. This scheme provides an
analysis of the static byte-level file contents. overall detection accuracy of 96.88%.
The rest of the paper is organized as follows. In Section The results of their experiments reveal that Naı̈ve Bayes
2 we provide a brief overview of related work in the domain algorithm with strings is the most effective approach for de-
of data mining malware detection techniques. We describe tecting the unseen malicious executables with reasonable
the detailed architecture of our malware detection technique processing overheads. The authors acknowledge the fact
in Section 3. We discuss the dataset in Section 4. We re- that the string features are not robust and can be easily
port the results of pilot studies in Section 5. In Section 6, defeated. Multi-Naı̈ve Bayes with byte sequences also pro-
we discuss the knowledge discovery process of the proposed vides a relatively high detection accuracy, however, it has
technique by visualizing the learning models of data mining large processing and memory requirements. Byte sequence
algorithms used for classification. Finally, we conclude the technique was later improved by Kolter et al and is explained
paper with an outlook to our future work. below.

2. RELATED WORK 2.2 Kolter et al — KM

In this section we explain the details of most relevant mal- In [8], Kolter et al use n-gram analysis and data min-
ware detection techniques: (1) ‘strings’ by Schultz et al [7], ing approaches to detect malicious executables in the wild.
(2) ‘KM’ by Kolter et al [8] and (3) ‘NG’ by Stolfo et al [16], They use n-gram analysis to extract features from 1, 971 be-
[17]. In our comparative study, we use ‘strings’ and ‘KM’ nign and 1, 651 malicious PE files. The PE files have been
as benchmarks for comparison, whereas ‘NG’ effectively uses collected from machines running Windows 2000 and XP op-
just one of the many statistical features used in our proposed erating systems. The malicious PE files are taken from an
technique. older version of the VX Heavens Virus Collection [27].
The authors evaluate their approach for two classification
2.1 Schultz et al — Strings problems: (1) classification between the benign and mali-
In [7], Schultz et al use several data mining techniques to cious executables, and (2) categorization of executables as
distinguish between the benign and malicious executables in a function of their payload. The authors have categorized
Windows or MS-DOS format. They have done experiments only three types — mailer, backdoor and virus — due to the
1
limited number of malware samples.
Throughout this text, the terms detection accuracy and Top n-grams with the highest information gain are taken
Area Under ROC Curve (AUC) are used interchangeably.
The AUC (0 ≤ AUC ≤ 1) is used as a yardstick to determine as binary features (T if present and F if absent) for every
the detection accuracy. Higher values of AUC mean high PE file. The authors have done pilot studies to determine
true positive (tp) rate and low false positive (fp) rate [30]. the size of n-grams, the size of words and the number of
At AUC = 1, tp rate = 1 and fp rate = 0. top n-grams to be selected as features. A smaller dataset
consisting of 561 benign and 476 malicious executables is of the system. In this study, we have set the block size to
considered in this study. They have used 4-grams, one byte 1024 bytes (= 1K). For instance, if the file size is 100K
word and top 500 n-grams as features. bytes then the file is split into 100 blocks. The frequency
Several inductive learning methods, namely instance-based histograms for 1-, 2-, 3-, and 4-gram of byte values are cal-
learner, Naı̈ve Bayes, support vector machines, decision trees culated for each block. These histograms are given as input
and boosted versions of instance-based learner, Naı̈ve Bayes, to the feature extraction module.
support vector machines and decision trees are used for clas-
sification. The same features are provided as input to all 3.2 Feature Extraction Module (F)
classifiers. They report the detecting accuracy as the area Feature extraction module computes a number of statis-
under an ROC curve (AUC) which is a more complete mea- tical and information-theoretic features on the histograms
sure compared with the detection accuracy [29]. AUCs show for each block generated by the previous module. Overall,
that the boosted decision trees outperform rest of the clas- the features’ set consist of 13 diverse features, which are
sifiers for both classification problems. separately computed on 1, 2, 3, and 4 gram frequency his-
tograms.2 This brings the total size of features’ set to 52
2.3 Stolfo et al — NG features per block. We now provide brief descriptions of
In a seminal work, Stolfo et al have used n-gram analysis every feature.
for filetype identification [15] and later for malware detection
[16], [17]. Their earlier work, called fileprint analysis, uses 3.2.1 Simpson’s Index
1-gram byte distribution of a whole file and compares it with The Simpson’s index [31] is a measure defined in an ecosys-
different filetype models to determine the filetype. In their tem, which quantifies the diversity of species in a habitat.
later work, they detect malware embedded in DOC and PDF It is calculated using the following equation:
files using three different models single centroid, multi cen- P n(n − 1)
troid, and exemplar of benign byte distribution of the whole Si = , (1)
files. A distance measure, called Mahanalobis Distance, is N (N − 1)
calculated between the n-gram distribution of these models
where n is the frequency of byte-values of consecutive n-
and a given test file. They also use 1-gram and 2-gram dis-
grams and N is the total number of bytes in a block i.e, 1000
tributions to test their approach on a dataset comprising of
in our case. A value of zero shows no significant difference
31 benign application executables, 331 benign executables
between frequency of n-grams in a block. Similarly, as the
from the System32 folder and 571 viruses. The experimen-
value of Si increases, the variance in frequency of n-grams
tal results have shown that their proposed technique is able
in a block also increases.
to detect a considerable proportion of the malicious files.
In all subsequent feature definitions, we represent with Xi
However, their proposed technique is specific to embedded
the frequency of j th n-gram in ith block, where j is varying
malware and does not deal with detection of stand-alone
from 0 − 255 for 1-gram, 0 − 65535 for 2-grams, 0 − 16777215
malware.
for 3-grams, and 0 − 4294967295 for 4-grams.

3. ARCHITECTURE OF PROPOSED TECH- 3.2.2 Canberra Distance

NIQUE This distance measures the sum of series of fractional dif-
ferences between coordinates of a pair of objects [31]. Math-
In this section, we explain our data mining based malware ematically we represent it as:
detection. It is important to emphasize that our technique
does not require any a priori information about the filetype X
n
| Xj − Xj+1 |
of a given file; as a result, our scheme is robust to subtle file CA(i) = (2)
j=0
| Xj | + | Xj+1 |
header obfuscations crafted by an attacker. Moreover, our
technique is also able to classify the malware as a function
of their payload, i.e. it can detect the family of a given 3.2.3 Minkowski Distance of Order
malware. This is a generalized metric to measure the differences
The architecture of proposed technique is shown in Figure between absolute magnitude of differences between a pair of
objects:
v
1. It consists of four modules: (1) block generator (B), (2)
feature extractor (F), (3) data miner (D), and (4) correla- u
uX | X
= t
n
tion (C). It can accept any type of file as an input. We now
mi λ
j − Xj+1 , |λ (3)
discuss the details of every module.
j=0

3.1 Block Generator Module (B) where we have used λ = 3 as suggested in [31].
The block generator module divides the byte-level con-
tents of a given file into fixed-sized chunks — known as 3.2.4 Manhattan Distance
blocks. We have used blocks for reducing the processing It is a special case of Minkowski distance [31] with λ = 1.
overhead of the module. In future, we want to analyze the
benefit of using variable-sized blocks as well. Remember
that using a suitable block size plays a critical role in defin-
Xn
mhi = | Xj − Xj+1 | (4)
ing the accuracy of our framework because it puts a lower j=0
limit on the minimum size of malware that our framework
can detect. We have to compromise a trade-off between the 2
In rest of the paper, we use the generic term n-grams once
amount of available information per block and the accuracy we want to refer to all 4-grams separately.
 Figure 1: Architecture of our proposed technique

3.2.5 Chebyshev Distance 3.2.8 Correlation Coefficient

Chebyshev distance measure is also called maximum value The standard angular separation between two vectors which
distance. It measures the absolute magnitude of differences is centered around the mean of the its magnitude values is
between coordinates of a pair of objects: called correlation coefficient [31]. This again measures the
similarity between two values:
chi = maxj | Xj − Xj+1 | (5)
P n
− X i ) · (Xj+1 − X i )
j=0 (Xj

It is a special case of Minkowski difference [31] with λ =

CCi =
(
P n
(Xj − X i ) ·
2 P n 2 1/2
(Xj+1 − X i ) )
, (8)
j=0 j=0
∞.
where X i is the mean of frequencies of n-grams in a given
3.2.6 Bray Curtis Distance block i.
It is a normalized distance measure which is defined as the
ratio of absolute difference of frequencies of n-grams and the 3.2.9 Entropy
sum of their frequencies [31]: Entropy measures the degree of dispersal or concentration
P of a distribution. In information-theoretic terms, entropy
of a probability distribution defines the minimum average
P
n
j=0 | Xj − Xj+1 |
bci = n (6) number of bits that a source requires to transmit symbols
j=0 (Xj + Xj+1 ) according to that distribution [31]. Let R be a discrete ran-
dom variable such that R = {ri , i ∈ ∆n }, where ∆n is the
3.2.7 Angular Separation image of a random variable. Then entropy of R is defined
This feature models the similarity of two vectors by taking
cosine of the angle between them [31]. A higher value of
as:
X
angular separation between two vectors shows that they are E(R) = − t(ri ) log2 t(ri ), (9)
similar. i∈∆n

P n
Xj · Xj+1
where t(ri ) is the frequency of n-grams in a given block.

ASi = P
( n
j=0

Xj2 ·
P n 2
Xj+1 )1/2
(7)
j=0 j=0
3.2.10 Kullback - Leibler Divergence
Table 1: Statistics of benign files used in this study
KL Divergence is a measure of the difference between two Filetype Qty. Avg. Size Min. Size Max. Size
probability distributions [31]. It is often referred to as a dis- (kilo-bytes) (kilo-bytes) (kilo-bytes)
tance measure between two distributions. Mathematically, DOC 300 1, 015.2 44 7, 706
EXE 300 4, 095.0 48 15, 005
it is represented as: JPG 300 1, 097.8 3 1, 629
X
n
Xj
MP3
PDF
300
300
3, 384.4
1, 513.1
654
25
6, 210
20, 188
KLi (Xj || Xj+1 ) = Xj log (10)
j=0
Xj+1 ZIP 300 1, 489.6 8 9, 860

3.2.11 Jensen-Shannon Divergence

It is a popular measure in probability theory and statistics 3.3.2 Boosting (AdaBoostM1)
and measures the similarity between two probability distri- We have used AdaBoostM1 algorithm implemented in WEKA
butions [31]. It is also known as Information Radius (IRad). [24]. As the name suggests, it is a meta algorithm which is
Mathematically, it is represented as: designed to improve the performance of base learning algo-
1 1 rithms. AdaBoostM1 keeps calling the weak classifier until
JSDi (Xj || Xj+1 ) = D(Xj || M ) + D(Xj+1 || M ) (11) a pre-defined number of times and tweaks those instances
2 2
that have resulted in misclassification. In this way, it keeps
where M = 12 (Xj + Xj+1 ). on adapting itself with the ongoing classification process. It
is known to be sensitive to outliers and noisy data.
3.2.12 Itakura-Saito Divergence
Itakura-Saito Divergence is a special form of Bregman dis- 3.4 Correlation Module (C)
tance [31]. The correlation module gets the per block classification
Xn
Xj Xj
results in the form of n or pm. It then calculates the corre-
lation among the blocks which are labeled as n or pm. De-
BF (Xj , Xj+1 ) = ( − log − 1) (12)
Xj+1 Xj+1 pending upon the fraction of n and pm blocks in a file, the
P log X .
j=0
file is classified as malicious or benign. We can also set a
which is generated by the convex function Fi (Xj ) = − j threshold for tweaking the final classification decision. For
instance if we set the threshold to 0.5, a file having 4 benign
3.2.13 Total Variation and 6 malicious blocks will be classified as malicious, and
It measures the largest possible difference between two vice-versa.
probability distributions Xj and Xj+1 [31]. It is defined as:
1 X 4. DATASET
δi (Xj , Xj+1 ) = | Xj − Xj+1 | (13)
2 In this section, we present an overview of the dataset used
j
in our study. We first describe the benign and then the
malware dataset used in our experiments. It is important
3.3 Data Mining based Classification Module to note that in this work we are not differentiating between
(D)
packed and non-packed files. Our scheme works regardless
The classification module gets as an input the feature vec- of the packed/non-packed nature of the file.
tor in the form of an arff file [26]. This feature file is then
further presented for classification to 6 sub-modules. The 4.1 Benign Dataset
six modules actually contain learnt models of six types of The benign dataset for our experiments consists of six dif-
malicious files: backdoor, virus, worm, trojan, constructor ferent filetypes: DOC, EXE, JPG, MP3, PDF and ZIP. These file-
and miscellaneous. The feature vector file is presented to all types encompass a broad spectrum of commonly used files
sub-modules in parallel and they produce an output of n or ranging from compressed to redundant and from executa-
pm per block. In addition to this, the output of the classi- bles to document files. Each set of benign files contains 300
fication sub-modules provide us insights into the payload of typical samples of the corresponding filetype, which provide
the malicious file. us with 1, 800 benign files in total. We have ensured the
Boosted decision tree is used for classifying each block as generality of the benign dataset by randomizing the sam-
n or pm. We have used AdaBoostM1 algorithm for boost- ple sources. More specifically, we queried well-known search
ing decision tree (J48) [24]. We have selected this classifier engines with random keywords to collect these files. In ad-
after extensive pilot studies which are detailed in Section dition, typical samples are also collected from the local net-
5. We provide brief explanations of decision tree (J48) and work of our virology lab.
boosting algorithm (AdaBoostM1) below. Some pertinent statistics of the benign dataset used in
this study are tabulated in Table 1. It can be observed
3.3.1 Decision Tree (J48) from Table 1 that the benign files have very diverse sizes
We have used C4.5 decision tree (J48) implemented in varying from 3 KB to 20 MB, with an average file size of
Waikato Environment for Knowledge Acquisition (WEKA) approximately 2 MB. The divergence in sizes of benign files
[26]. It uses the concept of information entropy to build the is important as malicious programs are inherently smaller in
tree. Every feature is used to split the dataset into smaller size for ease of propagation.
subsets and the normalized information gain is calculated.
The feature with highest information gain is selected for
decision making.
 Table 2: Statistics of malware used in this study Table 3: AUC’s for detecting malicious executables
vs benign files. Bold entries in every column repre-
Maj. Category Qty. Avg. Size Min. Size Max. Size
(kilo-bytes) (bytes) (kilo-bytes) sent the best results.
Backdoor 3,444 285.6 56 9, 502
Classifier Back Cons Misc Troj Virus Worm
Constructor 172 398.5 371 5, 971
Trojan 3114 135.7 12 4, 014 Strings
Virus 1048 50.7 175 1, 332 RIPPER 0.591 0.611 0.690 0.685 0.896 0.599
Worm 1471 72.3 44 2, 733 NB 0.615 0.610 0.714 0.751 0.944 0.557
Miscellaneous 1062 197.7 371 14, 692 M-NB 0.615 0.606 0.711 0.755 0.952 0.575
B-J48 0.625 0.652 0.765 0.762 0.946 0.642
KM
SMO 0.720 − 0.611 0.755 0.865 0.750
B-SMO 0.711 − 0.611 0.766 0.931 0.759
4.2 Malware Dataset NB 0.715 − 0.610 0.847 0.947 0.750
We have used ‘VX Heavens Virus Collection’ [27] data- B-NB 0.715 − 0.606 0.821 0.939 0.760
base which is available for free download in the public do- J48 0.712 − 0.560 0.805 0.850 0.817
B-J48 0.795 − 0.652 0.851 0.921 0.820
main. Malware samples, especially recent ones, are not eas- IBk 0.752 − 0.611 0.850 0.942 0.841
ily available on the Internet. Computer security corpora- Proposed solution with 4 features
tions do have an extensive malware collection, but unfortu- J48 0.835 0.812 0.863 0.837 0.909 0.880
nately they do not share their malware databases for re- B-J48 0.849 0.841 0.883 0.839 0.933 0.884
search purposes. This is a comprehensive database that NB 0.709 0.716 0.796 0.748 0.831 0.715
B-NB 0.709 0.722 0.794 0.746 0.807 0.707
contains a total of 37, 420 malware samples. The samples IBk 0.779 0.817 0.844 0.791 0.917 0.812
consist of backdoors, constructors, flooders, bots, nukers, B-IBk 0.782 0.817 0.844 0.791 0.918 0.812
sniffers, droppers, spyware, viruses, worms and trojans etc. Proposed solution with 52 features
We only consider Win32 based malware in PE file format. B-J48 0.979 0.965 0.950 0.985 0.970 0.932
The filtered dataset used in this study contains 10, 311 Win32
malware samples. To make our study more comprehensive,
we divide the malicious executables based on the function
of their payload. The malicious executables are divided into nuker based malware allow an attacker to launch malicious
six major categories such as backdoor, trojan, virus, worm, activities at a victim’s computer system that can possibly
constructor, and miscellaneous (malware like nuker, flooder, result in a denial of service attack. These activities can
virtool, hacktool etc). We now provide a brief explanation result in slow down, restart, crash or shutdown of a com-
of each of the six malware categories. puter system. Exploit and hacktool exploit vulnerabilities
in a system’s implementation which most commonly results
4.2.1 Backdoor in buffer overflows. Flooder initiates unwanted information
A backdoor is a program which allows bypassing of stan- floods such as email, instant messaging and SMS floods.
dard authentication methods of an operating system. As a The detailed statistics of the malware used in our study
result, remote access to computer systems is possible with- is provided in Table 2. The average malware size in this
out explicit consent of the users. Information logging and dataset is 64.2 KB. The sizes of malware samples used in
sniffing activities are possible using the gained remote ac- our study vary from 4 bytes to more than 14 MB. Intuitively
cess. speaking, small sized malware are harder to detect than the
larger ones.
4.2.2 Constructor
This category of malware mostly includes toolkits for au-
tomatically creating new malware by varying a given set of
5. PILOT STUDIES
input parameters. In our initial set of experiments, we have conducted an ex-
tensive search in the design space to select the best features’
4.2.3 Worm set and classifier for our scheme. The experiments are done
The malware in this category spreads over the network by on 5% of total dataset to have small design cycle for our
replicating itself. approach. Recall that in Section 3, we have introduced 13
different statistical and information-theoretic features. The
4.2.4 Trojan pilot studies are aimed to convince ourselves that we need
A trojan is a broad term that refers to stand alone pro- all of them. Moreover, we have also evaluated well-known
grams which appear to perform a legitimate function but data mining algorithms on our dataset in order to find the
covertly do possibly harmful activities such as providing re- best classification algorithm for our problem. We have used
mote access, data destruction and corruption. Instance based (IBk) [22], Decision Trees (J48) [25], Naı̈ve
Bayes [23] and the boosted versions of these classifiers in
4.2.5 Virus our pilot study. For boosting we have used AdaBoostM1
A virus is a program that can replicate itself and attach algorithm [24]. We have utilized implementations of these
itself with other benign programs. It is probably the most algorithms available in WEKA [26].
well-known type of malware and has different flavors.
5.1 Discussion on Pilot Studies
4.2.6 Miscellaneous The classification results of our experiments are tabulated
The malware in this category include DoS (denial of ser- in Table 3 which show that the boosted Decision Tree (J48)
vice), nuker, exploit, hacktool and flooders. The DoS and significantly outperforms other classifiers in terms of detec-
Table 4: Feature Analysis. AUC’s for detecting 1

virus executables vs benign files using boosted J48.

F1,F2, F3 and F4 correspond to 4 different features. 0.8
Feature No. of Gram/feature AUC
F1 1 0.823
F2 1 0.839 0.6

tp rate
F3 1 0.866
F4 2 0.891
F1-F2 1, 1 0.940 0.4 Boosted IBK
F1-F3 1, 1 0.928 IBK
F1-F4 1, 2 0.932 Boosted NB
0.2 NB
F2-F4 1, 2 0.929
Boosted J48
F1-F2-F4 1, 1, 2 0.962
J48
F3-F2 1, 1 0.954 0
F3-F4 1, 2 0.913 0 0.2 0.4 0.6 0.8 1
fp rate
F1-F2-F3-F4 1, 1, 1, 2 0.956

Figure 2: ROC plot for virus-benign classification

tion accuracy. Similarly we also evaluate the role of num-

ber of features and the number of n-grams on the accuracy
files each from malware and benign datasets respectively.
of proposed approach. In Table 3, we first use four fea-
We create six separate training samples for each malware
tures namely: Simpson’s index (F1), Entropy Rate (F2),
category. We use these samples to train boosted decision
Canberra distance (F3) on 1-gram and Simpson’s index on
tree and consequently get 6 classification models for each
2-grams (F4). Our scheme achieves a significantly higher de-
malware type.
tection accuracy compared with strings and KM. We then
For an easier understanding of the classification process,
tried different combinations of features and n-grams and tab-
we take backdoor as an example. We have selected 50 back-
ulated the results in Table 4. It is obvious from Table 4 that
door files and 50 benign files to get a training sample for clas-
once we move from a single feature from (F1) on 1-gram to
sifying backdoor and benign files. We train the data mining
(F4) on 2-grams the detection accuracy improves from 0.82
algorithm with this sample and as a result get the training
to 0.891. Once we use combination of features computed on
model to classify a backdoor. Moreover, we further test,
1-gram and 2-grams the accuracy approaches to 0.962 (see
using this model, all six benign filetypes and backdoor mal-
F1-F2-F4). This provided us the motivation to use all 13
ware files. It is important to note that this model is specifi-
features on 1-, 2-, 3- and 4-grams resulting in a total of 52
cally designed to distinguish backdoor files from six benign
features. The ROC curve for the virus-benign classification
filetypes (one-vs-all classification), where only 50 backdoor
using F1, F2, F3 and F4 features is shown in Figure 2.
samples are taken in training. This number is considerably
It is clear in Table 3 that strings approach has a signifi-
small keeping in view the problem at hand. Nonetheless, the
cantly higher accuracy for virus types compared with other
backdoor classification using a single classifier completes in
malware types. We analyzed the reason behind this by look-
seven runs: six for benign filetypes and one for itself. In
ing at the signatures used by this method. We observed that
a similar fashion, one can classify malware types i.e., virus,
typically viruses carry some common strings like “Chinese
worm, trojan, constructor and miscellaneous. Once our so-
Hacker” and for strings approach they become its signatures.
lution is trained for each category of malware, we test it
Since similar strings do not appear in other malware types;
on the benign dataset of 1800 files and the malware dataset
therefore, strings accuracy degrades to as low as 0.62 in case
of 10, 311 files from VX Heavens. We ensure that the be-
of backdoors.
nign and malware files used in the training phase are not
Recall that KM uses 4-grams as binary features. KM
included in the testing phase to verify our claim of zero-day
follows the same pattern of higher accuracy for detecting
malware detection. The classification results are shown in
viruses and relatively lower accuracy for other malware types.
Figure 3 in the form of AUCs. It is interesting to see that
However, its accuracy results are significantly higher com-
viruses, as expected, are easily classified by our scheme. In
pared with the strings approach. Note that our proposed so-
comparison trojans are programs that look and behave like
lution with 52 features not only provides the best detection
benign programs, but perform some illegitimate activities.
accuracy for virus category but its accuracy remains consis-
As expected, the classification accuracy for trojans is 0.88
tent across all malware types. This shows the strength of
— significantly smaller compared with viruses.
using diverse features’ set with a boosted J48 classifier.
In order to get a better understanding, we have plotted the
ROC graphs of classification results for each malware type in
6. RESULTS & DISCUSSION Figure 3. The classification results show that malware files
Remember that our approach is designed to achieve a chal- are quite distinct from benign files in terms of byte-level
lenging objective: to distinguish between malicious files of file content. The ROC plot further confirms that virus are
type backdoor, virus, worm, trojan, constructor from be- easily classified compared with other malware types while
nign files of types DOC, EXE, JPG, MP3, PDF and ZIP just on trojan and backdoors are relatively difficult to classify. Ta-
the basis of byte-level information. ble 5 shows portions of the developed decision trees for all
We now explain our final experimental setup. The pro- malware categories. As decision trees provide a simple and
posed scheme, as explained in Section 3.2, computes features robust method of classification for a large dataset, it is in-
on n-grams of each block of a given file. We create the train- teresting to note that malicious files are inherently different
ing samples by randomly selecting 50 malicious and benign from the benign ones even at the byte-level.
 Table 5: Portions of developed decision trees 1
| | | | KL1 > 0.022975
| | | | | Entropy2 <= 6.822176
| | | | | | Manhattan1 <= 0.005411 0.8
| | | | | | | Manhattan4 <= 0.000062: Malicious (20.0)
| | | | | | | Manhattan4 > 0.000062: Benign (8.0/2.0)
| | | | | | Manhattan1 > 0.005411: Malicious (538.0/6.0) 0.6

tp rate
(a) between Backdoor and Benign files
0.4 Virus (AUC = 0.945)
| | CorelationCoefficient1 > 0.619523 Worm (AUC = 0.919)
| | | Chebyshev4 <=1.405 Trojan (AUC = 0.881)
| | | | Itakura2 <= 87.231983: Malicious (352.0/9.0) 0.2 Backdoor (AUC = 0.849)
| | | | Itakura2 > 87.231983 Constructor (AUC = 0.925)
| | | | | TotalVariation1 <= 0.3415: Malicious (11.0) Miscellaneous (AUC = 0.903)
| | | | | TotalVariation1 > 0.3415: Benign (8.0) 0
0 0.2 0.4 0.6 0.8 1
fp rate
(b) between Trojan and Benign files

| | CorelationCoefficient4 > 0.187794 Figure 3: ROC plot for detecting malware from be-
| | | Simpson_Index_1 <= 0.005703
| | | | CorelationCoefficient3 <= 0.17584: Malicious (32.0)
nign files
| | | | CorelationCoefficient3 > 0.17584
| | | | | Entropy1 <= 4.969689: Malicious (4.0/1.0)
| | | | | Entropy1 > 4.969689: Benign (5.0)
| | | Simpson_Index_1 > 0.005703: Benign (11.0) 7. CONCLUSION & FUTURE WORK
(c) between Virus and Benign files In this paper we have proposed a non-signature based
technique which analyzes the byte-level file content. We
| | | Entropy2 > 3.00231 argue that such a technique provides implicit robustness
| | | | Canberra2 <= 49.348481
| | | | | Canberra1 <= 14.567909: Malicious (161.0) against common obfuscation techniques — especially repacked
| | | | | Canberra1 > 14.567909 malware to obfuscate signatures. An outcome of our re-
| | | | | | Itakura3 <= 126.178699: Malicious (5.0) search is that malicious and benign files are inherently dif-
| | | | | | Itakura3 > 126.178699: Benign (5.0)
| | | | Canberra2 > 49.348481: Benign (13.0/1.0) ferent even at the byte-level.
The proposed scheme uses a rich features’ set of 13 differ-
(d) between Worm and Benign files ent statistical and information-theoretic features computed
CorelationCoefficient3 <= -0.013832 on 1-, 2-, 3- and 4-grams of each block of a file. Once we
| Itakura2 <= 5.905754 have calculated our features’ set, we give it as an input
| | Itakura3 <= 5.208592 to the boosted decision tree (J48) classifier. The choice of
| | | CorelationCoefficient4 <= -0.155078: Malicious (7.0)
| | | CorelationCoefficient4 > -0.155078: Benign (43.0/6.0) features’ set and classifier is an outcome of extensive pilot
| | Itakura3 > 5.208592: Benign (303.0/5.0) studies done to explore the design space. The pilot stud-
ies demonstrate the benefit of our approach compared with
(e) between Constructor and Benign files
other well-known data mining techniques: strings and KM
| Entropy4 > 6.754364 approach. We have tested our solution on an extensive ex-
| | KL1 <= 0.698772
| | | Manhattan3 <= 0.001003
ecutable dataset. The results of our experiments show that
| | | | Entropy2 <= 6.411063: Malicious (29.0) our technique achieves 90% detection accuracy for different
| | | | Entropy2 > 6.411063 malware types. Another important feature of our framework
| | | | | KL1 <= 0.333918: Malicious (2.0) is that it can also classify the family of a given malware file
| | | | | KL1 > 0.333918: Benign (2.0)
| | | Manhattan3 > 0.001003: Benign (3.0) i.e. virus, trojan etc.
In future, we would like to evaluate our scheme on a larger
(f ) between Miscellaneous and Benign files dataset of benign and malicious executables and reverse en-
gineer the features’ set for further improving the detection
accuracy. Moreover, we plan to evaluate the robustness of
The novelty of our scheme lies in the way the selected
our proposed technique on a customized dataset containing
features have been computed — per block n-gram analysis,
manually packed executable files.
and the correlation between the blocks classified as benign
or potentially malicious. The features used in our study are
taken from statistics and information theory. Many of these
features have already been used by researchers in other fields Acknowledgments
for similar classification problems. The chosen set of features This work is supported by the National ICT R&D Fund,
is not, by any means, the optimal collection. The selection of Ministry of Information Technology, Government of Pak-
optimal number of features remains an interesting problem istan. The information, data, comments, and views detailed
which we plan to explore in our future work. Moreover, the herein may not necessarily reflect the endorsements of views
executable dataset used in our study contained both packed of the National ICT R&D Fund.
and non-packed PE files. We plan to evaluate the robustness We acknowledge M.A. Maloof and J.Z. Kolter for their
of our proposed technique on manually crafted packed file valuable feedback regarding the implementation of strings
dataset. and KM approaches. Their comments were of great help in
establishing the experimental testbed used in our study. We
also acknowledge the anonymous reviewers for their valuable
suggestions pertaining to possible extensions of our study.
8. REFERENCES [16] S.J. Stolfo, K. Wang, W.J. Li, “Towards Stealthy
[1] Symantec Internet Security Threat Reports I-XI (Jan Malware Detection”, Advances in Information Security,
2002—Jan 2008). Vol. 27, pp. 231-249, Springer, USA, 2007.
[2] F-Secure Corporation, “F-Secure Reports Amount of [17] W.J. Li, S.J. Stolfo, A. Stavrou, E. Androulaki, A.D.
Malware Grew by 100% during 2007”, Press release, Keromytis, “A Study of Malcode-Bearing Documents”,
2007. International Conference on Detection of Intrusions &
[3] A. Stepan, “Improving Proactive Detection of Packed Malware, and Vulnerability Assessment (DIMVA), pp.
Malware”, Virus Buletin, March 2006, available at 231-250, Springer, Switzerland, 2007.
http://www.virusbtn.com/virusbulletin/ [18] M.Z. Shafiq, S.A. Khayam, M. Farooq, “Embedded
archive/2006/03/vb200603-packed.dkb Malware Detection using Markov n-Grams”,
[4] R. Perdisci, A. Lanzi, W. Lee, “Classification of Packed International Conference on Detection of Intrusions &
Executables for Accurate Computer Virus Detection”, Malware, and Vulnerability Assessment (DIMVA), pp.
Pattern Recognition Letters, 29(14), pp. 1941-1946, 88-107, Springer, France, 2008.
Elsevier, 2008. [19] M. Christodorescu, S. Jha, and C. Kruegal, “Mining
[5] AVG Free Antivirus, available at Specifications of Malicious Behavior”, European
http://free.avg.com/. Software Engineering Conference and the ACM
SIGSOFT Symposium on the Foundations of Software
[6] Panda Antivirus, available at
Engineering (ESEC/FSE 2007), pp. 5-14, Croatia, 2007.
http://www.pandasecurity.com/.
[20] Frans Veldman, “Heuristic Anti-Virus Technology”,
[7] M.G. Schultz, E. Eskin, E. Zadok, S.J. Stolfo, “Data
International Virus Bulletin Conference, pp. 67-76,
mining methods for detection of new malicious
USA, 1993, available at http://mirror.sweon.net/
executables”, IEEE Symposium on Security and
madchat/vxdevl/vdat/epheurs1.htm.
Privacy, pp. 38-49, USA, IEEe Press, 2001.
[21] Jay Munro, “Antivirus Research and Detection
[8] J.Z. Kolter, M.A. Maloof, “Learning to detect malicious
Techniques”, Antivirus Research and Detection
executables in the wild”, ACM SIGKDD International
Techniques, ExtremeTech, 2002, available at
Conference on Knowledge Discovery and Data Mining,
http://www.extremetech.com/article2/0,2845,
pp. 470-478, USA, 2004.
367051,00.asp.
[9] J. Kephart, G. Sorkin, W. Arnold, D. Chess, G.
[22] D.W. Aha, D. Kibler, M.K. Albert, “Instance-based
Tesauro, S. White, “Biologically inspired defenses
learning algorithms”, Journal of Machine Learning, Vol.
against computer viruses”, International Joint
6, pp. 37-66, 1991.
Conference on Artificial Intelligence (IJCAI), pp.
985-996, USA, 1995. [23] M.E. Maron, J.L. Kuhns, “On relevance, probabilistic
indexing and information retrieval”, Journal of the
[10] R.W. Lo, K.N. Levitt, R.A. Olsson, “MCF: A
Association of Computing Machinery, 7(3), pp.216-244,
malicious code filter”, Computers & Security,
1960.
14(6):541-566, Elseveir, 1995.
[24] Y. Freund, R. E. Schapire, “A decision-theoretic
[11] O. Henchiri, N. Japkowicz, “A Feature Selection and
generalization of on-line learning and an application to
Evaluation Scheme for Computer Virus Detection”,
boosting”, Journal of Computer and System Sciences,
IEEE International Conference on Data Mining
No. 55, pp. 23-37, 1997
(ICDM), pp. 891-895, USA, IEEE Press, 2006.
[25] J.R. Quinlan, “C4.5: Programs for machine learning”,
[12] P. Kierski, M. Okoniewski, P. Gawrysiak, “Automatic
Morgan Kaufmann, USA, 1993.
Classification of Executable Code for Computer Virus
Detection”, International Conference on Intelligent [26] I.H. Witten, E. Frank, “Data mining: Practical
Information Systems, pp. 277-284, Springer, Poland, machine learning tools and techniques”, Morgan
2003. Kaufmann, 2nd edition, USA, 2005.
[13] T. Abou-Assaleh, N. Cercone, V. Keselj, R. Sweidan. [27] VX Heavens Virus Collection, VX Heavens website,
“Detection of New Malicious Code Using N-grams available at http://vx.netlux.org
Signatures”, International Conference on Intelligent [28] J. Oberheide, E. Cooke, F. Jahanian. “CloudAV:
Information Systems, pp. 193-196, Springer, Poland, N-Version Antivirus in the Network Cloud”, USENIX
2003. Security Symposium, pp. 91-106, USA, 2008.
[14] J.H. Wang, P.S. Deng, “Virus Detection using Data [29] T. Fawcett, “ROC Graphs: Notes and Practical
Mining Techniques”, IEEE International Carnahan Considerations for Researchers”, TR HPL-2003-4, HP
Conference on Security Technology, pp. 71-76, IEEE Labs, USA, 2004.
Press, 2003. [30] S.D. Walter, “The partial area under the summary
[15] W.J. Li, K. Wang, S.J. Stolfo, B. Herzog, “Fileprints: ROC curve”, Statistics in Medicine, 24(13), pp.
identifying filetypes by n-gram analysis”, IEEE 2025-2040, 2005.
Information Assurance Workshop, USA, IEEE Press, [31] T.M. Cover, J.A. Thomas, “Elements of Information
2005. Theory”, Wiley-Interscience, 1991.