Malicious Traffic Detection in Local Networks With Snort: Lo Ic Etienne / EPFL - SSC
Malicious Traffic Detection in Local Networks With Snort: Lo Ic Etienne / EPFL - SSC
Malicious Traffic Detection in Local Networks With Snort: Lo Ic Etienne / EPFL - SSC
This paper will explore how Snort, an open source Net- #Protocol % of traffic
TCP 93 %
work Intrusion Detection System, can be used to secure UDP 6%
and monitor such a very large corporate network. Other 1%
SSH 26 %
This thesis will start by a brief tour of horizon of net- HTTP 12 %
work security in Chapters 3 and 4; Chapters 5 and 6 will SSL 2%
X11 1%
present Snort’s functionalities and rule system. It will SMTP <1%
show that Snort’s default classification is inadequate for Other 59 %
large network rule selection.
Chapter 7 will present findings concerning commonly Tab. 1: Protocol breakdown of traffic sample
available rules, starting with rules related to CERN poli-
cies. A solution to improve Emule detection when Skype
is present is proposed. Rules that significantly improve The IDS typically sees 600Mbits/s on average, and
the detection of infected and compromized devices at more than 1Gbit/s during peak hours.
CERN are also proposed. With such a broad variety of devices, origins, and
Chapter 8 will introduce a tool to handle rule up- amount of traffic, this network offers a perfect frame-
dates, and Chapter 9 will suggest possible performance work for IDS tuning. Clearly, naive approaches such as
improvements to the Snort platform. blocking all non-http traffic would be unacceptable in
this context.
2 About CERN
3 Network Traffic Analysis as Part of an
2.1 CERN IDS System
CERN, the European Organization for Nuclear Research, 3.1 IDS
is the largest particle physics laboratory in the world.
Commonly referred to as the birthplace of the world- An Intrusion Detection System (or IDS) is composed of
wide web, it currently hosts scientists from some 580 in- software and/or hardware designed to detect unwanted
stitutes and counts 20 European member states. attempts of accessing, manipulating, and/or disabling of
CERN’s missions are Research, Technology, Collabo- computer systems. An IDS is used to detect several types
ration, and Education [1]. of malicious behaviors that can compromize the security
and trust of a computers system. These threats are vari-
ous, and include network attacks against vulnerable ser-
2.2 CERN Public Network vices, data driven attacks on applications, host based at-
The CERN Public Network offers a great playground for tacks such as privilege escalation, unauthorized accesses,
any IDS system. or malware (viruses, worms) [2].
CERN hosts more than 10’000 visitors each year, most
of whom bring their own, unmanaged and/or unpatched,
1
4 State of the Art - From Raw Packet Capture to Advanced Detection Mechanisms 2
Structure of an IDS An IDS is typically composed of NetFlow is an embedded instrumentation within Cisco
three main parts: a sensor, an engine, and a console. IOS Software to characterize network operation. It gives
The sensor’s main task is to analyze all the data ac- the administrators the tools to understand who, what,
cording to some configuration data, and create events when, where, and how network traffic is flowing.
accordingly. The engine records events logged by the Data is collected directly by switches, and can be ex-
sensors in a database, and uses a system of rules to gen- ported to a reporting server. A flow is a n-tuple1 , usually
erate alerts from security events received. The console identified by the source and destination IPs and ports,
monitors events and alerts, and allows to interact with the protocol, and the unilateral number of bytes trans-
the latter. ferred.
In many cases, the three components are combined in a This data can then be aggregated and analyzed to de-
simple device or appliance. A more detailed explanation tect unwanted or malicious behavior. One could, for
of the components can be found in [3]. example, count the number of SMTP servers (destina-
tion port 25), the number of peers, or the number of
SSH servers contacted by any host to detect misbehav-
Different Types of IDS There are two main types of
ing hosts (sending spam mail, using P2P, or doing ssh
IDS working at different points in the infrastructure:
scans).
Network IDS (NIDS): The NIDS scans all network traf- A worm detection system based on netflow data is pro-
fic that is fed to it. The NIDS typically functions in posed in [5].
the same way as an antivirus software: every single
packet is scanned for patterns which may indicate a High Level Analysis With this technique, the payload
problem. of every single packet is inspected. This requires much
more resources than NetFlow based inspection tech-
Host-based IDS (HIDS): HIDS are typically installed on niques, as all packets have to be opened up to the 7th
every host. HIDS are more platform specific, and are layer of the OSI model to be analyzed. However, this
focused on the target computer. HIDS can capture is obviously much more useful, as it can detect proto-
all the traffic generated by a host, which a NIDS cols running on any ports, as well as any byte pattern
typically cannot do in a switched network. HIDS regardless of the underlying protocol.
are not necessarily traffic based, but also look at the
system’s state. Snort is an open source NIDS software [6]. Combining
the benefits of signature, protocol and anomaly based
4 State of the Art - From Raw Packet inspection Snort is the most widely deployed IDS/IPS
Capture to Advanced Detection technology worldwide. It is able to perform “high level
analysis” on all the traffic flowing through its sensor.
Mechanisms Snort is available in two different version: Snort itself,
and the Snort Security Platform (Snort SP). Snort SP
Network Traffic Analysis can be performed in many
is an extension of Snort, using the same engine, but al-
different ways. Here is a list of the features that charac-
lowing much more options, such as multithreading, an
terizes network traffic. Each of these features is part of
the OSI model [4]. 1 n depending on the version
5 Snort in a Large Corporate Network 3
interactive shell, and performance improvements. Snort One of the drawbacks of this solution is that every
SP is still in early beta phase, and is very likely to im- unexpected traffic or unexpected exchange of information
prove its performance before the final release. will generate an alert. This solution could be efficient for
This paper will focus on the SnortSP-3.0.0b2, as there small, well-defined networks, but are likely to do more
were stability problems with the latest beta (SnortSP- harm than good in a large heterogenous network.
3.0.0b3). An excellent tour of horizon in this field can be found
Figure 1 presents the software architecture. SnortSP in [10]. Recent work attempts to bound the number of
is designed to act as an “operating system” for packet- false alarms while optimizing the correct results [11].
based network security applications, providing common
functionality that all programs need. 5 Snort in a Large Corporate Network
From a developers point of view, SnortSP is what gath-
ers data and handles any evasive techniques or other 5.1 Deployment
conditions that occur in suspicious and malicious traf-
fic. SnortSP normalizes the data and then provides this Sensor location is important. Typically, a good entry
cleaned up high level data to the engines for inspection. point is at the border between the LAN and the Internet.
Snort SP includes a new command line interface Placing Snort at this strategic point, allows the analy-
backed by the LUA embeddable programming language. sis of all traffic coming in and out of the local network.
This language allows to extend Snort functionalities with For this study, the Snort sensor was placed between the
a new scripting language [7]. CERN Public Network and the Internet.
The engines are analysis modules that plug into Snort This is a compromize, because it does not allow to
SP. Multiple engines can run simultaneously on the same scan inside-to-inside traffic. Such an analysis could be
traffic, in the same Snort SP instance. performed for example using sflow (which is a statisti-
The great advantage of this platform, is that it gives cal Netflow), or dumping all unauthorized traffic at the
the opportunity to run multiple analyzers in parallel, switch level.
thus increasing significantly the amount of traffic that It is also important to define what to detect with Snort.
can be analyzed. This will be presented in Chapter 9.1. As Snort rules are able to detect anything in the traffic,
it is important to clearly define the needs.
Is it enough to detect compromized hosts? Are there
policies that need to be enforced? Is it useful to record
all incoming attacks towards the network?
Those are all questions that need to be answered before
deploying Snort rules.
The P2P sets contain signatures to detect all kind of Skype is without doubts the most popular VoIP ap-
traffic, and there are some rules that need to be disabled plication currently used on the Internet. It uses its
before this set gives usable results. own proprietary protocols, and all traffic is encrypted.
Skype’s understanding and detection has been the sub-
6.4.3 Attacks and others ject of many research papers in the last few years [15]
[16] [17].
Other source files fall into this large category. There are [18] concluded that Skype was made by clever people,
other policies such as Instant Messenging (IM), informa- that they made a good use of cryptography, and that
tion about incoming attacks towards CERN, or sets to Skype is incompatible with traffic monitoring and IDS
detect hosts running adware and other badware. systems. This fact has been confirmed in this paper.
This last set was initially part of the “compromized” At CERN, Skype is used daily by more than 1’000
category, but due to the very large amount of devices hosts and users are required to run it on a specific port
running adware (during one day, the sensor detected 35 in order to avoid being affected by corresponding IDS
IPs running “Fun Web Products”, and about the same alerts.
number of IPs running a dozen of other adwares. Given Being encrypted and having its first bytes serving as
that adware do not pose a threat per se, the related rules sequence number [18], Skype traffic is likely, after enough
were disabled. time, to generate alerts on all Snort rules based on only
few bytes.
7 Snort Rules Evaluation There are many rules triggering Skype alerts, and it
seems that keeping only rule 5998 is sufficient to reduce
After the initial classification of the files, all sets of rules the noise while keeping enough information to detect
were thoroughly evaluated. Each of them was deployed Skype usage. This result was attained by running the
and analyzed according to the metrics defined in Chap- tool presented as Appendix E.2.
ter 6.3. The CERN Public Network is ideal for such Rule 5998 detects Skype logins, which are mandatory.
an analysis, because it provides means to interact and This login process is periodically repeated, so keeping
cross check with the users, and offers a great variety of only this rule also allows to record the timeframe of
devices and network traffic. With the large number of Skype’s usage.
users bringing their own unmanaged laptops, there are a
lot of infected devices helping to tweak the IDS. Emule is the file sharing protocol having the most rules
in VRT and ET. However it also is the most difficult to
7.1 CERN Policies detect file sharing protocol. With all rules based on two
or four bytes, it has the weakest rules.
7.1.1 Peer-to-Peer Emule rules pose problem on many levels; they are
Introduction This paragraph will present the findings weak and computationally expensive: Most of them only
concerning P2P detection using Snort rules. check for patterns of two bytes in all UDP traffic. With
The first big surprise here was the amount of different random traffic, a two bytes pattern triggers an alert every
protocols seen in the traffic. It seems that even if Bit- 65’536 packet on average. With more than 100’000 IP
torrent is the dominant P2P network, many users still packets per second going through the IDS during the
rely on old and less wide-spread protocols. Many foreign day, this clearly poses a problem. Analyzing all traffic
users were detected using localized P2P software, with for small patterns, they also are quite computationally
names unknown to most Europeans. expensive. The two most time consuming P2P rules are
Snort’s efficiency in detecting these protocols varies 2003322 and 2003321. According to Snort’s performance
from case to case. Some P2P protocols are very easily profiling tool, each of them requires ten times more CPU
recognizable, while others trigger too many false positives time than other P2P rules.
to provide useful data regarding P2P usage at CERN. With its default configuration and all Emule rules ac-
tive, there were more than 230 devices detected as run-
The rules ET and VRT both provide a file called ning Emule during a day. Almost all of those were also
p2p.rules containing all kind of rules detecting P2P traf- detected running Skype. An in-depth analysis of the
fic. Table 3 summarizes the content of these two files. All alerts showed that the very large majority of Emule alerts
of these protocols were seen at CERN in a one-month were in fact generated by legit Skype traffic. Table 4
time period. The “Others” rules detect 13 other file- quickly presents the number of Emule alerts seen on the
sharing protocols, out of which six were seen at CERN Skype port, and on other ports for 15 randomly selected
over that same period. Over a year, this number would hosts. More than 75% of the Emule alerts seem to be
probably increase significantly. triggered by Skype. In almost all cases, a detailed anal-
ysis of the 25 remaining percents led to the conclusion
Protocol # rules % of rules that it was Skype running on an arbitrary port.
Emule 26 29 A Python tool was developed to try to find some pat-
Bittorrent 13 15
Napster 9 10 terns in the alerts. The idea was to find a subset of all
Gnutella 5 5 the rules that successfully detected Emule traffic while
KaZaA 5 5
Skype 5 5 keeping False Positives to a minimum.
Others 28 31 While the perfect subset was not found, there was one
rule that was almost always present in Emule traffic and
Tab. 3: P2P Rules summary did not seem to trigger too many false positives: 2001298
(presented below). This rule triggers on E2DK Server
Status Request messages. The Emule protocol states
7 Snort Rules Evaluation 7
Shellcode Another file that was entirely disabled after One probable explanation of these differences is that
some research is shellcode.rules. Most of the rules con- ET is community driven, whereas VRT rules are written
tained in this file were triggering regular alerts for many by a team of Sourcefire experts. Being internationally
different hosts, and the amount of false positives it was spread, the community is probably much more efficient
generating was deemed excessive. Most of the rules in in writing rules detecting all kind of threats they have
this file looked for specific binary pattern that may indi- seen on their network, while Sourcefire team focuses on
cate a successful exploit. However, most of the byte pat- known exploits based on security bulletins. This doesn’t
terns it was looking for were always regularly contained mean that ET rules are better, but that they seem to be
in legit files. more oriented on detecting malware, whereas VRT rules
A quick example that can be easily checked is rule seem to be more oriented on detecting known exploited
1394, which looks for a series of 31 consecutive ’A’s vulnerabilites.
(NOOP, byte value 0x41) in all traffic. Even if very spe-
cific, it seems that this string is used in many JPEG
images to align the data and fill fields with placeholder 7.3 Attacks
data. A check was run against a web gallery, and out of 7.3.1 Introduction
the 1900 JPEG pictures it contained, 40 contained this
specific pattern. So there are probably millions of legit This set of rule tries to group all rules indicating that an
images triggering this specific alert around the web. attack is in progress. As any other big organization or
Similar checks were conducted with the other shell- company, the main problem with this set is that CERN
code.rules rules, and the results were always the same: is constantly under attack, and therefore there are con-
there were normal files triggering alerts. Therefore this stantly hundreds of alerts triggered by Snort.
file was disabled. Snort’s attack coverage is very wide. There are rules
aimed at detecting specific vulnerabilities, rules analyz-
7.2.3 Modifications done to the Set ing abnormal use of a protocol, rules detecting brute force
attempts, rules detecting abnormal traffic, etc. Table 8
Deploying the remaining set of files untouched triggers of Appendix A.2 lists all the files initially included in this
many alerts due to rules not complex enough and/or un- set.
needed alerts. While incoming attacks are known and taken care of,
This section will summarize the changes done to these outgoing attacks are much more interesting and could
files to optimize them for CERN environment. The mod- indicate compromized hosts.
ification details can be found as Appendix B.1. To generate such a set, a program was written to “re-
Some rules were disabled because of their resource con- verse” rules, so that they would consider the CERN Pub-
sumption, some because of their lack of complexity (they lic Network as the potential source of attacks.
were triggering too many false positives), and others be- Both attack sets (normal and reversed) were deployed,
cause they did not provide any useful results at CERN. and the same iterative process was started. Each alert
The process of selection was an iterative process. All was analyzed and the source rule disabled in case of false
of the rules were deployed, and all alerts were manually positive or unwanted alert.
looked at and analyzed. All rules that were not meeting However the amount of alerts was huge3 , and after
the requirements were disabled, and the new set was re- three weeks of intensive sorting and processing the idea
deployed. This process was repeated until the fraction was abandoned (see details in the next sections).
of false positives or unwanted alerts compared to real The number of alerts had been considerably reduced,
threats was acceptable. but in the three weeks these sets have been running, no
All of these modifications are detailed as Ap- useful alert had been seen.
pendix B.1, and are available in Oinkmaster format as The list of modifications done to this set is proposed
Appendix C.2. as Appendix B.3.
Having a timestamp on the rules could allow to easily In one day, there were more than 80 IPs “attacking”
deactivate old and deprecated rules. Unfortunately this the IMAP protocol, 90 “attacking” the POP protocol,
field does not yet exists. and 38 “attacking” the FTP protocol.
There also are a lot of rules targeted at specific web A sample of these alerts was chosen and studied, and
servers and applications. However for these rules to there was no confirmed case of attack. All these alerts
be useful, it is mandatory to know which webserver seemed to have been triggered by perfectly standard soft-
runs which operating system, and which webserver hosts ware and traffic.
which web application. This knowledge is difficult to
have in a very large network with hundreds of web 7.3.4 Rule Scoring
servers, and blindly enabling everything creates way too
many false positives or unwanted alerts. In order to improve the selection of rules, a new strategy
Snort also offer some DDOS rules detecting brute was devised to try to sort the rules and keep only the
force attemps, scans, or blind large scale attacks against interesting alerts.
the network, but there are other means to detect these The idea was to compute a “rule score” depending on
threats (Chapter 4). each rule complexity, classtype, and specificities. If the
Another interesting fact is that there are hundreds of rule score was above a threshold, then the rule would be
rules detecting perfectly harmless and normal traffic and kept, and if it was below the rule would be dismissed.
classified as “attempts”. The most noticeable rules do- Here are the different factors that were measured by
ing this are the “ping” rules. There are tens of rules the rule analyzer:
detecting all kind of pings. One quick example is the
rule 480, which is labeled ”ICMP ping speedera”, which • The number of bytes it is checking. The more the
is triggered by normal windows update behavior. better.
In a network of reasonable size these rules could pro-
vide useful information about incoming attacks. They • The placement of these bytes. Looking for a byte at
just seem to be not fit to be used at CERN. a specific position is obviously better than looking
for a byte anywhere in a packet.
7.3.3 Reversed attacks • The number of ports concerned by the rule. The
more specific the better.
Using reverse attack rules did not really give more conclu-
sive results. There were a lot of unforeseen consequences. • The packet size. If the rule specifies a packet size, it
The main results for this set are given below. is obviously better than checking all traffic.
Web attacks There are eight rule files targeted at web • Penalty for certain classtypes. Certain very specific
attaks, each being specific to a web server, to a specific rules are only classified as “not-suspicious” or “icmp-
type of traffic, or to known web vulnerabilities. event”. To dismiss them as well some keywords were
All of the web alerts seen with the reversed set were associated a penalty value to dismiss them.
legitimate. The main problem with those were the search
• Other features, such as flowbits, or PCREs, which
engines. Request containing potentially malicious strings
both improve a rule.
were always triggered by legitimate users querying search
engines. After some trial and error process while trying to find
To illustrate this with a very simple example, imagine the ideal weights for the different parameters, it seemed
that a user is trying to insert something in a database. that there was no direct correlation between complexity
This user is very likely to query google for “INSERT of a rule and its usefulness.
INTO (...)”. This query will be posted in the URL via Starting again from scratch, by putting all the weights
the GET method, and Snort’s SQL injection rules will to 0 except the “content” score, thus reducing signifi-
think this is an injection attempt, and trigger an alert. cantly the dimension space of the problem, traffic was
The inefficiency of these reversed web rules seems to be gathered and alerts analyzed. Figure 2 presents the num-
mainly due to the search engines. One could imagine to ber of alerts opposed to the complexity of the rules trig-
create exceptions for all the known search engines IPs, gering alerts over a day. There seem to be no direct re-
but unfortunately there is no such list and there will lation between the complexity of a rule and the number
probably never be. Even if it existed, the list would be of alerts it triggers.
too long for Snort; Snort being slow to process IP lists Figure 3 presents the the number of CERN IPs trig-
in rules. gering a rule opposed to the complexity (the score) of
There were also a lot of rules triggering when a user ac- the rules. Here again, there seem to be no direct rela-
cesses a potentially dangerous directory such as /cgi-bin tion between the number of IPs triggered by a rule and
or /viewtopic.php. Obviously a lot of websites meet these its complexity. Also, this plot does not reflect the “use-
requirements on the web, therefore generating unwanted fulness” of the rules. Rules with high scores that could
alerts. be selected by this process were mainly unneeded and
informational rules.
Specific protocols attacks The reverse rules analyzing As there seemed to be no direct correlation between
the SMTP, POP, IMAP, FTP, and other protocols did the score of a rule and its efficiency/usefulness, this idea
not give more conclusive results. For example, the only was abandoned too.
hosts triggering SMTP attack alerts were CERN mail The python code computing the scores is joined as Ap-
servers, which were obviously not attacking anyone. pendix E.1.
8 Rules Management 10
80000
A set of rules detecting compromized and infected
70000
hosts has been proposed and tested thoroughly on the
CERN Public Network.
Number
of
alerts
60000
50000
It was much more difficult to obtain a clean set de-
40000
tecting attacks, either incoming or outgoing. Without
30000
20000
knowing exactly what to look for, attack rules are too
10000
noisy to be useful.
0
There seem to be no correlation between a rule com-
0
2
4
6
8
10
12
14
16
18
20
Rule
score
plexity and its usefulness. A tool was written to evaluate
the rules based on their content, but the correct balance
of weights has not been found to successfully use this tool
Fig. 2: Number of alerts vs score
to generate the wished ruleset.
Number
of
CERN
IPs
vs.
score
400
8 Rules Management
350
250
200
It has been seen that there are many different sources for
150
finding rules that are regularly updated, and that rules
cannot be deployed as they are. Many rules need to
100
be disabled or modified by hand before they can be de-
50
ployed, and these modifications have to be re-done every
0
0
2
4
6
8
10
12
14
16
18
20
time a new ruleset becomes available.
Rule
score
So there is a need to properly handle the new set re-
leases while keeping the specific modifications done to
Fig. 3: Number of IPs vs score the sets.
Pulled Pork, on the other hand, provides moreless the modifysid * "^alert (\S+) (\S+) (\S+) (\S+) (\S+)" | \
same functionalities as Oinkmaster. However it is still in "alert ${1} ${5} ${3} ${4} ${2}"
modifysid * "msg(\s?):\"" | "msg:\"REVERSED "
an early development phase; the project began in May
2009. Therefore this tool was not evaluated in this paper. This will switch the local and remote addresses, and
Oinkmaster still remains the reference tool in this area. change to message alert to reflect the change.
Once the configuration file is complete, there are two
8.3 Update Process parameters that need to be passed to Oinkmaster to ex-
ecute the update process:
With the help of Oinkmaster, the update process is
simple. The user only has to download the two rules -C
sets tarballs from VRT and Emerging Threats, and run This will tell Oinkmaster where to find the config-
Oinkmaster on those sets. uration file. If it is omitted Oinkmaster will try to
The Oinkmaster configuration file is relatively easy to use /etc/oinkmaster.conf.
follow. It starts by defining where to find the rule sets,
and then lists all the modifications that have to be done -o
to these sets. The script is then called with two parame- The ouput directory. This is followed by the path
ters, the location of the configuration file, and the output where the rules are to be put.
directory where it should put the final rules.
The aforementioned functionalities make Oinkmaster
With the appropriate openings in the firewall,
an efficient and flexible tool to manage Snort rules.
Oinkmaster could even automatically download the lat-
est tarballs via HTTP, FTP, or even SSH.
After its run, Oinkmaster provides a short summary 9 Snort Optimizations
of the rules that were added/deleted/modified since the
last update that can easily be reviewed by hand. 9.1 Using Snort SP: Multiple Analyzers
A sample report is included as Appendix C.1
The whole process could be put in a cron job and run Snort SP claims to be much more performant than its
automatically every day or week. predecessor due to multithreading. However, by default,
Snort SP does not take full advantage of this possibility.
The Snort SP platform is built with three different lay-
8.4 Oinkmaster Configuration ers, the source, the engine, and the analyzers (see Chap-
Here is a short description of Oinkmaster’s most common ter 4). The source is responsible for capturing the traffic,
options. Its configuration file is split in three different and handing it to the engine. The engine preprocesses
parts. The first part starts by defining where to find the the traffic, and gives it to the analyzers, which, only then,
rules tarballs and some other constants. The second tells will try to match the traffic with the enabled rules.
Oinkmaster which files are irrelevant in those tarballs, Due to the very large amount of traffic seen at CERN’s
and the last part applies modifications to the remaining Public Network, one analyzer is not enough to try and
rules. match all the wanted rules with every pre-processed
For this last part, there are a few statements that need packet. A proposed optimization is to split the rules in
to be known: different analyzers, so that each of them is able to handle
the flow of traffic and work on the traffic in parallel.
disablesid: This is done by modifying the LUA configuration file
This is the most common. It completely disables a in such a way that multiple analyzers are created, and
rule that is by default enabled in the tarballs. attached to the engine. Each analyzer has its own con-
figuration file, telling it what rules should be enabled.
enablesid:
This can enable some rules that are by default dis- The benefits of such a configuration are obvious, the
abled in the tarballs. amount of traffic that can be analyzed increases signifi-
cantly with each new analyzer. With most high-end pro-
modifysid: cessors having now 16 cores, this architecture allows up
This one offers the greatest flexibility, and allows to to 14 analyzers (one thread per core).
do pretty much anything with the rules. The stan- The only disadvantage is that this has to be done man-
dard syntax is modifysid 1234 ”foo” | ”bar”. This ually and that Snort can not automatically split the rules
will replace foo with bar in rule 1234. Complex regu- in an optimal way. It also adds some overhead to the
lar expressions can be used in these statements. It is analysis.
important to note that this modification is applied An example configuration script is provided as Ap-
only once for each rule. For a modification to be pendix D.1
applied twice on a rule (to remove a keyword for ex- Unfortunately, there is no documentation on those pos-
ample), it is needed to duplicate the statement (ie. sibilities at the time of this writing.
with modifysid 1234,1234 ).
localsid: 9.2 Libpcap Modification
This one marks a rule as “modified locally”. If a As seen in 9.1, there is only one thread in Snort SP that
new version of the rule is downloaded, it will not be captures the traffic and feeds the Snort SP engine.
enabled, and the old one will be kept unharmed. To capture the traffic, Snort relies on the libpcap li-
To reverse all the alert rules, for example, the following brary [23]. Before Snort can access the data, it has to
statements can be used: be copied many times between the NIC, the kernel, and
9 Snort Optimizations 12
80000000
radically changing the way data is passed around before
75000000
entering the Snort engine. 70000000
55000000
With this new library, the data is immediately copied 50000000
09:52
10:02
10:12
10:22
10:32
10:42
10:52
11:02
11:12
11:22
11:32
11:42
11:52
12:02
12:12
12:22
12:32
12:42
12:52
13:02
13:12
13:22
13:33
13:43
13:53
14:03
14:13
14:23
14:33
14:43
14:53
15:03
15:13
15:23
15:33
15:43
15:53
16:03
16:12
16:22
16:32
16:42
16:52
17:02
17:12
17:12
17:22
access it. The use of this library greatly reduces the Day
1
‐
New
libpcap
Day
2
‐
Default
libpcap
30
20
fication. 25
For the first one, there were no Snort rules active, and 20
content‐replace.rules
emerging‐emerging.rules
dns.rules
rservices.rules
porn.rules
netbios.rules
web‐coldfusion.rules
web‐a:acks.rules
icmp.rules
exploit.rules
scan.rules
icmp‐info.rules
info.rules
bad‐traffic.rules
finger.rules
chat.rules
web‐cgi.rules
experimental.rules
telnet.rules
other‐ids.rules
smtp.rules
web‐client.rules
imap.rules
local.rules
Cp.rules
web‐iis.rules
ddos.rules
mulDmedia.rules
oracle.rules
rpc.rules
sql.rules
shellcode.rules
virus.rules
pop3.rules
web‐acDvex.rules
p2p.rules
mysql.rules
backdoor.rules
pop2.rules
a:ack‐responses.rules
dos.rules
specific‐threats.rules
web‐frontpage.rules
snmp.rules
voip.rules
web‐misc.rules
policy.rules
Ftp.rules
web‐php.rules
scada.rules
nntp.rules
x11.rules
emerging‐botcc.rules
emerging‐exploit.rules
emerging‐rbn.rules
emerging‐scan.rules
emerging‐tor.rules
emerging‐compromised.rules
emerging‐dshield.rules
emerging‐web.rules
emerging‐inappropriate.rules
emerging‐virus.rules
emerging‐p2p.rules
emerging‐malware.rules
emerging‐dos.rules
emerging‐drop.rules
emerging‐web_sql_injecDon.rules
emerging‐voip.rules
emerging‐a:ack_response.rules
emerging‐game.rules
but the percentage of dropped packets was not. We can
see that when the number of captured packets exceeds
6M/minute, the default libpcap in unable to handle the
traffic and starts loosing packets. The amount of pack- with
the
New
libpcap
with
the
Default
libpcap
Running Snort with this option does not seem to sig- To further improve the amount of traffic that can be
nificantly impact snort performance. In fact no difference analyzed by Snort SP, its ability to use multiple analyzers
was noticed with and without it. on the same traffic has been tested, and it’s efficiency is
Here is an exemple of an unexpected expensive rule: undeniable as it allows to multiply the number of rules
deployed without increasing the packet loss ratio.
alert tcp $EXTERNAL_NET 1024: -> $HOME_NET 1024: (msg:"ET
TROJAN Beizhu/Womble/Vipdataend Controller Keepalive"; This paper also addressed different problems that arise
flowbits:isset,ET.vipde; flow:established,from_server; dsize:1; when trying to deal with the rules. There is a clear lack
content:"d"; classtype:trojan-activity; reference:(...);
sid:2008335; rev:6;) of classification among all the available rules, and it is
difficult to choose the ones that will be useful in a par-
This rule relies on a flowbit, and if the flowbit is set, ticular environment.
then checks if the packet is of size 1, and that the byte is An in-depth analysis of all the available rules was done,
“d”. This rule was reported as one of the most time con- and the rules have been re-classified in three main cate-
suming. A lot of rules with FlowBits showed the same gories: policies, compromized, and attacks.
behavior. It seems that Snort spends a lot of time check- For the first two categories, a working set is proposed
ing for FlowBits. and tested on the CERN Public Network. These sets
A few other things that Snort seems to not like in have been deployed, and have significantly improved the
terms of CPU time is IP filtering, Perl regular expressions efficiency at detecting infected hosts, while reducing the
(PCRE), and sliding windows in all traffic (ie. checking number of false positives.
for a series of bytes anywhere in all packets. The problem of the coexistence of Skype and Emule
All IP-based rules (rules that were not relying on con- rules has been addressed and some optimizations to bit-
tent, but only on a list of known remote IPs) performed torrent’s detection have been proposed.
very badly on our Snort sensor. This is probably due to Concerning the attacks, it seems that adapting the
the way Snort analyzes the traffic. The use of netflow set to a very large environment requires a tremendous
data to do the exact same thing a posteriori seems to amount of work and extensive knowledge of the network,
be much more efficient, as Snort can analyze much more which is clearly not possible in a very large and heteroge-
traffic for details that netflow data does not contain. nous corporate network.
PCRE on traffic were also slowing considerably the Once the rules have been chosen, there is no easy way
sensor. A few rules were modified to check for specific to update the rulesets while keeping the changes intact.
strings before checking the PCREs. Adding checks re- To take care of the update process, it is proposed to
duced the overall CPU time consumption. It is recom- use the “Oinkmaster” software, which allows Snort’s ad-
mended to always check for the maximum possible spe- ministrators to easily and seamlessly update their snort
cific strings before checking for PCRE, and to only vali- rules.
date the findings with the PCRE.
An example of a rule that has been modified in this
way is the following: 10.1 Outlook
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:" There is a clear lack of classification in the default rule-
(...)"; flow:to_server,established; content:"User-Agent\:"; sets available. It has been seen that a better classification
nocase; pcre:"/User-Agent\:[^\n]+DEBUT\.TMP/i"; sid:2003427;
rev:3;) will help many users to get the best out of their Snort
sensor.
This rule was running a PCRE check for all web An additional field “date” on all the rules would signif-
requests. By simply adding content:“DEBUT.TMP”; icantly help on selecting current events rules and dismiss
before the PCRE, the load was significantly reduced. all the deprecated rules.
After the modification only packets containing both Another classification system, such as a tagging sys-
strings were checked with the PCRE. tem, could also help in selecting rules if it were available.
The final rule becomes: With the correct weights, the rule score system pro-
posed in Chapter 7.3.4 could also be of use. Unfortu-
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:" nately they have not been found.
(...)"; flow:to_server,established; content:"User-Agent\:";
nocase; content:‘‘DEBUT.TMP"; nocase; Another possibilty to improve Snort’s performance
pcre:"/User-Agent\:[^\n]+DEBUT\.TMP/i"; sid:2003427; rev:3;) could be to recompile Snort with another optimized com-
piler. Snort is currently built with GCC, but the Intel
10 Conclusion compiler has been known to produce faster code on Intel
machines [26].
This thesis presented Snort SP capabilities as an IDS
in a large corporate network. The two main aspects
of the software configuration were covered in this doc-
ument: The rules and the performance problems. Sev-
eral options to improve Snort’s performance have been
proposed.
The bottleneck on a high-speed network with the de-
fault installation is the packet capture library. A re-
placement has been proposed and its efficiency has been
shown. With its default configuration and no rule active,
our Snort instance was dropping more than 15% of the
traffic. This number was reduced close to zero with the
new library.
A Additional Data 14
A Additional Data
A.1 Classtypes
Table 5 presents the default classtypes available in Snort
with their description.
Classtype Description File
not-suspicious Not Suspicious Traffic emerging-virus.rules
unknown Unknown Traffic emerging-malware.rules
bad-unknown Potentially Bad Traffic emerging-attack response.rules
attempted-recon Attempted Information Leak ddos.rules
successful-recon-limited Information Leak virus.rules
successful-recon- Large Scale Information Leak backdoor.rules
largescale spyware-put.rules
attempted-dos Attempted Denial of Service attack-responses.rules
successful-dos Denial of Service shellcode.rules
attempted-user Attempted User Privilege Gain
unsuccessful-user Unsuccessful User Privilege Gain
successful-user Successful User Privilege Gain Tab. 7: Files containing compromized rules
attempted-admin Attempted Administrator Privi-
lege Gain
successful-admin Successful Administrator Privi-
lege Gain
rpc-portmap-decode Decode of an RPC Query
shellcode-detect Executable code was detected
string-detect A suspicious string was detected
suspicious-filename- A suspicious filename was de-
detect tected
suspicious-login An attempted login using a sus-
picious username was detected
system-call-detect A system call was detected
tcp-connection A TCP connection was detected
trojan-activity A Network Trojan was detected
unusual-client-port- A client was using an unusual
connection port
network-scan Detection of a Network Scan
denial-of-service Detection of a Denial of Service
Attack
non-standard-protocol Detection of a non-standard pro-
tocol or event
protocol-command- Generic Protocol Command De-
decode code File
web-application- access to a potentially vulnerable emerging-exploit.rules
activity web application emerging-scan.rules
web-application-attack Web Application Attack emerging-web.rules
misc-activity Misc activity emerging-dos.rules
misc-attack Misc Attack emerging-web sql injection.rules
icmp-event Generic ICMP event emerging-voip.rules
kickass-porn SCORE! Get the lotion! dns.rules
policy-violation Potential Corporate Privacy Vi- rservices.rules
olation web-coldfusion.rules
default-login-attempt Attempt to login by a default icmp.rules
username and password exploit.rules
scan.rules
finger.rules
Tab. 5: Snort’s description of the classtypes web-cgi.rules
telnet.rules
smtp.rules
web-client.rules
imap.rules
ftp.rules
A.2 File mappings web-iis.rules
ddos.rules
These tables present how the different available files are oracle.rules
rpc.rules
split into the three categories: policies, compromized, sql.rules
and attacks. (Tables 6, 7 and 8) shellcode.rules
pop3.rules
web-activex.rules
File mysql.rules
chat.rules dos.rules
content-replace.rules specific-threats.rules
policy.rules snmp.rules
porn.rules voip.rules
voip.rules web-misc.rules
multimedia.rules web-php.rules
p2p.rules web-frontpage.rules
chat.rules scada.rules
emerging-game.rules nntp.rules
emerging-inappropriate.rules
emerging-p2p.rules
emerging-policy.rules Tab. 8: Files containing attacks rules
#+++############################################################################
# #
# IRC detection rules #
# #
#---############################################################################
# note: to avoid performance problems, we only check the first 256 bytes
12077 : 2009345 :
This looks for c99shell command requests. There Triggers when a web server replies 401 unauthorized.
are a lot of incoming requests, but there is no way This does not mean that a computer has been com-
to sort between successful and attempted attacks. promized.
The number of false positives outnumbers the real
cases (zero found over one month). 2009346 :
This does not report a compromized host. It detects
2000345, 2000348, 2000347, 2000352 : http bruteforce (many 401 errors during a short pe-
These detect IRC messages on non standard ports riod).
(nick change, join, privmsg, and dns). This is cov-
ered by CERN IRC rules. B.1.3 Resource consuming rules
2000562 : These rules were commented because they consumed a
Detects file attachments, which are normal. lot of resources, and their usefulness was discussable.
The performance was evaluated using Snort’s perfor-
2001689 : mance profiling tool. Here is the list of the disabled rules
This rule looks for potential bots scanning for SQL and the reasons:
server. This rule does not report compromized
hosts, and there are hundreds of bots at any time 7101 & 7103 :
trying to scan the CERN Public Network from out- Dependant on 7102. No use if 7102 is disabled.
side.
7102 :
2001795 : It detects a Spyware dated from 2004. And the load
This rule triggers when an IP is sending more than is very high.
30 mail per minute towards CERN.
7716 :
2001920 : 7715 has been modified to trigger an alert instead.
Looks in all SMTP traffic and catches gif.exe in in-
7761 :
coming mail. But all .exe attachments are rejected
This rule was the winner in times of resource con-
by CERN mail servers.
sumption. It tracks a malware dated 2004. Any
2002322 : antivirus should detect it.
This rule looks at all incoming MSN messages con- 13509 :
taining links ending in “.php”. Any link to “in- 13508 is complex enough and has been modified to
dex.php” would trigger an alert. trigger an alert to reduce load. Therefore the mod-
2002323 : ified version of 13508 already covers this threat.
This rule detects exe files sent via msn. 2002031 :
2002894, 2002892, 2002895, 2001919 : IRC - potential download or upload. IRC is covered
These rules detect viruses incoming or outgoing via by the CERN rules.
SMTP. The only IPs triggering these alerts are 2002032 :
CERN mail servers, which already drop incoming IRC - potential bad command. IRC is covered by
and outgoing viruses. the CERN rules.
2003484 : 2003176 :
This rule should be called ”malformed http request” Detects a packet of 4 null bytes. The load is very
instead of “virus...”. It is triggered regularely by high, and it detects a mail-spreading worm of 2006.
linux computers.
2003380 :
2007866 : Looks for a suspicious User-Agent. It checks for
This one tries to detect gadu-gadu, which is not a PCRE on all User Agent strings, and the rule is
trojan. impossible to modify to improve its performance.
2008221, 2008222 : 2003427 :
And these two detects incoming potential phishing RxToolbar. Very high load and it is only adware, so
e-mails. no real threat.
2008333, 2007774 : 2007583 :
Detects the “swizzor” adware, which is not a trojan. Looks for User-Agent “IEbar”. Induces a very high
load.
2008411 :
This rule looks for people sending e-mails with “The 2008178 :
Bat” mail client and having attachments. 2008177 is already covering this threat and it is con-
suming too much resources.
2008576 :
Looks for tinype windows executables. Has triggered 2008335 :
many times on legit remote hosts hosting normal 2008334 is already covering this threat and it is con-
files. suming too much resources.
B Detailed results 18
2009026 :
2009025 is already covering this threat and it is con-
suming too much resources.
2003427, 2007583 :
These two signatures were modified in order to in-
crease the system performance. Both signatures
were checking if the packet contained a “User-
Agent”, and if found were trying to match it using
PCRE. Both were modified to also check for the spe-
cific User-Agent before doing the PCRE check. The
additions were respectively “content:’DEBUT.tmp’”
“and “content:’iebar’.
7118 :
This rule is new, and uses a token that Snort SP
beta 2 does not understand: “http header”. It has
been removed from the rule.
This modification will be removed with a new Snort
SP version.
2008335, 2009026, 2003176 :
These rules were modified to improve the perfor-
mance. These 3 rules check for a flowbit, then if
the flowbit is found check that the packet has a cer-
tain size. The order of these two instructions was
reversed.
13508, 7715:
These two rules were modified to produce alerts.
They were initially created to set a flowbit that an-
other rule would check, but these other rules were
disabled to increase performance. These rules are
complex enough, and are reliable enough to create
alerts. The “noalert” keyword was removed from
them.
498 :
This rule checked all traffic for root, uid 0. It was
modified to only trigger an alert on outgoing traffic.
B Detailed results 19
Thread 2:
- ddos.rules
- emerging-attack_response.rules
- virus.rules
- attack-responses.rules
- backdoor.rules
480: 823:
“ICMP ping speedera” This is normal windows up- Triggers on all “cvweb.cgi” accesses.
date behavior.
2001219:
8428: SSH scan. Can be detected by other means.
Https traffic with some flags not set. The flags are
probably not set due to Snort’s packet drop. 2003068:
SSH scan outbound. Can be detected by other
485: means.
“ICMP Destination Unreachable Communication
Administratively Prohibited” 2517:
“IMAP PCT Client Hello overflow attempt” . 16739
882: alerts were triggered by a perfectly normal xchange
URI contains ”Calendar”. server.
466: 72517:
“ICMP L3retriever Ping”. This is apparently nor- Same rule reversed.
mal with windows
1042:
1394: Triggers when the string “Translate: F” is in http
“AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA” requests.
in payload.. should detect shellcode noop, but is
contained in many normal images. 71042:
Same reversed.
13514:
Triggered when the words “update” and “set” are 2000536, 2000537,2000538, 2000540, 2000543, 2000544,
present on the same line 2000545, 2000546:
Those are NMAP scans. There are too many of
2002922: them and not much can be done to prevent them.
“ET POLICY VNC Authentication Successful”.
This is allowed at CERN. 2003099:
Triggers when a null byte is found in the URI of a
2002912, 2002913, 2002914, 2002915, 2002916, 2002917, web request. There were more than 4000 alerts per
2002918, 2002919, 2002920, 2002921, 2002923, day coming from lots of IPs.
2002924 :
Real VNC stuff, induces high load, and rules are too 72003099:
targeted. Same reversed.
2001090: 483:
Triggers when javascript is found on a web page but Triggers on ICMP traffic containing
not enclosed by the appropriate “javascript” tag. In- “AAAAAAAA”. Cyberkit triggers those alerts.
duces a very high load.
10995:
2001091: SMTP possible BDAT DoS attempt. 3620 alerts on
Same for Visual Basic CERN mail server on one day.
2001092: 710995:
Same for Microsoft Access shell code. And its reverse.
B Detailed results 21
13948: 1288:
“DNS large number of NXDOMAIN replies - possi- “/ vti bin/” request.
ble DNS cache poisoning”. Triggers on normal traf-
fic. 78734:
“REVERSED WEB-PHP Pajax arbitrary command
72001621: execution attempt” . Triggers on normal traffic with
Potential PHP SQL injection attack. “Potential”. google servers.
895: web-misc.rules :
Triggers when “/redirect” is found in URLs. Why This file contained too many rules not suited for a
not? large environment.
7895: 2006445:
Same reversed. Triggers when “SELECT” and “FROM” were found
in a packet. There were many false positives.
1968, 1998, 1999, 2000, 2229, 72229:
Normal “.php” file access (such as viewtopic.php). 72006445:
Same reversed.
all SIDs having “web-application-activity” as classtype:
They provided no real information about attacks 72001087:
and were informational. Reversed “ET WEB-MISC cross site scripting at-
tempt to execute Javascript code”. No use for re-
712007: mote sites.
“401 Unauthorized” in SIP/2.0 protocol. Many
web-frontpage.rules:
alerts.
Mainly alerts related to accesses to some files. Too
853: many unwanted alerts.
When “/wrap” is contained in url. Lots of normal
7969:
web requests.
reversed “WEB-IIS WebDAV file lock attempt”.
7853: This is a normal feature of webdav.
Same reversed. All rules having “access” in their name in all the web*
2329: files:
SQL overflow attempt , checks only 2 bytes on any Access rules did not provide useful information.
ports. Whole sets of rules:
72329: • emerging-web.rules
Same reversed.
• emerging-web sql injection.rules
2002851: • emerging-voip.rules
FTP LIST without login. There are lots of FTPs.
• web-coldfusion.rules
72007873: • web-cgi.rules
Triggers on GET requests for files ending in .exe, • web-client.rules
.bat, .dll, ...
• web-iis.rules
1156: • web-activex.rules
When “////////” is contained in a packet. Re-
ported as ”apache directory disclosure attempt”. • snmp.rules
• voip.rules
478:
Triggered by ICMP Broadscan Smurf Scanner. Not • web-misc.rules
much can be done about it. • web-php.rules
• web-frontpage.rules
72002997:
Reversed ET WEB PHP Remote File Inclusion 474:
(monster list http). We don’t really care if an exter- ICMP traffic containing “—00 00 00 00 00 00 00
nal website is potentially vulnerable. 00—”. Why not?
8440: 2002995:
Too many alerts / IPs to be usable. Potential IMAP scan. Can be detected with some
other means.
72002992, 72002993, 72002994, 72002995:
Triggers on 10 mail connections in 2 minutes outgo- 2006546:
ing. SSH bruteforce. Can be detected with other means.
11969: 2006435:
VOIP-SIP inbound 401 unauthorized message. SSH bruteforce. Can be detected with other means.
B Detailed results 22
73072:
Imap status overflow attempt. Always triggered by
legitimate traffic.
2590:
Smtp mail from overflow attempt. Idem.
2183:
“Content transfer encoding” overflow attempt. Al-
ways triggered by legitimate traffic.
713513:
Reversed web traffic containing ”insert * into”. Trig-
gered by search engine requests.
2003:
SQL worm propagation attempt. Triggered by 52
cern machines. No virus found on some of those.
Probably false positives.
2050:
SQL overflow attempt. Checks for one byte in all
packets of size bigger than 100 bytes to port 1434.
Lots of false positives.
72250:
Checks for POP3 user with % at the end. Triggered
by legitimate traffic... (OVH and some physics lab
in Japan)
13512:
Checks for “exec master” SQL in traffic. Lots of
false positives.
13513:
Checks for “insert into” SQL statement in traffic.
Lots of false positives.
713695:
“Reversed Real Helix server 2002 vulnerability”.
This reversed alerts is triggered by legitimate rtsp
akamai servers.
C Rules Management 23
C Rules Management
C.1 Oinkmaster Sample Report
This is report provided by Oinkmaster after a run. It provides a summary of the modifications apported to the
rules: tells you which one have been modified, which ones have been disabled, etc.
It even warns you when a file is added or deleted, so that the changes can be reflected in the snort configuration
file.
This is very useful to merge the local changes apported to the rules, with the new rules provided by different
sources.
[***] Results from Oinkmaster started 20090721 15:52:29 [***]
old: alert tcp any any -> any any (msg:"ET ATTACK_RESPONSE Bindshell2 Decoder Shellcode"; content:"|53 53 53 53 53 43 |";\
content:"|66 53 89|"; distance:0; classtype:shellcode-detect; reference:url,doc.emergingthreats.net/2009246;\
reference:url,www.emergingthreats.net/cgi-bin/cvsweb.cgi/sigs/ATTACK_RESPONSE/ATTACK_RESPONSE_Common_ShellCode;\
sid:2009246; rev:1;)
new: alert tcp any any -> any any (msg:"ET ATTACK_RESPONSE Bindshell2 Decoder Shellcode"; \
content:"|53 53 53 53 53 43 53 43 53 FF D0 66 68|"; content:"|66 53 89 E1 95 68 A4 1A|"; distance:0;\
classtype:shellcode-detect; reference:url,doc.emergingthreats.net/2009246; \
reference:url,www.emergingthreats.net/cgi-bin/cvsweb.cgi/sigs/ATTACK_RESPONSE/ATTACK_RESPONSE_Common_ShellCode;\
sid:2009246; rev:2;)
url = file://snortrules-snapshot-2.8.tar.gz
url = file://emerging.rules.tar.gz
path = /bin:/usr/bin:/usr/local/bin
#######################################################################
# Files to totally skip (i.e. never update or check for changes) #
# #
# Syntax: skipfile filename #
# or: skipfile filename1, filename2, filename3, ... #
#######################################################################
# The file deleted.rules contains rules that have been deleted from
# other files, so there is usually no point in updating it.
skipfile deleted.rules
skipfile bad-traffic.rules
skipfile chat.rules
skipfile content-replace.rules
C Rules Management 24
skipfile dns.rules
skipfile dos.rules
skipfile emerging-botcc-BLOCK.rules
skipfile emerging-botcc.excluded
skipfile emerging-botcc.rules
skipfile emerging-compromised-BLOCK.rules
skipfile emerging-compromised.rules
skipfile emerging-dos.rules
skipfile emerging-drop-BLOCK.rules
skipfile emerging-drop.rules
skipfile emerging-dshield-BLOCK.rules
skipfile emerging-dshield.rules
skipfile emerging-exploit.rules
skipfile emerging-inappropriate.rules
skipfile emerging-malware.rules
skipfile emerging-policy.rules
skipfile emerging-rbn-BLOCK.rules
skipfile emerging-rbn.rules
skipfile emerging-scan.rules
skipfile emerging-sid-msg.map
skipfile emerging-sid-msg.map.txt
skipfile emerging-tor-BLOCK.rules
skipfile emerging-tor.rules
skipfile emerging-voip.rules
skipfile emerging-web.rules
skipfile emerging-web_sql_injection.rules
skipfile emerging.conf
skipfile emerging.rules
skipfile experimental.rules
skipfile exploit.rules
skipfile finger.rules
skipfile ftp.rules
skipfile icmp-info.rules
skipfile icmp.rules
skipfile imap.rules
skipfile info.rules
skipfile misc.rules
skipfile multimedia.rules
skipfile mysql.rules
skipfile netbios.rules
skipfile nntp.rules
skipfile oracle.rules
skipfile other-ids.rules
skipfile policy.rules
skipfile pop2.rules
skipfile pop3.rules
skipfile porn.rules
skipfile rpc.rules
skipfile rservices.rules
skipfile scada.rules
skipfile scan.rules
skipfile shellcode.rules
skipfile smtp.rules
skipfile snmp.rules
skipfile specific-threats.rules
skipfile spyware-put.rules
skipfile sql.rules
skipfile telnet.rules
skipfile tftp.rules
skipfile voip.rules
skipfile web-activex.rules
skipfile web-attacks.rules
skipfile web-cgi.rules
skipfile web-client.rules
skipfile web-coldfusion.rules
skipfile web-frontpage.rules
skipfile web-iis.rules
skipfile web-misc.rules
skipfile web-php.rules
skipfile x11.rules
##########################################################################
# SIDs to modify after each update (only for the skilled/stupid/brave). #
# Don’t use it unless you have to. There is nothing that stops you from #
# modifying rules in such ways that they become invalid or generally #
# break things. You have been warned. #
# If you just want to disable SIDs, please skip this section and have a #
# look at the "disablesid" keyword below. #
# #
# You may specify multiple modifysid directives for the same SID (they #
# will be processed in order of appearance), and you may also specify a #
# list of SIDs on which the substitution should be applied. #
# If the argument is in the form something.something it’s regarded #
# as a filename and the substitution will apply on all rules in that #
C Rules Management 25
# file. The wildcard ("*") can be used to apply the substitution on all #
# rules regardless of the SID or file. Please avoid using #comments #
# at the end of modifysid lines, they may confuse the parser in some #
# situations. #
# #
# Syntax: #
# modifysid SID "replacethis" | "withthis" #
# or: #
# modifysid SID1, SID2, SID3, ... "replacethis" | "withthis" #
# or: #
# modifysid file "replacethis" | "withthis" #
# or: #
# modifysid * "replacethis" | "withthis" #
# #
##########################################################################
#the two following statements removes "noalert" from rules that were not producing alerts but only setting flowbits.
#Rules reading the flowbits have been disabled
modifysid 13508 "flowbits:noalert;" | "" #generate an alert, because 13509 has been disabled below
modifysid 7715 "flowbits:noalert;" | "" #generate an alert, 7716 has been disabled because of resource consumption
# rule detecting root uid0 in traffic. Modified so that only outgoing triggers an alert
modifysid 498 "any any -> any any" | "\$HOME_NET any -> any any"
########################################################################
# SIDs that we don’t want to update. #
# If you for some reason don’t want a specific rule to be updated #
# (e.g. you made local modifications to it and you never want to #
# update it and don’t care about changes in the official version), you #
# can specify a "localsid" statement for it. This means that the old #
# version of the rule (i.e. the one in the rules file on your #
# harddrive) is always kept, regardless if the official version has #
# been updated. Please do not use this feature unless in special #
# cases as it’s easy to end up with many signatures that aren’t #
# maintained anymore. See the FAQ for details about this and hints #
# about better solutions regarding customization of rules. #
########################################################################
########################################################################
# SIDs to enable after each update. #
# Will simply remove all the leading ’#’ for a specified SID (if it’s #
# a multi-line rule, the leading ’#’ for all lines are removed.) #
# These will be processed after all the modifysid and disablesid #
# statements. Using ’enablesid’ on a rule that is not disabled is a #
# NOOP. #
# #
# Syntax: enablesid SID #
# or: enablesid SID1, SID2, SID3, ... #
########################################################################
enablesid 2003427
########################################################################
# SIDs to comment out, i.e. disable, after each update by placing a #
# ’#’ in front of the rule (if it’s a multi-line rule, it will be put #
# in front of all lines). #
# #
# Syntax: disablesid SID #
# or: disablesid SID1, SID2, SID3, ... #
########################################################################
disablesid 2003380 # Suspicious User-Agent - pcre on all UA, difficult to modifysid efficiently
disablesid 2003380 # does pcre on ALL UAs... disabled
disablesid 2003427 # RxToolbar UA - Very high load and only adware
#disablesid 2007583 # iebar spyware. reenabled
disablesid 2008178 # TROJAN Ceckno Keepalive from Controller - covered by 2008177
disablesid 2008335 # 2008334 already covers this threat
disablesid 2009026 # 2009026 already covers this threat
#Those are the emule rules that trigger way too many FPs to be useful for CERN when Skype is around.
#Chapter 7 of the thesis explains this in detail.
disablesid 2003316,2003310,2003317,2003308,2003309,2003311,2003312,2003313,2003314,2003315,2003318,2003319,2003320,2003321,2003322,2003323,
disablesid 2003324,2587,2000330,2000332,2000333,2001295,2001296,2001297,2001299
# The only emule rule interesting to us is 2001298
enablesid 2001298
#This is a bittorent rule triggering too many unwanted alerts and dependent on another rule (2181)
disablesid 2000334
#This is a bittorrent rule detecting the same machines with less alerts
enablesid 2181
D Improvements to Snort 27
D Improvements to Snort
D.1 Sample Script to Create Mutiple Analyzers
Below is a sample of the configuration file creating two analyzers for a Snort instance.
opttab1={
conf="/opt/snort/etc/snortsp1.conf",
dynamic_engine_lib="(...)",
dynamic_preprocessor_lib_dir="(...)",
l="/opt/snort/log/current"
}
opttab2={
conf="/opt/snort/etc/snortsp2.conf",
dynamic_engine_lib="(...)",
dynamic_preprocessor_lib_dir="(...)",
l="/opt/snort/log/current"
}
function init ()
eng.new("e1")
eng.add_analyzer({
engine=engine_id,
analyzer="a1", order=1,
module=snort_module,
data=opttab1, bpf=""
})
eng.add_analyzer({
engine=engine_id,
analyzer="a2", order=2,
module=snort_module,
data=opttab2, bpf=""
})
end
Upon initialization, Snort SP creates a new engine, named “e1”, and attaches two analyzers. Each having a
separate options array.
E Tools
E.1 Python Script Used to Compute a Rule Score
This script computes a score for each rule contained in the file passed as first argument. A rule will be placed either
in keep.rules or in dismiss.rules, depending on a fixed threshold.
In the below version, only the content is used to compute a general score. This is very easily modifiable with the
different weight on the first lines.
import re
import os
import sys
# We create two output files. One where the kept rules will be put, and one where the dismissed rules will be put
of = open("keep.rules", ’a’)
dismiss = open("dismiss.rules", ’a’)
# The scores that will multiply with the weights (above). All initialized to 0.
scoreNumberOfPorts = 0 #from 0 (all ports) to 1 (1 port)
scoreContent = 0 #from 0 when no content checked to 20 depending on the sum of bytes + bytes^2/depth, \
#=2 if 1 byte is checked, =2, if 1 byte at specific position, =10 if 10 bytes, =20 if 10 bytes at specific position
scorePacketSize = 0 #from 0 (no size) to 1 (exact size)
scoreIPs = 0 #unimplemented yet
scoreClasstype = 0 #remove unwanted classtypes with negative score
scoreFlowBits = 0 #Increase confidence in rules having flowbit set
scoreMessage = 0 #Reduce confidence when the message contains some keywords
scorePcre = 0 #Increase confidence when a rule has a PCRE
# Number of ports. This one was the most difficult to implement due to the large variety of possiblities
# in specifying ports. It can be a single port (80), a port range (80:100), an unbounded port range (80:),
# a list of ports (80,81), a combination of the above (80-,22), or a variable ($HTTP_PORTS), and all the above negated (!80:100)
# The coding is not very nice, because each of the above case was added one by one... But it works.
# The score is then computer for the packet size, which is given by: (1518.0-(BIGGEST-SMALLEST))/1518.0;
# This gives a score of 1 if the port is specific, 0 if any.
# And now the big part, the content score. There can be many content statements, each can be followed by a depth.
# The total score is given by: Sum ( # bytes checked + (# bytes checked ^ 2 / depth) ). If 10 bytes are to be found anywhere
# in the packet, the score will be 10, if those 10 bytes are at an exact position, the score will be 20.
# The score is maxed to 20 at the end.
hasContent = False
length = 0.0
for word in payload:
if word.find("content:") != -1:
#not the first content. If true, then some content already encountered but not summed, so we sum it with no depth.
if hasContent:
scoreContent = scoreContent + length
hasContent = False
content = word.split(":",1)[1].strip("\"").replace("\\","")
length = 0.0
bytes = False #0 = char, 1 = bytes
for char in content:
if char is "|":
bytes = not bytes
continue
if bytes:
length = length + 0.5
else:
length = length + 1.0
hasContent = True
elif word.find("depth:") != -1 and hasContent:
hasContent = False
depth = word.split(":")[1]
scoreContent = scoreContent + length + length * length / int(depth)
if hasContent:
scoreContent = scoreContent + length
hasContent = False
# And now the Classtype score. If it is mentioned below, the score will be set to 1
for word in payload:
if word.find("classtype:") != -1:
classtype = word.split(":",1)[1]
if classtype.find("web-application-activity") != -1:
scoreClasstype = 1
elif classtype.find("misc-activity") != -1:
scoreClasstype = 1
elif classtype.find("web-application-attack") != -1:
scoreClasstype = 1
elif classtype.find("not-suspicious") != -1:
scoreClasstype = 1
elif classtype.find("icmp-event") != -1:
scoreClasstype = 1
elif classtype.find("suspicious-filename-detect") != -1:
scoreClasstype = 1
print "score Classtype is", scoreClasstype
# We place the rule in one or the other file depending on the score.
# In this version, the limit is set to 7, so that 3 bytes at specific position are dismissed, but 4 kept.
if totalScore >= 7:
of.write("#"+str(totalScore)+"\n"+line)
else:
dismiss.write("#"+str(totalScore)+"\n"+line)
except Exception:
print "ErrorLine",line
of.close()
dismiss.close()
# Min number of IPs for considering the set when plotting the dependency graph.
# Explained in detail below
MIN_THRESHOLD = 2
# These are the two regexp to match alerts. The first one extracts the SID and message, and the second one the IPs, time and ports.
#
name = dict()
if m is not None:
# At this point, we have the two interesting lines for an alert, and therefore all the necessary data.
port1 = m.group(’port1’)
port2 = m.group(’port2’)
name[sid] = descr
nodes = {}
# This is valid only if there are at least MIN_THRESHOLD IPs. If there are less then we do not plot.
References
[1] “CERNs mission.” http://public.web.cern.ch/public/en/About/Mission-en.html, cited June 2009.
[2] “Intrusion detection system.” http://en.wikipedia.org/wiki/Intrusion detection system, cited June 2009.
[3] “Intrusion Detection Systems (IDS).” http://www.windowsecurity.com/articles/Intrusion Detection Systems
IDS Part I network intrusions attack symptoms IDS tasks and IDS architecture.html, cited June 2009.
[4] “OSI model.” http://en.wikipedia.org/wiki/OSI model, cited August 2009.
[5] Y.-T. Chan, C. Shoniregun, and G. Akmayeva, “A netflow based internet-worm detecting system in large
network,” in Digital Information Management, 2008. ICDIM 2008. Third International Conference on, pp. 581–
586, Nov. 2008.
[6] “Snort homepage.” http://www.snort.org, cited July 2009.
[7] “The Programming Language Lua.” http://www.lua.org/about.html, cited June 2009.
[8] “Snort 3.0 Architecture Series Part 2: Changes and Betas.” http://securitysauce.blogspot.com/2008/08/
snort-30-architecture-series-part-2.html, cited August 2009.
[9] P. Garcia-Teodoroa, J. Diaz-Verdejo, G. Macia-Fernandez, and E. Vazquez, “Anomaly-based network intrusion
detection: Techniques, systems and challenges,” Computers & Security, vol. 28, pp. 18–28, 2009.
[10] T. T. T. Nguyen and G. Armitage, “A survey of techniques for internet traffic classification using machine
learning,” Communications Surveys & Tutorials, IEEE, vol. 10, no. 4, pp. 56–76, 2008.