Article

Fast and memory-efficient regular expression matching for deep packet inspection

Authors:

T. V. Lakshman,

Randy H. KatzAuthors Info & Claims

ANCS '06: Proceedings of the 2006 ACM/IEEE symposium on Architecture for networking and communications systems

Pages 93 - 102

https://doi.org/10.1145/1185347.1185360

Published: 03 December 2006 Publication History

Abstract

Packet content scanning at high speed has become extremely important due to its applications in network security, network monitoring, HTTP load balancing, etc. In content scanning, the packet payload is compared against a set of patterns specified as regular expressions. In this paper, we first show that memory requirements using traditional methods are prohibitively high for many patterns used in packet scanning applications. We then propose regular expression rewrite techniques that can effectively reduce memory usage. Further, we develop a grouping scheme that can strategically compile a set of regular expressions into several engines, resulting in remarkable improvement of regular expression matching speed without much increase in memory usage. We implement a new DFA-based packet scanner using the above techniques. Our experimental results using real-world traffic and patterns show that our implementation achieves a factor of 12 to 42 performance improvement over a commonly used DFA-based scanner. Compared to the state-of-art NFA-based implementation, our DFA-based packet scanner achieves 50 to 700 times speedup.

References

[1]

J. Levandoski, E. Sommer, and M. Strait, "Application Layer Packet Classifier for Linux." http://l7-filter.sourceforge.net/.

[2]

"SNORT Network Intrusion Detection System." http://www.snort.org.

[3]

"Bro Intrusion Detection System." http://bro-ids.org/Overview.html.

[4]

L. Tan and T. Sherwood, "A High Throughput String Matching Architecture for Intrusion Detection and Prevention," Proc. LISA, 2005.

[5]

Y. Cho and W. Mangione-Smith, "Deep packet filter with dedicated logic and read only memories," Proc. FCCM, 2004.

Digital Library

[6]

Z. K. Baker and V. K. Prasanna, "Time and area efficient pattern matching on FPGAs," Proc. FPGAs, 2004.

Digital Library

[7]

Z. K. Baker and V. K. Prasanna, "A methodology for synthesis of efficient intrusion detection systems on FPGAs.," Proc. FCCM, 2004.

Digital Library

[8]

M. Aldwairi, T. Conte, and P. Franzon, "Configurable string matching hardware for speedup up intrusion detection," Proc. WASSA, 2004.

Digital Library

[9]

S. Dharmapurikar, M. Attig, and J. Lockwood, "Deep packet inspection using parallel bloom filters," IEEE Micro, 2004.

Digital Library

[10]

F. Yu, R. H. Katz, and T. V. Lakshman, "Gigabit Rate Packet Pattern Matching with TCAM," Proc. ICNP, 2004.

Digital Library

[11]

Y. H. Cho and W. H. MangioneSmith, "A Pattern Matching Coprocessor for Network Security," Proc. DAC, 2005.

Digital Library

[12]

T. J. Green, A. Gupta, G. Miklau, M. Onizuka, and D. Suciu, "Processing XML Streams with Deterministic Automata and Stream Indexes," ACM TODS, vol. 29, 2004.

Digital Library

[13]

Y. Diao, M. Altinel, M. J. Franklin, H. Zhang, and P. Fischer, "Path Sharing and Predicate Evaluation for High-Performance XML Filtering," ACM TODS, 2003.

Digital Library

[14]

J. E. Hopcroft, R. Motwani, and J. D. Ullman, Introduction to Automata Theory, Languages, and Computation, Addison Wesley, 2001.

Digital Library

[15]

R. Sommer and V. Paxson, "Enhancing Byte-Level Network Intrusion Detection Signatures with Context," Proc. CCS, 2003.

Digital Library

[16]

J. Moscola, J. Lockwood, R. P. Loui, and Michael Pachos, "Implementation of a Content-Scanning Module for an Internet Firewall," Proc. FCCM, 2003.

Digital Library

[17]

R. Sidhu and V. K. Prasanna, "Fast regular expression matching using FPGAs," Proc. FCCM, 2001.

Digital Library

[18]

R. Franklin, D. Carver, and B. Hutchings, "Assisting network intrusion detection with reconfigurable hardware," Proc. FCCM, 2002.

Digital Library

[19]

C. R. Clark and D. E. Schimmel, "Scalable pattern matching for high speed networks," Proc FCCM, 2004.

Digital Library

[20]

S. Kumar, S. Dharmapurikar, F. Yu, P. Crowley, J. Turner., "Algorithms to accelerate Multiple Regular Expression Matching for Deep Packet Inspection," ACM Sigcomm 2006.

Digital Library

[21]

"Standard for Information Technology, Portable Operating System Interface (POSIX)," Portable Applications Standards Committee of IEEE Computer Society and the Open Group.

[22]

C. L. A. Clarke and G. V. Cormack, "On the use of regular expressions for searching text," Technical Report CS-95-07, Department of Computer Science, University of Waterloo, 1995.

[23]

J. A. Kahle, M. N. Day, H. P. Hofstee, C. R. Johns, T. R. Maeurer, and D. Shippy, "Introduction to the Cell multiprocessor," IBM J. RES. & DEV., vol. 49, JULY/SEPTEMBER 2005.

Digital Library

[24]

"MIT DARPA Intrusion Detection Data Sets." http://www.ll.mit.edu/IST/ideval/data/2000/2000_data_index.html.

[25]

V. Paxson et al., "Flex: A fast scanner generator." http://www.gnu.org/software/flex/.

[26]

Perl compatible Regular Expression, http://www.pcre.org/

[27]

F. Yu, Z. Chen, Y. Diao, T. V. Lakshman and R. H. Katz, "Fast and Memory-Efficient Regular Expression Matching for Deep Packet Inspection," UC Berkeley technical report, May 2006.

[28]

B. C. Brodie, R. K. Cytron, and D. E. Taylor, "A Scalable Architecture for High-Throughput Regular-Expression Pattern Matching," ISCA 2006.

Digital Library

Cited By

Mamouras KLe Glaunec ALi WChattopadhyay A(2024)Static Analysis for Checking the Disambiguation Robustness of Regular ExpressionsProceedings of the ACM on Programming Languages10.1145/36564618:PLDI(2073-2097)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3656461
Mamouras KChattopadhyay A(2024)Efficient Matching of Regular Expressions with Lookaround AssertionsProceedings of the ACM on Programming Languages10.1145/36329348:POPL(2761-2791)Online publication date: 5-Jan-2024
https://dl.acm.org/doi/10.1145/3632934
Wen ZKong LLe Glaunec AMamouras KYang KTsafrir DMusuvathi MGupta RAbu-Ghazaleh N(2024)BVAP: Energy and Memory Efficient Automata Processing for Regular Expressions with Bounded RepetitionsProceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 210.1145/3620665.3640412(151-166)Online publication date: 27-Apr-2024
https://dl.acm.org/doi/10.1145/3620665.3640412
Show More Cited By

Index Terms

Fast and memory-efficient regular expression matching for deep packet inspection
1. Security and privacy
  1. Network security

Recommendations

Algorithms to accelerate multiple regular expressions matching for deep packet inspection
Proceedings of the 2006 conference on Applications, technologies, architectures, and protocols for computer communications

There is a growing demand for network devices capable of examining the content of data packets in order to improve network security and provide application-specific services. Most high performance systems that perform deep packet inspection implement ...
An improved DFA for fast regular expression matching

Modern network devices need to perform deep packet inspection at high speed for security and application-specific services. Finite Automata (FAs) are used to implement regular expressions matching, but they require a large amount of memory. Many recent ...
Efficient regular expression pattern matching for network intrusion detection systems using modified word-based automata
SIN '12: Proceedings of the Fifth International Conference on Security of Information and Networks

Network Intrusion Detection Systems (NIDS) intercept the traffic at an organization's network periphery to thwart intrusion attempts. Signature-based NIDS compares the intercepted packets against its database of known vulnerabilities and malware ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ANCS '06: Proceedings of the 2006 ACM/IEEE symposium on Architecture for networking and communications systems

December 2006

202 pages

ISBN:1595935800

DOI:10.1145/1185347

General Chair:
Laxmi N. Bhuyan
University of California, Riverside, USA
,
Program Chairs:
Michel Dubois
University of Southern California, USA
,
Will Eatherton
Cisco, USA

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 December 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ANCS06

Sponsor:

ANCS06: Symposium on Architecture for Networking and Communications Systems

December 3 - 5, 2006

California, San Jose, USA

Acceptance Rates

Overall Acceptance Rate 88 of 314 submissions, 28%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

359
Total Citations
View Citations
2,278
Total Downloads

Downloads (Last 12 months)37
Downloads (Last 6 weeks)5

Reflects downloads up to 12 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mamouras KLe Glaunec ALi WChattopadhyay A(2024)Static Analysis for Checking the Disambiguation Robustness of Regular ExpressionsProceedings of the ACM on Programming Languages10.1145/36564618:PLDI(2073-2097)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3656461
Mamouras KChattopadhyay A(2024)Efficient Matching of Regular Expressions with Lookaround AssertionsProceedings of the ACM on Programming Languages10.1145/36329348:POPL(2761-2791)Online publication date: 5-Jan-2024
https://dl.acm.org/doi/10.1145/3632934
Wen ZKong LLe Glaunec AMamouras KYang KTsafrir DMusuvathi MGupta RAbu-Ghazaleh N(2024)BVAP: Energy and Memory Efficient Automata Processing for Regular Expressions with Bounded RepetitionsProceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 210.1145/3620665.3640412(151-166)Online publication date: 27-Apr-2024
https://dl.acm.org/doi/10.1145/3620665.3640412
Lin JLin WLin HZhu LZhang DWu C(2024)P4Rex: Accelerating regular expression matching with programmable switchesComputer Networks10.1016/j.comnet.2024.110662252(110662)Online publication date: Oct-2024
https://doi.org/10.1016/j.comnet.2024.110662
Zhang LDeep SFloratou AGruenheid APatel JZhu Y(2023)Exploiting Structure in Regular Expression QueriesProceedings of the ACM on Management of Data10.1145/35892971:2(1-28)Online publication date: 20-Jun-2023
https://dl.acm.org/doi/10.1145/3589297
Le Glaunec AKong LMamouras K(2023)Regular Expression Matching using Bit Vector AutomataProceedings of the ACM on Programming Languages10.1145/35860447:OOPSLA1(492-521)Online publication date: 6-Apr-2023
https://dl.acm.org/doi/10.1145/3586044
Wang XGong LCao JLou WWang WWang CZhou XIenne PZhang Z(2023)hAP: A Spatial-von Neumann Heterogeneous Automata Processor with Optimized Resource and IO Overhead on FPGAProceedings of the 2023 ACM/SIGDA International Symposium on Field Programmable Gate Arrays10.1145/3543622.3573190(185-196)Online publication date: 12-Feb-2023
https://dl.acm.org/doi/10.1145/3543622.3573190
Li ZChen SDai HXu DChu CXiao B(2023)Abnormal Traffic Detection: Traffic Feature Extraction and DAE-GAN With Efficient Data AugmentationIEEE Transactions on Reliability10.1109/TR.2022.320434972:2(498-510)Online publication date: Jun-2023
https://doi.org/10.1109/TR.2022.3204349
Wang SZhang MLi GLiu CWang ZLiu YXu M(2023)Bolt: Scalable and Cost-Efficient Multistring Pattern Matching With Programmable SwitchesIEEE/ACM Transactions on Networking10.1109/TNET.2022.320252331:2(846-861)Online publication date: Apr-2023
https://doi.org/10.1109/TNET.2022.3202523
Qiu TYang XWang BZong CZhu RXia X(2023)Efficient Index-Based Regular Expression Matching with Optimal Query Plan TreeDatabase Systems for Advanced Applications10.1007/978-3-031-30637-2_3(35-45)Online publication date: 14-Apr-2023
https://doi.org/10.1007/978-3-031-30637-2_3
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents