Detecting network-based obfuscated code injection attacks using sandboxing

Andersson, Stig, Clark, Andrew J., & Mohay, George M. (2005) Detecting network-based obfuscated code injection attacks using sandboxing. In Clark, Andrew J., Kerr, Kathryn, & Mohay, George M. (Eds.) AusCERT Asia Pacific Information Technology Security Conference : Refereed R&D Stream, 22-26 May 2005, Gold Coast, Queensland.

View at publisher


Intrusion detection systems (IDSs) are widely recognised as the last line of defence often used to enable incident response when intrusion prevention mechanisms are ineffective, or have been compromised. A signature based network IDS (NIDS) which operates by comparing network traffic to a database of suspicious activity patterns (known as signatures) is a popular solution due to its ease of deployment and relatively low false positive (incorrect alert) rate. Lately, attack developers have focused on developing stealthy attacks designed to evade NIDS. One technique used to accomplish this is to obfuscate the shellcode (the executable component of an attack) so that it does not resemble the signatures the IDS uses to identify the attacks but is still logically equivalent to the clear-text attacks when executed. We present an approach to detect obfuscated code injection attacks, an approach which compensates for efforts to evade IDSs. This is achieved by executing those network traffic segments that are judged potentially to contain executable code and monitoring the execution to detect operating system calls which are a necessary component of any such code. This detection method is based not on how the injected code is represented but rather on the actions it performs. Correct configuration of the IDS at deployment time is crucial for correct operation when this approach is taken, in particular, the examined executable code must be executed in an environment identical to the execution environment of the host the IDS is monitoring with regards to both operating system and architecture. We have implemented a prototype detector that is capable of detecting obfuscated shellcodes in a Linux environment, and demonstrate how it can be used to detect new or previously unseen code injection attacks and obfuscated attacks as well as well known attacks.

Impact and interest:

Search Google Scholar™

Citation counts are sourced monthly from Scopus and Web of Science® citation databases.

These databases contain citations from different subsets of available publications and different time periods and thus the citation count from each is usually different. Some works are not in either database and no count is displayed. Scopus includes citations from articles published in 1996 onwards, and Web of Science® generally from 1980 onwards.

Citations counts from the Google Scholar™ indexing service can be viewed at the linked Google Scholar™ search.

Full-text downloads:

403 since deposited on 14 Jun 2009
36 in the past twelve months

Full-text downloads displays the total number of times this work’s files (e.g., a PDF) have been downloaded from QUT ePrints as well as the number of downloads in the previous 365 days. The count includes downloads for all files if a work has more than one.

ID Code: 21168
Item Type: Conference Paper
Refereed: Yes
Additional URLs:
Keywords: intrusion detection, code injection attack
ISBN: 1864997990
Subjects: Australian and New Zealand Standard Research Classification > INFORMATION AND COMPUTING SCIENCES (080000) > COMPUTER SOFTWARE (080300) > Computer System Security (080303)
Divisions: Past > QUT Faculties & Divisions > Faculty of Science and Technology
Past > Institutes > Information Security Institute
Copyright Owner: Copyright 2005 [please consult the authors]
Deposited On: 14 Jun 2009 23:37
Last Modified: 29 Feb 2012 13:13

Export: EndNote | Dublin Core | BibTeX

Repository Staff Only: item control page