Automated feature engineering for HTTP tunnel detection

Davis, Jonathan J. & Foo, Ernest (2016) Automated feature engineering for HTTP tunnel detection. Computers & Security, 59, pp. 166-185.

[img] Accepted Version (PDF 2MB)
Administrators only until May 2018 | Request a copy from author

View at publisher


Generating discriminative input features is a key requirement for achieving highly accurate classifiers. The process of generating features from raw data is known as feature engineering and it can take significant manual effort. In this paper we propose automated feature engineering to derive a suite of additional features from a given set of basic features with the aim of both improving classifier accuracy through discriminative features, and to assist data scientists through automation. Our implementation is specific to HTTP computer network traffic. To measure the effectiveness of our proposal, we compare the performance of a supervised machine learning classifier built with automated feature engineering versus one using human-guided features. The classifier addresses a problem in computer network security, namely the detection of HTTP tunnels. We use Bro to process network traffic into base features and then apply automated feature engineering to calculate a larger set of derived features. The derived features are calculated without favour to any base feature and include entropy, length and N-grams for all string features, and counts and averages over time for all numeric features. Feature selection is then used to find the most relevant subset of these features. Testing showed that both classifiers achieved a detection rate above 99.93% at a false positive rate below 0.01%. For our datasets, we conclude that automated feature engineering can provide the advantages of increasing classifier development speed and reducing development technical difficulties through the removal of manual feature engineering. These are achieved while also maintaining classification accuracy.

Impact and interest:

0 citations in Scopus
Search Google Scholar™

Citation counts are sourced monthly from Scopus and Web of Science® citation databases.

These databases contain citations from different subsets of available publications and different time periods and thus the citation count from each is usually different. Some works are not in either database and no count is displayed. Scopus includes citations from articles published in 1996 onwards, and Web of Science® generally from 1980 onwards.

Citations counts from the Google Scholar™ indexing service can be viewed at the linked Google Scholar™ search.

ID Code: 94291
Item Type: Journal Article
Refereed: Yes
Keywords: Feature engineering, HTTP, Tunnel detection, Supervised machine learning, Bro
DOI: 10.1016/j.cose.2016.01.006
ISSN: 0167-4048
Subjects: Australian and New Zealand Standard Research Classification > INFORMATION AND COMPUTING SCIENCES (080000) > COMPUTER SOFTWARE (080300) > Computer System Security (080303)
Australian and New Zealand Standard Research Classification > INFORMATION AND COMPUTING SCIENCES (080000) > DISTRIBUTED COMPUTING (080500) > Networking and Communications (080503)
Divisions: Current > QUT Faculties and Divisions > Science & Engineering Faculty
Copyright Owner: Crown copyright 2016
Deposited On: 29 Mar 2016 23:33
Last Modified: 03 Apr 2016 04:40

Export: EndNote | Dublin Core | BibTeX

Repository Staff Only: item control page