Enhanced n-gram extraction using relevance feature discovery

Albathan, Mubarak, Li, Yuefeng, & Algarni, Abdulmohsen (2013) Enhanced n-gram extraction using relevance feature discovery. In Cranefield, Stephen & Nayak, Abhaya (Eds.) Proceedings of the 26th Australasian Joint Conference : AI2013 Advances in Artificial Intelligence, Springer, Dunedin, New Zealand, pp. 453-465.

View at publisher


Guaranteeing the quality of extracted features that describe relevant knowledge to users or topics is a challenge because of the large number of extracted features. Most popular existing term-based feature selection methods suffer from noisy feature extraction, which is irrelevant to the user needs (noisy). One popular method is to extract phrases or n-grams to describe the relevant knowledge. However, extracted n-grams and phrases usually contain a lot of noise. This paper proposes a method for reducing the noise in n-grams. The method first extracts more specific features (terms) to remove noisy features. The method then uses an extended random set to accurately weight n-grams based on their distribution in the documents and their terms distribution in n-grams. The proposed approach not only reduces the number of extracted n-grams but also improves the performance. The experimental results on Reuters Corpus Volume 1 (RCV1) data collection and TREC topics show that the proposed method significantly outperforms the state-of-art methods underpinned by Okapi BM25, tf*idf and Rocchio.

Impact and interest:

0 citations in Scopus
Search Google Scholar™

Citation counts are sourced monthly from Scopus and Web of Science® citation databases.

These databases contain citations from different subsets of available publications and different time periods and thus the citation count from each is usually different. Some works are not in either database and no count is displayed. Scopus includes citations from articles published in 1996 onwards, and Web of Science® generally from 1980 onwards.

Citations counts from the Google Scholar™ indexing service can be viewed at the linked Google Scholar™ search.

ID Code: 67089
Item Type: Conference Paper
Refereed: Yes
Additional Information: Lecture Notes in Computer Science
Keywords: Feature selection, N-gram, Terms weight, Relevance feedback
DOI: 10.1007/978-3-319-03680-9_46
ISBN: 97830319036793
ISSN: 0302-9743
Divisions: Current > Schools > School of Electrical Engineering & Computer Science
Current > QUT Faculties and Divisions > Science & Engineering Faculty
Copyright Owner: Copyright 2013 Springer International Publishing Switzerland
Deposited On: 10 Feb 2014 01:33
Last Modified: 11 Feb 2014 01:42

Export: EndNote | Dublin Core | BibTeX

Repository Staff Only: item control page