Enhanced n-gram extraction using relevance feature discovery
Albathan, Mubarak, Li, Yuefeng, & Algarni, Abdulmohsen (2013) Enhanced n-gram extraction using relevance feature discovery. In Cranefield, Stephen & Nayak, Abhaya (Eds.) Proceedings of the 26th Australasian Joint Conference : AI2013 Advances in Artificial Intelligence, Springer, Dunedin, New Zealand, pp. 453-465.
Guaranteeing the quality of extracted features that describe relevant knowledge to users or topics is a challenge because of the large number of extracted features. Most popular existing term-based feature selection methods suffer from noisy feature extraction, which is irrelevant to the user needs (noisy). One popular method is to extract phrases or n-grams to describe the relevant knowledge. However, extracted n-grams and phrases usually contain a lot of noise. This paper proposes a method for reducing the noise in n-grams. The method first extracts more specific features (terms) to remove noisy features. The method then uses an extended random set to accurately weight n-grams based on their distribution in the documents and their terms distribution in n-grams. The proposed approach not only reduces the number of extracted n-grams but also improves the performance. The experimental results on Reuters Corpus Volume 1 (RCV1) data collection and TREC topics show that the proposed method significantly outperforms the state-of-art methods underpinned by Okapi BM25, tf*idf and Rocchio.
Impact and interest:
Citation counts are sourced monthly from and citation databases.
These databases contain citations from different subsets of available publications and different time periods and thus the citation count from each is usually different. Some works are not in either database and no count is displayed. Scopus includes citations from articles published in 1996 onwards, and Web of Science® generally from 1980 onwards.
Citations counts from theindexing service can be viewed at the linked Google Scholar™ search.
|Item Type:||Conference Paper|
|Additional Information:||Lecture Notes in Computer Science|
|Keywords:||Feature selection, N-gram, Terms weight, Relevance feedback|
|Divisions:||Current > Schools > School of Electrical Engineering & Computer Science
Current > QUT Faculties and Divisions > Science & Engineering Faculty
|Copyright Owner:||Copyright 2013 Springer International Publishing Switzerland|
|Deposited On:||10 Feb 2014 01:33|
|Last Modified:||11 Feb 2014 01:42|
Repository Staff Only: item control page