Improving the performance of model-order selection criteria by partial-model selection search

Abstract

The traditional searching method for model-order selection in
linear regression is a nested full-parameters-set searching procedure
over the desired orders, which we call full-model order
selection. On the other hand, a method for model-selection
searches for the best sub-model within each order. In this paper,
we propose using the model-selection searching method
for model-order selection, which we call partial-model order
selection. We show by simulations that the proposed searching
method gives better accuracies than the traditional one,
especially for low signal-to-noise ratios over a wide range
of model-order selection criteria (both information theoretic based
and bootstrap-based). Also, we show that for some
models the performance of the bootstrap-based criterion improves
significantly by using the proposed partial-model selection
searching method.
Index Terms— Model order estimation, model selection,
information theoretic criteria, bootstrap
1. INTRODUCTION
Several model-order selection criteria can be applied to find
the optimal order. Some of the more commonly used information
theoretic-based procedures include Akaike’s information
criterion (AIC) [1], corrected Akaike (AICc) [2], minimum
description length (MDL) [3], normalized maximum likelihood
(NML) [4], Hannan-Quinn criterion (HQC) [5], conditional
model-order estimation (CME) [6], and the efficient
detection criterion (EDC) [7].
From a practical point of view, it is difficult to decide
which model order selection criterion to use. Many of them
perform reasonably well when the signal-to-noise ratio (SNR)
is high. The discrepancies in their performance, however,
become more evident when the SNR is low. In those situations,
the performance of the given technique is not only
determined by the model structure (say a polynomial trend
versus a Fourier series) but, more importantly, by the relative
values of the parameters within the model. This makes
the comparison between the model-order selection algorithms
difficult as within the same model with a given order one
could find an example for which one of the methods performs
favourably well or fails [6, 8].
Our aim is to improve the performance of the model order
selection criteria in cases where the SNR is low by considering
a model-selection searching procedure that takes into
account not only the full-model order search but also a partial model
order search within the given model order. Understandably,
the improvement in the performance of the model order
estimation is at the expense of additional computational
complexity.