Model order selection (MOS) in linear regression models is a widely studied problem in signal processing. Penalized log likelihood techniques based on information theoretic criteria (ITC) are algorithms of choice in MOS problems. Recently, a number of model selection problems have been successfully solved with explicit finite sample guarantees using a concept called residual ratio thresholding (RRT). This paper proposes to use RRT for MOS in linear regression models and provide rigorous mathematical analysis of RRT. RRT is numerically shown to deliver a highly competitive performance when compared to popular MOS criteria, such as Akaike information criterion, Bayesian information criterion, and penalized adaptive likelihood, especially when the sample size is small. We also analytically establish an interesting interpretation for RRT based on ITC thereby linking these two model selection principles. © 2018 IEEE.

Sheetal Kalyani

Department of Electrical Engineering

Covariance matrix

Linear Regression

Mathematical models

Numerical models

Sampling

signal processing

ADAPTATION MODELS

Consistency

Covariance matrices

Indexes

INFORMATION CRITERION

MODEL-ORDER SELECTION

Signal to noise ratio

Fulltext

IIT Madras is a public technical and research university located in Chennai, Tamil Nadu. Founded in 1959, it is recognised as an Institute of National Importance.

IIT Madras has been ranked as the top engineering institute in India for four years in a row by the National Institutional Ranking Framework of the MHRD

It currently offers undergraduate, postgraduate and research degrees across 16 disciplines in Engineering, Sciences, Humanities and Management. About 596 faculty belonging to science and engineering departments and centres of the Institute are engaged in teaching, research and industrial consultancy.

IIT Madras

Residual Ratio Thresholding for Linear Model Order Selection

IEEE Transactions on Signal Processing

Recovering the support of sparse vectors in underdetermined linear regression models, aka, compressive sensing is important in many signal processing applications. High SNR consistency (HSC), i.e., the ability of a support recovery technique to correctly identify the support with increasing signal to noise ratio (SNR) is an increasingly popular criterion to qualify the high SNR optimality of support recovery techniques. The HSC results available in literature for support recovery techniques applicable to underdetermined linear regression models like least absolute shrinkage and selection operator (LASSO), orthogonal matching pursuit (OMP) etc. assume a priori knowledge of noise variance or signal sparsity. However, both these parameters are unavailable in most practical applications. Further, it is extremely difficult to estimate noise variance or signal sparsity in underdetermined regression models. This limits the utility of existing HSC results. In this article, we propose two techniques, viz., residual ratio minimization (RRM) and residual ratio thresholding with adaptation (RRTA) to operate OMP algorithm without the a priroi knowledge of noise variance and signal sparsity and establish their HSC analytically and numerically. To the best of our knowledge, these are the first and only noise statistics oblivious algorithms to report HSC in underdetermined regression models. © 2019 Elsevier B.V.

Signal Processing

High SNR consistent compressive sensing without signal and noise statistics

Linear regression models contaminated by Gaussian noise (inlier) and possibly unbounded sparse outliers are common in many signal processing applications. Sparse recovery inspired robust regression (SRIRR) techniques are shown to deliver high-quality estimation performance in such regression models. Unfortunately, most SRIRR techniques assume a priori knowledge of noise statistics like inlier noise variance or outlier statistics like number of outliers. Both inlier and outlier noise statistics are rarely known a priori, and this limits the efficient operation of many SRIRR algorithms. This paper proposes a novel noise statistics oblivious algorithm called residual ratio thresholding GARD (RRT-GARD) for robust regression in the presence of sparse outliers. RRT-GARD is developed by modifying the recently proposed noise statistics dependent greedy algorithm for robust denoising (GARD). Both finite sample and asymptotic analytical results indicate that RRT-GARD performs nearly similar to GARD with a priori knowledge of noise statistics. Numerical simulations in real and synthetic data sets also point to the highly competitive performance of RRT-GARD. © 1991-2012 IEEE.

Noise Statistics Oblivious GARD for Robust Regression with Sparse Outliers

High SNR consistency of model order selection criteria in linear regression models has attracted a lot of attention in the signal processing community recently. It is now known that Exponentially Embedded Family, versions of Minimum Description Length etc. are high SNR consistent. However, a general framework for high SNR consistency in linear regression is still missing. This paper fills this gap by deriving necessary and sufficient conditions (NSCs) for model order selection criteria in both known and unknown noise variance situations to be high SNR consistent. Many popular model order selection criteria are proved to be high SNR consistent using these NSCs. We also provide a convergence rate analysis to discriminate between various high SNR consistent model order selection criteria. A direct application of model order selection techniques to the very important subset selection problem is computationally infeasible. This paper establish the high SNR consistency of an existing t-statistics based index ordering scheme that allows the conversion of a subset selection problem into a model order selection problem. Combining this index ordering with high SNR consistent model order selection criteria leads to a novel subset selection procedure (SSP) that is numerically shown to outperform existing high SNR consistent SSPs. © 1991-2012 IEEE.

High SNR consistent linear model order selection and subset selection

We describe an algorithm for sparse channel estimation applicable to orthogonal frequency division multiplexing systems. The proposed algorithm uses a least squares (LS) technique for channel estimation and a generalized Akaike information criterion to estimate the channel length and tap positions. This effectively reduces the signal space of the LS estimator, and hence improves the estimation performance as demonstrated using computer simulations. For example, the proposed modified LS with sparse channel-estimation algorithm has a 5-dB lower mean square error in channel estimation when compared to the conventional approach [1], which translates to approximately 0.5 dB improvement in signal-to-noise ratio at the receiver. © 2005 IEEE.

IEEE Signal Processing Letters

Improving channel estimation in OFDM systems for sparse multipath channels

A First Course in Linear Model Theory

Probability Density Function Estimation Using the EEF With Application to Subset/Feature Selection

Journal	Data powered by TypesetIEEE Transactions on Signal Processing
Publisher	Data powered by TypesetInstitute of Electrical and Electronics Engineers Inc.
ISSN	1053587X
Open Access	Yes