Abstract
•The asymptotic distribution of the projector associated with the SCM is given for arbitrary distributions of the useful signal and noise with finite fourth-order moments.•The asymptotic distribution of the DOA estimate given by the MUSIC algorithm is proved for both nonparametric non-Gaussian noise and parametric CES noise models. Theoretical evaluation of the loss of performance is given for heavy-tailed distributions of the noise which is significant for weak SNR and closely spaced sources.•For CES distributed observations, the asymptotic distribution of the projector associated with any M-estimate of the covariance matrix is given. This allows us to prove that this loss of performance can be alleviated.•These theoretical results are confirmed by some simulations performed with either a complex circular Student t-distribution or a complex generalized Gaussian distribution of the noise or the observations model.
This paper addresses the theoretical analysis of the robustness of subspace-based algorithms with respect to non-Gaussian noise distributions using perturbation expansions. Its purpose is twofold. It aims, first, to derive the asymptotic distribution of the estimated projector matrix obtained from the sample covariance matrix (SCM) for arbitrary distributions of the useful signal and the noise. It proves that this distribution depends only of the second-order statistics of the useful signal, but also on the second and fourth-order statistics of the noise. Second, it derives the asymptotic distribution of the estimated projector matrix obtained from any M-estimate of the covariance matrix for both real (RES) and complex elliptical symmetric (CES) distributed observations. Applied to the MUSIC algorithm for direction-of-arrival (DOA) estimation, these theoretical results allow us to theoretically evaluate the performance loss of this algorithm for heavy-tailed noise distributions when it is based on the SCM, which is significant for weak signal-to-noise ratio (SNR) or closely spaced sources. These results also make it possible to prove that this performance loss can be alleviated by replacing the SCM by an M-estimate of the covariance for CES distributed observations, which has been observed until now only by numerical experiments.