Title:Local signal search enhanced RMT estimator for estimating the number of signals based on random matrix theory

Abstract:Estimating the number of signals is a fundamental problem in many scientific and engineering fields. As a well-known estimator based on the random matrix theory, the RMT estimator estimates the number of signals via detecting the largest noise eigenvalues. Since the RMT estimator considers the distribution of the largest noise eigenvalues but does not consider the distribution of the sample signal eigenvalues, it may down-estimate the number of signals when there are multiple weak signals or when there exist both very strong signals and very weak signals. In order to solve this problem, in this paper we propose a local signal search enhanced RMT estimator by further utilizing the asymptotic distribution of sample signal eigenvalues. In the proposed estimator, a local signal component search algorithm is presented to detect the weak signals that the RMT estimator fails to detect, which is the main contribution of this paper. The main ideas of this local signal component search algorithm are as follows: (a) Firstly, we obtain the sample signal strength by utilizing the asymptotic distribution of the sample signal eigenvalues, and derive its distribution; and (b) Then, based on the distribution of the sample signal strength, a local signal component search algorithm is proposed to further detect the weak signals via sequentially testing whether the sample signal strength is above a threshold. Finally, simulation results show that the proposed local signal search enhanced RMT estimator significantly outperforms the existing estimators including the RMT estimator, the classic AIC and MDL estimators, and the modified AIC estimator, especially for the cases when there are multiple weak signals or when there exist both very strong signals and very weak signals.