Two-Stage Decision Making Policy Using Bayesian Multi-armed Bandit Algorithm for Opportunistic Spectrum Access

Recently, various paradigms, for instance, device-to-device (D2D) communications, LTE-unlicensed and cognitive radio are being envisioned to improve the average spectrum utilization as well as energy efficiency of the decentralized wireless communication networks. Such paradigms are based on an opportunistic spectrum access technique in which secondary (unlicensed) users (SUs) can use the temporarily unoccupied spectrum without any interference to the primary (licensed) users. SUs need decision making policies (DMPs) to identify optimum sub-bands and to minimize collisions with other SUs. Design of such DMP, especially for the decentralized networks, is a challenging problem where there is no communication among SUs and is the motivation behind the work presented in this paper. We have proposed a new two-stage DMP which consists of Bayesian Multiarmed Bandit algorithm for accurate estimation of sub-band statistics (i.e. probability of being vacant) independently at each SU and sub-band access scheme for orthogonalization among SUs. The simulation results indicate that the proposed DMP leads to 45% improvement in terms of average spectrum utilization compared to 36--39% in the existing DMPs. Furthermore, the number of collisions are 58.5 % lower in the proposed DMP making it energy efficient. We also show that sensing errors don't have significant effect on the performance of DMPs.