Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

A deep reinforcement learning-based D2D spectrum allocation underlaying a cellular network

  • Original Paper
  • Published:
Wireless Networks Aims and scope Submit manuscript

Abstract

We develop a deep reinforcement learning-based (DRL) spectrum access scheme for device-to-device communications in an underlay cellular network. Based on the DRL scheme, the base station aims to maximize the overall system throughput of both the D2D and cellular communications by learning an optimal spectrum allocation strategy. While D2D pairs dynamically access the time slots (TSs) of a shared spectrum belonging to a dedicated cellular user (CU). In particular, to ensure that the quality of service (QoS) requirement of cell-edge CUs, this paper addresses the various positions of CUs and D2D pairs by dividing the cellular area into shareable and un-shareable areas. Then, a double deep Q-network is adopted for the BS to decide whether and which D2D pair can access each TS within a shared spectrum. The proposed DDQN spectrum allocation not only enjoys low computational complexity since just current state information is utilized as input, but also approaches the throughput of exhaustive search method since received signal-to-noise ratios are utilized as inputs. Numerical results show that the proposed deep learning-based spectrum access scheme outperforms the state-of-art algorithms in terms of throughput.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. The transmit powers of both the D2D and cellular users also affect the D2D-CU pairing, while this issue is not considered here but left for a future topic.

  2. The extension to the case of multiple D2D pairs is left for further research.

  3. The number of neurons of the hidden layer should be greater than the number of inputs to the DQN to prevent information loss during training, however the optimal tradeoff between the number of neurons and the computational complexity is beyond the scope of this work.

References

  1. Cotton, D., & Chaczko, Z. (2021). Gymd2d: A device-to-device underlay cellular offload evaluation platform. In IEEE wireless communications and networking conference (WCNC), 2021, 1–7.

  2. Liang, L., Li, G. Y., & Xu, W. (2017). Resource allocation for D2D-enabled vehicular communications. IEEE Transactions on Communications, 65, 3186–3197.

    Article  MATH  Google Scholar 

  3. Li, Z., & Guo, C. (2019). Multi-agent deep reinforcement learning based spectrum allocation for D2D underlay communications. IEEE Transactions on Vehicular Technology, 69(2), 1828–1840.

    Article  MATH  Google Scholar 

  4. Najla, M., Becvar, Z., & Mach, P. (2021). Reuse of multiple channels by multiple d2d pairs in dedicated mode: A game theoretic approach. IEEE Transactions on Wireless Communications., 20, 4313–4327.

    Article  MATH  Google Scholar 

  5. Kai, C., Wu, Y., Peng, M., & Huang, W. (2021). Joint uplink and downlink resource allocation for NOMA-enabled D2D communications. IEEE Wireless Communications Letters, 10, 1247–1251.

    Article  MATH  Google Scholar 

  6. Van Hasselt, H., Guez, A., & Silver, D. (2016). Deep reinforcement learning with double q-learning. Proceedings of the AAAI Conference on Artificial Intelligence, 30(1), 2094–2100.

    MATH  Google Scholar 

  7. Wang, D., Qin, H., Song, B., Du, X., & Guizani, M. (2019). Resource allocation in information-centric wireless networking with D2D-enabled MEC: A deep reinforcement learning approach. IEEE Access, 7, 114935–114944.

    Article  Google Scholar 

  8. Vu, H. V., Liu, Z., Nguyen, D. H. N., Morawski, R., & Le-Ngoc, T. (2020). Multi-agent reinforcement learning for joint channel assignment and power allocation in platoon-based c-v2x systems. arXiv:2011.04555.

  9. Huang, J., Yang, Y., He, G., Xiao, Y., & Liu, J. (2021). Deep reinforcement learning-based dynamic spectrum access for d2d communication underlay cellular networks. IEEE Communications Letters, 25(8), 2614–2618.

    Article  MATH  Google Scholar 

  10. Ji, Z., Qin, Z., & Parini, C. G. (2022). Reconfigurable intelligent surface aided cellular networks with device-to-device users. IEEE Transactions on Communications, 70, 1808–1819.

    Article  MATH  Google Scholar 

  11. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013). Playing Atari with deep reinforcement learning, pp. 1–9. arXiv:1312.5602

  12. Liang, Y.-J., & Lin, Y.-S. (2017). A non-iterative resource allocation strategy for device-to-device communications in underlaying cellular networks. Wireless Network, 23, 2485–2497.

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electronic Engineering, National Ilan University, Yilan, Taiwan

    Yao-Jen Liang

  2. Department of Electrical Engineering, National Chiayi University, Chiayi, Taiwan

    Yu-Chan Tseng

  3. Department of Electrical Engineering, National Chung Cheng University, Chiayi, Taiwan

    Chi-Wen Hsieh

Authors
  1. Yao-Jen Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yu-Chan Tseng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chi-Wen Hsieh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yao-Jen Liang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liang, YJ., Tseng, YC. & Hsieh, CW. A deep reinforcement learning-based D2D spectrum allocation underlaying a cellular network. Wireless Netw 31, 435–441 (2025). https://doi.org/10.1007/s11276-024-03766-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11276-024-03766-6

Keywords

Search

Navigation