Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Iris: Deep Reinforcement Learning Driven Shared Spectrum Access Architecture for Indoor Neutral-Host Small Cells

Published: 01 August 2019 Publication History

Abstract

We consider indoor mobile access, a vital use case for current and future mobile networks. For this key use case, we outline a vision that combines a neutral-host-based shared small-cell infrastructure with a common pool of spectrum for dynamic sharing as a way forward to proliferate indoor small-cell deployments and open up the mobile operator ecosystem. Toward this vision, we focus on the challenges pertaining to managing access to shared spectrum [e.g., 3.5-GHz U.S. Citizen Broadband Radio Service (CBRS) spectrum]. We propose <italic>Iris</italic>, a practical shared spectrum access architecture for indoor neutral-host small-cells. At the core of <italic>Iris</italic> is a deep reinforcement learning-based dynamic pricing mechanism that efficiently mediates access to shared spectrum for diverse operators in a way that provides incentives for operators and the neutral-host alike. We then present the <italic>Iris</italic> system architecture that embeds this dynamic pricing mechanism alongside cloud-RAN and RAN slicing design principles in a practical neutral-host design tailored for the indoor small-cell environment. Using a prototype implementation of the <italic>Iris</italic> system, we present the extensive experimental evaluation results that not only offer insight into the <italic>Iris</italic> dynamic pricing process and its superiority over alternative approaches but also demonstrate its deployment feasibility.

References

[1]
Vision 5G: Thriving Indoors, Cisco, San Jose, CA, USA, Feb. 2017.
[2]
Small Cell Indoor Coverage Solutions, Analysys Mason, London, U.K., Mar. 2016.
[3]
DAS Deployment Overview: Streamlined Approaches for DAS Deployments, Viavi Solutions, Milpitas, CA, USA, Mar. 2017.
[4]
Multi-carrier small cell solutions for in-building wireless,” Wireless 20|20, White Paper, Feb. 2017. [Online]. Available: https://www.ipaccess.com/uploads/wysiwyg_editor/files/2017/Multi-Carrier-Small-Cell-Solutions.pdf
[5]
J. Zander and P. Mähönen, “Riding the data tsunami in the cloud: Myths and challenges in future wireless access,” IEEE Commun. Mag., vol. 51, no. 3, pp. 145–151, 2013.
[6]
Multi-operator and neutral host small cells: Drivers, architecture, planning and regulation,” 5G Americas, White Paper, Dec. 2016. [Online]. Available: http://www.5gamericas.org/files/4914/8193/1104/SCF191_Multi-operator_neutral_host_small_cells.pdf
[7]
Network Sharing Makes Sense in-Building, not Outdoors, says AT&T. Accessed: Aug. 2, 2018. [Online]. Available: https://enterpriseiotinsights.com/20180523/channels/news/network-sharing-in-building-tag40
[8]
Small Cell ‘Neutral Hosting’ is it the Future?. Accessed: Aug. 2, 2018. [Online]. Available: https://tinyurl.com/ya6g2fpy
[9]
Dense Air. Accessed: Aug. 2, 2018. [Online]. Available: http://denseair.net/
[10]
IP.access Viper. Accessed: Aug. 2, 2018. [Online]. Available: https://tinyurl.com/ydabffvc
[11]
Baicells NeutralCell. Accessed: Aug. 2, 2018. [Online]. Available: http://www.baicells.com/neutralcell.html
[12]
Accessed: Aug. 2, 2018. Crown Castle Indoor Small Cells. [Online]. Available: https://tinyurl.com/y93vl78s
[13]
I. Giannoulakiset al., “The emergence of operator-neutral small cells as a strong case for cloud computing at the mobile edge,” Trans. Emerg. Telecommun. Technol., vol. 27, no. 9, pp. 1152–1159, 2016.
[14]
M. Matinmikko, M. Latva-Aho, P. Ahokangas, S. Yrjölä, and T. Koivumäki, “Micro operators to boost local service delivery in 5G,” Wireless Pers. Commun. J., vol. 95, no. 1, pp. 69–82, May 2017.
[15]
P. Ahokangaset al., “Future micro operators business models in 5G,” Bus. Manage. Rev., vol. 7, no. 5, p. 143, 2016.
[16]
Making neutral host a reality with OnGo,” Mobile Experts, CBRS Alliance, White Paper, Dec. 2018. [Online]. Available: https://www.cbrsalliance.org/wp-content/uploads/2018/12/ME-Neutral-Host-with-OnGo-WP-Final.pdf
[17]
P. Ahokangaset al., “Business models for local 5G micro operators,” in Proc. IEEE Int. Symp. Dynamic Spectr. Access Netw. (DYSPAN), Oct. 2018, pp. 1–8.
[18]
B. Nguyenet al., “ECHO: A reliable distributed cellular core network for hyper-scale public clouds,” in Proc. 24th ACM MobiCom, 2018, pp. 163–178.
[19]
Neutral host solutions for multi-operator wireless coverage in managed spaces,” ATIS, White Paper, Sep. 2016. [Online]. Available: https://access.atis.org/apps/group_public/download.php/31137/ATIS-I-0000052.pdf
[20]
Bringing the Sharing Economy to the Airwaves Will Boost Your Bandwidth. Accessed: Aug. 2, 2018. [Online]. Available: https://tinyurl.com/y9vsvsue
[21]
FCC Rule Making on 3.5 GHz Band/Citizens Broadband Radio Service, FCC, Washington, DC, USA, Apr. 2015.
[22]
3.8 GHz to 4.2 GHz Band: Opportunities for Innovation, Ofcom, London, U.K., Apr. 2016.
[23]
CBRS: New shared spectrum enables flexible indoor and outdoor mobile solutions and new business models,” Mobile Experts, White Paper, Mar. 2017. [Online]. Available: https://www.federatedwireless.com/wp-content/uploads/2017/09/Mobile-Experts-CBRS-Overview.pdf
[24]
M. Matinmikko, H. Okkonen, M. Palola, S. Yrjola, P. Ahokangas, and M. Mustonen, “Spectrum sharing using licensed shared access: The concept and its workflow for LTE-advanced networks,” IEEE Wireless Commun., vol. 21, no. 2, pp. 72–79, Apr. 2014.
[25]
M. Matinmikko-Blue, S. Yrjölä, V. Seppänen, P. Ahokangas, H. Hämmäinen, and M. Latva-Aho, “Analysis of spectrum valuation approaches: The viewpoint of local 5G networks in shared spectrum bands,” in Proc. IEEE Int. Symp. Dyn. Spectr. Access Netw. (DYSPAN), Oct. 2018, pp. 1–9.
[26]
F. P. Kelly, A. K. Maulloo, and D. K. H. Tan, “Rate control for communication networks: Shadow prices, proportional fairness and stability,” J. Oper. Res. Soc., vol. 49, no. 3, pp. 237–252, Mar. 1998.
[27]
S. Sen, C. Joe-Wong, S. Ha, and M. Chiang, “Smart data pricing: Using economics to manage network congestion,” Commun. ACM, vol. 58, no. 12, pp. 86–93, Nov. 2015.
[28]
N. Li, L. Chen, and S. H. Low, “Optimal demand response based on utility maximization in power networks,” in Proc. IEEE Power Energy Soc. Gen. Meeting, Jul. 2011, pp. 1–8.
[29]
Z. Liu, I. Liu, S. Low, and A. Wierman, “Pricing data center demand response,” ACM SIGMETRICS Perform. Eval. Rev., vol. 42, no. 1, pp. 111–123, 2014.
[30]
C.-L. I, J. Huang, R. Duan, C. Cui, J. Jiang, and L. Li, “Recent progress on C-RAN centralization and cloudification,” IEEE Access, vol. 2, pp. 1030–1039, 2014.
[31]
V. Sciancalepore, K. Samdanis, X. Costa-Perez, D. Bega, M. Gramaglia, and A. Banchs, “Mobile traffic forecasting for maximizing 5G network slicing resource utilization,” in Proc. IEEE INFOCOM, May 2017, pp. 1–9.
[32]
P. Caballeroet al., “Multi-tenant radio access network slicing: Statistical multiplexing of spatial loads,” IEEE/ACM Trans. Netw., vol. 25, no. 5, pp. 3044–3058, Oct. 2017.
[33]
D. Bega, M. Gramaglia, A. Banchs, V. Sciancalepore, K. Samdanis, and X. Costa-Perez, “Optimising 5G infrastructure markets: The business of network slicing,” in Proc. IEEE INFOCOM, May 2017, pp. 1–9.
[34]
R. Kokku, R. Mahindra, H. Zhang, and S. Rangarajan, “NVS: A substrate for virtualizing wireless resources in cellular networks,” IEEE/ACM Trans. Netw., vol. 20, no. 5, pp. 1333–1346, Oct. 2012.
[35]
R. Kokku, R. Mahindra, H. Zhang, and S. Rangarajan, “CellSlice: Cellular wireless resource slicing for active RAN sharing,” in Proc. IEEE 5th Int. Conf. Commun. Syst. Netw. (COMSNETS), Jan. 2013, pp. 1–10.
[36]
M. Jiang, M. Condoluci, and T. Mahmoodi, “Network slicing management & prioritization in 5G mobile systems,” in Proc. 22nd Eur. Wireless Conf., 2016, pp. 1–6.
[37]
M. R. Crippaet al., “Resource sharing for a 5G multi-tenant and multi-service architecture,” in Proc. 23rd Eur. Wireless Conf., 2017, pp. 1–6.
[38]
M. G. Kibria, G. P. Villardi, K. Nguyen, W.-S. Liao, K. Ishizu, and F. Kojima, “Shared spectrum access communications: A neutral host micro operator approach,” IEEE J. Sel. Areas Commun., vol. 35, no. 8, pp. 1741–1753, Aug. 2017.
[39]
J. O. Fajardoet al., “Introducing mobile edge computing capabilities through distributed 5G cloud enabled small cells,” Mobile Netw. Appl., vol. 21, no. 4, pp. 564–574, 2016.
[40]
I. P. Chochliouroset al., “A novel architectural concept for enhanced 5G network facilities,” in Proc. MATEC Web Conf., vol. 125, 2017, p. 03012.
[41]
X. Foukas, M. K. Marina, and K. Kontovasilis, “Orion: RAN slicing for a flexible and cost-effective multi-service mobile network architecture,” in Proc. 23rd ACM MobiCom, 2017, pp. 127–140.
[42]
S. H. Low and D. E. Lapsley, “Optimization flow control. I. Basic algorithm and convergence,” IEEE/ACM Trans. Netw., vol. 7, no. 6, pp. 861–874, Dec. 1999.
[43]
S. Ha, S. Sen, C. Joe-Wong, Y. Im, and M. Chiang, “TUBE: Time-dependent pricing for mobile data,” ACM SIGCOMM Comput. Commun. Rev., vol. 42, no. 4, pp. 247–258, 2012.
[44]
M. G. Kibria, G. P. Villardi, K. Ishizu, F. Kojima, and H. Yano, “Resource allocation in shared spectrum access communications for operators with diverse service requirements,” EURASIP J. Adv. Signal Process., vol. 2016, p. 83, Jul. 2016.
[45]
M. G. Kibria, G. P. Villardi, K. Nguyen, K. Ishizu, and F. Kojima, “Heterogeneous networks in shared spectrum access communications,” IEEE J. Sel. Areas Commun., vol. 35, no. 1, pp. 145–158, Jan. 2017.
[46]
J. Luo, J. Eichinger, Z. Zhao, and E. Schulz, “Multi-carrier waveform based flexible inter-operator spectrum sharing for 5G systems,” in Proc. IEEE Int. Symp. Dyn. Spectr. Access Netw. (DYSPAN), Apr. 2014, pp. 449–457.
[47]
T. Sanguanpuak, S. Guruacharya, E. Hossain, N. Rajatheva, and M. Latva-Aho, “On spectrum sharing among micro-operators in 5G,” in Proc. IEEE Eur. Conf. Netw. Commun. (EuCNC), Jun. 2017, pp. 1–6.
[48]
B. Singh, K. Koufos, O. Tirkkonen, and R. Berry, “Co-primary inter-operator spectrum sharing over a limited spectrum pool using repeated games,” in Proc. IEEE Int. Conf. Commun. (ICC), Jun. 2015, pp. 1494–1499.
[49]
B. Singh, K. Koufos, O. Tirkkonen, and R. Jäntti, “Repeated spectrum sharing games in multi-operator heterogeneous networks,” in Proc. IEEE Int. Symp. Dyn. Spectr. Access Netw. (DySPAN), Sep./Oct. 2015, pp. 221–228.
[50]
C. Hasan and M. K. Marina, “Communication-free inter-operator interference management in shared spectrum small cell networks,” in Proc. IEEE Int. Symp. Dyn. Spectr. Access Netw. (DySPAN), 2018., Oct. 2018, pp. 1–10.
[51]
X. Zhou, S. Gandhi, S. Suri, and H. Zheng, “eBay in the sky: Strategy-proof wireless spectrum auctions,” in Proc. ACM 14th ACM MobiCom, 2008, pp. 2–13.
[52]
F. Fu and U. C. Kozat, “Stochastic game for wireless network virtualization,” IEEE/ACM Trans. Netw., vol. 21, no. 1, pp. 84–97, Feb. 2013.
[53]
K. Zhu and E. Hossain, “Virtualization of 5G cellular networks as a hierarchical combinatorial auction,” IEEE Trans. Mobile Comput., vol. 15, no. 10, pp. 2640–2654, Oct. 2016.
[54]
X. Feng, P. Lin, and Q. Zhang, “FlexAuc: Serving dynamic demands in a spectrum trading market with flexible auction,” IEEE Trans. Wireless Commun., vol. 14, no. 2, pp. 821–830, Feb. 2015.
[55]
T. Le, M. Beluri, M. Freda, J.-L. Gauvreau, S. Laughlin, and P. Ojanen, “On a new incentive and market based framework for multi-tier shared spectrum access systems,” in Proc. IEEE Int. Symp. Dyn. Spectr. Access Netw. (DYSPAN), Apr. 2014, pp. 477–488.
[56]
S. Sengupta and M. Chatterjee, “Designing auction mechanisms for dynamic spectrum access,” Mobile Netw. Appl., vol. 13, no. 5, pp. 498–515, 2008.
[57]
S. Gandhi, C. Buragohain, L. Cao, H. Zheng, and S. Suri, “Towards real-time dynamic spectrum auctions,” Comput. Netw., vol. 52, no. 4, pp. 879–897, 2008.
[58]
J. Jia, Q. Zhang, Q. Zhang, and M. Liu, “Revenue generation for truthful spectrum auction in dynamic spectrum access,” in Proc. 10th ACM Int. Symp. Mobile Ad Hoc Netw. Comput., 2009, pp. 3–12.
[59]
L. Gao, J. Huang, Y.-J. Chen, and B. Shou, “An integrated contract and auction design for secondary spectrum trading,” IEEE J. Sel. Areas Commun., vol. 31, no. 3, pp. 581–592, Mar. 2013.
[60]
W. Dong, S. Rallapalli, L. Qiu, K. K. Ramakrishnan, and Y. Zhang, “Double auctions for dynamic spectrum allocation,” IEEE/ACM Trans. Netw., vol. 24, no. 4, pp. 2485–2497, Aug. 2016.
[61]
nFAPI and FAPI specifications,” Small Cell Forum, White Paper, May 2017. [Online]. Available: http://scf.io/en/documents/082_-_nFAPI_and_FAPI_specifications.php
[62]
MulteFire release 1.0 technical paper: A new way to wireless,” MulteFire Alliance, White, Paper, 2017. [Online]. Available: https://www.multefire.org/wp-content/uploads/MulteFire-Release-1.0-whitepaper_FINAL.pdf
[63]
R. Irmeret al., “Coordinated multipoint: Concepts, performance, and field trial results,” IEEE Commun. Mag., vol. 49, no. 2, pp. 102–111, Feb. 2011.
[64]
G. Salami, S. Burley, O. Durowoju, and C. Kellett, “LTE indoor small cell capacity and coverage comparison,” in Proc. IEEE 24th Int. Symp. Pers., Indoor Mobile Radio Commun. (PIMRC Workshops), Sep. 2013, pp. 66–70.
[65]
M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming. Hoboken, NJ, USA: Wiley, 2014.
[66]
C. J. C. H. Watkins and P. Dayan, “Q-learning,” Mach. Learn., vol. 8, nos. 3–4, pp. 279–292, 1992.
[67]
L. Busoniu, R. Babuska, B. De Schutter, and D. Ernst, Reinforcement Learning and Dynamic Programming Using Function Approximators, vol. 39. Boca Raton, FL, USA: CRC Press, 2010.
[68]
D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, and M. Riedmiller, “Deterministic policy gradient algorithms,” in Proc. ICML, 2014, pp. 1–9.
[69]
T. P. Lillicrapet al., “Continuous control with deep reinforcement learning,” 2015, arXiv:1509.02971. [Online]. Available: https://arxiv.org/abs/1509.02971
[70]
J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel, “High-dimensional continuous control using generalized advantage estimation,” 2015, arXiv:1506.02438. [Online]. Available: https://arxiv.org/abs/1506.02438
[71]
Y. Duan, X. Chen, R. Houthooft, J. Schulman, and P. Abbeel, “Benchmarking deep reinforcement learning for continuous control,” in Proc. Int. Conf. Mach. Learn., 2016, pp. 1329–1338.
[72]
S. Gu, E. Holly, T. Lillicrap, and S. Levine, “Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), May/Jun. 2017, pp. 3389–3396.
[73]
Y. Tassaet al., “DeepMind control suite,” 2018, arXiv:1801.00690. [Online]. Available: https://arxiv.org/abs/1801.00690
[74]
N. Nikaein, M. K. Marina, S. Manickam, A. Dawson, R. Knopp, and C. Bonnet, “OpenAirInterface: A flexible platform for 5G research,” ACM SIGCOMM Comput. Commun. Rev., vol. 44, no. 5, pp. 33–38, 2014.
[75]
N. Makris, P. Basaras, T. Korakis, N. Nikaein, and L. Tassiulas, “Experimental evaluation of functional splits for 5G cloud-RANs,” in Proc. IEEE Int. Conf. Commun. (ICC), May 2017, pp. 1–6.
[76]
C.-Y. Chang, N. Nikaein, R. Knopp, T. Spyropoulos, and S. S. Kumar, “FlexCRAN: A flexible functional split framework over Ethernet fronthaul in cloud-RAN,” in Proc. IEEE Int. Conf. Commun. (ICC), May 2017, pp. 1–7.
[77]
X. Foukas, F. Sardis, F. Foster, M. K. Marina, M. A. Lema, and M. Dohler, “Experience building a prototype 5G Testbed,” in Proc. ACM 1st Int. Workshop Experimentation Meas. 5G (EM-5G), 2018.
[78]
(2018). DDPG Implementation. [Online]. Available: https://github.com/stevenpjg/ddpg-aigym
[79]
G. Brockmanet al., “Openai gym,” 2016, arXiv:1606.01540. [Online]. Available: https://arxiv.org/abs/1606.01540
[80]
A. Botta, A. Dainotti, and A. Pescapé, “A tool for the generation of realistic network workload for emerging networking scenarios,” Comput. Netw., vol. 56, no. 15, pp. 3531–3547, Oct. 2012.
[81]
H. Wang, F. Xu, Y. Li, P. Zhang, and D. Jin, “Understanding mobile traffic patterns of large scale cellular towers in urban environment,” in Proc. ACM Internet Meas. Conf., 2015, pp. 225–238.
[82]
X. Foukas, M. K. Marina, and K. Kontovasilis, “Iris: Deep reinforcement learning driven shared spectrum access architecture for indoor neutral-host small cells,” 2018, arXiv:1812.06183. [Online]. Available: https://arxiv.org/abs/1812.06183
[83]
T. Degris, P. M. Pilarski, and R. S. Sutton, “Model-free reinforcement learning with continuous action in practice,” in Proc. IEEE Amer. Control Conf. (ACC), Jun. 2012, pp. 2177–2182.
[84]
R. Lowe, Y. Wu, A. Tamar, J. Harb, O. P. Abbeel, and I. Mordatch, “Multi-agent actor-critic for mixed cooperative-competitive environments,” in Proc. Adv. Neural Inf. Process. Syst., 2017, pp. 6382–6393.

Cited By

View all
  • (2024)Survey on Machine Learning-Enabled Network Slicing: Covering the Entire Life CycleIEEE Transactions on Network and Service Management10.1109/TNSM.2023.328765121:1(994-1011)Online publication date: 1-Feb-2024
  • (2024)Learning With Side Information: Elastic Multi-Resource Control for the Open RANIEEE Journal on Selected Areas in Communications10.1109/JSAC.2023.333917442:2(295-309)Online publication date: 1-Feb-2024
  • (2024)Empowering the 6G Cellular Architecture With Open RANIEEE Journal on Selected Areas in Communications10.1109/JSAC.2023.333461042:2(245-262)Online publication date: 1-Feb-2024
  • Show More Cited By

Index Terms

  1. Iris: Deep Reinforcement Learning Driven Shared Spectrum Access Architecture for Indoor Neutral-Host Small Cells
            Index terms have been assigned to the content through auto-classification.

            Recommendations

            Comments

            Information & Contributors

            Information

            Published In

            cover image IEEE Journal on Selected Areas in Communications
            IEEE Journal on Selected Areas in Communications  Volume 37, Issue 8
            Aug. 2019
            256 pages

            Publisher

            IEEE Press

            Publication History

            Published: 01 August 2019

            Qualifiers

            • Research-article

            Contributors

            Other Metrics

            Bibliometrics & Citations

            Bibliometrics

            Article Metrics

            • Downloads (Last 12 months)0
            • Downloads (Last 6 weeks)0
            Reflects downloads up to 26 Jan 2025

            Other Metrics

            Citations

            Cited By

            View all
            • (2024)Survey on Machine Learning-Enabled Network Slicing: Covering the Entire Life CycleIEEE Transactions on Network and Service Management10.1109/TNSM.2023.328765121:1(994-1011)Online publication date: 1-Feb-2024
            • (2024)Learning With Side Information: Elastic Multi-Resource Control for the Open RANIEEE Journal on Selected Areas in Communications10.1109/JSAC.2023.333917442:2(295-309)Online publication date: 1-Feb-2024
            • (2024)Empowering the 6G Cellular Architecture With Open RANIEEE Journal on Selected Areas in Communications10.1109/JSAC.2023.333461042:2(245-262)Online publication date: 1-Feb-2024
            • (2023)Taking 5G RAN Analytics and Control to a New LevelProceedings of the 29th Annual International Conference on Mobile Computing and Networking10.1145/3570361.3592493(1-16)Online publication date: 10-Jul-2023
            • (2023)NeutRAN: An Open RAN Neutral Host Architecture for Zero-Touch RAN and Spectrum SharingIEEE Transactions on Mobile Computing10.1109/TMC.2023.331172823:5(5786-5798)Online publication date: 4-Sep-2023
            • (2023)Optimal Network Protocol Selection for Competing Flows via Online LearningIEEE Transactions on Mobile Computing10.1109/TMC.2022.316288022:8(4822-4836)Online publication date: 1-Aug-2023
            • (2022)Dynamic spectrum access and sharing through actor-critic deep reinforcement learningEURASIP Journal on Wireless Communications and Networking10.1186/s13638-022-02124-42022:1Online publication date: 4-Jun-2022
            • (2021)ConcordiaProceedings of the 2021 ACM SIGCOMM 2021 Conference10.1145/3452296.3472894(580-596)Online publication date: 9-Aug-2021
            • (2020)Collaborative Flow Control in the DARPA Spectrum Collaboration ChallengeIEEE Transactions on Network and Service Management10.1109/TNSM.2020.303107817:4(2024-2038)Online publication date: 1-Dec-2020

            View Options

            View options

            Figures

            Tables

            Media

            Share

            Share

            Share this Publication link

            Share on social media