research-article

Near-Storage Processing for Solid State Drive Based Recommendation Inference with SmartSSDs®

Authors:

Mohammadreza Soltaniyeh,

Veronica Lagrange Moutinho Dos Reis,

Richard P. Martin,

Santosh NagarakatteAuthors Info & Claims

ICPE '22: Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering

Pages 177 - 186

https://doi.org/10.1145/3489525.3511672

Published: 09 April 2022 Publication History

Abstract

Deep learning-based recommendation systems are extensively deployed in numerous internet services, including social media, entertainment services, and search engines, to provide users with the most relevant and personalized content. Production scale deep learning models consist of large embedding tables with billions of parameters. DRAM-based recommendation systems incur a high infrastructure cost and limit the size of the deployed models. Recommendation systems based on solid-state drives (SSDs) are a promising alternative for DRAM-based systems. Systems based on SSDs can offer ample storage required for deep learning models with large embedding tables. This paper proposes SmartRec, an inference engine for deep learning-based recommendation systems that utilizes Samsung SmartSSD, an SSD with an on-board FPGA that can process data in-situ. We evaluate SmartRec with state-of-the-art recommendation models from Facebook and compare its performance and energy efficiency to a DRAM-based system on a CPU. We show SmartRec improves the energy efficiency of the recommendation inference task up to 10x in comparison to the baseline CPU implementation. In addition, we propose a novel application-specific caching system for SmartSSDs that allows the kernel on the FPGA to use its DRAM as a cache to minimize high latency SSD accesses. Finally, we demonstrate the scalability of our design by offloading the computation to multiple SmartSSDs to further improve performance.

References

[1]

Assaf Eisenman, Maxim Naumov, Darryl Gardner, Misha Smelyanskiy, Sergey Pupyrev, Kim Hazelwood, Asaf Cidon, and Sachin Katti. 2018. Bandana: Using Non-volatile Memory for Storing Deep Learning Models. arxiv: 1811.05922 [cs.LG]

[2]

Phil Francisco. 2011. The Netezza Data Appliance Architecture: A Platform for High Performance Data Warehousing and Analytics. IBM Redbook (2011).

[3]

Udit Gupta, Samuel Hsia, Vikram Saraph, Xiaodong Wang, Brandon Reagen, Gu-Yeon Wei, Hsien-Hsin S. Lee, David Brooks, and Carole-Jean Wu. 2020 a. DeepRecSys: A System for Optimizing End-To-End At-Scale Neural Recommendation Inference. In 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA). 982--995. https://doi.org/10.1109/ISCA45697.2020.00084

Digital Library

[4]

Udit Gupta, Carole-Jean Wu, Xiaodong Wang, Maxim Naumov, Brandon Reagen, David Brooks, Bradford Cottel, Kim Hazelwood, Mark Hempstead, Bill Jia, Hsien-Hsin S. Lee, Andrey Malevich, Dheevatsa Mudigere, Mikhail Smelyanskiy, Liang Xiong, and Xuan Zhang. 2020 b. The Architectural Implications of Facebook's DNN-Based Personalized Recommendation. In 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA). 488--501. https://doi.org/10.1109/HPCA47549.2020.00047

[5]

Malay Haldar, Mustafa Abdool, Prashant Ramanathan, Tao Xu, Shulin Yang, Huizhong Duan, Qing Zhang, Nick Barrow-Williams, Bradley C. Turnbull, Brendan M. Collins, and et al. 2019. Applying Deep Learning to Airbnb Search. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (Jul 2019). https://doi.org/10.1145/3292500.3330658

Digital Library

[6]

Kim Hazelwood, Sarah Bird, David Brooks, Soumith Chintala, Utku Diril, Dmytro Dzhulgakov, Mohamed Fawzy, Bill Jia, Yangqing Jia, Aditya Kalro, James Law, Kevin Lee, Jason Lu, Pieter Noordhuis, Misha Smelyanskiy, Liang Xiong, and Xiaodong Wang. 2018. Applied Machine Learning at Facebook: A Datacenter Infrastructure Perspective. In 2018 IEEE International Symposium on High Performance Computer Architecture (HPCA) . 620--629. https://doi.org/10.1109/HPCA.2018.00059

[7]

Ranggi Hwang, Taehun Kim, Youngeun Kwon, and Minsoo Rhu. 2020. Centaur: A Chiplet-Based, Hybrid Sparse-Dense Accelerator for Personalized Recommendations .IEEE Press, 968--981. https://doi.org/10.1109/ISCA45697.2020.00083

Digital Library

[8]

Wenqi Jiang, Zhenhao He, Shuai Zhang, Thomas B. Preuß er, Kai Zeng, Liang Feng, Jiansong Zhang, Tongxuan Liu, Yong Li, Jingren Zhou, Ce Zhang, and Gustavo Alonso. 2021 a. MicroRec: Efficient Recommendation Inference by Hardware and Data Structure Solutions. In Proceedings of Machine Learning and Systems, A. Smola, A. Dimakis, and I. Stoica (Eds.), Vol. 3. 845--859.

[9]

Wenqi Jiang, Zhenhao He, Shuai Zhang, Kai Zeng, Liang Feng, Jiansong Zhang, Tongxuan Liu, Yong Li, Jingren Zhou, Ce Zhang, and Gustavo Alonso. 2021 b. FleetRec: Large-Scale Recommendation Inference on Hybrid GPU-FPGA Clusters .Association for Computing Machinery, New York, NY, USA, 3097--3105. https://doi.org/10.1145/3447548.3467139

Digital Library

[10]

Liu Ke, Udit Gupta, Carole-Jean Wu, Benjamin Youngjae Cho, Mark Hempstead, Brandon Reagen, Xuan Zhang, David Brooks, Vikas Chandra, Utku Diril, Amin Firoozshahian, Kim Hazelwood, Bill Jia, Hsien-Hsin S. Lee, Meng Li, Bert Maher, Dheevatsa Mudigere, Maxim Naumov, Martin Schatz, Mikhail Smelyanskiy, and Xiaodong Wang. 2019. RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing. arxiv: 1912.12953 [cs.DC]

[11]

H.T. Kung, Bradley McDanel, and Sai Qian Zhang. 2019. Packing Sparse Convolutional Neural Networks for Efficient Systolic Array Implementations: Column Combining Under Joint Optimization. In Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems (Providence, RI, USA) (ASPLOS '19). Association for Computing Machinery, New York, NY, USA, 821--834. https://doi.org/10.1145/3297858.3304028

Digital Library

[12]

Youngeun Kwon, Yunjae Lee, and Minsoo Rhu. 2019. TensorDIMM: A Practical Near-Memory Processing Architecture for Embeddings and Tensor Operations in Deep Learning. In Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture (Columbus, OH, USA) (MICRO '52). Association for Computing Machinery, New York, NY, USA, 740--753. https://doi.org/10.1145/3352460.3358284

Digital Library

[13]

Veronica Lagrange Moutinho dos Reis, Harry (Huan) Li, and Anahita Shayesteh. 2020. Modeling Analytics for Computational Storage. In Proceedings of the ACM/SPEC International Conference on Performance Engineering (Edmonton AB, Canada) (ICPE '20). Association for Computing Machinery, New York, NY, USA, 88--99. https://doi.org/10.1145/3358960.3375794

Digital Library

[14]

Yang Li and Zhitao Dai. 2019. Design and Implementation of Hardware Accelerator for Recommendation System based on Heterogeneous Computing Platform. https://doi.org/10.2991/icmeit-19.2019.150

[15]

Maxim Naumov, Dheevatsa Mudigere, Hao-Jun Michael Shi, Jianyu Huang, Narayanan Sundaraman, Jongsoo Park, Xiaodong Wang, Udit Gupta, Carole-Jean Wu, Alisson G. Azzolini, Dmytro Dzhulgakov, Andrey Mallevich, Ilia Cherniavskii, Yinghai Lu, Raghuraman Krishnamoorthi, Ansha Yu, Volodymyr Kondratenko, Stephanie Pereira, Xianjie Chen, Wenlin Chen, Vijay Rao, Bill Jia, Liang Xiong, and Misha Smelyanskiy. 2019. Deep Learning Recommendation Model for Personalization and Recommendation Systems. CoRR, Vol. abs/1906.00091 (2019). https://arxiv.org/abs/1906.00091

[16]

Jongsoo Park, Maxim Naumov, Protonu Basu, Summer Deng, Aravind Kalaiah, Daya Khudia, James Law, Parth Malani, Andrey Malevich, Satish Nadathur, Juan Pino, Martin Schatz, Alexander Sidorov, Viswanath Sivakumar, Andrew Tulloch, Xiaodong Wang, Yiming Wu, Hector Yuen, Utku Diril, Dmytro Dzhulgakov, Kim Hazelwood, Bill Jia, Yangqing Jia, Lin Qiao, Vijay Rao, Nadav Rotem, Sungjoo Yoo, and Mikhail Smelyanskiy. 2018. Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications. arxiv: 1811.09886 [cs.LG]

[17]

Sahand Salamat, Armin Haj Aboutalebi, Behnam Khaleghi, Joo Hwan Lee, Yang Seok Ki, and Tajana Rosing. 2021. NASCENT: Near-Storage Acceleration of Database Sort on SmartSSD. In The 2021 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (Virtual Event, USA) (FPGA '21). Association for Computing Machinery, New York, NY, USA, 262--272. https://doi.org/10.1145/3431920.3439298

Digital Library

[18]

Mohammadreza Soltaniyeh, Richard P. Martin, and Santosh Nagarakatte. 2021 a. SPOTS: An Accelerator for Sparse CNNs Leveraging General Matrix-Matrix Multiplication. arxiv: 2107.13386 [cs.AR]

[19]

Mohammadreza Soltaniyeh, Veronica Lagrange Moutinho Dos Reis, Matthew Bryson, Richard Martin, and Santosh Nagarakatte. 2021 b. Near-Storage Acceleration of Database Query Processing with SmartSSDs. In 2021 IEEE 29th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM). 265--265. https://doi.org/10.1109/FCCM51124.2021.00052

[20]

Moshe Tennenholtz and Oren Kurland. 2019. Rethinking Search Engines and Recommendation Systems: A Game Theoretic Perspective. Commun. ACM, Vol. 62, 12 (Nov. 2019), 66--75. https://doi.org/10.1145/3340922

Digital Library

[21]

Jizhe Wang, Pipei Huang, Huan Zhao, Zhibo Zhang, Binqiang Zhao, and Dik Lun Lee. 2018. Billion-Scale Commodity Embedding for E-Commerce Recommendation in Alibaba. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery ; Data Mining (London, United Kingdom) (KDD '18). Association for Computing Machinery, New York, NY, USA, 839--848. https://doi.org/10.1145/3219819.3219869

Digital Library

[22]

Mark Wilkening, Udit Gupta, Samuel Hsia, Caroline Trippel, Carole-Jean Wu, David Brooks, and Gu-Yeon Wei. 2021. RecSSD: Near Data Processing for Solid State Drive Based Recommendation Inference. In Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (Virtual, USA) (ASPLOS 2021). Association for Computing Machinery, New York, NY, USA, 717--729. https://doi.org/10.1145/3445814.3446763

Digital Library

[23]

Xilinx-Samsung. 2021. SmartSSD. https://www.xilinx.com/applications/data-center/computational-storage/smartssd.html

[24]

Weijie Zhao, Jingyuan Zhang, Deping Xie, Yulei Qian, Ronglai Jia, and Ping Li. 2019 b. AIBox: CTR Prediction Model Training on a Single Node. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management (Beijing, China) (CIKM '19). Association for Computing Machinery, New York, NY, USA, 319--328. https://doi.org/10.1145/3357384.3358045

Digital Library

[25]

Zhe Zhao, Lichan Hong, Li Wei, Jilin Chen, Aniruddh Nath, Shawn Andrews, Aditee Kumthekar, Maheswaran Sathiamoorthy, Xinyang Yi, and Ed Chi. 2019 a. Recommending What Video to Watch next: A Multitask Ranking System. In Proceedings of the 13th ACM Conference on Recommender Systems (Copenhagen, Denmark) (RecSys '19). Association for Computing Machinery, New York, NY, USA, 43--51. https://doi.org/10.1145/3298689.3346997

Digital Library

[26]

Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. 2018a. Deep Interest Evolution Network for Click-Through Rate Prediction. arxiv: 1809.03672 [stat.ML]

[27]

Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018b. Deep Interest Network for Click-Through Rate Prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery; Data Mining (London, United Kingdom) (KDD '18). Association for Computing Machinery, New York, NY, USA, 1059--1068. https://doi.org/10.1145/3219819.3219823

Digital Library

Cited By

Ait-Oucheggou LRubini SBattou ABoukhobza J(2025)QM-ARC: QoS-aware Multi-tier Adaptive Cache Replacement StrategyFuture Generation Computer Systems10.1016/j.future.2024.107548163(107548)Online publication date: Feb-2025
https://doi.org/10.1016/j.future.2024.107548
Khadirsharbiyani SElyasi NAboutalebi ALiu CChoi CKandemir M(2024)SmartGraph: A Framework for Graph Processing in Computational StorageProceedings of the ACM Symposium on Cloud Computing10.1145/3698038.3698538(737-754)Online publication date: 20-Nov-2024
https://dl.acm.org/doi/10.1145/3698038.3698538
Jang HSong JJung JPark JKim YLee J(2024)Smart-Infinity: Fast Large Language Model Training using Near-Storage Processing on a Real System2024 IEEE International Symposium on High-Performance Computer Architecture (HPCA)10.1109/HPCA57654.2024.00034(345-360)Online publication date: 2-Mar-2024
https://doi.org/10.1109/HPCA57654.2024.00034
Show More Cited By

Index Terms

Near-Storage Processing for Solid State Drive Based Recommendation Inference with SmartSSDs®
1. Computer systems organization
  1. Architectures
    1. Other architectures
      1. Neural networks
      2. Reconfigurable computing

Recommendations

RecSSD: near data processing for solid state drive based recommendation inference
ASPLOS '21: Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems

Neural personalized recommendation models are used across a wide variety of datacenter applications including search, social media, and entertainment. State-of-the-art models comprise large embedding tables that have billions of parameters requiring ...
Towards a deep learning model for hybrid recommendation
WI '17: Proceedings of the International Conference on Web Intelligence

The deep learning wave is propagating through many research areas and communities. In the last years it quickly propagated to Recommendation Systems, a research area which aims to recommend items to users. Indeed, many deep learning models and ...
A General Rating Recommended Weight-Aware Model for Recommendation System
HCC 2016: Revised Selected Papers of the Second International Conference on Human Centered Computing - Volume 9567

In recommendation system, the ratings represent the users' preference and play an important role in recommending items to users. However, the ratings of items may be influenced by many factors, such as time the latest ratings are more able to reflect ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICPE '22: Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering

April 2022

242 pages

ISBN:9781450391436

DOI:10.1145/3489525

General Chairs:
Dan Feng
Huazhong University of Science and Technology, China
,
Steffen Becker
University of Stuttgart, Germany
,
Program Chairs:
Nikolas Herbst
University of Würzburg, Germany
,
Philipp Leitner
Chalmers and University of Gothenburg
,
Publications Chair:
Alessandro Papadopoulos
Mälardalen University, Sweden

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICPE '22

Sponsor:

ICPE '22: ACM/SPEC International Conference on Performance Engineering

April 9 - 13, 2022

Beijing, China

Acceptance Rates

ICPE '22 Paper Acceptance Rate 14 of 58 submissions, 24%;

Overall Acceptance Rate 252 of 851 submissions, 30%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
605
Total Downloads

Downloads (Last 12 months)151
Downloads (Last 6 weeks)17

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ait-Oucheggou LRubini SBattou ABoukhobza J(2025)QM-ARC: QoS-aware Multi-tier Adaptive Cache Replacement StrategyFuture Generation Computer Systems10.1016/j.future.2024.107548163(107548)Online publication date: Feb-2025
https://doi.org/10.1016/j.future.2024.107548
Khadirsharbiyani SElyasi NAboutalebi ALiu CChoi CKandemir M(2024)SmartGraph: A Framework for Graph Processing in Computational StorageProceedings of the ACM Symposium on Cloud Computing10.1145/3698038.3698538(737-754)Online publication date: 20-Nov-2024
https://dl.acm.org/doi/10.1145/3698038.3698538
Jang HSong JJung JPark JKim YLee J(2024)Smart-Infinity: Fast Large Language Model Training using Near-Storage Processing on a Real System2024 IEEE International Symposium on High-Performance Computer Architecture (HPCA)10.1109/HPCA57654.2024.00034(345-360)Online publication date: 2-Mar-2024
https://doi.org/10.1109/HPCA57654.2024.00034
Karimzadeh FImani MAsgari BCao NLin YFang Y(2023)Memory-Based Computing for Energy-Efficient AI: Grand Challenges2023 IFIP/IEEE 31st International Conference on Very Large Scale Integration (VLSI-SoC)10.1109/VLSI-SoC57769.2023.10321880(1-8)Online publication date: 16-Oct-2023
https://doi.org/10.1109/VLSI-SoC57769.2023.10321880
Ait-Oucheggou LRubini SBattou ABoukhobza J(2023)Investigating Multi-Tier and QoS-Aware Caching Based on ARC2023 31st International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS)10.1109/MASCOTS59514.2023.10387601(1-4)Online publication date: 16-Oct-2023
https://doi.org/10.1109/MASCOTS59514.2023.10387601
Fakhry DAbdelsalam MEl-Kharashi MSafar M(2023)A review on computational storage devices and near memory computing for high performance applicationsMemories - Materials, Devices, Circuits and Systems10.1016/j.memori.2023.1000514(100051)Online publication date: Jul-2023
https://doi.org/10.1016/j.memori.2023.100051

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents