Random Forests over normalized data in CPU-GPU DBMSes
Pages 98 - 101
Abstract
This short paper studies query execution based on message passing on CPU-GPU systems, using random forests training as the workload. We investigate different data placement and query execution strategies and find that the unique properties of training ML models using message passing necessitates different design decisions. We show that with proper data placement and CPU-GPU co-execution, training random forest models using pure SQL can outperform the leading LightGBM ML library by 1.5 × on SSB SF=10.
References
[1]
2021. 2021 Kaggle Data Science and Machine Learning Survey. https://www.kaggle.com/code/paultimothymooney/2021-kaggle-data-science-machine-learning-survey/notebook.
[2]
2023. cuDF - GPU DataFrames. https://github.com/rapidsai/cudf.
[3]
Yuki Asada, Victor Fu, Apurva Gandhi, Advitya Gemawat, Lihao Zhang, Dong He, Vivek Gupta, Ehi Nosakhare, Dalitso Banda, Rathijit Sen, and Matteo Interlandi. 2022. Share the Tensor Tea: How Databases Can Leverage the Machine Learning Ecosystem. Proc. VLDB Endow. 15, 12 (aug 2022), 3598–3601. https://doi.org/10.14778/3554821.3554853
[4]
Sebastian Breß. 2014. The design and implementation of CoGaDB: A column-oriented GPU-accelerated DBMS. Datenbank-Spektrum 14 (2014), 199–209.
[5]
Jiashen Cao, Rathijit Sen, Matteo Interlandi, Joy Arulraj, and Hyesoon Kim. 2023. Revisiting Query Performance in GPU Database Systems. arXiv preprint arXiv:2302.00734 (2023).
[6]
Tianqi Chen and Carlos Guestrin. 2016. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. 785–794.
[7]
Ashish Gupta, Venky Harinarayan, and Dallan Quass. 1995. Aggregate-query processing in data warehousing environments. (1995).
[8]
Jiong He, Mian Lu, and Bingsheng He. 2013. Revisiting co-processing for hash joins on the coupled cpu-gpu architecture. arXiv preprint arXiv:1307.1955 (2013).
[9]
Zezhou Huang, Rathijit Sen, Jiaxiang Liu, and Eugene Wu. 2022. (Technical Report) Joinboost: Grow trees over normalized data using only SQL. https://github.com/zachary62/JoinBoost/blob/main/technical/JoinBoost_tech.pdf.
[10]
Zezhou Huang and Eugene Wu. 2022. Calibration: A Simple Trick for Wide-table Delta Analytics. arXiv e-prints (2022).
[11]
Alekh Jindal, K. Venkatesh Emani, Maureen Daum, Olga Poppe, Brandon Haynes, Anna Pavlenko, Ayushi Gupta, Karthik Ramachandra, Carlo Curino, Andreas Mueller, Wentao Wu, and Hiren Patel. 2021. Magpie: Python at Speed and Scale using Cloud Backends. In Conference on Innovative Data Systems Research.
[12]
Mahmoud Abo Khamis, Hung Q Ngo, XuanLong Nguyen, Dan Olteanu, and Maximilian Schleich. 2018. AC/DC: In-database learning thunderstruck. In Proceedings of the Second Workshop on Data Management for End-To-End Machine Learning. 1–10.
[13]
Aaftab Munshi. 2009. The opencl specification. In 2009 IEEE Hot Chips 21 Symposium (HCS). IEEE, 1–314.
[14]
Patrick O’Neil, Elizabeth O’Neil, Xuedong Chen, and Stephen Revilak. 2009. The star schema benchmark and augmented fact table indexing. In Performance Evaluation and Benchmarking: First TPC Technology Conference, TPCTC 2009, Lyon, France, August 24-28, 2009, Revised Selected Papers 1. Springer, 237–252.
[15]
The pandas development team. 2020. pandas-dev/pandas: Pandas. https://doi.org/10.5281/zenodo.3509134
[16]
Mark Raasveldt and Hannes Mühleisen. 2019. DuckDB: an embeddable analytical database. In Proceedings of the 2019 International Conference on Management of Data. 1981–1984.
[17]
Viktor Rosenfeld, Sebastian Breß, and Volker Markl. 2022. Query processing on heterogeneous CPU/GPU systems. ACM Computing Surveys (CSUR) 55, 1 (2022), 1–38.
[18]
Maximilian Schleich, Dan Olteanu, Mahmoud Abo Khamis, Hung Q Ngo, and XuanLong Nguyen. 2019. A layered aggregate engine for analytics workloads. In Proceedings of the 2019 International Conference on Management of Data. 1642–1659.
[19]
Glenn R Shafer and Prakash P Shenoy. 1990. Probability propagation. Annals of mathematics and Artificial Intelligence 2, 1 (1990), 327–351.
[20]
Anil Shanbhag, Samuel Madden, and Xiangyao Yu. 2020. A study of the fundamental performance characteristics of GPUs and CPUs for database analytics. In Proceedings of the 2020 ACM SIGMOD international conference on Management of data. 1617–1632.
[21]
Anil Shanbhag, Samuel Madden, and Xiangyao Yu. 2020. A Study of the Fundamental Performance Characteristics of GPUs and CPUs for Database Analytics. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data (Portland, OR, USA) (SIGMOD ’20). 1617–1632. https://doi.org/10.1145/3318464.3380595
[22]
Diego G Tomé, Tim Gubner, Mark Raasveldt, Eyal Rozenberg, and Peter A Boncz. 2018. Optimizing Group-By and Aggregation using GPU-CPU Co-Processing. In ADMS@ VLDB. 1–10.
[23]
Swarup Acharya Phillip B Gibbons Viswanath and Poosala Sridhar Ramaswamy. 1998. Join Synopses for Approximate Query Answering. (1998).
[24]
Bobbi W Yogatama, Weiwei Gong, and Xiangyao Yu. 2022. Orchestrating data placement and query execution in heterogeneous CPU-GPU DBMS. Proceedings of the VLDB Endowment 15, 11 (2022), 2491–2503.
[25]
Huan Zhang, Si Si, and Cho-Jui Hsieh. 2017. GPU-acceleration for Large-scale Tree Boosting. arXiv preprint arXiv:1706.08359 (2017).
[26]
Yansong Zhang, Yu Zhang, Jiaheng Lu, Shan Wang, Zhuan Liu, and Ruichen Han. 2020. One size does not fit all: accelerating OLAP workloads with GPUs. Distributed and Parallel Databases 38 (2020), 995–1037.
Recommendations
Mersenne Twister Random Number Generation on FPGA, CPU and GPU
AHS '09: Proceedings of the 2009 NASA/ESA Conference on Adaptive Hardware and SystemsRandom number generation is a very important operation in computational science e.g. in Monte Carlo simulations methods. It is also a computationally intensive operation especially for high quality random number generation. In this paper, we present the ...
Optimized HPL for AMD GPU and multi-core CPU usage
The installation of the LOEWE-CSC ( http://csc.uni-frankfurt.de/csc/__ __51 ) supercomputer at the Goethe University in Frankfurt lead to the development of a Linpack which can fully utilize the installed AMD Cypress GPUs. At its core, a fast DGEMM for ...
Mining data with random forests: current options for real-world applications
Random Forests are fast, flexible, and represent a robust approach to mining high-dimensional data. They are an extension of classification and regression trees CART. They perform well even in the presence of a large number of features and a small ...
Comments
Information & Contributors
Information
Published In
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 18 June 2023
Check for updates
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- Amazon
- Adobe
- NSF
Conference
SIGMOD/PODS '23
Sponsor:
Acceptance Rates
DaMoN '23 Paper Acceptance Rate 17 of 23 submissions, 74%;
Overall Acceptance Rate 94 of 127 submissions, 74%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 152Total Downloads
- Downloads (Last 12 months)47
- Downloads (Last 6 weeks)4
Reflects downloads up to 25 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format