research-article

Streams on wires: a query compiler for FPGAs

Authors:

Gustavo AlonsoAuthors Info & Claims

Proceedings of the VLDB Endowment, Volume 2, Issue 1

Pages 229 - 240

https://doi.org/10.14778/1687627.1687654

Published: 01 August 2009 Publication History

Abstract

Taking advantage of many-core, heterogeneous hardware for data processing tasks is a difficult problem. In this paper, we consider the use of FPGAs for data stream processing as coprocessors in many-core architectures. We present Glacier, a component library and compositional compiler that transforms continuous queries into logic circuits by composing library components on an operator-level basis. In the paper we consider selection, aggregation, grouping, as well as windowing operators, and discuss their design as modular elements.

We also show how significant performance improvements can be achieved by inserting the FPGA into the system's data path (e.g., between the network interface and the host CPU). Our experiments show that queries on the FPGA can process streams at more than one million tuples per second and that they can do this directly from the network, removing much of the overhead of transferring the data to a conventional CPU.

References

[1]

N. Bandi, A. Metwally, D. Agrawal, and A. El Abbadi. Fast Data Stream Algorithms Using Associative Memories. In Proc. of the ACM SIGMOD Int'l Conference on Management of Data, Beijing, China, 2007.

Digital Library

[2]

Netezza Corp. http://www.netezza.com/.

[3]

D. DeWitt. DIRECT---A Multiprocessor Organization for Supporting Relational Database Management Systems. IEEE Trans. on Computers, c-28(6), 1979.

Digital Library

[4]

FIX Protocol Specification. http://fixprotocol.org/specifications.

[5]

C. A. Galindo-Legaria and M. Joshi. Orthogonal Optimization of Subqueries and Aggregation. In Proc. of the ACM SIGMOD Int'l Conference on Management of Data, Santa Barbara, CA, USA, 2001.

Digital Library

[6]

B. Gedik, P. S. Yu, and R. Bordawekar. Executing Stream Joins on the Cell Processor. In Proc. of the 33rd Int'l Conference on Very Large Databases (VLDB), Vienna, Austria, 2007.

Digital Library

[7]

B. T. Gold, A. Ailamaki, L. Huston, and B. Falsafi. Accelerating Database Operations Using a Network Processor. In Workshop on Data Management on New Hardware (DaMoN), Baltimore, MD, USA, 2005.

Digital Library

[8]

N. K. Govindaraju, J. Gray, R. Kumar, and D. Manocha. GPUTeraSort: High Performance Graphics Co-processor Sorting for Large Database Management. In Proc. of the 2006 ACM SIGMOD Int'l Conference on Management of Data, Chicago, IL, USA, 2006.

Digital Library

[9]

J. Gray, A. Bosworth, A. Lyaman, and H. Pirahesh. Data Cube: A Relational Aggregation Operator Generalizing GROUP-BY, CROSS-TAB, and SUB-TOTALS. In Proc. of the 12th Int'l Conference on Data Engineering, New Orleans, LA, USA, 1996.

Digital Library

[10]

D. Greaves and S. Singh. Kiwi: Synthesis of FPGA Circuits from Parallel Programs. In IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM), 2008.

Digital Library

[11]

T. Grust and J. Teubner. Relational Algebra: Mother Tongue---XQuery: Fluent. In Proc. of the 1st Twente Data Management Workshop (TDM), Enschede, The Netherlands, 2004.

[12]

S. A. Guccione, D. Levi, and D. Downs. A Reconfigurable Content Addressable Memory. In 7th Reconfigurable Architectures Workshop (RAW 2000), Cancún, Mexico, 2000.

Digital Library

[13]

S. Héman, M. Zukowski, A. de Vries, and P. Boncz. Efficient and Flexible Information Retrieval Using MonetDB/X100. In 3rd Biennial Conf. on Innovative Data Systems Research (CIDR), Asilomar, CA, USA, 2007.

[14]

Kickfire. http://www.kickfire.com/.

[15]

Options Price Reporting Authority (OPRA). Traffic Projections 2009/2010.

[16]

StreamBase Systems, Inc. http://www.streambase.com/.

[17]

N. Tatbul, U. Çetintemel, S. B. Zdonik, M. Cherniack, and M. Stonebraker. Load Shedding in a Data Stream Manager. In Proc. of the 29th Int'l Conference on Very Large Databases (VLDB), Berlin, Germany, 2003.

Digital Library

[18]

Xilinx Inc. An Overview of Multiple CAM Designs in Virtex Family Devices. Application Note 201, September 1999.

Cited By

Zhang CTang XPeng Y(2024)Enhancing Regular Expression Processing through Field-Programmable Gate Array-Based Multi-Character Non-Deterministic Finite AutomataElectronics10.3390/electronics1309163513:9(1635)Online publication date: 24-Apr-2024
https://doi.org/10.3390/electronics13091635
Östgren MSourdis I(2024)A Parallel Hash Table for Streaming ApplicationsProceedings of the 2024 International Conference on Parallel Architectures and Compilation Techniques10.1145/3656019.3676951(297-308)Online publication date: 14-Oct-2024
https://dl.acm.org/doi/10.1145/3656019.3676951
Najafi MQadah TSadoghi MJacobsen H(2024)DIBA: A Re-Configurable Stream ProcessorIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.338119236:9(4550-4566)Online publication date: 25-Mar-2024
https://dl.acm.org/doi/10.1109/TKDE.2024.3381192
Show More Cited By

Index Terms

Streams on wires: a query compiler for FPGAs

Recommendations

Sliding-window top-k queries on uncertain streams

Recently, due to the imprecise nature of the data generated from a variety of streaming applications, such as sensor networks, query processing on uncertain data streams has become an important problem. However, all the existing works on uncertain data ...
Exploiting predicate-window semantics over data streams

The continuous sliding-window query model is used widely in data stream management systems where the focus of a continuous query is limited to a set of the most recent tuples. In this paper, we show that an interesting and important class of queries ...
Condensative stream query language for data streams
ADC '07: Proceedings of the eighteenth conference on Australasian database - Volume 63

In contrast to traditional database queries, a query on stream data is continuous in that it is periodically evaluated over fractions (sliding windows) of the data stream. This introduces challenges beyond those encountered when processing traditional ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the VLDB Endowment

Proceedings of the VLDB Endowment Volume 2, Issue 1

August 2009

1293 pages

ISSN:2150-8097

Issue’s Table of Contents

Publisher

VLDB Endowment

Publication History

Published: 01 August 2009

Published in PVLDB Volume 2, Issue 1

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

115
Total Citations
View Citations
565
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)0

Reflects downloads up to 02 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang CTang XPeng Y(2024)Enhancing Regular Expression Processing through Field-Programmable Gate Array-Based Multi-Character Non-Deterministic Finite AutomataElectronics10.3390/electronics1309163513:9(1635)Online publication date: 24-Apr-2024
https://doi.org/10.3390/electronics13091635
Östgren MSourdis I(2024)A Parallel Hash Table for Streaming ApplicationsProceedings of the 2024 International Conference on Parallel Architectures and Compilation Techniques10.1145/3656019.3676951(297-308)Online publication date: 14-Oct-2024
https://dl.acm.org/doi/10.1145/3656019.3676951
Najafi MQadah TSadoghi MJacobsen H(2024)DIBA: A Re-Configurable Stream ProcessorIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.338119236:9(4550-4566)Online publication date: 25-Mar-2024
https://dl.acm.org/doi/10.1109/TKDE.2024.3381192
Hahn TSchüll DWildermann STeich J(2024)ABACUS: ASIP-Based Avro Schema-Customizable Parser Acceleration on FPGAs2024 27th International Symposium on Design & Diagnostics of Electronic Circuits & Systems (DDECS)10.1109/DDECS60919.2024.10508904(79-85)Online publication date: 3-Apr-2024
https://doi.org/10.1109/DDECS60919.2024.10508904
Ottimo AMencagli GDanelutto M(2024)Boosting general-purpose stream processing with reconfigurable hardwareThe Journal of Supercomputing10.1007/s11227-024-05894-480:9(13048-13078)Online publication date: 27-Feb-2024
https://dl.acm.org/doi/10.1007/s11227-024-05894-4
Lohmann STutsch D(2023)The Doubly Linked Tree of Singly Linked Rings: Providing Hard Real-Time Database Operations on an FPGAComputers10.3390/computers1301000813:1(8)Online publication date: 24-Dec-2023
https://doi.org/10.3390/computers13010008
Geethakumari PSourdis I(2023)Stream Aggregation with Compressed Sliding WindowsACM Transactions on Reconfigurable Technology and Systems10.1145/359077416:3(1-28)Online publication date: 20-Jun-2023
https://dl.acm.org/doi/10.1145/3590774
Moghaddamfar MMay NFärber CLehner WKumar AIenne PZhang Z(2023)A Study of Early Aggregation in Database Query Processing on FPGAsProceedings of the 2023 ACM/SIGDA International Symposium on Field Programmable Gate Arrays10.1145/3543622.3573194(55-65)Online publication date: 12-Feb-2023
https://dl.acm.org/doi/10.1145/3543622.3573194
Ottimo AMencagli GDanelutto M(2023)FSP: a Framework for Data Stream Processing Applications targeting FPGAs2023 31st Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP)10.1109/PDP59025.2023.00021(92-99)Online publication date: Mar-2023
https://doi.org/10.1109/PDP59025.2023.00021
Hahn TWildermann STeich J(2023)SPEAR-JSON: Selective Parsing of JSON to Enable Accelerated Stream Processing on FPGAs2023 33rd International Conference on Field-Programmable Logic and Applications (FPL)10.1109/FPL60245.2023.00034(189-196)Online publication date: 4-Sep-2023
https://doi.org/10.1109/FPL60245.2023.00034
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents