Abstract
High throughput DNA sequencing made individual genome profiling possible and produces very large amounts of data. Today data and associated metadata are stored in FASTQ text file assemblies carrying the information of genome fragments called reads. Current techniques rely on mapping these reads to a common reference genome for compression and analysis. However, about 10% of the reads do not map to any known reference making them difficult to compress or process. These reads are of high importance because they hold information absent from any reference. Finding overlaps in these reads can help subsequent processing and compression tasks tremendously. Within this context clustering is used to find overlapping unmapped reads and sort them in groups. Clustering being an extremely time consuming task a modular multi-FPGA pipeline was designed and is the focus of this paper. A pipeline with 6 FPGAs was created and has shown a speed-up of \(\times 5\) compared to existing FPGA implementations. Resulting enriched files encoding reads and clustering results show file sizes within a 10% margin of the best DNA compressors while providing valuable extra information.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
This artificial human genome was built as an average of multiple human genomes.
- 2.
- 3.
References
Cox, A.J., Bauer, M.J., Jakobi, T., Rosone, G.: Large-scale compression of genomic sequence databases with the Burrows-Wheeler transform. Bioinformatics 28, 1415–1419 (2012)
Deorowicz, S., Grabowski, S.: Compression of DNA sequence reads in FASTQ format. Bioinformatics 27, 860–862 (2011)
Du, K.L.: Clustering: a neural network approach. Neural Networks 23, 89–107 (2010)
Fritz, M.H.Y., Leinonen, R., Cochrane, G., Birney, E.: Efficient storage of high throughput DNA sequencing data using reference-based compression. Genome Res. 21, 734–740 (2011)
Gouin, A., Nouhaud, P., Legeai, F., Rizk, G., Simon, J.C., Lemaitre, C.: Whole genome re-sequencing: lessons from unmapped reads. Journées Ouvertes Biologie Informatique Mathématiques (2013)
Guerra, A., Lotero, J., Isaza, S.: Performance comparison of sequential and parallel compression applications for DNA raw data. J. Supercomput. 72, 4696–4717 (2016)
Hussain, H.M., Benkrid, K., Seker, H., Erdogan, A.T.: FPGA implementation of K-means algorithm for bioinformatics application: An accelerated approach to clustering Microarray data. In: Adaptive Hardware and Systems, pp. 248–255 (2011)
Jain, A.K.: Data clustering: 50 years beyond K-means. Pattern Recogn. Lett. 31, 651–666 (2010)
Li, R., Zhu, H., Ruan, J., Qian, W., Fang, X., Shi, Z., Li, Y., Li, S., Shan, G., Kristiansen, K., Li, S., Yang, H., Wang, J., Wang, J.: De novo assembly of human genomes with massively parallel short read sequencing. Genome Res. 20(2), 265–272 (2010)
Petraglio, E., Wertenbroek, R., Capitao, F., Guex, N., Iseli, C., Thoma, Y.: Genomic data clustering on FPGAs for compression. In: Wong, S., Beck, A.C., Bertels, K., Carro, L. (eds.) ARC 2017. LNCS, vol. 10216, pp. 229–240. Springer, Cham (2017). doi:10.1007/978-3-319-56258-2_20
Pinho, A.J., Pratas, D., Garcia, S.P.: GReEn: a tool for efficient compression of genome resequencing data. Nucleic Acids Res. 40(4), e27 (2011)
Pollard, K.S., van der Laan, M.J.: Cluster analysis of genomic data. Bioinformatics and Computational Biology Solutions Using R and Bioconductor, pp. 208–228. Springer, New York (2005)
Stephens, Z.D., Lee, S.Y., Faghri, F., Campbell, R.H., Zhai, C., Efron, M.J., Iyer, R., Schatz, M.C., Sinha, S., Robinson, G.E.: Big data: astronomical or genomical? PLoS Biol. 13, e1002195 (2015)
Winterstein, F., Bayliss, S., Constantinides, G.A.: FPGA-based K-means clustering using tree-based data structures. In: 23rd International Conference on Field Programmable Logic and Applications, pp. 1–6 (2013)
Acknowledgments
The research presented in this paper was funded by the Swiss PASC initiative in the framework of the PoSeNoGap (Portable Scalable Concurrency for Genomic Data Processing) project. The authors would like to thank all the participants for the fruitful discussions, namely Ioannis Xenarios, Nicolas Guex, Christian Iseli, Thierry Schüpbach and Daniel Zerzion from SIB, Marco Mattavelli, and Claudio Alberti from EPFL, Flavio Capitao, and Roberto Rigamonti from HEIG-VD.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Wertenbroek, R., Petraglio, E., Thoma, Y. (2017). Pipelined Multi-FPGA Genomic Data Clustering. In: Ibrahim, S., Choo, KK., Yan, Z., Pedrycz, W. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2017. Lecture Notes in Computer Science(), vol 10393. Springer, Cham. https://doi.org/10.1007/978-3-319-65482-9_41
Download citation
DOI: https://doi.org/10.1007/978-3-319-65482-9_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-65481-2
Online ISBN: 978-3-319-65482-9
eBook Packages: Computer ScienceComputer Science (R0)