Big Data in HEP: A comprehensive use case study

O Gutsche, M Cremonesi, P Elmer… - Journal of Physics …, 2017 - iopscience.iop.org
O Gutsche, M Cremonesi, P Elmer, B Jayatilaka, J Kowalkowski, J Pivarski, S Sehrish
Journal of Physics: Conference Series, 2017iopscience.iop.org
Abstract Experimental Particle Physics has been at the forefront of analyzing the worlds
largest datasets for decades. The HEP community was the first to develop suitable software
and computing tools for this task. In recent times, new toolkits and systems collectively called
Big Data technologies have emerged to support the analysis of Petabyte and Exabyte
datasets in industry. While the principles of data analysis in HEP have not changed (filtering
and transforming experiment-specific data formats), these new technologies use different …
Abstract
Experimental Particle Physics has been at the forefront of analyzing the worlds largest datasets for decades. The HEP community was the first to develop suitable software and computing tools for this task. In recent times, new toolkits and systems collectively called Big Data technologies have emerged to support the analysis of Petabyte and Exabyte datasets in industry. While the principles of data analysis in HEP have not changed (filtering and transforming experiment-specific data formats), these new technologies use different approaches and promise a fresh look at analysis of very large datasets and could potentially reduce the time-to-physics with increased interactivity.
iopscience.iop.org