-
Data Preservation in High Energy Physics
Authors:
T. Basaglia,
M. Bellis,
J. Blomer,
J. Boyd,
C. Bozzi,
D. Britzger,
S. Campana,
C. Cartaro,
G. Chen,
B. Couturier,
G. David,
C. Diaconu,
A. Dobrin,
D. Duellmann,
M. Ebert,
P. Elmer,
J. Fernandes,
L. Fields,
P. Fokianos,
G. Ganis,
A. Geiser,
M. Gheata,
J. B. Gonzalez Lopez,
T. Hara,
L. Heinrich
, et al. (29 additional authors not shown)
Abstract:
Data preservation is a mandatory specification for any present and future experimental facility and it is a cost-effective way of doing fundamental research by exploiting unique data sets in the light of the continuously increasing theoretical understanding. This document summarizes the status of data preservation in high energy physics. The paradigms and the methodological advances are discussed…
▽ More
Data preservation is a mandatory specification for any present and future experimental facility and it is a cost-effective way of doing fundamental research by exploiting unique data sets in the light of the continuously increasing theoretical understanding. This document summarizes the status of data preservation in high energy physics. The paradigms and the methodological advances are discussed from a perspective of more than ten years of experience with a structured effort at international level. The status and the scientific return related to the preservation of data accumulated at large collider experiments are presented, together with an account of ongoing efforts to ensure long-term analysis capabilities for ongoing and future experiments. Transverse projects aimed at generic solutions, most of which are specifically inspired by open science and FAIR principles, are presented as well. A prospective and an action plan are also indicated.
△ Less
Submitted 9 September, 2023; v1 submitted 7 February, 2023;
originally announced February 2023.
-
HEP Software Foundation Community White Paper Working Group -- Data Organization, Management and Access (DOMA)
Authors:
Dario Berzano,
Riccardo Maria Bianchi,
Ian Bird,
Brian Bockelman,
Simone Campana,
Kaushik De,
Dirk Duellmann,
Peter Elmer,
Robert Gardner,
Vincent Garonne,
Claudio Grandi,
Oliver Gutsche,
Andrew Hanushevsky,
Burt Holzman,
Bodhitha Jayatilaka,
Ivo Jimenez,
Michel Jouvin,
Oliver Keeble,
Alexei Klimentov,
Valentin Kuznetsov,
Eric Lancon,
Mario Lassnig,
Miron Livny,
Carlos Maltzahn,
Shawn McKee
, et al. (13 additional authors not shown)
Abstract:
Without significant changes to data organization, management, and access (DOMA), HEP experiments will find scientific output limited by how fast data can be accessed and digested by computational resources. In this white paper we discuss challenges in DOMA that HEP experiments, such as the HL-LHC, will face as well as potential ways to address them. A research and development timeline to assess th…
▽ More
Without significant changes to data organization, management, and access (DOMA), HEP experiments will find scientific output limited by how fast data can be accessed and digested by computational resources. In this white paper we discuss challenges in DOMA that HEP experiments, such as the HL-LHC, will face as well as potential ways to address them. A research and development timeline to assess these changes is also proposed.
△ Less
Submitted 30 November, 2018;
originally announced December 2018.
-
A Roadmap for HEP Software and Computing R&D for the 2020s
Authors:
Johannes Albrecht,
Antonio Augusto Alves Jr,
Guilherme Amadio,
Giuseppe Andronico,
Nguyen Anh-Ky,
Laurent Aphecetche,
John Apostolakis,
Makoto Asai,
Luca Atzori,
Marian Babik,
Giuseppe Bagliesi,
Marilena Bandieramonte,
Sunanda Banerjee,
Martin Barisits,
Lothar A. T. Bauerdick,
Stefano Belforte,
Douglas Benjamin,
Catrin Bernius,
Wahid Bhimji,
Riccardo Maria Bianchi,
Ian Bird,
Catherine Biscarat,
Jakob Blomer,
Kenneth Bloom,
Tommaso Boccali
, et al. (285 additional authors not shown)
Abstract:
Particle physics has an ambitious and broad experimental programme for the coming decades. This programme requires large investments in detector hardware, either to build new facilities and experiments, or to upgrade existing ones. Similarly, it requires commensurate investment in the R&D of software to acquire, manage, process, and analyse the shear amounts of data to be recorded. In planning for…
▽ More
Particle physics has an ambitious and broad experimental programme for the coming decades. This programme requires large investments in detector hardware, either to build new facilities and experiments, or to upgrade existing ones. Similarly, it requires commensurate investment in the R&D of software to acquire, manage, process, and analyse the shear amounts of data to be recorded. In planning for the HL-LHC in particular, it is critical that all of the collaborating stakeholders agree on the software goals and priorities, and that the efforts complement each other. In this spirit, this white paper describes the R&D activities required to prepare for this software upgrade.
△ Less
Submitted 19 December, 2018; v1 submitted 18 December, 2017;
originally announced December 2017.
-
The LCG POOL Project, General Overview and Project Structure
Authors:
Dirk Duellmann
Abstract:
The POOL project has been created to implement a common persistency framework for the LHC Computing Grid (LCG) application area. POOL is tasked to store experiment data and meta data in the multi Petabyte area in a distributed and grid enabled way. First production use of new framework is expected for summer 2003. The project follows a hybrid approach combining C++ Object streaming technology su…
▽ More
The POOL project has been created to implement a common persistency framework for the LHC Computing Grid (LCG) application area. POOL is tasked to store experiment data and meta data in the multi Petabyte area in a distributed and grid enabled way. First production use of new framework is expected for summer 2003. The project follows a hybrid approach combining C++ Object streaming technology such as ROOT I/O for the bulk data with a transactionally safe relational database (RDBMS) store such as MySQL. POOL is based a strict component approach - as laid down in the LCG persistency and blue print RTAG documents - providing navigational access to distributed data without exposing details of the particular storage technology. This contribution describes the project breakdown into work packages, the high level interaction between the main pool components and summarizes current status and plans.
△ Less
Submitted 16 June, 2003;
originally announced June 2003.
-
The POOL Data Storage, Cache and Conversion Mechanism
Authors:
D. Dullmann,
M. Frank,
G. Govi,
I. Papadopoulos,
S. Roiser
Abstract:
The POOL data storage mechanism is intended to satisfy the needs of the LHC experiments to store and analyze the data from the detector response of particle collisions at the LHC proton-proton collider. Both the data rate and the data volumes will largely differ from the past experience. The POOL data storage mechanism is intended to be able to cope with the experiment's requirements applying a…
▽ More
The POOL data storage mechanism is intended to satisfy the needs of the LHC experiments to store and analyze the data from the detector response of particle collisions at the LHC proton-proton collider. Both the data rate and the data volumes will largely differ from the past experience. The POOL data storage mechanism is intended to be able to cope with the experiment's requirements applying a flexible multi technology data persistency mechanism. The developed technology independent approach is flexible enough to adopt new technologies, take advantage of existing schema evolution mechanisms and allows users to access data in a technology independent way. The framework consists of several components, which can be individually adopted and integrated into existing experiment frameworks.
△ Less
Submitted 11 June, 2003;
originally announced June 2003.
