-
The RTApipe framework for the gamma-ray real-time analysis software development
Authors:
N. Parmiggiani,
A. Bulgarelli,
D. Beneventano,
V. Fioretti,
A. Di Piano,
L. Baroncelli,
A. Addis,
M. Tavani,
C. Pittori,
I. Oya
Abstract:
In the multi-messenger era, coordinating observations between astronomical facilities is mandatory to study transient phenomena (e.g. Gamma-ray bursts) and is achieved by sharing information with the scientific community through networks such as the Gamma-ray Coordinates Network. The facilities usually develop real-time scientific analysis pipelines to detect transient events, alert the astrophysi…
▽ More
In the multi-messenger era, coordinating observations between astronomical facilities is mandatory to study transient phenomena (e.g. Gamma-ray bursts) and is achieved by sharing information with the scientific community through networks such as the Gamma-ray Coordinates Network. The facilities usually develop real-time scientific analysis pipelines to detect transient events, alert the astrophysical community, and speed up the reaction time of science alerts received from other observatories. We present in this work the RTApipe framework, designed to facilitate the development of real-time scientific analysis pipelines for present and future gamma-ray observatories. This framework provides pipeline architecture and automatisms, allowing the researchers to focus on the scientific aspects and integrate existing science tools developed with different technologies. The pipelines automatically execute all the configured analyses during the data acquisition. This framework can be interfaced with science alerts networks to perform follow-up analysis of transient events shared by other facilities. The analyses are performed in parallel and can be prioritised. The workload is highly scalable on a cluster of machines. The framework provides the required services using containerisation technology for easy deployment. We present the RTA pipelines developed for the AGILE space mission and the prototype of the SAG system for the ground-based future Cherenkov Telescope Array observatory confirming that the RTApipe framework can be used to successfully develop pipelines for the gamma-ray observatories, both space and ground-based.
△ Less
Submitted 26 February, 2023;
originally announced February 2023.
-
Preliminary Results of a New Deep Learning Method to Detect and Localize GRBs in the AGILE/GRID Sky Maps
Authors:
N. Parmiggiani,
A. Bulgarelli,
A. Macaluso,
V. Fioretti,
A. Di Piano,
L. Baroncelli,
A. Addis,
M. Landoni,
C. Pittori,
F. Verrecchia,
F. Lucarelli,
A. Giuliani,
F. Longo,
D. Beneventano,
M. Tavani
Abstract:
AGILE is an ASI space mission launched in 2007 to study X-ray and gamma-ray phenomena in the energy range from $\sim20$ keV to $\sim10$ GeV. The AGILE Team developed a real-time analysis pipeline for the fast detection of transient sources, and the follow-up of external science alerts received through networks such as the General Coordinates Network. We developed a new Deep Learning method for det…
▽ More
AGILE is an ASI space mission launched in 2007 to study X-ray and gamma-ray phenomena in the energy range from $\sim20$ keV to $\sim10$ GeV. The AGILE Team developed a real-time analysis pipeline for the fast detection of transient sources, and the follow-up of external science alerts received through networks such as the General Coordinates Network. We developed a new Deep Learning method for detecting and localizing Gamma-Ray Bursts (GRB) in the AGILE/GRID sky maps. We trained the model using sky maps with GRBs simulated in a radius of 20 degrees from the center of the map, which is larger than 99.5 \% of the error region present in the GRBWeb catalog. We also plan to apply this method to search for counterparts of gravitational wave events, which typically have a wider localization error region. The method comprises two Deep Learning models implemented with two Convolutional Neural Networks. The first model detects and filters sky maps containing a GRB, while the second model localizes its position. We trained and tested the models using simulated data. The detection model achieves an accuracy of 95.7 \%, and the localization model has a mean error lower than 0.8 degrees. We configured a Docker container with all the required software for data simulation and deployed it using the Amazon Web Service to calculate the p-value distribution under different conditions. With the p-value distribution, we can calculate the statistical significance of a detection.
△ Less
Submitted 21 February, 2023;
originally announced February 2023.
-
The AGILE real-time analysis pipelines in the multi-messenger era
Authors:
N. Parmiggiani,
A. Bulgarelli,
A. Ursi,
V. Fioretti,
L. Baroncelli,
A. Addis,
A. Di Piano,
C. Pittori,
F. Verrecchia,
F. Lucarelli,
M. Tavani,
D. Beneventano
Abstract:
In the multi-messenger era, space and ground-based observatories usually develop real-time analysis (RTA) pipelines to rapidly detect transient events and promptly share information with the scientific community to enable follow-up observations. These pipelines can also react to science alerts shared by other observatories through networks such as the Gamma-Ray Coordinates Network (GCN) and the As…
▽ More
In the multi-messenger era, space and ground-based observatories usually develop real-time analysis (RTA) pipelines to rapidly detect transient events and promptly share information with the scientific community to enable follow-up observations. These pipelines can also react to science alerts shared by other observatories through networks such as the Gamma-Ray Coordinates Network (GCN) and the Astronomer's Telegram (ATels). AGILE is a space mission launched in 2007 to study X-ray and gamma-ray phenomena. This contribution presents the technologies used to develop two types of AGILE pipelines using the RTApipe framework and an overview of the main scientific results. The first type performs automated analyses on new AGILE data to detect transient events and automatically sends AGILE notices to the GCN network. Since May 2019, this pipeline has sent more than 50 automated notices with a few minutes delay since data arrival. The second type of pipeline reacts to multi-messenger external alerts (neutrinos, gravitational waves, GRBs, and other transients) received through the GCN network and performs hundreds of analyses searching for counterparts in all AGILE instruments' data. The AGILE Team uses these pipelines to perform fast follow-up of science alerts reported by other facilities, which resulted in the publishing of several ATels and GCN circulars.
△ Less
Submitted 11 August, 2021; v1 submitted 10 August, 2021;
originally announced August 2021.
-
A Deep Learning Method for AGILE-GRID GRB Detection
Authors:
N. Parmiggiani,
A. Bulgarelli,
V. Fioretti,
A. Di Piano,
A. Giuliani,
F. Longo,
F. Verrecchia,
M. Tavani,
D. Beneventano,
A. Macaluso
Abstract:
The follow-up of external science alerts received from Gamma-Ray Bursts (GRB) and Gravitational Waves (GW) detectors is one of the AGILE Team's current major activities. The AGILE team developed an automated real-time analysis pipeline to analyse AGILE Gamma-Ray Imaging Detector (GRID) data to detect possible counterparts in the energy range 0.1-10 GeV. This work presents a new approach for detect…
▽ More
The follow-up of external science alerts received from Gamma-Ray Bursts (GRB) and Gravitational Waves (GW) detectors is one of the AGILE Team's current major activities. The AGILE team developed an automated real-time analysis pipeline to analyse AGILE Gamma-Ray Imaging Detector (GRID) data to detect possible counterparts in the energy range 0.1-10 GeV. This work presents a new approach for detecting GRBs using a Convolutional Neural Network (CNN) to classify the AGILE-GRID intensity maps improving the GRBs detection capability over the Li&Ma method, currently used by the AGILE team. The CNN is trained with large simulated datasets of intensity maps. The AGILE complex observing pattern due to the so-called 'spinning mode' is studied to prepare datasets to test and evaluate the CNN. A GRB emission model is defined from the Second Fermi-LAT GRB catalogue and convoluted with the AGILE observing pattern. Different p-value distributions are calculated evaluating with the CNN millions of background-only maps simulated varying the background level. The CNN is then used on real data to analyse the AGILE-GRID data archive, searching for GRB detections using the trigger time and position taken from the Swift-BAT, Fermi-GBM, and Fermi-LAT GRB catalogues. From these catalogues, the CNN detects 21 GRBs with a significance $\geq 3 σ$, while the Li&Ma method detects only two GRBs. The results shown in this work demonstrate that the CNN is more effective in detecting GRBs than the Li&Ma method in this context and can be implemented into the AGILE-GRID real-time analysis pipeline.
△ Less
Submitted 18 May, 2021;
originally announced May 2021.
-
RTApipe, a framework to develop astronomical pipelines for the real-time analysis of scientific data
Authors:
N. Parmiggiani,
A. Bulgarelli,
D. Beneventano,
V. Fioretti,
L. Baroncelli,
A. Addis,
M. Tavani
Abstract:
In the multi-messenger era, astronomical projects share information about transients phenomena issuing science alerts to the Scientific Community through different communications networks. This coordination is mandatory to understand the nature of these physical phenomena. For this reason, astrophysical projects rely on real-time analysis software pipelines to identify as soon as possible transien…
▽ More
In the multi-messenger era, astronomical projects share information about transients phenomena issuing science alerts to the Scientific Community through different communications networks. This coordination is mandatory to understand the nature of these physical phenomena. For this reason, astrophysical projects rely on real-time analysis software pipelines to identify as soon as possible transients (e.g. GRBs), and to speed up external alerts' reaction time. These pipelines can share and receive the science alerts through the Gamma-ray Coordinates Network. This work presents a framework designed to simplify the development of real-time scientific analysis pipelines. The framework provides the architecture and the required automatisms to develop a real-time analysis pipeline, allowing the researchers to focus more on the scientific aspects. The framework has been successfully used to develop real-time pipelines for the scientific analysis of the AGILE space mission data. It is planned to reuse this framework for the Super-GRAWITA and AFISS projects. A possible future use for the Cherenkov Telescope Array (CTA) project is under evaluation.
△ Less
Submitted 18 May, 2021;
originally announced May 2021.
-
The On-Site Analysis of the Cherenkov Telescope Array
Authors:
Andrea Bulgarelli,
Valentina Fioretti,
Andrea Zoli,
Alessio Aboudan,
Juan José Rodríguez-Vázquez,
Giovanni De Cesare,
Adriano De Rosa,
Gernot Maier,
Etienne Lyard,
Denis Bastieri,
Saverio Lombardi,
Gino Tosti,
Sonia Bergamaschi,
Domenico Beneventano,
Giovanni Lamanna,
Jean Jacquemier,
Karl Kosack,
Lucio Angelo Antonelli,
Catherine Boisson,
Jerzy Borkowski,
Sara Buson,
Alessandro Carosi,
Vito Conforti,
Pep Colomé,
Raquel de los Reyes
, et al. (32 additional authors not shown)
Abstract:
The Cherenkov Telescope Array (CTA) observatory will be one of the largest ground-based very high-energy gamma-ray observatories. The On-Site Analysis will be the first CTA scientific analysis of data acquired from the array of telescopes, in both northern and southern sites. The On-Site Analysis will have two pipelines: the Level-A pipeline (also known as Real-Time Analysis, RTA) and the level-B…
▽ More
The Cherenkov Telescope Array (CTA) observatory will be one of the largest ground-based very high-energy gamma-ray observatories. The On-Site Analysis will be the first CTA scientific analysis of data acquired from the array of telescopes, in both northern and southern sites. The On-Site Analysis will have two pipelines: the Level-A pipeline (also known as Real-Time Analysis, RTA) and the level-B one. The RTA performs data quality monitoring and must be able to issue automated alerts on variable and transient astrophysical sources within 30 seconds from the last acquired Cherenkov event that contributes to the alert, with a sensitivity not worse than the one achieved by the final pipeline by more than a factor of 3. The Level-B Analysis has a better sensitivity (not be worse than the final one by a factor of 2) and the results should be available within 10 hours from the acquisition of the data: for this reason this analysis could be performed at the end of an observation or next morning. The latency (in particular for the RTA) and the sensitivity requirements are challenging because of the large data rate, a few GByte/s. The remote connection to the CTA candidate site with a rather limited network bandwidth makes the issue of the exported data size extremely critical and prevents any kind of processing in real-time of the data outside the site of the telescopes. For these reasons the analysis will be performed on-site with infrastructures co-located with the telescopes, with limited electrical power availability and with a reduced possibility of human intervention. This means, for example, that the on-site hardware infrastructure should have low-power consumption. A substantial effort towards the optimization of high-throughput computing service is envisioned to provide hardware and software solutions with high-throughput, low-power consumption at a low-cost.
△ Less
Submitted 7 September, 2015;
originally announced September 2015.
-
The Agile Alert System For Gamma-Ray Transients
Authors:
A. Bulgarelli,
M. Trifoglio,
F. Gianotti,
M. Tavani,
N. Parmiggiani,
V. Fioretti,
A. W. Chen,
S. Vercellone,
C. Pittori,
F. Verrecchia,
F. Lucarelli,
P. Santolamazza,
G. Fanari,
P. Giommi,
D. Beneventano,
A. Argan,
A. Trois,
E. Scalise,
F. Longo,
A. Pellizzoni,
G. Pucella,
S. Colafrancesco,
V. Conforti,
P. Tempesta,
M. Cerone
, et al. (4 additional authors not shown)
Abstract:
In recent years, a new generation of space missions offered great opportunities of discovery in high-energy astrophysics. In this article we focus on the scientific operations of the Gamma-Ray Imaging Detector (GRID) onboard the AGILE space mission. The AGILE-GRID, sensitive in the energy range of 30 MeV-30 GeV, has detected many gamma-ray transients of galactic and extragalactic origins. This wor…
▽ More
In recent years, a new generation of space missions offered great opportunities of discovery in high-energy astrophysics. In this article we focus on the scientific operations of the Gamma-Ray Imaging Detector (GRID) onboard the AGILE space mission. The AGILE-GRID, sensitive in the energy range of 30 MeV-30 GeV, has detected many gamma-ray transients of galactic and extragalactic origins. This work presents the AGILE innovative approach to fast gamma-ray transient detection, which is a challenging task and a crucial part of the AGILE scientific program. The goals are to describe: (1) the AGILE Gamma-Ray Alert System, (2) a new algorithm for blind search identification of transients within a short processing time, (3) the AGILE procedure for gamma-ray transient alert management, and (4) the likelihood of ratio tests that are necessary to evaluate the post-trial statistical significance of the results. Special algorithms and an optimized sequence of tasks are necessary to reach our goal. Data are automatically analyzed at every orbital downlink by an alert pipeline operating on different timescales. As proper flux thresholds are exceeded, alerts are automatically generated and sent as SMS messages to cellular telephones, e-mails, and push notifications of an application for smartphones and tablets. These alerts are crosschecked with the results of two pipelines, and a manual analysis is performed. Being a small scientific-class mission, AGILE is characterized by optimization of both scientific analysis and ground-segment resources. The system is capable of generating alerts within two to three hours of a data downlink, an unprecedented reaction time in gamma-ray astrophysics.
△ Less
Submitted 15 January, 2014;
originally announced January 2014.
