Alvise Dorigo
Istituto Nazionale di Fisica Nucleare, Sezione di Padova, Department Member
Research Interests:
Research Interests:
The European Middleware Initiative (EMI) project aims to deliver a consolidated set of middleware products based on the four major middleware providers in Europe -ARC, dCache, gLite and UNICORE. The CREAM (Computing Resource Execution And... more
The European Middleware Initiative (EMI) project aims to deliver a consolidated set of middleware products based on the four major middleware providers in Europe -ARC, dCache, gLite and UNICORE. The CREAM (Computing Resource Execution And Management) Service, a service for job management operation at the Computing Element (CE) level, is a software product which is part of the EMI middleware distribution. In this paper we discuss about some new functionality in the CREAM CE introduced with the first EMI major release (EMI-1, codename Kebnekaise). The integration with the Argus authorization service is one of these implementations: the use of a unique authorization system, besides simplifying the overall management, allows also to avoid inconsistent authorization decisions. An improved support for complex deployment scenarios (e.g. for sites having multiple CE head nodes and/or having heterogeneous resources) is another new achievement. The improved support for resource allocation in a multi-core environment, and the initial support of version 2.0 of the Glue specification for resource publication are other new functionalities introduced with the first EMI release.
Research Interests:
... S Monforte4,5, F Prelz3, D Rebatto3, M Sgaravatto1, L Zangrando1 1 INFN Padova, Via Marzolo 8, I-35131 Padova, Italy 2 INFN-CNAF, Viale Berti Pichat 6/2, I-40127 Bologna Italy 3 INFN Milano, Via Celoria 16, I-20133 Milano, Italy 4... more
... S Monforte4,5, F Prelz3, D Rebatto3, M Sgaravatto1, L Zangrando1 1 INFN Padova, Via Marzolo 8, I-35131 Padova, Italy 2 INFN-CNAF, Viale Berti Pichat 6/2, I-40127 Bologna Italy 3 INFN Milano, Via Celoria 16, I-20133 Milano, Italy 4 INFN Catania, Via Santa Sofia 64, I-95123 ...
The EU-funded project EMI, now at its second year, aims at providing a unified, high quality middleware distribution for e-Science communities. Several aspects about workload management over diverse distributed computing environments are... more
The EU-funded project EMI, now at its second year, aims at providing a unified, high quality middleware distribution for e-Science communities. Several aspects about workload management over diverse distributed computing environments are being challenged by the EMI roadmap: enabling seamless access to both HTC and HPC computing services, implementing a commonly agreed framework for the execution of parallel computations and supporting interoperability models between Grids and Clouds. Besides, a rigourous requirements collection process, involving the WLCG and various NGIs across Europe, assures that the EMI stack is always committed to serving actual needs. With this background, the gLite Workload Management System (WMS), the meta-scheduler service delivered by EMI, is augmenting its functionality and scheduling models according to the aforementioned project roadmap and the numerous requirements collected over the first project year. This paper is about present and future work of the EMI WMS, reporting on design changes, implementation choices and longterm vision.
Research Interests:
The Logging and Bookkeeping service tracks jobs passing through the Grid. It collects important events generated by both the grid middleware components and applications, and processes them at a chosen LB server to provide the job state.... more
The Logging and Bookkeeping service tracks jobs passing through the Grid. It collects important events generated by both the grid middleware components and applications, and processes them at a chosen LB server to provide the job state. The events are transported through secure and reliable channels. Job tracking is fully distributed and does not depend on a single information source, the robustness is achieved through speculative job state computation in case of reordered, delayed or lost events. The state computation is easily adaptable to modified job control flow.
Research Interests:
This paper describes the achievements of the H2020 project INDIGO-DataCloud. The project has provided e-infrastructures with tools, applications and cloud framework enhancements to manage the demanding requirements of scientific... more
This paper describes the achievements of the H2020 project INDIGO-DataCloud. The project has provided e-infrastructures with tools, applications and cloud framework enhancements to manage the demanding requirements of scientific communities, either locally or through enhanced interfaces. The middleware developed allows to federate hybrid resources, to easily write, port and run scientific applications to the cloud. In particular, we have extended existing PaaS (Platform as a Service) solutions, allowing public and private e-infrastructures, including those provided by EGI, EUDAT, and Helix Nebula, to integrate their existing services and make them available through AAI services compliant with GEANT interfederation policies, thus guaranteeing transparency and trust in the provisioning of such services. Our middleware facilitates the execution of applications using containers on Cloud and Grid based infrastructures, as well as on HPC clusters. Our developments are freely downloadable ...
Research Interests:
Research Interests:
Research Interests:
Research Interests:
Research Interests:
When dealing with the concurrent access from a multitude of clients to petabyte-scale data repositories, high performance, fault tolerance, robustness, and scalability are four very important issues. This paper describes the choices and... more
When dealing with the concurrent access from a multitude of clients to petabyte-scale data repositories, high performance, fault tolerance, robustness, and scalability are four very important issues. This paper describes the choices and the work done to address the client side of high demand data access needs of modern physics experiments, such as the BaBar experiment at SLAC, and of any other field in which a reliable data access is a primary issue. For this purpose a highly scalable architecture has been designed and deployed which allows thousands of batch jobs and interactive sessions to effectively access the data repositories with as few fails as possible. ROOT REMOTE DATA ACCESS ROOT provides a remote file access mechanism via a TCP/IP-based data server daemon known as rootd, and its only purpose is to serve opaque data. rootd and the ROOT framework allow an analysis job to get access to local or remote files in a transparent way without any change to the source code. In fact...
Research Interests:
Abstract:- When dealing with the concurrent access from a multitude of clients to petabyte-scale data repositories, high performance, fault tolerance, robustness, and scalability are four very important issues. This work describes the... more
Abstract:- When dealing with the concurrent access from a multitude of clients to petabyte-scale data repositories, high performance, fault tolerance, robustness, and scalability are four very important issues. This work describes the architecture and the choices done in designing the xrootd file access system. The first goal of the system was to provide access to over 10^7 files representing several petabytes of experimental physics data. This work addresses the high demand data access needs of modern physics experiments, such as the BaBar experiment at SLAC, and covers method and tools useful to any other field in which reliability, performance and scalability in data access are a primary issue. Key-Words:- Scalability, Fault tolerance, load balancing, peer to peer, XROOTD, TXNetFile, ROOT 1- From a storage system to a data access architecture The BaBar experiment [1] at the Stanford Linear Accelerator Center 1 produces a huge amount of data to be accessed by a high number of anal...
Research Interests:
Research Interests:
Research Interests:
... E-mail: glite-jobmgmt-devel@lists.infn.it 1 INFN Padova, Via Marzolo 8, I-35131 Padova, Italy 2 INFN-CNAF, Viale Berti Pichat 6/2, I-40127 Bologna Italy 3 INFN Milano, Via Celoria 16, I-20133 Milano, Italy 4 INFN Catania, Via Santa... more
... E-mail: glite-jobmgmt-devel@lists.infn.it 1 INFN Padova, Via Marzolo 8, I-35131 Padova, Italy 2 INFN-CNAF, Viale Berti Pichat 6/2, I-40127 Bologna Italy 3 INFN Milano, Via Celoria 16, I-20133 Milano, Italy 4 INFN Catania, Via Santa Sofia 64, I-95123, Italy Abstract. ...
Research Interests:
Research Interests:
Job execution and management is one of the most important functionality provided by every modern Grid middleware. In this paper we describe how the problem of job management has been addressed in the gLite middleware by means of the CREAM... more
Job execution and management is one of the most important functionality provided by every modern Grid middleware. In this paper we describe how the problem of job management has been addressed in the gLite middleware by means of the CREAM and CEMonitor services. CREAM (Computing Resource Execution and Management) provides a job execution and management capability for Grid systems, while CEMonitor is a generalpurpose asynchronous event notification framework. Both services expose a Web Service interface allowing conforming clients to submit, manage and monitor computational jobs to a Local Resource Management System. PACS: 89.20.Ff Computer Science and Technology Published by SIS-Pubblicazioni
Research Interests:
Research Interests:
Research Interests:
The CMS experiment will produce several Pbytes of data every year, to be distributed over many computing centers geographically distributed in different countries. Analysis of this data will be also performed in a distributed way, using... more
The CMS experiment will produce several Pbytes of data every year, to be distributed over many computing centers geographically distributed in different countries. Analysis of this data will be also performed in a distributed way, using grid infrastructure. CRAB (CMS Remote Analysis Builder) is a specific tool, designed and developed by the CMS collaboration, that allows a transparent access to distributed data to end physicist. Very limited knowledge of underlying technicalities are required to the user. CRAB interacts with the local user environment, the CMS Data Management services and with the Grid middleware. It is able to use WLCG, gLite and OSG middleware. CRAB has been in production and in routine use by end-users since Spring 2004. It has been extensively used in studies to prepare the Physics Technical Design Report (PTDR) and in the analysis of reconstructed event samples generated during the Computing Software and Analysis Challenge (CSA06). This involved generating thousands of jobs per day at peak rates. In this paper we discuss the current implementation of CRAB, the experience with using it in production and the plans to improve it in the immediate future.