Workshop della Commissione Calcolo e Reti Castiadas, 25-27 maggio 2004 Il progetto EGEE Luciano Gaido.

Presentazione sul tema: "Workshop della Commissione Calcolo e Reti Castiadas, 25-27 maggio 2004 Il progetto EGEE Luciano Gaido."— Transcript della presentazione:

1 Workshop della Commissione Calcolo e Reti Castiadas, 25-27 maggio 2004 Il progetto EGEE Luciano Gaido

2 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004 Il contesto 01/01 – 03/04 12/01 – 03/04 2002-2005 + 2006-2008 06/00 -...... Grid.it (FIRB) 01/03 – 12/05

3 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004 Risultati dei progetti precedenti Release di middleware EDG 2.x LCG 2.0 (basata su EDG 2.0) INFN-GRID 2.0 (basata su LCG 2.0) EGEE-0 (aka LCG 2.0) Infrastrutture Testbed EDG (development, application) Testbed INFN-GRID Testbed DataTAG Grid di produzione italiana (INFN-GRID, GRID-IT) Infrastruttura di produzione LCG

4 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004 Le infrastrutture TB0 TB1 TB2 TB3 Mar 2004 2001 2002 2003 Globus 1 EDG 1 EDG 1.4EDG 2 2004 Assessment R&D Jan 2001 LCG-1LCG-2 INFNGRID-1INFNGRID-2 Figura presa dalla pres. di M. Parsons a Cork

5 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004 Il progetto EGEE Obiettivo: “ EGEE aims to integrate current national, regional and thematic Grid efforts in order to create a seamless European Grid Infrastructure. This infrastructure will build on the underlying EU Research Network GEANT and exploit Grid expertise that has been generated by projects such as the European Union DataGrid project, other EU supported Grid projects and the national Grid initiatives such as UK e-Science, InfnGrid and NorduGrid. ” Iniziato il 1 aprile 2004, con durata biennale Previsto un rinnovo per un secondo biennio 70 partner, 32 MEuro Kick-off meeting a Cork (19-22 aprile 2004) Applications Geant network Grid infrastr.

6 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004 Organizzazione di EGEE I 70 partner sono stati divisi in 12 federazioni: 1.CERN 2.Central Europe including Austria, Czech Republic, Hungary, Poland, Slovakia and Slovenia 3.France 4.Germany and Switzerland 5.Ireland and the United Kingdom 6.Italy 7.Northern Europe including Belgium, Denmark, Finland, the Netherlands, Norway and Sweden 8.Russia 9.South-East Europe including Bulgaria, Cyprus, Greece, Israel and Romania 10.South-West Europe including Portugal and Spain 11.NRENS (National Research and Education Networks) 12.United States

7 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004 Struttura di EGEE Tre filoni di attivit à : Joint Research Activities: JRA1 – Middleware Engineering and IntegrationJRA1 – Middleware Engineering and Integration JRA2 – Quality Assurance JRA3 – Security JRA4 - Network Services Development Service Activities: SA1 – Grid Operation and SupportSA1 – Grid Operation and Support SA2 – Network Resource Provision Networking Activities: NA1 – Management of I3 (Integrated Infrastructure Initiative) NA2 – Dissemination and Outreach NA3 – User Training and Induction NA4 – Application Identification and SupportNA4 – Application Identification and Support NA5 – Internal Cooperation

8 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004 EGEE Implementation From day 1 (1 st April 2004) Production grid service based on the LCG infrastructure running LCG-2 grid middleware (SA) LCG-2 will be maintained until the new generation has proven itself (fallback solution) In parallel develop a “next generation” grid facility (JRA) Produce a new set of grid services according to evolving standards (Web Services) Run a development service providing early access for evaluation purposes Will replace LCG-2 on production facility in 2005 Globus 2 basedWeb services based EGEE-2EGEE-1LCG-2LCG-1 EDGVDT... LCG EGEE...AliEn Dalla pres. di Bob Jones a Cork

9 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004 EGEE and LCG EGEE builds on the work of LCG to establish a grid operations service LCG: a worldwide collaboration of The LHC experiments The Regional Computing Centres Physics institutes Mission: Prepare and deploy the computing environment that will be used by the experiments to analyse the LHC data Strategy: Integrate thousands of computers at dozens of participating institutes worldwide into a global computing resource Rely on software being developed in advanced grid technology projects, both in Europe and in the USA Status : LCG service up and running with LCG-2 mware – successfully being used for LHC data challenges Dalla pres. di Bob Jones a Cork

10 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004 JRA1: Middleware Engineering and Integration L ’ attivit à di sviluppo è distribuita su vari cluster: UK IT/CZ (F. Prelz, M. Sgaravatto) CERN Nordic Ogni cluster ha un suo development testbed distribuito L ’ integrazione e testing del m/w e lo sviluppo di tools è responsabilit à del CERN. Del cluster italo/ceco fa parte anche Datamat

11 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004 JRA1: Middleware Engineering and Integration Punto di partenza per la reingegnerizzazione del m/w: Design e componenti grid basati su quanto deciso dall ’ RTAG* ARDA (A Realization of Distributed Analysis) Design Team costituito da S. Fisher (UK), F. Prelz (IT/CZ), D. Groep (Nordic), M. Livny (VDT), P. Buncic (CERN), P. Kunszt (CERN), F. Hemmer (CERN) e E. Laure (CERN) Responsabilit à del CE (IT/CZ) Esperienza e componenti da AliEn, VDT e EDG Web Services * Requirements and Technical Assessment Group (LCG)

12 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004 JRA1: Middleware Engineering and Integration Key deliverables/milestones Jun’04 Tools for mware engineering and integration deployed (MJRA1.1) Jun’04 Architecture and Planning Document - Rel 1 (DJRA1.1) Jun’04 SW cluster develop. and testing infrastructure available (MJRA1.2) Aug’04 Integration and testing infrastructure in place with test plans – Rel 1 (MJRA1.3) Aug’04 Design Document for grid services - Rel 1 (DJRA1.2)

13 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004 NA4 Work Breakdown Structure http://alipc1.ct.infn.it/grid/egee/na4 Dalla pres. di R. Barbera a Cork

14 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004 NA4 Generic Applications goals Activity NA4 focuses on the identification and support of early-user and established applications for use on the EGEE infrastructure. It has the following objectives:  To identify through the dissemination partners and a well defined integration process a portfolio of early user applications from a broad range of application sectors from academia, industry and commerce.  To support development and production use of all of these applications on the EGEE infrastructure and thereby establish a strong user base on which to build a broad EGEE user community.  To initially focus on two well-defined application areas – Particle Physics and Life sciences. The expected outcome of the activity will be the establishment of a broad portfolio of applications across a wide range of sectors suited to execution on the EGEE infrastructure meeting the needs of a broad collection of user groups from many sectors across Europe as illustrated in Figure (note the timeline for the introduction of each application domain is purely illustrative). Dalla pres. di R. Barbera a Cork

15 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004 Conclusions and outlook for GILDA GILDA is a complete suite of grid elements (test-bed, CA, VO, monitoring system, web portal) and applications fully dedicated to dissemination purposes. GILDA runs and will run the last production (stable) version of the grid middleware (currently Grid.it 2.0.0 based on LCG 2.0.0). GILDA is the dissemination tool which will be used by NA3 during induction courses and tutorials (first in Edinburgh next week!). Next future (2-3 weeks): adding new sites: NeSC, InterQuanta (private company in Spain), any other interested; finalize and advertise the “24x7” public “grid demonstrator”: a customized version of the full GENIUS web portal from where all users belonging to the GILDA VO can submit a pre-defined set of applications to the GILDA Testbed; publish all the documentation on the GILDA web site. Dalla pres. di R. Barbera a Cork

16 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004 The GILDA Tutorials (1/2) Edinburgh, 7 April 2004 Tunis, 22-23 April 2004 Edinburgh, 26-28 April 2004, registration and programregistration and program CERN, 17-19 May 2004 Dalla pres. di R. Barbera a Cork

17 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004 SA1: obiettivi Core Infrastructure services: Operate essential grid services Grid monitoring and control: Proactively monitor the operational state and performance, Initiate corrective action Middleware deployment and resource induction: Validate and deploy middleware releases Set up operational procedures for new resources Resource provider and user support: Coordinate the resolution of problems from both Resource Centres and users Filter and aggregate problems, providing or obtaining solutions Grid management: Coordinate Regional Operations Centres (ROC) and Core Infrastructure Centres (CIC) Manage the relationships with resource providers via service-level agreements. International collaboration: Drive collaboration with peer organisations in the U.S. and in Asia-Pacific Ensure interoperability of grid infrastructures and services for cross-domain VO’s Participate in liaison and standards bodies in wider grid community

18 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004 SA1: strutture (1) RC (Resource Centres): Provide computing and storage resources for the grid OMC (Operation Management Center): Manage the operations of the entire EGEE grid from a single centralised location (CERN) CIC (Core Infrastructure Center): Provide the basic service infrastructure of the Grid Operate the key services which connect users with resources Support the Regional Operation Centers Numeri: RC (ad libitum), OMC (1), ROC (9), CIC (4)

19 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004 SA1: strutture (2) ROC (Regional Operation Center): Customize and certificate the middleware releases Provide the middleware release and up-to-date documentation to a set of Resource Centers (RC) Validate the set of grid services and components to form a functioning Grid to serve user communities Validate installation/upgrade procedures and documentation Develop/Run test suites for components and Grid services to certify a site installation Refer and escalate middleware problems to developers Collaborate with CIC to have monitoring systems and performance evaluator running Define the tools to measure the service level provided by resource centers Manage the access policies and sharing of the resources Provide 24x7 support service for users and local administrators

20 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004 SA1: distribuzione geografica delle strutture 10 federazioni: CERN OMC + CIC Francia ROC + CIC Italia ROC + CIC Germania e Svizzera ROC UK e Irlanda ROC + CIC Northern EuropeROC South West EuropeROC South East EuropeROC Central EuropeROC RussiaROC

21 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004 SA1 in Italia: da dove partiamo La grid in italia (InfnGrid) esiste dal 2001: E ’ diventata una infrastruttura di produzione (Grid-It) nel 2003 22 siti operativi (alcuni fanno parte anche dell ’ infrastruttura LCG2) Usata per le attivit à degli esperimenti (Data Challenge e produzioni): CMS preproduction (INFN-GRID_1.0 aka LCG_1.0, winter 2003) ALICE DC prima fase (febbraio-maggio 2004) ATLAS DC e produzioni di VIRGO e BABAR in partenza Prove di produzione di LHCb in corso Release: si usa INFN-GRID 2.0.0 (basata su LCG_2.0.0, 100% compatibile)

22 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004 http://grid-it.cnaf.infn.it

23 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004

24

25

26

27 SA1 in Italia Chi siamo: INFN-GRID Firb (Grid.it) tramite INFN ENEA UniLE* UniNA* UniCal* * SPACI ( S outhern P artnership for A dvanced C omputational I nfrastructures) Responsabili ROC italiano: C. Vistoli, L. Gaido

28 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004 SA1-IT: da dove partiamo The existing italian Grid Operation Center is the first step towards the EGEE ROC + CIC. It will evolve according to the EGEE plans. Existing teams: Release and Documentation Team - “builds” the INFN-GRID release, starting from the LCG release and (up to now) - provides the LCFGng profiles - provides documentation and a sw repository Central Management Team - Takes care of the deployment of the release, certificates the sites and manages the grid services

29 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004 SA1-IT: da dove partiamo (cont.) Existing teams (cont.): Monitoring Team GridICE development and support Provides a customized view for – VO, – Sites, – Grid Operation Center (+ services), – Users Support Team Provides support through a web portal (interim solution: OneorZero with a ticketing system and a knowledge base system)

30 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004 SA1: dove andiamo Per costruire una grid europea (EGEE), risultato dell’interconnessione delle grid nazionali, le strutture di operation, management e supporto esistenti nelle differenti regioni/federazioni devono armonizzarsi ed integrarsi evolvendo in una struttura comune. Percorso: definire un modello generale per ROC e CIC identificare le differenti esigenze dei ROC (e quindi dei Rersouce Center) Trovare il minimo comune denominatore (dei requirement) per definire il contenuto della release EGEE (in collaborazione con JRA1) Organizzarsi perche’ il resto (quello che non farà parte delle release ufficiale) venga prodotto e supportato da SA1 e/o da uno o più ROC  fondamentale evitare divergenze

31 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004 SA1: cosa stiamo facendo? I primi passi: Continuiamo a gestire le infrastrutture nazionali esistenti Creato gruppo ROC managers: Riunione telefonica settimanale Primo meeting ‘de visu’: milano 10-11 maggio Questionario sulle piattaforme hw e sistemi operativi Molte altre cose da definire..... In via di definizione i ruoli di CIC e ROC Definizione Execution Plan Definizione dei diversi testbed necessari (vedi figura)

32 EGEE Certification, Testing and Release Cycle CERTIFICATION TESTING SERVICES Integrate Basic Functionality Tests Run tests C&T suites Site suites Run Certification Matrix Release candidate tag PRE-PRODUCTION PRODUCTION APP INTEGR Certified release tag DEVELOPMENT & INTEGRATION UNIT & FUNCTIONAL TESTING Dev Tag JRA1 HEP EXPTS BIO-MED OTHER TBD APPS SW Installation DEPLOYMENT PREPARATION Deployment release tag DEPLOY SA1 Production tag

33 Workshop della Commissione Calcolo e Reti - 25-27 maggio 2004 Domande?