CRESCO Meeting Portici 30.Sett. 2008 ITALIAN NATIONAL AGENCY FOR NEW TECNOLOGY, ENERGY AND THE ENVIRONMENT

ENEA-GRID and CRESCO project Silvio Migliori, Giovanni Bracco, Salvatore Podda ENEA-Dipartimento Tecnologie Fisiche e Nuovi Materiali (FIM) ENEA-GRID and CRESCO project Contents ENEA-GRID infrastructure (www.afs.enea.it/project/eneagrid) CRESCO project and the “Virtual Lab” (www.cresco.enea.it)

Ringraziamenti (1) Ministero della Ricerca e dell’Università (MUR) Direzione dell’ENEA Capo Dipartimento Ing. Sandro Taglienti Comitato Scientifico: J. Dongarra, A. Mathis, D. Ferrari Macrostruttura di progetto : Andrea Quintiliani, Vittorio Rosato, Sandro Bologna Collaboratori direttamente coinvolti : Giovanni Bracco, Salvatore Podda, Samuele Pierattini, Pietro D’Angelo, Daniele Remoli, Pasquale Cuomo, Francesco Beone, Antonio Perozziello, Carlo Pinto, Erminia Mattei, Gloria Tabili, Nadia Mencucci, CaterinaVinci, Dante Giammattei, Alessandro De Stefano,Vincenzo Colaci,Gaziano Furini, Matteo De Rosa Il personale dell’Unità FIM-INFOGER coordinato da Roberto Guadagni

Ringraziamenti (2) Il personale del Centro ENEA di Portici e il Direttore Ezio Terzini I ricercatori assegnisti di ricerca del progetto CRESCO Università, Consorzi e Società Partner di progetto Tutti i Ricercatori ENEA ed i rispettivi responsabili coinvolti (circa 60 ricercatori dei Centri di Bologna, Casaccia, Frascati, Portici, Trisaia, Brindisi) I ricercatori che per primi hanno utilizzato il sistema ed hanno condiviso con noi le problematiche iniziali Il personale della ditta Computer Var e IBM per la realizzazione dell’infrastruttura

ENEA-GRID Born in 1998 with 6 Italian geographic clusters Main aim Give to ENEA researchers a unified environment to use in an easy way heterogeneous computers and applications software Build an homogeneous ICT infrastructure but with a distributed and delegated control Integrate the ENEA-GRID with National and International GRIDS Application Fabric Connectivity Resource Collective User Data archives Colleagues & 3D Software catalogs Computers ENEA-GRID www.afs.enea.it/project/eneagrid Sensor nets

ENEA-GRID from the user point of view Main feature: Access from any kind of connection Sharing data in world wide areas (geographical file system AFS) Access to the data from any kind of digital device client Running any kind of programs Access to National and International GRIDS Computers Computers Computers Computers Data archives ENEA-GRID Sensor nets Computers Sensor nets Software catalogs EU… Trisaia Brindisi Portici ( Roma-1 Roma-2 Bologna

ENEA-GRID from application point of view Potentiality: Running all Software Model for any kind of computer architecture Only one installation for each software for all geographical computers All authorized users can have access the software from local site Any application can share the data in the common geographical file system All sensors can share the data easily using the geographical file systems Computers Software Model 1 Computers Computers Rules Software Model 2 ENEA-GRID Sensor nets Software Model 3 Computers Sensor nets Software Model .. Software catalogs EU… Trisaia Brindisi Portici ( Roma-1 Roma-2 Bologna

LSF multi-cluster as integrator ENEA GRID WEB ICA Application Portal (ICA-Protocol) Graphical User Interface User programs and commercial codes LSF multi-cluster as integrator Monitoring, Automatic recovery Quality of service Accaunting XTERM LSF AFS Geographical cross platform & File System AIX SGI … Linux Alpha SUN HP Win Linux

ENEA GRID A Simple view of ENEA GRID Cluster 2° (Casaccia) Load Leveler LSF Graphic User Interface LSF multi-cluster as integrator Telnet User programs & commercial code ICA WEB Cluster 5° ( Portici) AFS Geographical cross platform& File System Load Leveler LSF Graphic User Interface LSF multi-cluster as integrator Telnet User programs & commercial code ICA WEB Cluster 2° (Casaccia) & File System Load Leveler LSF Graphic User Interface Telnet ICA WEB Cluster 6° ( Brindisi) ENEA GRID AFS Geographical cross platform& File System Load Leveler LSF Graphic User Interface LSF multi-cluster as integrator Telnet User programs & commercial code ICA WEB Cluster 1° ( Bologna) AFS Geographical cross platform& File System Load Leveler LSF Graphic User Interface LSF multi-cluster as integrator Telnet User programs & commercial code ICA WEB Cluster 3° (Frascati) AFS Geographical cross platform& File System Load Leveler LSF Graphic User Interface LSF multi-cluster as integrator Telnet User programs & commercial code ICA WEB Cluster 4° (Trisaia )

ENEA-GRID Computational & 3D Centers Ispra Saluggia BOLOGNA 30 S.Teresa #CPU/Core CASACCIA 140 400 Manfredonia FRASCATI 2750 PORTICI 90 BRINDISI 45 TRISAIA

ENEA-GRID Computational Resources OS #CPU/Core Gflops AIX >300 3000 Frascati(258), Bologna(24), Portici(18),Brindisi(2) Linux x86 32/64 >3000 25000 Frascati(140), Casaccia(54), Portici(2700),Trisaia(20), Brindisi(84) Linux IA64 (Altix) 64 300 Casaccia IRIX 26 40 Frascati(8), Casaccia(4), Portici(1),Trisaia(8), Brindisi(1), Bologna(5) Solaris 8 10 Trisaia(4), Casaccia(2), Bologna(2) Windows 32/64 46 100 Frascati(4), Portici(34),Trisaia(4), Brindisi(4) Mac OS X 14 60 Frascati(1), Trisaia(13) Computational resources of Portici (NA) , Brindisi e Trisaia (MT) have been financed by Ministry of Research PON TELEGRID (Avv. MIUR 68) CRESCO (Avv. MUR 1575)

HPC CRESCO INFRASTRUCTURE 17 TFlops Backup system 300 TByte IBM Tape Library TS3500 with 4 drives SERVERS BACKUP 3 Nodi IBM 3650 FC IB Interconnections InfiniBand 4XDDR IB SERVERS GPFS 4 Nodi IBM 3650 FC High speed disc 2 GByte/s 160 TByte IBM/DDN 9550 Section 1 (Large memory) 42 Nodi SMP IBM x3850-M2 with 4 Xeon Quad-Core Tigerton E7330 (32/64 GByte RAM 2.4GHz/ 1066MHz/6MB L2) total 672 cores Intel Tigerton Section 3 (Special) 4 Nodi blades IBM QS21 con 2 Cell BE Processors 3.2 Ghz each. 6 Nodi IBM x3755, 8 Core AMD 8222 FPGA VIRTEX5 4 Nodi IBM x 3755, 8 core AMD 8222 with NVIDIA Quadro FX 4500 X2 4 Nodi windows 8 core 16 Byte RAM Double interconnection at 1 Gbit & 1 network at 1 Gbit for management Section 2 (MPP) 256 Nodi blades IBM HS21 with 2 Xeon Quad-Core Clovertown E5345 (2.33GHz/1333MHz/8MB L2), 16 GB RAM total 2048 cores Intel Clovertown 4x10 Gbits 1000 Mbits 35 Node for services Servers for : Front-end installations AFS … Portici LAN GARR (WAN) 1000 Mbits

CRESCO HPC hall

CRESCO HPC details Sections 1 & 3 Section 2 & DDN network: IB/GB Corridor hot Sections 1 &2 Control room network: IB/GB backup system

Computational ability ENEA-GRID & CRESCO An heterogeneous system with more than 3000 CPU/cores Code Risorsa Serial Section 1 & ENEA-GRID Parallel hardly coupled Section 2 Embarrassing parallel Section 1, 2, ENEA-GRID System with several parallel codes Section 1, 2, Section 1+2 Special ( FPGA, GB, CELL) Section 3 windows (serial and parallel) Not LINUX ENEA-GRID Large date “Data Base” Section 1,2,3 e GPFS

CRESCO in the Top500 List June 2008 Italy (06/2008) 90 CINECA 125 ENEA 131 ENI 135 CILEA 156 226 Telecom Jack Dongarra 2002

ENEA GRID and experimental facilities DNA Sequence system ( ABI Prism 3700) Trisaia DB_2 ENEA GRID CPUS WEB ICA SSH DB_1 Controlled Nuclear Fusion: FTU Frascati Tokamak Upgrade Video Acquisition DB_3 Electron Microscope (Brindisi) 300 KeV (sept. 2004)

? La ENEA-GRID interoperability with others GRID ENEA has been developing the “shared proxy” solution Maintain the GRID internal architecture and autonomy Allow multiplatform impementations Under production on EGEE Under production on GRISU Required by EFDA for EGEE EGEE ? EFDA PI2S2 GARR GARR Others Entity

Italian GRID Infrastructure ENEA-GRID & GRISU & IGI IGI Italian GRID Infrastructure Ispra Saluggia INFN-GRID BOLOGNA ENEA-GRID S.Teresa CASACCIA Manfredonia FRASCATI PORTICI BRINDISI TRISAIA GRISU SPACI

CRESCO for EUROPEAN FUSION IB System for development and test of computational codes for ITER (EFDA contract) 1 Tflops peak Cresco 25 Tflops peak

Gateway to 2 0 0 8 Brindisi EGEE & GRISU GEANT Accesso Accesso WEB boquad.bologna.enea.it dafne.casaccia.enea.it sp4_-1.frascati.enea.it Portici Kleos.portici.enea.it infocal.trisia.enea.it Gateway to EGEE & GRISU Brindisi ercules.brindisi.enea.it Accesso WEB Accesso Client Accesso Diretto www.afs.enea.it/project/eneagrid

CRESCO is totally integrate ENEA-GRID & CRESCO CRESCO is totally integrate in ENEA-GRID

Example of use of FLUENT on ENEA-GRID Sicurezza industriale Sicurezza ENEA External Users Submissions options Fluent options

ENEA-GRID & Virtual Lab architecture Li ENEA University and research institute Enterprise New collaborations ENEA-GRID

Per lavorare con CRESCO www.cresco.enea.it

Thanks for attention ITALIAN NATIONAL AGENCY FOR NEW TECNOLOGY, ENERGY AND THE ENVIRONMENT