Michele Punturo INFN Perugia 1. Riunione di oggi Mattina: problemi immediati – Trasferimento dati da LIGO (Luca Rei) – Trasferimento dati da.

Presentazione sul tema: "Michele Punturo INFN Perugia 1. Riunione di oggi Mattina: problemi immediati – Trasferimento dati da LIGO (Luca Rei) – Trasferimento dati da."— Transcript della presentazione:

1 Virgo@CNAF Michele Punturo INFN Perugia 1

2 Riunione di oggi Mattina: problemi immediati – Trasferimento dati da LIGO (Luca Rei) – Trasferimento dati da EGO/Virgo (Livio Salconi) Pomeriggio: necessità ed evoluzioni (tempo scala: pochi mesi) – Adattare una pipeline di analisi alla infrastruttura del CNAF (Lisa Zangrando) – Adattare l’infrastruttura del CNAF alle pipelines di LIGO/Virgo 2

3 Data Transfer 3

4 Data Transfer (DT) We defined a DT team There are different type of data to be transferred EGO-to-CC (Bulk data, 2- 4TB/day, trend 4GB/day, …): – L.Salconi (EGO) LIGO-to-CNAF(RDS+h(t), 75GB/day): – L.Rei (INFN-Ge) LIGO-to-CCIN2P3 (RDS+h(t), 75GB/day): – M.A. Bizouard (LAL) O1 DT still to start Some reliability improvement needed in the “pyton” based DT from EGO to CC: – Discussion with CNAF will start in November – Discussion with CCIN2P3 immediately after Low latency DT (few GB/day): – B. Mours (LAPP) 4

5 Scopo di questo meeting (1) Per la data trasfer il target è uscire da questo meeting con le soluzioni tecniche che ci occorrono per rendere più affidabile il trasferimento dei bulk data e per far funzionare il trasferimento di dati da LIGO 5

6 Running @ CNAF 6

7 Virgo@CNAF At CNAF can run at the moment few “local” pipelines: – CW_FrequencyHough (Roma1) – CW_FDStat (Polgraw) – MBTA (development) “Global” LSC-Virgo pipelines cannot run at CNAF because are based on LSC environment – Static file system – I/O files in well defined directories – See Patrick Brady and Luca Rei talks However, Frequency Hough over-saturated the resources reserved to Virgo 7

8 CPU energy Virgo is the 3 rd CPU consumer of the CNAF, just after the large LHC experiments HS06.day Source: CdG CNAF Oct. 2015 Sept.2015 data 8

9 Jobs Big fluctuations 9

10 Virgo at CCIN2P3 Different is the situation at CCIN2P3 In 2015 only 1.2M HS06.hour have been used of the 24M HS06.hour requested Only 3 pipelines are running: cosmic strings, EM follow-up and Omicron The weight of Virgo wrt the other experiments is negligible in CCIN2P3 and small wrt Virgo@CNAF 10

11 Two steps strategy How to increase the role of “Virgo” computing facilities/centers allowing to run “global” pipelines? Short term (O2) approach – Select a global pipeline where Virgo scientists play a relevant role In cWB Padova-Trento INFN groups have a crucial role – Adapt the pipeline to the CNAF infrastructure Very short time attempt: use internal CNAF scheduler Short time attempt: use GRID architecture at CNAF – Reusable in the other facilities in Europe 11

12 cWB WMS Coherent Wave Burst Workflow Management System – Target: Enable to run cWB on GRID – Realization of a client layer (CWB WMS) that prepare, submit and monitor a cWB job for the GRID environment Lisa Zangrando (INFN-Pd) 12

13 Accounting Se portiamo più di una pipeline “pesante” al CNAF avremo bisogno di un sistema di accounting che ci permetta di capire chi fa cosa Non credo che sia necessario un sistema alla LIGO: Ma avere una coda per pipeline? 13

14 Adapting CC to pipelines Ideas: – Trick: Sophisticated login script for users that want to run a LSC like job – It mounts a LSC-like environment on the working nodes – Cloud: Static: we subtract a fraction of the “Virgo” machines to the common poll and we configure as LSC like – Drawbacks: » Violation of the resource sharing logic and reduction of the facilities available for “Virgo” pipelines Dynamic: – LSC like and GRID like environments are created by the requirements of the job All these hypothesis need a strong interaction, collaboration and availability of CC staff people Timeline (target): 2017 14

15 Scopo di questo meeting (2) Per l’uso delle risorse di calcolo del CNAF scopo di questo meeting è: – 1) definire ed ottenere tutti i requirements che ci permettano di implementare la soluzione a breve termine entro Aprile 2016 – 2) mettere su il progetto pilota che ci permetta di capire come poter processare al CNAF tutte le pipelines globali, senza modifiche delle medesime 15