ATLAS HLT/DAQ Stato e prospettive Valerio Vercesi CSN1 Settembre 2005.

Presentazione sul tema: "ATLAS HLT/DAQ Stato e prospettive Valerio Vercesi CSN1 Settembre 2005."— Transcript della presentazione:

1 ATLAS HLT/DAQ Stato e prospettive Valerio Vercesi CSN1 Settembre 2005

2 V. Vercesi - INFN Pavia 2Outline  Pre-series  Status in USA15/SDX1  Commissioning and exploitation  Large Scale Test  Activities, experiences  Lessons learnt  Activities  Monitoring, ROD Crate DAQ  Algorithms development and deployment  Finance  Accounting  2006 requests

3 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 3  S. Falciano (Roma1) Coordinatore Commissioning HLT  A. Negri (Pavia) Coordinatore Event Filter Dataflow  A. Nisati (Roma1) TDAQ Institute Board chair e Coordinatore Muon Slice PESA  F. Parodi (Genova) Coordinatore b-tagging PESA  V. Vercesi (Pavia) Deputy HLT leader e Coordinatore PESA (Physics and Event Selection Architecture)  Attività italiane  Trigger di Livello-1 muoni barrel (Napoli, Roma1, Roma2)  Trigger di Livello-2 muoni (Pisa, Roma1)  Trigger di Livello-2 pixel (Genova)  Event Filter Dataflow (LNF, Pavia)  Selection software steering (Genova)  Event Filter Muoni (Lecce, Napoli, Pavia, Roma1)  DAQ (LNF, Pavia, Roma1)  Monitoring (Cosenza, Napoli, Pavia, Pisa)  Pre-series commissioning (LNF, Pavia, Roma1)

4 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 4 ATLAS TDAQ system Muon ROD LVL1 LVL2 Event builder network Storage: ~ 300 MB/s ROBROBROB CaloInner Pipeline Memories Readout Drivers Readout Buffers ~1600 High-Level Trigger LEVEL-1 TRIGGER Hardware-Based Coarse granularity from calorimeter & muon systems LEVEL-2 TRIGGER Regions-of-Interest “seeds” Full granularity for all subdetector systems Fast Rejection “steering” EVENT FILTER Possibly “seeded” by Level 2 Full event access Algorithms inherited by offline RoI EF farm ~1000 CPUs 1 selected event every million TDAQ ≅ Rates 40 MHz ~75 kHz ~2 kHz ~200 Hz ~ 2 ms ~ 10 ms ~ 1 s Latency EF LVL2 farm ( )

5 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 5TDAQ  Trigger e Data Acquisition hanno da sempre in fase di commissioning un doppio ruolo  Come “server” per il commissioning dei rivelatori  Come “client” per utilizzare le informazioni realistiche dell’esperimento per i propri studi di funzionalità e performance  La situazione si è già presentata durante il Combined Testbeam 2004  Il TDAQ di ATLAS è un progetto in piena evoluzione in cui development/commissioning/exploitation sono ancora fasi molto miscelate  Presentazione di risultati e indicazione delle prospettive  Maggiore enfasi alle componenti con forte partecipazione italiana  Descrizione del piano di commissioning generale

6 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 6 Pre-series design “Module-0” of final system 8 racks (~10% of final dataflow)

7 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 7 Pre-series reality LVL2 rack EF rack Switch rack Online rack 6 racks SDX1 ROS rack

8 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 8 Commissioning and exploitation  Fully functional, small scale, version of the complete HLT/DAQ  Equivalent to a detector’s ‘module 0’  Purpose and scope of the pre-series system  Pre-commissioning phase  To validate the complete, integrated, HLT/DAQ functionality  To validate the infrastructure, needed by HLT/DAQ, at point-1  Commissioning phase  To validate a component (e.g. a ROS) or a deliverable (e.g. a Level-2 rack) prior to its installation and commissioning  TDAQ post-commissioning development system  Validate new components (e.g. their functionality when integrated into a fully functional system)  Validate new software elements or software releases before moving them to the experiment

9 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 9 Pre-Series Commissioning

10 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 10 Commissioning LVL2+ROS First measurements with full LVL2 rack feeded by ROS data Using separate Control and Data networks

11 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 11 Commissioning EF

12 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 12  Pre-series  Status in USA15/SDX1  Commissioning and exploitation  Large Scale Test  Activities, experiences  Lessons learnt  Activities  Monitoring, ROD Crate DAQ  Algorithms development and deployment  Finance  Accounting  2006 requests

13 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 13 Large Scale Tests  Pre-serie work will help understanding the TDAQ system in terms of functionality  Forms the basis for future deployments/exploitations  Complexity of ATLAS TDAQ system arises also from the size of bulk components involved  Topology of communications, size of LVL2/EF farms, software, …  Test scalability of HLT system using presently available large installations  Understand issues like configuration, startup time, communication, control, error reporting, …  UCB/TRIUMF WestGrid Cluster (http://www.westgrid.ca)  60 racks x 14 nodes = 840 Dual-CPU nodes (3 GHz CPUs / 2-4 GB RAM)  CERN LXSHARE Cluster (http://batch.web.cern.ch/batch)  Up to ~700 nodes (various flavours)  Reference page for all tests  http://atlas-tdaq-large-scale-tests.web.cern.ch http://atlas-tdaq-large-scale-tests.web.cern.ch

14 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 14 Success, problems, …  High test activity at both system and component level  Problem finding – solving or workaround – find new problems  General feeling that  We are well advancing in the understanding of the system on a large scale  In some areas we have come further than hoped (HLT offline)  A lot remains to be done for final Atlas  Repeated questions: when are the next large scale tests planned?  Comprehensive report is planned to be available for the Atlas Week in October

15 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 15 State transitions Luke Warm Start Luke Warm Stop USR_RUNNING_TIME (default is 30 s) RUNNING CONFIGURED INITIAL ABSENT Configure Boot Unconfigure Shutdown Setup Close Cold Start Cold Stop configure: load configure start: prepareForRun startTrigger stop: stopTrigger stopFrontEnd stopDataCollection stopEventFilter stopRecording unconfigure: unconfigure unload

16 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 16 LVL2 transition times  State transistion timing quite acceptable  No significant differences between 2 and 3 tier Run Control

17 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 17 EF results @ LST  Effect of realistic algorithm: TrigMoore vs HelloWorld  EF standalone 1EFD+2PTs / node  up to 200 nodes  MySQL as geometry DB used  significant slow down due to access/reading geom. DB  MySQL vs Oracle DB in TrigMoore  EF standalone, 1EFD+2PTs / node  Oracle DB – up to 160 nodes  MySQL DB – up to 200 hosts  MySQL works faster at “small scales”, while Oracle looks better at higher scales - to be investigated more  not able to on higher than 200 nodes with any of both partitions – to be investigate further (do we need to replicated DBs ?)

18 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 18  Pre-series  Status in USA15/SDX1  Commissioning and exploitation  Large Scale Test  Activities, experiences  Lessons learnt  Activities  Monitoring, ROD Crate DAQ  Algorithms development and deployment  Finance  Accounting  2006 requests

19 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 19 Cosmics Tile setup MobiDAQ (Mobile DAQ): read out of 8 drawers in the pit with temporary RODemu but real TDAQ (tdaq-01-02-00), tests of electronics, cosmic muons runs

20 GNAM Monitoring chain  Framework per monitoring on-line a basso livello  Core: trasporto di eventi, istogrammi e comandi  Plugin dinamici: decodifica e istogrammazione  Possibilita’ di correlazione fra diversi rivelatori  Comandi asincroni (reset, rebin, update)  Status  Incluso in TDAQ da aprile  Utilizza i servizi disponibili per il monitoring  Validato al CTB04; in uso in alcuni siti di commissioning

21 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 21 GNAM al commissioning  Acquisizione dati nell’ambito del software TDAQ  Commissioning di MDT usa GNAM per monitoring online ed analisi dati  Sampling completo a livello di ROS [~2 KHz per noise test, ~200 Hz per pulser]  Integrazione di librerie: completato per MDT, in via di sviluppo per RPC  Stato del monitoring MDT:  Richiesta minimale di informazioni allo shifter (nomi camere)  Output: file di istogrammi e file di testo con risultati di analisi dati per ciascuna camera e per ciascun run  In sviluppo:  ottimizzazione dell’analisi dati on-line  installazione del presenter per la visualizzazione degli istogrammi  on-line event display

22 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 22 PMP Presenter  Visualizzazione asincrona on-line di istogrammi  Interattivo (reset, rebin, zoom, fitting, ecc...)  Operazioni grafiche su istogrammi (ROOT canvas)  Grafica configurabile  Status  Incluso in TDAQ da settembre  Utilizza i servizi disponibili per il monitoring  Pienamente funzionale al CTB04  Riprogettato per nuove funzionalità e maggiore scalabilità

23 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 23 GNAM&PMP: sviluppi futuri  GNAM  Completare l’integrazione nel framework del TDAQ  Configurazione di Core e librerie dal database del TDAQ  Messaggistica e gestione degli errori software  Supporto per la generazione di allarmi automatici  Livelli di severità, routing  Verifica delle prestazioni e delle risorse necessarie  CPU, Memoria, Banda  Studio della scalabilità  PMP  Completare la nuova versione  Minimizzazione del traffico di rete  Adattamento della IGUI alla nuova struttura  Generazione allarmi  Plugin di analisi degli istogrammi

24 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 24 ROD Crate DAQ  RCD is used as interface with the RODs for  Control  Configuration  Monitoring  Statistics  Event sampling  Data readout (through VME)  User guide for detectors developers available  Validation system in Bld. 40  DAQ Commissioning – Phase 1:  The ROD Emulator system will be used in order to validate all common RCD software and infrastructure  After adding and validating the detector sw and hw specific items, multi crate event building will be used in the absence of the full DAQ chain VMEbus memory + CORBO = Memory + Registers + Interrupt capability RCCRCC MEMORYMEMORY CORBOCORBO Config & Control Data readout REBREB Event Fragments ROD Emulator ROSROS

25 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 25 RCD exploitation  Experience from Combined Testbeam extremely useful  Recall almost all detector used it in the CTB  Successful workshop to put forward new requirements  As a consequence, several improvements during last months  Configurable interrupt handling  Simplified user interface to access ordered event fragments  Data driven event building for multicrate acquisition in the commissioning phase  Simplified ROD emulation  Hardware trigger distribution  All sub-detector commissioning (but LAr…) sites use RCD  MDT and RPC on the forefront  BB5 integration, Point 1 with MROD, Lab testing with ROD emulators

26 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 26 Algorithms: Muon slice  LVL1 simulation is the fundamental input for the measurement of the full muon vertical slice performance  LVL2 and EF Muon algorithms have been extensively tested on data simulated in ATLAS  Rome Physics Workshop: June 2005  LVL2:  Fast  Confirm the LVL1 trigger with a more precise P T estimation within a Region of Interest (RoI)  Global pattern recognition, track fit, fast P T estimate via Look Up Table (LUT) with no use of time consuming fit methods  Event Filter: TrigMoore  Based on offline reconstruction algorithm Moore  Can run seeded (reconstruction starting from RoI of previous levels)  Precise P T determination  General goal is now to achieve more realistic estimate of trigger selections and corresponding rates  Real geometry, configuration and conditions database usage, …

27 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 27 LVL1 Coincidence Windows  Athena release 10.0.4  Low p T 6 GeV Threshold Low-pT Inefficiency map Efficiency curve

28 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 28 MuFast Radius approach  Use of curvature radius instead of sagitta investigated  More suitable for the endcap, recover efficiency in the barrel  Same algorithm across ± 2.4 in  Barrel, Muon layout Q Standard sectors All sectors |  | < 1.1 Tail extension over 2  : 15% @ 6 GeV 25% @ 20 GeV 35% @ 100 GeV misposition of the fit segment w.r.t. the MDT wire, increases tails at higher p T.

29 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 29 MuFast endcap extension  Early stages of project  Endcap differs from Barrel  M and O station are outside B field  Inhomogeneous B field – bending is local  Algorithm  Pattern recognition and fit in TGC → position and slope in EM  Extrapolate segment into MDT EM/EO → Roads in EM/EO, find hits, fit  Next  pattern recognition and fit in MDT as in mFast – not done yet  Extrapolation into EI and LUT

30 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 30 MuFast: MDT miscalibration  Commissioning the algorithms: realistic approach to data handling  The plot shows the muFast resolution for two different scenarios:  the correct MDT r-t function is used, red points  a systematic shift of + 0.2 mm is added to the radius returned by the correct r-t function, blue points +10% degradation @ 50 GeV, to be compared with a + 5% expected by a naive calculation

31 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 31 TrigMOORE updates  A check on the detectors hash ids selected by RoIs is now performed in order to avoid “duplicated” Moore tracks, that can be reconstructed twice if starting from equal or similar RoIs.  The RegionSelector can be initialised with maps from MDT, RPC, TGC for different layouts  CSC recently added: checks in progress  TrigMoore can be seeded in the barrel and in the endcaps  Work in progress to establish robustness and fake rate in presence of cavern background

32 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 32 p T resolution (Moore) The relative degradation in  (p T ) is +5% for muons with a 6 GeV/c transverse momentum, increases to +13% around 50 GeV/c This MDT miscalibration leaves almost unaffected  resolutions. TrigMoore: MDT miscalibration  Single muons (with p T = 6, 10, 20, 40, 100 GeV/c, produced for the Rome Initial Layout) have been reconstructed in two different scenarios  Using the correct MDT r-t relation function (red squares in pictures)  Applying a systematic +0.2 mm shift on the radius obtained with the correct MDT r-t function (blue circles in pictures)

33 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 33 LVL2 tracking: SiTrack  Preliminary results obtained on DC1 b-jet samples at initial luminosity

34 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 34 B-tagging @ LVL2  Results obtained with the “standard” SiTrack algorithm on DC1 data  b-tagging: likelihood ratio using transverse and longitudinal impact parameters  Upgraded version to be tested soon: should improve both efficiency and track parameters resolution and hence significantly improve the b-tagging performance

35 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 35 Algorithms steering

36 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 36 ATLAS Commissioning Phases  Commissioning means bringing ATLAS systems from “just installed” to “operational”. It is broken in 4 phases  Subsystem standalone commissioning  Integrate subsystems into full detector  Cosmic rays, recording data, analyze/understand, distribute to remote sites  Single beam, first collisions, increasing rates, etc…  A consistent part of commissioning activities will be done during the installation itself  Phases will overlap since different systems may be at different development levels  For the barrel calorimeter commissioning will start soon  Tile calorimeter is already taking data

37 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 37 HLT Commissioning  Commissioning is a set of activities which spans the time interval from the installation of the HLT racks and nodes …  A rack is the elementary unit for commissioning  OS, Dataflow and Online software are installed ... to the phase when the HLT is filtering physics data and recording them  HLT selection algorithms are installed and running stably  The complete trigger menu (at least for early physics) is configured  The trigger selection efficiencies and background rejection rates are understood and can serve as input for physics measurements  Phase-1 Commissioning definition is the most urgent  Heavily use the Pre-series to exercise the procedures for installation and commissioning  Important steps will cover the integration of detectors into full system  Involve operations that have a very strong coupling with the offline commissioning activities  Development of specific algorithms looking at simple data decoding (cabling,…)  Final commissioning phases extend far beyond the data-taking startup (interface with run coordinator team)

38 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 38 Cosmic muons in ATLAS Rock ~ Silicon 600m x 600m x 200m deep (2.33 g/cm 3 ) Air Concrete Surface building PX14/16 shielding (2.5 g/cm 3 ) PX14 (18.0 m Inner Ø) PX16 (12.6 m Inner Ø ) ATLAS Geant Simulation Initial detector

39 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 39 Overall plan

40 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 40Outlook  A lot of work during this year, system entering phase of complete deployment  Purchase plan proceeding as scheduled, with some minor delays  (Wo)manpower situation not always healthy  More help and support welcome  Three big tasks awaiting us in the next months  Commissioning the pre-series and extract a coherent and complete set of system performance measurements  Based on previous experiences and on already established partial results  On-line trigger selections evaluation (rates, efficiencies, physics coverage,…) evolving towards more realistic approach  Calibration, geometry “as installed”, mis-alignment, error handling, complete trigger menus, physics analysis based on trigger objects  Prepare for cosmic run next year  “Cosmic” slices and trigger menu (Tile, LVL1, “digital” LVL2)

41 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 41  Pre-series  Status in USA15/SDX1  Commissioning and exploitation  Large Scale Test  Activities, experiences  Lessons learnt  Activities  Monitoring, ROD Crate DAQ  Algorithms development and deployment  Finance  Accounting  2006 requests

42 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 42Accounting  Contributo INFN alla Pre-serie  Read-Out System: 51 kCHF (ROS Racks)  Online Computing System: 40 kCHF (Monitoring, Operations)  Online Network System: 44 kCHF (Switches, FileServer)  Inviati al CERN a Dicembre 2004  VV riceve in copia tutte le fatturazioni dei singoli acquisti ed un sommario mensile dello stato finanziario  Contributo CORE 2005  Online Computing System: 45 kCHF (Monitoring, Operations)  Inviati al CERN a Maggio 2005  Già acquisiti due file server  Read-Out System: 275 kCHF (ROS Racks)  Questo acquisto si espleta secondo una gara e non con un semplice market survey o price inquiry come fino ad ora avvenuto  Richiesta alla Giunta l’autorizzazione per partecipare alla gara Grazie a Speranza che si è prodigata per espletare le pratiche necessarie  Il CERN preferisce gestire la gara su un periodo di due anni Omogeneità dei componenti vs miglioramento delle prestazioni Si sommano i 275 kCHF del 2006

43 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 43 Cost Profile (kCHF) 200420052006200720082009Total Pre-series 14000000 Detector R/O 0275 000550 LVL2 Proc 0065195230160650 Event Builder 0050 11070280 Event Filter 001701805703801300 Online 045135000180 Infrastructure 0080 20 200 INFN Total 1403207755059306303300 TDR Total 10483357408745447522454325101 INFN Percentage(%) 13.49.519.011.112.413.913.1

44 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 44 Missioni Estere  LNF  Commissioning HLT/DAQ Pre-serie e pit: 6 m.u.  Ferrer, Kordas (+ Miscetti, Giovannella)  Pavia  VV coordinatore PESA e duputy HLT: 1 m.u.  Negri A. responsabile Event Filter: 1 m.u.  Scannicchio D. commissioning HLT: 2 m.u.  Roma1  Speranza responsabile commissioning HLT: 2 m.u.  Leandro chair IB, coordinatore slice mu: 1 m.u.  ROD crate DAQ e HLT/DAQ muoni : 4 m.u.  Pasqualucci, Di Mattia, …

45 CSN1 Settembre 2005 V. Vercesi - INFN Pavia 45Milestones  30/06/2005  TDAQ - Installazione, test e uso della "Pre-serie" (~ 10% TDAQ slice)  “ragionevolmente” raggiunta: ritardi accumulati soprattutto sugli acquisti delle componenti  24/12/2005  TDAQ - Installazione e test dei ROS di Pixel, LAr, Tile, Muon (interfacciamento al ROD Crate e integrazione nel DAQ)  Parte del piano di commissioning in esecuzione: piccola dipendenza dalla data di consegna dei ROS  30/04/2006  Completamento dei test sulla pre-serie e definizione delle funzionalità per il supporto al commissioning TDAQ  31/08/2006  Commissioning delle slice di ROS dei rivelatori utilizzando le funzionalità della pre-serie (modulo-0 del sistema finale)  31/12/2006  Presa dati integrata dei rivelatori nel pozzo con raggi cosmici