Overview: - Slim5 beam test DAQ - SuperB SVT DAQ DAQ for SVT Mauro Villa INFN & Università di Bologna Overview: - Slim5 beam test DAQ - SuperB SVT DAQ

Slim 5 sensors and chips APSEL4D DAQ main characteristics: Pixel detector Coordinate pitch: 50x50 um Time Stamp Step 200ns-20us (5us) Readout clock 2-100 MHz (20 MHz) Bandwidth: 420 Mbit/s/chip Completely DATA DRIVEN 32x128 pix - 50 mm pitch perif & spars logic Microstrip and FSSR2 DAQ main characteristics Strip detector n&p side (x&y) Pitch: 50 um Time Stamp Step 200ns-20us (5us) Readout clock 2-70 MHz (30 MHz) Bandwidth: 360 Mbit/s/chip Completely DATA DRIVEN Paris, 16/02/09 M. Villa

SLIM5 Beam test Main challenges: . Time alignment of sensors 2 APSEL4D DNW MAPS chips 6 strip/striplets modules with 36 FSSR2 RO chips 3-4 beam counter scintillators coupled to PMT LVL1 trigger capability with Associative Memories APSEL4D chip Main challenges: Time alignment of sensors Data collection before trigger Smart triggers on tracks APSEL4D chip 10 mm2 active area Paris, 16/02/09 M. Villa

DAQ Main schema SLIM Detector Beam Crate VME 9U DAQ PC1 DAQ PC2 EDRO2 Time, Synchronization, Trigger, Controls, and Event building on EDRO1 SLIM Detector Beam DAQ Setup, monitoring, & control Crate VME 9U DAQ PC1 DAQ PC2 EDRO2 EDRO1 AM Board CPU Intel Ctrl Ctrl triggers triggers eth eth eth hits S-link S-link VME Ctrl S-link data in S-link data in tracks Bus VME Paris, 16/02/09 M. Villa

EDRO Readout board Stratix To DAQ 160 MB/s VME Slink EPMC Tracks from Data from FE chips 4 Gbit/s EPMC FE chips Time and Configuration Stratix Tracks from AM EPMC Performance: 40 Mhz bus clocks 8 Gbit input rate 1 Gbit output rate TTCRQ Hits to AM Paris, 16/02/09 M. Villa

Few key points Board structure: mainboard+mezzanines Mezzanines help to decouple inputs and outputs from main logic Large main FPGA for maximal freedom in the algorithmic part: hit organization, trigger and event building Several trigger algorithms developed Almost off-the-shelf components: TTCrq and S-link mezzanine for clock and data out Paris, 16/02/09 M. Villa

DAQ performances Rate of Event collected AM tracks vs reco tracks Up to 2.5 Million/s Trigger latency: 1-20 us DAQ rate: Up to 40 kHz Total statistics: 90 ME in 12 days Triggers on Scintillators Hit multiplicities Tracks Low fake rate (<0.1%) AM tracks vs reco tracks Paris, 16/02/09 M. Villa

SuperB SVT Data Acquisition SVT layers Front end chip selection Data driven or triggered chips? Front end chips and data volume Layer0 module prototype options SVT DAQ in SuperB schemas Paris, 16/02/09 M. Villa

SuperB SVT 40 cm 30 cm 20 cm Layer0 Layer Radius 0 1.5 cm 1 3.3 cm 2 4.0 cm 3 5.9 cm 4 9.1 to 12.7 cm 5 11.4 to 14.6 cm Baseline: use an SVT similar to the BaBar one adding a Layer 0 Layer 0 options: striplets or thin pixels. Each layer has several modules, mechanically independent units (52+8) Each module has 2 half-modules, electrically independent units: Sensor, front-end chips, HDI with power/signal input and data output link Pixel half-module Strip(lets) half-module Pixel sensor + front-end chips HDI Si Wafers Data Power/Signal Front-end chips HDI Power/Signal Data Paris, 16/02/09 M. Villa

Front-end chips for SuperB Front-end chip requirements: amplify, shape, digitize sensor signals. Perform timestamp association Sparsified and fast readout (layer 0 bkgd rate: 5MHz/cm2) Striplets readout: existent chip FSSR2 (BteV) good candidate: Readout efficiency not an issue with background rate x 10  6% occupancy in 132 ns time window Data-driven architecture 2 main clocks: RDclock 70 MHz, BCclock for timestamp ~100 ns Could be used also for Layer1-5 MAPS sensor/readout: data-driven readout architecture under development: sparsification and timestamp information implemented in CMOS MAPS matrix. APSEL4D 4k pixels produced and tested on beam: very good! APSEL320x216 69k pixels needed for Layer 0 multichip modules Paris, 16/02/09 M. Villa

Data driven vs triggered FE chips Data Driven chips FSSR2: only minor modifications needed for SuperB MAPS: no storage of data in local high radiation environment Could allow to use SVT data for LV1 trigger Need very fast link to send all data from HDI to DAQ (layer 0 critical! See next tables) Triggered chips Sending off chip only data from LV1 evt reduces by ~ 1/10 the load to the data transmission Need to investigate if there is a good match for striplets! MAPS readout: data out of high rad desiderable. Data stored outside vertex region in custom high rad memory chips. Pixel sensor + front-end chips Off Detector HDI Power/Signal DAQ Data HDI Power/Signal Very Fast link DAQ On Detector Paris, 16/02/09 M. Villa link Data

SVT in numbers 6 layers: Layer0 + 5 layers like BaBar SVT Modules HDI ReadOut Section (ROS) chips/ROS chips channels 0- Striplets 8 16 32 6 192 2.46E+04 0- MAPS 12.59E+06 1 12 24 7 168 2.15E+04 2 3 10 240 3.07E+04 4 64 5 288 3.69E+04 18 36 72 324 4.15E+04 total 60 120   1316 12.97E+06 Track rate @ layer0: 5.3 GHz (safety factor 5 included) fully dominated by machine bkgd Paris, 16/02/09 M. Villa

Layer0 module prototype Option A: HDI with local rad-hard memory (+ optical links) L1 handled in the HDI Memory buffers And L1 logic optical Link Optical Link 2.5 Gbit/s Edro like ROM Off detector low rad area Counting room On detector High rad area Pro: data outside vertex area data flux reduced asap Cons: rad-hard logic rad-hard optical link (GBT?) no trigger on SVT data Paris, 16/02/09 M. Villa

Layer0 module prototype Option B: minimal HDI with local line drivers All events outside detector L1 handled off detector Buffers-Modulators followed by drivers Cu bus <20 Gbit/s Optical Link 2.5 Gbit/s Edro like ROM Off detector low rad area Counting room On detector High rad area pro: few rad-hard elements few space required full event readout (trigger) commercial optical link only Cons: rad-hard line drivers large data volume to ship out Paris, 16/02/09 M. Villa

SuperB-SVT DAQ SVT in SuperB DAQ (main key points): FCTS Counting room Detector side 60MHz clock Max jitter ~ 15ps rms & no phase spread. L1 accept 2.5Gbits/s Optical links Read event ROM FEC 60MHz clock L1 accept DAQ Read event (?) Setup and control 16 Setup and control Subsystems control SVT in SuperB DAQ (main key points): Timing: 60 MHz main clock OK, 0.2-1 us time stamp period FEC will accomodate both model 1 & model 2 trigger strategies Data driven chips: can help for triggering Data size will depend strongly on background rate @ layer 0! Paris, 16/02/09 M. Villa

Conclusions Successfully tested on beam two chips candidates for SVT readout: FSSR2, APSEL(4D) Proved full data driven characteristics till track triggering (@L1) SVT Layer0 module prototype outlined Two options under study SVT electronics fits well both triggering schema (model 1 and model 2) and global DAQ schema Paris, 16/02/09 M. Villa

Backups Paris, 16/02/09 M. Villa

APSEL6D Completely data driven architecture Space time coordinates Matrix for SuperB: 320x216 pixels a 40 um pitch. Active Area = 12.8x8.64 mm2 =110 mm2 Completely data driven architecture Space time coordinates Time granularity 0.8-5.0 us (1.0 us is the goal) Time stamping is EXTERNAL. Paris, 16/02/09 M. Villa

SuperB Layer 0 Module 6 Chip 1.1 cm^2 Rate 1 chip: 110 MHz Module Rates: 660 MHz Bandwidth: 20 Gbit/s (design parameters) 2009-2010 2-3 chip APSEL5D (0.6 cm^2) On a module for tests Policy: design for SuperB, produce in economy Paris, 16/02/09 M. Villa

EDRO Readout boards Stratix To DAQ 160 MB/s (waiting for GBT) VME Slink GigaBit Ethernet GbitEth Link ottici 4 Gbit/s 32 MB RAM EPMC Stratix Performance: 40 Mhz bus clocks 8 Gbit input rate 1 Gbit output rate Slim5 DAQ rate: 40 keventi/s Tracks from AM EPMC 60/80 MHz TTCRQ Hits to AM Paris, 16/02/09 M. Villa

ROS/ROM SVT schema To DAQ EDRO bidirectional Optical fiber EPMC EPMC 60/80 MHz Trigger logic, Timing Edro is a very flexible ROM/ROS board, can easily accomodate model1 & model2 triggering models Will be a buffer for high level triggers Will be developed in the next years for test beam activities but with SuperB in mind (GDB links, Gigabit Eth, RAM) Paris, 16/02/09 M. Villa

SVT electronic summary Data push architecture Data out of the high radiation region Triggering on SVT conceivable Time stamping (1.0 us) All hits/events (defined time stamp) will be available on electronic board within 3-4 us. Full flexibility for different trigger schema FrontEndChips  L1/HLT/TTC interaction mediated by a dedicated board Paris, 16/02/09 M. Villa

Overview: Scopo WP2 Realizzazione un sistema di DAQ per beam test, che possa sfruttare appieno le potenzialità dei chip prodotti. Sistema altamente flessibile, in grado di: Controllare da remoto la configurazione di un numero di chip variabile e di diverso tipo, in un ampio intervallo, registrare le hit dopo una opportuna raccolta in eventi, effettuare dei trigger sulla molteplicità di hit e sulle molteplicità di tracce. Aumento di rates rispetto a Slim5 Aumento prestazioni schede TDAQ (40->80 MHz) Paris, 16/02/09 M. Villa

Esperienza del Beam Test Molto soddisfacente dal punto di vista della TDAQ: Rate max di 40 kHz, fake rate minimo, dead-time basso (5-40%). Buon tracking delle condizioni di run e di DAQ. 6 Databases riempiti automaticamente: DAQ config (4), Table positions, Electronic-logbook. Da non ripetere per nessuna ragione: portare al beam test hardware o configurazioni non testate prima. Paris, 16/02/09 M. Villa

Organizzazione TDAQ Elettronica di read-out basata su schede EDRO (evoluzione) per APSEL/FSSR2/VIPIX-ApselLike Elettronica dedicata per MIMOROMA2 e RAPS da integrare nel sistema globale Elettronica analogica/FADC per altri sistemi Apsel3T e simili Triggers su scintillatori, molteplicità layer/hit Memorie associative per trigger su tracce Sistema di raccolta dati basata sull’infrastruttura TDAQ di ATLAS. BO RM PG PI TS BO PI BO Paris, 16/02/09 M. Villa

Modalità di integrazione Stessa temporizzazione (stesso BCO) Integrazione completa nello stesso data stream (FSSR2/Apsel?D/Scintillatori/Vipix) Temporizzazione molto diversa Stream dati multipli, trigger separati Condivisione del time-stamp (+altro) Correlazione off-line tra stream diversi (Apsel3T/Mimoroma2/RAPS … ?) Paris, 16/02/09 M. Villa

Schema DAQ Slim5-like DAQ Others elements Tele Front DUT 0 Tele Rear DUT 1,2 Sc0 Sc1 Slim5-like DAQ Others elements Timestamping Trigger Evoluzione di componenti specifiche. Inclusione di MIMOSA2/RAPS/Apsel3T. Maggiore attenzione al debug /monitoring online. Nessun cambiamento sostanziale rispetto a Slim5. Paris, 16/02/09 M. Villa

Modulo trigger CAEN 1495 USER FPGA Altera Cyclone 20k FPGA VME 64 Out LVDS 32 In LVDS 16 I/O LEMO 3 timers Gestione Clock Gestione segnali di trigger o di sincronizzazione per MIMOSA2/RAPS MAPS Analogiche Riduzione dell’elettronica digitale NIM Prima versione FW per marzo Di interesse anche per Trieste: DAQ FSSR2 Paris, 16/02/09 M. Villa

SuperB-Oriented DAQ I Modulo SuperB layer0: 6 chip, 660 MHit/s To DAQ EDRO Fibra ottica bidirezionale EPMC EPMC Modulo SuperB layer0: 6 chip, 660 MHit/s Rates: 20 Gbit/s full rate, 3 Gbit/s triggered rate (ottimizzabili) Events: 2.3 KBytes full, 400 Bytes triggered at 1 us di BCO Design for SuperB: Produzione di 2-3 EPMC con 2 link ottici ognuna Bo, 22/01/09 Paris, 16/02/09 M. Villa

Evoluzione schede EDRO Sull’hardware vecchio: Studio dei limiti di frequenza per il trasferimento dati (40 MHz 80 MHz) Ottimizzazione comunicazione EDRO-EDRO Piccole modifiche nel firmware Sull’hardware nuovo (possibilità): EPMC con fibra ottica (SuperB ??) Main EDRO con memoria ram esterna per massimizzare le performances di raccolta eventi nei beam tests con basso duty cycle (0.5 s ogni 60s) Eventuale sostituzione componenti lenti Interfaccia Gigabit Ethernet Paris, 16/02/09 M. Villa

SuperB Module Da accomodare: link ottico in zona di bassa radiazione; spazio disponibile; banda passante Line Drivers Cu bus 20 Gbit/s Link ottico 2.5 Gbit/s Edro like ROM Off detector low rad area Counting room On detector High rad area Vantaggi: pochi elementi in zona rad-hard, risoluzione problema dello spazio disponibile Lettura evento completo appena fuori dal rivelatore Link ottico commerciale (no GBT) Paris, 16/02/09 M. Villa

Rate half module SuperB Inputs: 6 chips, 1.1 cm2 5 MHz/cm2 particle rates cluster width = 4 Safety factor = 5 Bus width 30 bits/chip Timestamping @1 us Outputs: Hit rate: 660 Mhit/s/mod Bandwidth: 20 Gbit/s Event size: 2.5 kB Layer0 ev.size: 79 kB TOO BIG (x2 SuperB goal) UNUSABLE FOR TRIGGERING Al Bus very difficult Rethinking few numbers 4 MHz/cm2 mean particle rates cluster width = 1.2 Observed width Sparsification @ cluster level Loss-less data compression (-50%) -40% zonal compression Time stamp stripping New outputs: Data rate: 160 Mword/s/mod Bandwidth: 2.4 Gbit/s Event size: 0.3 kB Layer0 ev.size: 9.5 kB 30% SuperB goal USABLE FOR TRIGGERING Still in a safety factor of 5 Al bus requirement relaxed Paris, 16/02/09 M. Villa

second board assembled: understand Lamb0 specific problem Associative Memory Goals (most of the work expected in the second half of the year) Complete AMBslim timing optimization (40 MHz) to have a solid prototype. We work now on the GigaFitter for CDF learning about Virtex5 FPGA – useful experience for AMBslim also. second board assembled: understand Lamb0 specific problem Increase working clock frequency inside FPGA as much as possible (160 MHz would be optimum but now it is limited by serial links with EDRO) Paris, 16/02/09 M. Villa P. Giannetti

Associative memory board diagram NOW Blue = 40 MHz Red = 40 MHz AM GLUE FIFOS RECEIVERs & DRIVERs (ROAD bus + 6 HIT buses) LAMB CONNECTORs ROAD CONNECTOR HIT FPGA I/O control PIPELINE REGISTERs INDI [17:0] ADD OUT [30:0] TRACKs NEXT Blue = 80 MHz Red = 40 MHz Paris, 16/02/09 M. Villa

Future of a Level 1 application: manpower dependent (for sure missing during 2009) Simulation of the full architecture to define needed features: (a) input bandwidth (b) event buffering (c) computing power. We have software tools and new production of CDF AMchips (Gruppo 1 to CDF) for CDF AM enlargement and coprocessors that could speed up simulation Laura Sartori (OIF Marie Curie fellow coming back to Italy next summer) is going to design a 65 nm full custom cell @Ferrara with R.Tripiccione for the single patten bit for the new AMchip → larger pattern banks than what expected by technology scaling for Level-1 & 2 applications. Use of high rate serial links could be tested → important for new Lamb development. New complex boards development (selections of appropriate devices, placing & routing) is necessary for a new LAMB and a new AMBoard to reduce the latency and to increase the event buffering capability @ Level 1. At the test beam we used 4 LAMBs with 4 independent sets of input buses, being able to store a maximum number of 4 events. We need to store may be 8 or 16 events or more (simulation should state how many buffers we need). New LAMB should have AMchips directly connected to the control chip: no chip pipelines. If a Lamb has to store 4 events instead of one it needs 4 sets of independent input buses (now it has only one). Use of serial Links becomes very important. Otherwise Lambs have to be much smaller and we need to fit more than 4 Lambs on a single AMBoard New AMBoard consistent with new LAMBs able to support the necessary event storage capability Paris, 16/02/09 M. Villa P. Giannetti

Altre informazioni utili Vi è un beam test di ATLAS/LUCID all’SPS tra il 17 e il 30 Giugno. Useremo (con la massima cura) il telescopio di Slim5 Ringraziamo di cuore per la possibilità ed il supporto! In preparazione sistema di raffreddamento chip ad aria secca … riutilizzabile! Vi è la possibilità di fare misure in parassitaggio (APSEL4D1, Apsel3T ?), se serve/interessa. E’ da preparare… e vi sono limitazioni di spazio lungo la linea di fascio. Maggio: test di sistema a BO! Paris, 16/02/09 M. Villa

Single Electron Interference Obbiettivo: dimostrare con tecniche moderne l’interferenza di singolo elettrone Strumenti: APSEL4D (Space-time, data driven) Doppia fenditura FIB a 200 nm TEM: Transmission Electron Microscope (120 keV) Rischio: 100% ManPower: ε Priority: bassa Finanziamenti: 0 Tanamura et al. (a) (b) S M I P PO v Paris, 16/02/09 M. Villa

Single Electron Interference SE funziona: Alto ritorno di immagine per APSEL4D (5D,6D…) Physics World: SEI demonstration is the single most important physics experiment in the world history ! First group: Merli, Missiroli, Pozzi in Bologna (1974) Possibilità di testare chips in un TEM prima di fare un beam test. Valutazione performance sensore, qualche grandezza Stress tests per alti rates di DAQ; misure rate SEU Facility per studio di pixel chips, Moduli SuperB Anche per strips , striplets o altro ?? .. idee ancora non completamente formalizzate .. Possibili piccole richieste finanziarie (1-2 k€) su vipix nel 2010 Paris, 16/02/09 M. Villa

Conclusioni Attività principali WP2: Caratterizzazione/debug schede AM (+EDRO) Aumento clock EDRO, AM (2009) Integrazione elettronica analogica e/o senza time stamp (MIMOROMA2, RAPS, Apsel3T) (2009/10) Miglioramento performances DAQ Debug & on-line histogramming (2009) Software di integrazione (2009/10) Sviluppo schede EPMC/EDRO che siano SuperB-oriented (2009/2010) DAQ facility a Trieste per tests su FSSR2 (svincolati da Pomone et al.) Paris, 16/02/09 M. Villa

Paris, 16/02/09 M. Villa

Backups Backups Paris, 16/02/09 M. Villa

Attività a breve a Bologna Programmazione Trigger board Caen (Tesi) Valutazioni performaces scheda EDRO (tirocinio) Design EPMC con ricevitori ottici (??) (Avoni) Miglioramento firmware EDRO (M.V./Giorgi) Aggiornamento software TDAQ-02-00-00 (S. Valentinetti/C. Sbarra) Monitoraggi e slow control (borsista) Paris, 16/02/09 M. Villa

Finisar SFP Transceiver Fibre compatibili con S-Link (e GBT ?) Costo: 150 € Proposta preliminare Paris, 16/02/09 M. Villa

FPGA di gestione Xilinx Virtex 5, Altera Stratix II GX Costo: 500 € Proposta preliminare Paris, 16/02/09 M. Villa