WP3000 : Padova ICC Contribution 2007-2010 Mattia Vaccari (MV) Gabriele Mainetti (GM) & Giulia Rodighiero (GR) University of Padova – Department of Astronomy.

Presentazione sul tema: "WP3000 : Padova ICC Contribution 2007-2010 Mattia Vaccari (MV) Gabriele Mainetti (GM) & Giulia Rodighiero (GR) University of Padova – Department of Astronomy."— Transcript della presentazione:

1 WP3000 : Padova ICC Contribution 2007-2010 Mattia Vaccari (MV) Gabriele Mainetti (GM) & Giulia Rodighiero (GR) University of Padova – Department of Astronomy ASI Review Padova ICC Contribution 1 WP 3000.1 - Partecipazione all'ICC di PACS : fase di sviluppo WP 3000.2 - Partecipazione all'ICC di PACS : fase operativa WP 3000.3 - Partecipazione all'ICC di SPIRE : fase di sviluppo WP 3000.4 - Partecipazione all'ICC di SPIRE : fase operativa

2 PACS & SPIRE ICC Activities  Testing of PACS Instrument Simulator, Pipeline and AOTs  Definition of user requirements for Herschel Data Processing system  Development of Visualization Tools for SPIRE Interactive Analysis  Testing of Herschel Data Processing system at the astronomer level  Following the full Instrument and Data Processing Development Cycle ASI ReviewPadova ICC Contribution2

3 A)Test del simulatore strumentale di PACS per l’identificazione di eventuali bachi del software, dal punto di vista di un utente. B)Sono stati individuati e riportati al working group vari problemi negli output del simulatore. Le frequenti telecon hanno permesso di riportare in modo continuativo le problematiche individuate e di riportarle agli sviluppatori per individuare una soluzione. C)Le expertise acquisite dal team italiano (GR) nell’utilizzo del software hanno permesso di utilizzarlo come strumento nella preparazione delle osservazioni di Guaranteed e Open Time, creando mappe simulate con le diverse configurazioni richieste dai diversi proposal. 160 μ m simulated image PACS Instrument Simulator ASI Review3Padova ICC Contribution

4 PACS Pipeline ASI ReviewPadova ICC Contribution4 A)Utilizzo del simulatore per testare alcuni moduli della pipeline del fotometro di PACS B) - Generazione di timeline simulate corrispondenti alle osservazioni del GT PACS - Esecuzione e test della pipeline sulle timeline simulate - Applicazione dei moduli High-Pass Filtering e PhotProject per produrre mappe del cielo - Utilizzo di dati SOVT2 come input alla pipeline - Partecipazione ai meeting tecnici del GT per la validazione dei vari test della pipeline C) Questi impegni hanno contribuito ad un’estesa validazione sia del simulatore dello strumento PACS che al software di Data Processing dai raw data al Map Making

5 ASI ReviewPadova ICC Contribution5 A)Test degli Astronomical Observing Templates (i.e. Observing Modes) per ottimizzare l’esecuzione delle osservazioni per le survey extragalattiche sia nei programmi di Guaranteed Time che di Open Time B)Individuazione delle migliori strategie osservative, validate dalle successive osservazioni PEP C)Particolare contributo italiano alla pianificazione del campo COSMOS A scan mapping strategy with perpendicular “cross-linked” scans was adopted to homogeneously cover the field and remove correlated noise during data reduction Screenshot detailing the coverage of the COSMOS field as observed by the PEP Guaranteed Time Key Program PACS AOTs

6 Visualization Tools for SPIRE Interactive Analysis ASI ReviewPadova ICC Contribution6 Aim of the work : developing Visualization Tools for Interactive Analysis of SPIRE raw and reduced data (from level 0 to level 2). Understand the goodness of an observation Understand any problems of the detectors Use/manage/visualize the data for further processing DTMosaic DetectorTimelineExplorer is the default viewer for the SPIRE Detector Timeline product within HIPE. In HIPE from version 1.0. TimelineSimplePlot in HIPE from version 4.0. DTMosaic in HIPE from version 5.0 GM is the owner of the spire_ia_gui package TimelineSimplePlot DetectorTimelineExplorerComponent Intuitive view of detector arrays Powerful plotting tools No Jython involved Padova efforts focuesd on: 1.DetectorTimelineExplorer 2.TimelineSimplePlot 3.DTMosaic Level 0.5-1 product Detector Timelines Two ways : script (Jython) or Graphical User Interface (GUI)

7 One of the GUIs at a glance ASI ReviewPadova ICC Contribution7

8 SPIRE : Pipeline How-To ASI ReviewPadova ICC Contribution8 Herschel Science Archive http://herschel.esac.esa.int/Science_Archive.shtml Get the data Explore the data HIPE Level 0 - Raw data Level 0.5 Level 1 Level 2 - Reduced data Reprocess (if you need) HIPE + JYTHON Official Pipeline script Your pipeline scripts Science Image Credit : ESA / SPIRE / HerMES http://herschel.cf.ac.uk/results/lockman-hole

9 Herschel DP CCB & Users Group Activities  MV represented SPIRE within the Herschel Data Processing (DP) Configuration Control Board (CCB) and Users Group (UG) and chaired the SPIRE DP UG –Assembling and coordinating DP feedback from SPIRE consortium members –Participating into DP CCB telecons and DP UG meetings –Supervising (Mentoring) DP development related to a number of WPs ASI ReviewPadova ICC Contribution9  Development of Herschel DP has benefited from the exposure to a larger pool of dedicated users, stimulating ICC members to take action on issues raised by KPs  The Herschel DP UG has recently spawned a number of DP Interest Groups (IGs), including the “PACS and SPIRE Large Maps & Source Extraction” IG coordinated by Mattia Vaccari as mentor for the related DP WPs  Following SDP papers and the AO for OT, the Herschel DP UG will include new members from the IGs, supporting the newly established Herschel Users Group

10 Conclusion : Padova ICC Contribution  GM, GR, MV have represented Padova within the PACS and SPIRE ICCs throughout the 2007-2010 pre-to-post launch timeframe  They have provided a substantial contribution to: –pre-launch development and testing of the Herschel general DP and SPG system –post-launch improvements to general DP and SPG following community feedback –execution of PACS and SPIRE SPG pipelines as part of the HerMES and PEP GT KPs –involvement by the general astronomical community ASI ReviewPadova ICC Contribution10  Their work has ensured that Padova is recognized as an important player within the largest PACS and SPIRE GT KPs, thus continuing its long-time involvement in IRAS, ISO, Spitzer and now Herschel extragalactic survey science  They are now in a position to provide expert knowledge about the instruments and data processing system but also to carry out ground-breaking science with them  They have greatly contributed to providing the ICCs with early astronomers feedback on one hand and to popularizing Herschel and its data processing system within the extragalactic astronomical community at large on the other

11 Data Processing Acceptance Test – Cycle 1 : GR e’ stata coinvolta nelle campagne di validazione dell’interfaccia HIPE Feature testate in modo particolare includono:  SPIRE Spectrometer Pipeline  Spectral Visualization  Spectral Aritmethics Durante le fasi di PV e SDP, GR ha visitato MPE per circa una settimana al mese, svolgendo funzioni di Pipeline & Instrument Health Manager: verifica della situazione del database per la propagazione dei dati PACS dall'Herschel Science Center (HSC) agli ICC di MPE estrazione della telemetria dai file scaricati nel database trasferimento dei dati ausiliari dall’HSC all’ICC di MPE processamento automatico di tutte le osservazioni con la pipeline standard aggiornamento giornaliero delle Wiki page che descrivono lo stato del processamento delle osservazioni Monitoraggio dello stato di salute generico del satellite e dello strumento controllo dettagliato delle prestazioni del fotometro e dello spettrometro Other PACS Activities ASI Review11Padova ICC Contribution

12 Visualization Tools for SPIRE Interactive Analysis ASI ReviewPadova ICC Contribution12 DTMosaic DetectorTimelineExplorer/DetectorTimelineExplorerComponent (DTE/DTEC) is the default viewer for the Detector Timeline product within HIPE. In Hipe from version 1.0. TimelineSimplePlot is a useful tool to create simple plot from HIPE command line. In Hipe from version 4.0. DTMosaic is a tool to plot data recorded by many detectors during one observation on a single page to compare data more effectively (mosaic plot). The plots are arranged in the same pattern as the detectors are presented in the Bolometer Detector Array display, a honeycomb pattern. In HIPE from version 5.0 TimelineSimplePlot DetectorTimelineExplorerComponent

13 Visualization Tools for SPIRE Interactive Analysis ASI ReviewPadova ICC Contribution13 DTE/DTEC - Recent improvements Optimization of the code: created a new class for code shared between DTE and DTEC Color scale: now it’s possible to select the min/max value to display Mask visualization: it’s now possible to set a mask value from a preferences panel and visualize if a bolometer is flagged by this value Mosaic plot is allowed via right-click on the desired bolometer array A great effort has been made in increasing the package test coverage (from 50 to 81%). All the documentation for the GM class is up to date (javadoc, User Reference Manual) and a new User Manual has been written to be included in the Spire User Manual.

14 SPIRE Pipeline Task ASI ReviewPadova ICC Contribution14 Development and testing of tasks used in photometer pipeline script The tasks are used in the all “map” pipeline script: Large Map (POF5), Small Map (POF10), Parallel Mode The task are used to remove the median baseline and to join/cut the Photometer DetectorTimeline products, to join the Nominal House Keeping Product The test coverage for this package is 100% Javadoc and URM are up to date Written the SPIRE Pipeline Design Document Written the SPIRE Pipeline Test Plan Pipeline screenshot from PV data Javadoc example URM example PV data parallel mode pipeline screenshot

15 PACS : SW Testing & Instrument Monitoring (Rodighiero) ‏  GR continued Testing of DP pipeline on PV and SDP data, with particular attention to deglitching techniques on scan-maps, with a careful treatment of the masks that need to be applied on bright sources to avoid flux losses. Tuning and combination of photMMT deglitching and Second level deglitching provided the best results on fast scan maps.  Contribution to the analysis of calibration PACS data, for prime and parallel mode  GR continued duties activities as Pipeline Manager and Instrument Health Manager, with weekly shifts of Instrument Monitoring on behalf of the PACS ICC at MPE and remote operation from home institute. ASI ReviewPadova ICC Contribution15

16 SPIRE : DP Development (Mainetti & Vaccari) ASI ReviewPadova ICC Contribution16 New layout for DTE to optimize data visualization Implemented some tweaking based on users recommendation New package for all Visualization tools : spire_ia _gui. (WP Owner : Gabriele Mainetti)

17 SPIRE : DP Development (Mainetti & Vaccari) ASI ReviewPadova ICC Contribution17 New Control Panel : all the commands are now grouped here Larger plot area and new plotting option Value of the mask can be overplotted New arrangement for detector arrays

18 SPIRE : DP Development (Mainetti & Vaccari) New class: TimelineSimplePlot (TSP) TimelineSimplePlot class allows to create simple SPIRE Detector Timeline plots from HIPE command line. At least one instance of Timeline-type products must already be available in memory to launch DTE (e.g. output from pipeline script or product loaded from local store). ASI ReviewPadova ICC Contribution18 Gabriele Mainetti is also owner of the spire_ia_pipeline_common_util WP, containing miscellaneous tools used in the SPIRE data reduction pipeline