The MedIGrid PSE in an LCG/gLite environment S.Co.P.E. Project The MedIGrid PSE in an LCG/gLite environment Vania Boccia vania.boccia@unina.it

Workshop finale dei Progetti Grid del PON "Ricerca” Contributors Department of Math (UNINA): V.Boccia L.D’Amore G.Laccetti M.Lapegna CNR: L.Carracciuolo SPACI: A. Murli Workshop finale dei Progetti Grid del PON "Ricerca” 2000-2006 - Avviso 1575 10-12 Febbraio Catania

MedIGrid PSE: aim and motivations MedIGrid collects: physicians involved in nuclear medicine. computing scientists, in the field of distributed and parallel computing, devoted to the design and the deployment of new medical images reconstruction and denoising algorithms With aim of developing an high performance, freely accessible, grid-aware, PSE to enable clinicians to: manage process visualize medical images, coming from several acquisition systems, from any geographic location. Workshop finale dei Progetti Grid del PON "Ricerca” 2000-2006 - Avviso 1575 10-12 Febbraio Catania

Some Medical imaging algorithms ECODEN: Computational kernel for ecocardiographic sequences denoising μ-SEG: Computational kernel for medical images segmentation Execution environment Hardware: Cluster of nodes linked throught high speed connection Software: Operative System Unix-like, PETSc (based on MPI, BLAS, LAPACK) Workshop finale dei Progetti Grid del PON "Ricerca” 2000-2006 - Avviso 1575 10-12 Febbraio Catania

Workshop finale dei Progetti Grid del PON "Ricerca” ECODEN Denoised data Anisotropic diffusion (Perona-Malik PDE equation) iterative algorithms based on Krylov subspaces methods (CG, GMRES)‏ block diagonals preconditioners (bjacobi, asm) Original data Workshop finale dei Progetti Grid del PON "Ricerca” 2000-2006 - Avviso 1575 10-12 Febbraio Catania

μ-SEG + Original data Final data Segmentation function Level set formulation of Riemannian mean curvature flow equation Implicit schema using Newton-like linearization method Iterative algorithms based on Krylov subspaces methods (GMRES) block diagonals preconditioners (bjacobi) Original data + Final data Segmentation function Workshop finale dei Progetti Grid del PON "Ricerca” 2000-2006 - Avviso 1575 10-12 Febbraio Catania

Workshop finale dei Progetti Grid del PON "Ricerca” PSE layered structure The software architecture of MedIGrid PSE is composed by the following layers : The User Interface (GUI) includes tools to set image processing parameters, to visualize images and to make image post processing (i.e. rendering) The Grid Portal, based on LCG/gLite java library (WMS-Proxy, LFC/GFAL, JobMonitor), hides to the final user the Grid Environment Core and collective services: some services provided by LCG/gLite middleware, some components of application middleware (i.e. Scientific libraries) and some numerical kernels Workshop finale dei Progetti Grid del PON "Ricerca” 2000-2006 - Avviso 1575 10-12 Febbraio Catania

Workshop finale dei Progetti Grid del PON "Ricerca” PSE layered structure Workshop finale dei Progetti Grid del PON "Ricerca” 2000-2006 - Avviso 1575 10-12 Febbraio Catania

The GUI: Access the portal Authentication VOMS based Medical Imaging section choice Workshop finale dei Progetti Grid del PON "Ricerca” 2000-2006 - Avviso 1575 10-12 Febbraio Catania

The GUI: Job creation & submission Input data selection Parameters setting Algorithm selection Job submission Workshop finale dei Progetti Grid del PON "Ricerca” 2000-2006 - Avviso 1575 10-12 Febbraio Catania

The GUI: Job monitoring Workshop finale dei Progetti Grid del PON "Ricerca” 2000-2006 - Avviso 1575 10-12 Febbraio Catania

The GUI: Job monitoring Job selection State visualization Workshop finale dei Progetti Grid del PON "Ricerca” 2000-2006 - Avviso 1575 10-12 Febbraio Catania

The GUI: output visualization and post processing Output data selection Visualize & post process Workshop finale dei Progetti Grid del PON "Ricerca” 2000-2006 - Avviso 1575 10-12 Febbraio Catania

The GUI: Results visualization Workshop finale dei Progetti Grid del PON "Ricerca” 2000-2006 - Avviso 1575 10-12 Febbraio Catania

Job Submission: MedIGrid schema 1/4 Job definition (steps 1-2): The user, by using the Grid Portal, defines the job parameters (by chooising image processing algorithm, by browsing the File Catalogue to select input data, by fixing the output data logical name, ... ) through the Grafical Interface; Workshop finale dei Progetti Grid del PON "Ricerca” 2000-2006 - Avviso 1575 10-12 Febbraio Catania

Job Submission: MedIGrid schema 2/4 Job sumbission (steps 3-4): The submission service on Grid Portal, submits the defined job to the Resource Broker that choices a Computing Element; Job execution (steps 5-6): The first step of the job script asks to LFC for application binary and input data and then gets them directly from the Storage Element; Workshop finale dei Progetti Grid del PON "Ricerca” 2000-2006 - Avviso 1575 10-12 Febbraio Catania

Job Submission: MedIGrid schema 3/4 Data archiviation (step 7): At the end of the process the job script copies the output data directly on the SE and, at the same time, register files on the File Catalogue with the symbolic name chosen at step 1 from the user. Workshop finale dei Progetti Grid del PON "Ricerca” 2000-2006 - Avviso 1575 10-12 Febbraio Catania

Job Submission: MedIGrid schema 4/4 Output retrieval and visualization (steps 8-12): The execution log file moves from the Working Node to the User Interface through the Resource Broker to be visualized by the user; the user can require output files that, by means of their symbolic names, are moved from the SE directly on the Grid Portal and then visualized. Workshop finale dei Progetti Grid del PON "Ricerca” 2000-2006 - Avviso 1575 10-12 Febbraio Catania

Some strategies about job submission At execution time, parameter files reach computational resources passing through the Resource Broker (WMS), while binaries and input data are transferred directly from Storage to Computational Resource At the end of the execution output log file reaches the UI through theWMS, while the output image is directly moved on the Storage and registered on the File Catalogue. No bottlenecks, due to multiple and unuseful Resource Broker (WMS) transits, and no limitation imposed from WMS to data sizes Workshop finale dei Progetti Grid del PON "Ricerca” 2000-2006 - Avviso 1575 10-12 Febbraio Catania

Some details on other strategies We preserve execution from unattended behaviours (i.e. faults and performance degradation) Checkpointing techniques Workshop finale dei Progetti Grid del PON "Ricerca” 2000-2006 - Avviso 1575 10-12 Febbraio Catania

Checkpointing strategy 1/2 Numerical kernels are based on iterative algorithms The implemented schema is a “disk-based distributed checkponting” The key-steps are: the last ”correct” computed data is retrieved from remote storage resource by LFC service and brought on the WN the current data is computed the computed data is stored on remote storage resource and registered on the LFC Workshop finale dei Progetti Grid del PON "Ricerca” 2000-2006 - Avviso 1575 10-12 Febbraio Catania

Checkpointing strategy 2/2 In this schema: we can replicate data on more remote storages (as allowed by LFC/GFAL system) data are always available, even if storage and computing resources can become unavailable. Is implemented a storage service reliability, also paying for overhead in data transfer. Workshop finale dei Progetti Grid del PON "Ricerca” 2000-2006 - Avviso 1575 10-12 Febbraio Catania

Conclusion and feature work New kernels for medical imaging Optimize existing mechanisms for checkpointing (i.e. by combining diskless techniques) Efficient and “standard” based data and metadata management Workshop finale dei Progetti Grid del PON "Ricerca” 2000-2006 - Avviso 1575 10-12 Febbraio Catania

MedIGrid References L. Carracciuolo, L. D’Amore, and A. Murli. Towards a parallel component for imaging in PETSc programming environment: A case study in 3-D echocardiography. Parallel Computing, 32:67–83, 2006. L. D’Amore, D. Casaburi, L. Marcellino, and A. Murli Segmentazione di sequenze di immagini digitali mediante previsione del moto. Tech Rep. 62, Dip. di Matematica e Applicazioni, Università degli Studi di Napoli Federico II, 2006 Murli A., Boccia V., Carracciuolo L.,D'Amore L., Laccetti G. and Lapegna M. Montoring and Migration of a PETSc-based Parallel Application for Medical Imaging in a Grid computing PSE, in IFIP  International Federation for Information Processing, Volume 239, Grid-Based Problem Solving Environments, eds. Gaffney P.W., Pool, J.C.T., (Boston: Springer), pp 421-432., 2007 A. Murli, A. Boccia, L. Carracciuolo, L. D’Amore, G. Laccetti, M. Lapegna. The MedIGrid PSE in an LCG/gLite environment, Parallel and Distributed Processing with Applications, ISPA ’08. 10-12 Dec. Sidney (Australia) 2008.