HPC, Big Data, and AI Convergence Towards Exascale Challenge and Vision

HPC, Big Data, AI Convergence Towards Exascale provides an updated vision on the most advanced computing, storage, and interconnection technologies, that are at basis of convergence among the HPC, Cloud, Big Data, and artificial intelligence (AI) domains. Through the presentation of the solutions devised within recently founded H2020 European projects, this book provides an insight on challenges faced by integrating such technologies and in achieving performance and energy efficiency targets towards the exascale level. Emphasis is given to innovative ways of provisioning and managing resources, as well as monitoring their usage. Industrial and scientific use cases give to the reader practical examples of the needs for a cross-domain convergence. All the chapters in this book pave the road to new generation of technologies, support their development and, in addition, verify them on real-world problems. The readers will find this book useful because it provides an overview of currently available technologies that fit with the concept of unified Cloud-HPC-Big Data-AI applications and presents examples of their actual use in scientific and industrial applications.

1. Towards the Convergence of High-Performance Computing, Cloud and Big Data Domains Martin Golasowski, Jan Martinovic, Marc Levrier, Stephan Hachinger, Sophia Karagiorgou, Aikaterini Papapostolou, Spiros Mouzakitis, Ioannis Tsapelas, Monica Caballero, Marco Aldinucci, Jon Ander Gómez, Antony Chazapis, and Jean- Thomas Acquaviva 2. The LEXIS Platform for Distributed Workflow Execution and Data Management Martin Golasowski, Jan Martinovic, Jan Krenek, Katerina Slaninová, Marc Levrier, Piyush Harsh, Marc Derquennes, Frédéric Donnat, and Olivier Terzo 3. Enabling the HPC and Artificial Intelligence Cross-Stack Convergence at the Exascale Level Alberto Scionti, Paolo Viviani, Giacomo Vitali, Chiara Vercellino, and Olivier Terzo 4. Data System and Data Management in a Federation of HPC/Cloud Centers Johannes Munke, Mohamad Hayek, Martin Golasowski, Rubén J. García- Hernández, Frédéric Donnat, Cédric Koch- Hofer, Philippe Couvee, Stephan Hachinger, and Jan Martinovic 5. Distributed HPC Resources Orchestration for Supporting Large-Scale Workflow Execution Alberto Scionti, Paolo Viviani, Giacomo Vitali, Chiara Vercellino, Olivier Terzo, Stephan Hachinger, Lukáš Vojácek, Václav Svaton, Jan Krenek, Frédéric Donnat, and François Exertier 6. Advanced Engineering Platform Supporting CFD Simulations of Aeronautical Engine Critical Parts Donato Magarielli, Ennio Spano, Tommaso Fondelli, Daniele Pampaloni, Antonio Andreini, Paolo Savio, Michele Marconcini, and Francesco Poli 7. Event-driven, Time-Constrained Workflows: An Earthquake and Tsunami Pilot Rubén J. García- Hernández, Thierry Goubier, Danijel Schorlemmer, Natalja Rakowsky, Sven Harig, Andrea Ajmar, Lorenza Bovio, Stephane Louise, Tomáš Martinovic 8. Exploitation of Multiple Model Layers Within LEXIS Weather and Climate Pilot: An HPPC-Based Approach Paola Mazzoglio, Emanuele Danovaro, Laurent Ganne, Andrea Parodi, Stephan Hachinger, Antonella Galizia, Antonio Parodi, and Jan Martinovic 9. Data Convergence for High-Performance Cloud Christian Pinto, Antony Chazapis, Christos Kozanitis, Yiannis Gkoufas, Panos Koutsovasilis, Srikumar Venugopal, Jean- Thomas Acquaviva, and Angelos Bilas 10. The DeepHealth HPC Infrastructure: Leveraging Heterogenous HPC and Cloud Computing Infrastructures for IA-based Medical Solutions Eduardo Quiñones, Jesus Perales, Jorge Ejarque, Asaf Badouh, Santiago Marco, Fabrice Auzanneau, François Galea, David González, José Ramón Hervás, Tatiana Silva, Iacopo Colonnelli, Barbara Cantalupo, Marco Aldinucci, Enzo Tartaglione, Rafael Tornero, José Flich, Jose Maria Martínez, David Rodriguez, Izan Catalán, Jorge García, and Carles Hernández 11. Applications of AI and HPC in theHealth Domain Dana Oniga, Barbara Cantalupo, Daniele Perlo, Marco Grangetto, Federico Bolelli, Federico Pollastri, Michele Cancilla, Laura Canalini, Costantino Grana, Cristina Muñoz Alcalde, Franco Alberto Cardillo, Monica Florea, Enzo Tartaglione, and Marco Aldinucci 12. CYBELE: On the Convergence of HPC, Big Data Services, and AI Technologies Sophia Karagiorgou, Aikaterini Papapostolou, Zeginis Dimitris, Yannis Georgiou, Eugene Frimpong, Ioannis Tsapelas, Spiros Mouzakitis, and Konstantinos Tarabanis 13. CYBELE: A Hybrid Architecture of HPC and Big Data for AI Applications in Agriculture Naweiluo Zhou, Li Zhong, Dennis Hoppe, Branislav Pejak, Oskar Marko, Javier Cardona, Mikolaj Czerkawski, Ivan Andonovic, Craig Michie, Christos Tachtatzis, Emmanouil Alexakis, Philip Mavrepis, Dimosthenis Kyriazis, and Marcin Pospieszny 14. European Processor Initiative: Europe's Approach to Exascale Computing Mario Kovac, Jean- Marc Denis, Philippe Notton, Etienne Walter, Denis Dutoit, Frank Badstuebner, Stephan Stilkerich, Christian Feldmann, Benoît Dinechin, Renaud Stevens, Fabrizio Magugliani, Ricardo Chaves, Josip Knezovic´, Daniel Hofman, Mario Brcic´, Katarina Vukušic´, Agneza Šandic´, Leon Dragic´, Igor Piljic´, Mate Kovac, Branimir Malnar, and Alen Duspara

Olivier Terzo is currently the Head of the ACA Advance Computing and Applications Research Area with a staff of 16 Reseaerchers at the LINKS Foundation Research Center (Italy). Jan Martinovic is currently the Head of the Advanced Data Analysis and Simulations Lab at IT4Innovations National Supercomputing Center.