ENEA-CRESCO Computing Lab

The CRESCO computing laboratory at ENEA consists of a family of Linux HPC (High Performance Computing) Clusters, a large infrastructure for distributed computing. It offers production quality, service-oriented system for high performance and/or high throughput computing. All computing facilities in ENEA are integrated within ENEAGRID (https://www.eneagrid.enea.it), an infrastructure for distributed computing over 6 sites whose main facilities are the HPC CRESCO clusters in operation at ENEA. ENEA’s HPC supports R&D activities of several ENEA research groups and external associated partners, in sectors such as alternative energies, energy efficiency, climate and environment, new materials, manufacturing technologies, bioinformatics, transports, security and cultural heritage.

ENEAGRID is characterized by solid structural components, offering reliability and easy management, and by web interfaces which have been developed in–house and customized so as to ensure a friendly user environment: (a) authentication via Kerberos v5; (b) geographic filesystems: AFS/OpenAFS; (c) GPFS parallel filesystem (now IBM Spectrum Scale) also on WAN among different clusters and sites for total of 1.5 PB; (d)resource management via IBM-LSF; (e) system monitoring via Zabbix; (f) web access via in–house FARO interface;(g) user management via in–house WARC interface,. The computing resources currently provided to the users are x86–64 Linux systems for a total of 26000 cores.

The ENEA main computing facility is CRESCO6 installed in ENEA Portici. CRESCO6 is an HPC system consisting of 434 dual 24 cores Skylake cpu nodes, ranked 420th in the TOP500 list of Nov. 2018, 1.4 PFlops. It is a Lenovo ThinkSystem SD530 platform, an ultra-dense two-socket server in a 0.5 U rack form factor inserted in a 2U four-mode enclosure. The nodes are interconnected by an Intel Omni-Path network with 21 Intel Edge switches 100 series of 48 ports each, bandwidth equal to 100 GB/s, latency equal to 100ns. A second cluster is installed in the ENEA Frascati research centre of about 250 Tflop of peak computing power. It is water cooled and has 10 nodes (IBM Power 2x PPC8 + 4 GPU Tesla P100) with a total of 40 GPU.

Several services are offered by the infrastructure:

• Support to design large scale HPC simulations.
• Porting of numerical codes on CRESCO and their optimization for efficient exploitation of the CRESCO infrastructure.
• BigData and AI tools and large storage resources.
• Several numerical codes and tools are already available.

The collaboration with the ENEA staff is the key to design efficient and innovative high-end simulations. Most of the users are ENEA users and they collaborators in national and international projects. The most important results are in the CRESCO report, edited every year. The last year report is available for free at: https://ict.enea.it/category/rapporti-cresco/

• The available infrastructure is CRESCO6.
• The total time for the users is 6 months, extensions can be considered.
• The first 2 months are dedicated to the design and preparation of the simulations and benchmarking in collaboration with ENEA staff. In this period of time, users can use freely several queues on a small number of nodes and focused training activities can be activated.
• Then 4 months are dedicated to the large-scale simulations that are planned with the ENEA staff.
• The overall cost is: 2 PM + 20k€ for the infrastructure (20k node-hours).
• The users have to sign an agreement with ENEA where access policies are detailed and accepted.

Access costs via: unit costs (a), actual costs (b), or a combination (c): (b) the flat user access cost is in line with the current HPC market (1 euro/node) however it could vary depending on the requested support to port codes, develop ICT procedures, performance evaluation, code optimizations etc.

• Computer aided design of new materials and characterization of their physic-chemical properties.
• Several parallel numerical codes for atomic-scale modelling and visualization are available (Quantum Espresso suite, CP2K, LAMMPS, GROMACS).

All CRESCO users performing computational materials science on CRESCO are gathered in the CRESCO-CMAST (Computational Materials Science and Technologies) Virtual Lab. This laboraty is used to exchange information, best practices and promote new projects. Several European and national projects are currently running on the CRESCO platform.

• MiSE-ENEA-National Program agreement with Italian Ministry of industry and Economic development. Research in Innovative PV technologies (project B1.2) ENEA is contractor for the development of thin films and perovskite and kesterite, a-Si/c-Si hetero-junction solar cell, SMPPT converters, development of multifunctional PV modules based on innovative materials and technologies (2019-2021) https://www.enea.it/it/Ricerca_sviluppo/lenergia/ricerca-di-sistema-elettrico/accordo-di-programma-MiSE-ENEA-2019-2021/tecnologie/fotovoltaico-ad-alta-efficienza.
• H2020-AMPERE Automated photovoltaic cell and Module industrial Production to regain and secure European Renewable Energy market. ENEA Dissemination and Communication leader (WP7) and participate in New materials development (TCO and advanced metallization). Grant agreement ID: 745601 (2017-2020) http://www.ampere-h2020.eu/.
• FP7-CHEETAH photovoltaic modules - joining Europe’s Research and Development efforts in support of its PV industry. ENEA WP2 leader Fostering the use of existing facilities and expertise and it participated in thin film and OPV, materials and devices realization and characterization, testing of devices and modules, standardization of efficiency and stability measurements, life cycles analysis. Grant agreement ID: 609788 (2014-2017) https://www.cheetah-exchange.eu/.
• FP7-SOPHiA - Photovoltaic European Research Infrastructure. 1/02/2011-31/01/2015. ENEA WP Leader WP4 - Exchange of personnel, spreading of good practices and training courses to new users, summer school. Grant agreement ID: 262533 (2011-2014).
• H2020-EoCoE-II - Energy Oriented Center of Excellence: toward exascale for energy –European Data Infrastructure (RIA) (2019-2021) https://www.eocoe.eu/