Interest Group High-Performance Computing in Earth System Sciences (IG HPC in ESS)
This NFDI4Earth Interest Group provides a forum for High-Performance Computing (HPC) centres and users working towards a simplified re-use and FAIR (Findable, Accessible, Interoperable, Reusable) handling of Earth System Science (ESS) data on large computing systems. Interested? You are always welcome to sign up, as indicated below.
Motivation
There is an ever-increasing demand for using HPC infrastructures for solving geoscientific questions in different domains. Common examples for this can be found, e.g., in the weather and climate forecasting and the remote sensing community which need to cope with the generation, management and analysis of a rapidly increasing amount of high-resolution multidimensional data sets. However, the distribution and community-driven reuse of such large data sets that are hosted and produced at HPC facilities following the FAIR principles still poses a considerable challenge.
Our Aims
With the FAIR principles in mind, we have defined two focus areas for the interest group:
1) "Metadata, Interoperability and Reproducibility": We will help to devise and harmonize methods for the automatic enrichment of ESS data on HPC systems with a sufficient and standardized set of metadata, following NFDI4Earth and global standards. Metadata schemas will be recommended such that simulation data will be increasingly reproducible (by sufficient description of the execution environment and methodology) and interoperable (by sufficient description of data formats, etc.).
1) "Federated access, Findability and Accessibility": In order to make practical use of FAIR data, easy access to data storage and analysis facilities in the HPC centres is a must. Working - beyond NFDI4Earth - with national and international initiatives, we aim at making efficient "in-place" ESS data analysis possible by providing access to geographically-distributed computing systems (e.g. via federated identities). Where necessary, data collection from different systems has to be facilitated. Orchestrated, distributed ESS data analysis will be the logical next step.
The participants of this group, being mostly HPC providers or users, see themselves as a bridge between NFDI4Earth Measures, other NFDI consortia tackling similar issues (e.g. NFDI4Ing TA Doris, NFDIxCS) and common NFDI activities on the topic (in coordination with NFDI4Earth Measure 3.3 "NFDI Commons").
How to Participate and Contact Us
We have monthly virtual meetings, discussing the outlined topics in an iterative manner. In addition, physical meetings and workshops will be co-organized with regular NFDI4Earth events (such as the NFDI4Earth conferences, plenary meetings, etc.). Just contact us (see below) when interested.
Contacts / Sign-Up:
Please write to the current group leads for sign-up:
Meeting Time
We meet every second Monday of the month at 9:00 am.
Meeting Minutes
https://docs.google.com/document/d/1iwbonNFE48ZKD_t9lmsZH5zMMwl4NJ1JVgDsD0HLsmo/edit?usp=sharing
Concept Paper and Further Information
More on Our Vision and Aims
We have a concept paper (to be updated every 18 months), describing our aims with respect to the current situation in the field, here: https://doi.org/10.5281/zenodo.6565404.
A Few Further Selected Resources / Past Activities
2022
Our first HPC-NFDI4Earth-Workshop was held on November 10, 2022, 2:00 pm - 5:30 pm (Online): Federated and FAIR Data in HPC
2020
N4E-Konferenz_Block-III_SIG-1_HPC_Earth_Kurtz (slides of 1st N4E conference Nov-2020)