Javascript must be enabled to continue!
A workflow for cloud-based and HPC simulations with the NEMO ocean model using containers
View through CrossRef
In this work, we explored the deployment and execution of the NEMO ocean model using Singularity containers within the EDITO Model Lab, implementing the European Digital Twin of the Ocean. The Auto-NEMO workflow, a fork of Auto-EC-Earth used to run NEMO workflows using the NEMO Community reference code, was adapted to run simulations using containers. The use of a Singularity container ensures consistent execution by packaging all dependencies, making it easier to deploy the model across various HPC systems.The containerized approach was tested on multiple HPC platforms, including MareNostrum5 and LUMI, to evaluate scaling performance. Our tests compared the use of mpich and openmp libraries, providing insights into how communication strategies impact the computational performance of the model in containerized setups. In addition, the runs are orchestrated by a content workflow manager, in this case Autosubmit, deployed in a cloud infrastructure in EDITO-Infra, making the entire solution (workflow manager and workflow itself) portable end-to-end. The benefits of portability and reproducibility make containers an attractive solution for streamlining workflows in diverse computational environments.A comparison between containerized and non-containerized runs highlights the trade-offs involved. Direct execution may provide slightly better performance in some cases, but the containerized approach greatly reduces setup complexity. These findings demonstrate the potential of containerization to enhance efficiency and accessibility in large-scale ocean modeling efforts.
Title: A workflow for cloud-based and HPC simulations with the NEMO ocean model using containers
Description:
In this work, we explored the deployment and execution of the NEMO ocean model using Singularity containers within the EDITO Model Lab, implementing the European Digital Twin of the Ocean.
The Auto-NEMO workflow, a fork of Auto-EC-Earth used to run NEMO workflows using the NEMO Community reference code, was adapted to run simulations using containers.
The use of a Singularity container ensures consistent execution by packaging all dependencies, making it easier to deploy the model across various HPC systems.
The containerized approach was tested on multiple HPC platforms, including MareNostrum5 and LUMI, to evaluate scaling performance.
Our tests compared the use of mpich and openmp libraries, providing insights into how communication strategies impact the computational performance of the model in containerized setups.
In addition, the runs are orchestrated by a content workflow manager, in this case Autosubmit, deployed in a cloud infrastructure in EDITO-Infra, making the entire solution (workflow manager and workflow itself) portable end-to-end.
The benefits of portability and reproducibility make containers an attractive solution for streamlining workflows in diverse computational environments.
A comparison between containerized and non-containerized runs highlights the trade-offs involved.
Direct execution may provide slightly better performance in some cases, but the containerized approach greatly reduces setup complexity.
These findings demonstrate the potential of containerization to enhance efficiency and accessibility in large-scale ocean modeling efforts.
Related Results
Democratising HPC Training: co-creating an Industrial HPC Nano Online Course
Democratising HPC Training: co-creating an Industrial HPC Nano Online Course
Abstract
Purpose: Disruptive technologies (AI, IoT, etc) unlock new frontiers of data-centric innovation. This increases the computational needs, pushing more and more comp...
A paradigm shift of HPC for geosciences: a novel HPC service model for geosciences applications
A paradigm shift of HPC for geosciences: a novel HPC service model for geosciences applications
(English) The Oil and Gas (O&G) industry ranks prominently among the leading commercial users of powerful supercomputers worldwide, as indicated by global High-Performance Comp...
HPC Cloud Architecture to Reduce HPC Workflow Complexity in Containerized Environments
HPC Cloud Architecture to Reduce HPC Workflow Complexity in Containerized Environments
The complexity of high-performance computing (HPC) workflows is an important issue in the provision of HPC cloud services in most national supercomputing centers. This complexity p...
CLOUD COMPUTING - NAVIGATING THE DIGITAL SKY
CLOUD COMPUTING - NAVIGATING THE DIGITAL SKY
“Cloud Computing – Navigating the Digital Sky” is an extensive guide designed to provide a thorough understanding of cloud computing, an essential technology in today’s digital age...
TABular Semantic Enhancement Blueprint (TAB-SEB) v1
TABular Semantic Enhancement Blueprint (TAB-SEB) v1
Project website link: https://ariannamorettj.github.io/tab_seb/ Overview Purpose. The workflow blueprint supports semantic enhancement of Cultural Heritage and GLAM metadata by c...
Access impact of observations
Access impact of observations
The accuracy of the Copernicus Marine Environment and Monitoring Service (CMEMS) ocean analysis and forecasts highly depend on the availability and quality of observations to be as...
Performance evaluation of NEMO4.2 with Paraver
Performance evaluation of NEMO4.2 with Paraver
The last release of the NEMO v4.2 ocean model includes many modifications that have a significant impact on the model performance. The goal of the work is to assess NEMO performanc...
LLM as HPC Expert: Extending RAG Architecture for HPC Data
LLM as HPC Expert: Extending RAG Architecture for HPC Data
High-Performance Computing (HPC) is crucial for performing advanced computational tasks, yet their complexity often challenges users, particularly those unfamiliar with HPC-specifi...

