OpenMPI on MeluXina
The MeluXina system environment provides the OpenMPI implementation.
EasyBuild module description
The Open MPI Project is an open source Message Passing Interface implementation that is developed and maintained by a consortium of academic, research, and industry partners. Open MPI is therefore able to combine the expertise, technologies, and resources from all across the High Performance Computing community in order to build the best MPI library available. Open MPI offers advantages for system and software vendors, application developers and computer science researchers.
You can use OpenMPI to build your MPI programs written in C/C++ and fortran.
Reserve an interactive session:
salloc -A COMPUTE_ACCOUNT -t 01:00:00 -q dev --res cpudev -p cpu -N 1
The example above will allocate one CPU node in interactive mode (dev QoS with cpudev reservation). Load the OpenMPI (Will load default version if not specified, we recommend to load a specific version) module as in below script.
module load OpenMPI mpicc hello.c -o ./HelloTest_C
You can run your MPI program in a batch job using Slurm. The script below compile a simple C HelloWorld test on one CPU node and 5 tasks allocated for 5 minutes.
#!/bin/bash -l #SBATCH -N 1 #SBATCH --ntasks=5 #SBATCH --cpus-per-task=1 #SBATCH -p cpu #SBATCH -q test #SBATCH --time 00:05:00 #Load OpenMPI module module load OpenMPI #Check OpenMPI version mpicc --version #Compile C program with OpenMPI mpicc hello.c -o ./hello_c #Execute the program srun hello_c
It is recommended to use a Makefile to compile your program. Your Makefile should look like the example below:
#Defines compiler CC=mpicc # These are the options we pass to the compiler. # -std=c++14 means we want to use the C++14 standard. # -stdlib=libc++ specifies that we want to use the standard library implementation called libc++ # -g specifies that we want to include "debugging symbols" which allows us to use a debugging program. # -O0 specifies to do no optimizations on our code. CFLAGS = -std=c++14 -stdlib=libc++ -g -O0 all: hello hello: hello.o $CC $CFLAGS -o hello_c hello.o hello.o: hello.cpp $CC $CFLAGS -c hello.c
It is also possible to compile via a Makefile inside a batch job:
#!/bin/bash -l #SBATCH -N 1 #SBATCH --ntasks=5 #SBATCH --cpus-per-task=1 #SBATCH -p cpu #SBATCH -q test #SBATCH --time 00:05:00 #Load OpenMPI module module load OpenMPI #Check OpenMPI version mpicc --version #Compile C program with OpenMPI (in parallel) make -j #Execute the program srun hello_c
When using our OpenMPI build, you may encounter the following issue related to
[1637941300.411897] [mel2037:9795 :0] mm_posix.c:194 UCX ERROR open(file_name=/proc/9791/fd/41 flags=0x0) failed: No such file or directory [1637941300.411932] [mel2037:9795 :0] mm_ep.c:154 UCX ERROR mm ep failed to connect to remote FIFO id 0xc000000a4000263f: Shared memory error [1637941300.412053] [mel2037:9790 :0] mm_posix.c:194 UCX ERROR open(file_name=/proc/9791/fd/41 flags=0x0) failed: No such file or directory
The problem will be solved by exporting the environment variable: