Skip to content

Python

In development

JupyterLab is accessible for testing, and the service may be subject to disruption. To request access, or to provide feedback, please use our service desk. We are glad to hear from you and learn from your experience!

Overview

In the following, we use the following variables:

Variable Description Example value
${kernel_python} The base Python module to use Python/3.12.0-GCCcore-13.3.0
${kernel_root} The root path to the kernels / virtual environments ${yourProjectDirectory}/.venvs
${kernel_name} The name of the kernel / virtual environment MyTestVenv

What you will need to do is sourcing the script hereafter (you might have to adjust the variables mentioned above, as well as change the modules you want to bring along in your kernel)

Script to source

#!/bin/bash 

# Author:       Marco Magliulo
# Organization: LuxProvide
# Date:         2025-07-03
# Description:  This script streamlines the creation of a jupyter kernel on Meluxina 
#
# Copyright (c) 2025 Marco Magliulo, LuxProvide
#
# This script is licensed under the MIT License.

### Change the stack and the Python version you want 
### But be careful with the fact that you should pick the python version
### that is compliant with the JupyterLab and JupyterHub modules!!!
export stack='release/2024.1'
export pythonVersion='Python/3.12'
export kernel_python=$pythonVersion
### Change this path to the location where you put your venvs!
export kernel_root="/project/home/lxp/venvs"
export kernel_name="MyTestEnv"
export kernel_file="${kernel_root}/${kernel_name}/bin/kernel"
export pythonKernel="${kernel_root}/${kernel_name}/bin/python"

ml env/${stack}
ml JupyterLab JupyterHub
ml ${pythonVersion}
### Optional but recommended: here I unset the PYTHONPATH
### to ensure that the python installed in the site-package of the 
### python module is not going to interfere with my brand new venv! 
unset PYTHONPATH
rm -rf ${kernel_root}/${kernel_name} 
python3 -m venv ${kernel_root}/${kernel_name}

# Here I inject the content of the kernel file
### You can obviously change the LMOD modules that you want to bring along ! 
cat << EOF > ${kernel_file}
#!/bin/bash -li
echo "Kernel name: ${kernel_name}"
echo "Kernel root: ${kernel_root}"
echo "Kernel file: ${kernel_file}"
echo 'This is the initial module list'
module unuse /apps/USE/easybuild/release/latest/modules/all
module use /apps/USE/easybuild/${stack}/modules/all/
module list 
module load cuDF
module load PyTorch
${kernel_root}/${kernel_name}/bin/python "\$@"
EOF

chmod +x ${kernel_file}

source ${kernel_root}/${kernel_name}/bin/activate

$pythonKernel -m pip install -U jupyter-client 2>/dev/null
$pythonKernel -m pip install -U ipykernel 2>/dev/null
$pythonKernel -m ipykernel install --name "${kernel_name}" --user

# Run the command and count the lines
line_count=$(jupyter-kernelspec list | grep -i ${kernel_name}[^/] | wc -l)

# Check if the line count is 1
if [ $line_count -eq 1 ]; then
    echo "Assertion passed: Only one kernel found with the name '${kernel_name}'."
else
    echo "Assertion failed: Expected one kernel with the name '${kernel_name}', but found ${line_count}. This can lead to unexpected behavior."
fi

kernel_json="$(jupyter-kernelspec list | grep -i ${kernel_name}[^/] | awk '{print $2}')/kernel.json"
sed -i "s|/.*/bin/python.*\"|${kernel_root}/${kernel_name}/bin/kernel\"|" "${kernel_json}"

Install packages into the kernel virtual environment

First, activate the kernel's virtual environment:

source ${kernel_root}/${kernel_name}/bin/activate

Then install Python packages with pip:

python3 -m pip install PACKAGE_NAME

Package installation from notebook

We strongly advise against installing Python packages from a notebook cell, e.g.:

!pip install PACKAGE_NAME

This method does not guarantee that the packages are installed in the running kernel / virtual environment.