Summary and Setup

Apptainer (formerly known as Singularity) is a free and open-source container platform that allows you to create and run applications in isolated images (also called “containers”) in a simple, portable, fast, and secure manner. It performs operating system level virtualization known as containerization. Many container platforms are available, but Apptainer is designed to bring containers and reproducibility to the scientific community and High-Performance Computing (HPC) use cases. Using Apptainer, developers can work in reproducible environments of their choice and design, and these complete environments can be easily copied and executed on other platforms.

This is an introduction to Apptainer/Singularity, its motivations and applications in HEP.

Based on the Apptainer user guide.

Prerequisites

  • Basic knowledge of the Unix Shell, e.g., from the carpentry course.
  • Access to a computing system with Apptainer/Singularity available. It can either be installed locally, or the machine can have user namespaces enabled and access to CVMFS.

Apptainer and Singularity run only on Linux. So you will have to access a Linuz box, e.g. via ssh, or to run Linux on a VM on your computer. The Software Setup Requisites section will provide instructions for the different platforms. Once you have access to a Linux box you can proceed with the Software Setup section below.

Software Setup Requisites

Details

Different platforms provide different tools to connect to a remote Linux machine (PuTTY/ssh) or to run a Linux box in a virtual machine. Click on the dropdown corresponding to your platform to have access to a Linux box.

Use PuTTY to ssh to a Linux machine.

Or use WSL (the Windows Subsistem for Linux) to install a Linux system, e.g. Alma Linux 9

Use Terminal.app and the ssh command to connect to a Linux machine.

Or use VirtualBox to run a Linux VM like Alma Linux 9.

Use Terminal and go to the next section

Software Setup

In this document we use the names Apptainer and Singularity interchangeably. See the Introduction for more details about existing Apptainer and Singularity versions and the differences between them.

Option 1: Use pre-installed apptainer on a cluster

Apptainer/Singularity has become popular and usually it is available in the institutional computing resources. Check if apptainer or singularity are available with

BASH 

apptainer --version
singularity --version

If installed, you will see apptainer version ... or singularity version ..., depending on the flavor installed. Apptainer is preferable but either one is OK, so if apptainer is there, no need to check for singularity (which most likely will be a link to apptainer). This tutorial requires at least Apptainer 1.0.x or Singularity 3.5.x. Previous versions may not have all the required features. If none is in your $PATH or if the available version is too old, you may still be able to use an updated apptainer via CVMFS: check if you have user namespaces enabled and CVMFS to run singularity that way:

BASH 

[[ $(cat /proc/sys/user/max_user_namespaces) -gt 0 ]] && ls /cvmfs/oasis.opensciencegrid.org/mis/ &>/dev/null && { export PATH=/cvmfs/oasis.opensciencegrid.org/mis/apptainer/bin/:"$PATH"; echo "Success: Added to PATH"; singularity --version; } || echo "Failure: Unable to run Apptainer/Singularity via CVMFS"

If this works, it will be added to your path and you will see your apptainer/singularity version.

If your local computing system does not have Apptainer/Singularity installed, you may request it to your system administrator as suggested here.

Option 2: Install Apptainer/Singularity

You will need a Linux system (including WSL on Windows computers) to run Apptainer/Singularity natively. MacOS is not supported.

If you have root access

It is easiest to install if you have root access.

If not

If the above is not possible and you cannot use the CVMFS distribution you have still an option if user namespace is enabled on your system:

  1. Check if user namespaces are enabled:

    BASH 

    # on Debian/Ubuntu
grep -q 'kernel.unprivileged_userns_clone=1' /etc/sysctl.d/90-unprivileged_userns.conf && \
    echo "User namespaces enabled, continue the Apptainer installation" || \
    echo "User namespaces NOT enabled, your use of Apptainer will be very limited"
# on RHEL/CentOS cat /proc/sys/user/max_user_namespaces` is bigger than 0
[[ $(cat /proc/sys/user/max_user_namespaces) -gt 0 ]] && \
    echo "User namespaces enabled, continue the Apptainer installation" || \
    echo "User namespaces NOT enabled, your use of Apptainer will be very limited"

See these full instructions for more about checking for and enabling user namespaces.

  1. If enabled, install unprivileged Apptainer with one of these three methods (in order of preference):

    1. Chose your INSTALL_DIR and install there the relocatable Apptainer (recommended). Run:

      BASH 

      curl -s https://raw.githubusercontent.com/apptainer/apptainer/main/tools/install-unprivileged.sh | \
    bash -s - INSTALL_DIR

    2. Alternatively install from source without root privileges.

    3. Or use cvmfsexec to get CVMFS. This is a bit more complex, you can follow the instrictions summarized also in this paper.

If user namespaces are not enabled, apptainers/singularity is not installed, and you have no root access to the host, then your use of apptainer/singularity will be very limited even if you install it with one of the three methods above. You need to request to your system administrator to either install Apptainer/Singularity or to enable user namespaces.