Introduction to containers and Singularity

Almost all software depends on other software components to function, but these components have independent evolutionary paths.

Projects involving many software components can rapidly run into a combinatorial explosion in the number of software version configurations available, yet only a subset of possible configurations actually works as desired.

Virtualisation is an approach that can collect software components together and can help avoid software dependency problems.

Containers are a popular type of lightweight virtualisation that are widely used.

Docker and Singularity are two different software platforms that can create and manage containers and the resources they use.

Singularity is another container platform and it is often used in cluster/HPC/research environments.

Singularity has a different security model to other container platforms, one of the key reasons that it is well suited to HPC and cluster environments.

Singularity has its own container image format (SIF).

The singularity command can be used to pull images from Singularity Hub and run a container from an image file.

Singularity caches downloaded images so that an image isn’t downloaded again when it is requested using the singularity pull command.

The singularity exec and singularity shell commands provide different options for starting containers.

Singularity can start a container from a Docker image which can be pulled directly from Docker Hub.

Singularity definition files are used to define the build process and configuration for an image.

Singularity’s Docker container provides a way to build images on a platform where Singularity is not installed but Docker is available.

Existing images from remote registries such as Docker Hub can be used as a base for creating new Singularity images.

Singularity images containing MPI applications can be built on one platform and then run on another (e.g. an HPC cluster) if the two platforms have compatible MPI implementations.

When running an MPI application within a Singularity container, use the MPI executable on the host system to launch a Singularity container for each process.

Think about parallel application performance requirements and how where you build/run your image may affect that.

Singularity images require special access to GPUs due to dependency on libraries on the host operating system.

Understand the dependency between host operating system files and GPU runtime libraries in the container.

Understand the next steps for you in using containers.

Understand how you can access help and support to use containers.