Working with Singularity containers
Overview
Teaching: 25 min
Exercises: 15 minQuestions
How do I run a shell or different commands within a container?
Where does Singularity store images?
Objectives
Learn about Singularity’s image cache.
Understand how to run different commands when starting a container and open an interactive shell within a container environment.
Learn more about how singularity handles users and binds directories from the host filesystem.
Learn how to run Singularity containers based on Docker images.
Singularity’s image cache
While Singularity doesn’t have a local image repository in the same way as Docker, it does cache downloaded image files. As we saw in the previous episode, images are simply .sif files stored on your local disk.
If you delete a local .sif image that you have pulled from a remote image repository and then pull it again, if the image is unchanged from the version you previously pulled, you will be given a copy of the image file from your local cache rather than the image being downloaded again from the remote source. This removes unnecessary network transfers and is particularly useful for large images which may take some time to transfer over the network. To demonstrate this, remove the hello-world.sif file stored in your test directory and then issue the pull command again:
$ rm hello-world.sif
$ singularity pull hello-world.sif docker://hello-world
INFO: Using cached SIF image
As we can see in the above output, the image has been returned from the cache and we don’t see the output that we saw previously showing the image being downloaded from Singularity Hub.
How do we know what is stored in the local cache? We can find out using the singularity cache command:
$ singularity cache list
There are 1 container file(s) using 44.00 KiB and 3 oci blob file(s) using 3.36 KiB of space
Total space used: 47.36 KiB
This tells us how many container files are stored in the cache and how much disk space the cache is using but it doesn’t tell us what is actually being stored. To find out more information we can add the -v verbose flag to the list command:
$ singularity cache list -v
NAME DATE CREATED SIZE TYPE
0dcea989af054c9b5ab290 2023-12-11 22:16:39 0.57 KiB blob
719385e32844401d57ecfd 2023-12-11 22:16:39 2.40 KiB blob
ad7c8b818cf3016b2b6437 2023-12-11 22:16:39 0.39 KiB blob
e28fc75c4cbf64761ce6dd 2023-12-11 22:16:40 44.00 KiB oci-tmp
There are 1 container file(s) using 44.00 KiB and 3 oci blob file(s) using 3.36 KiB of space
Total space used: 47.36 KiB
This provides us with some more useful information about the actual images stored in the cache. In the TYPE column we can see that our image type is blob because it’s a docker image that has been pulled from Docker Hub.
Cleaning the Singularity image cache
We can remove images from the cache using the
singularity cache cleancommand. Running the command without any options will display a warning and ask you to confirm that you want to remove everything from your cache.You can also remove specific images or all images of a particular type. Look at the output of
singularity cache clean --helpfor more information.
Working with containers
Running specific commands within a container
We saw earlier that we can use the singularity inspect command to see the run script that a container is configured to run by default. What if we want to run a different command within a container, or we want to open a shell within a container that we can interact with?
If we know the path of an executable that we want to run within a container, we can use the singularity exec command. For example, using the hello-world.sif container that we’ve already pulled from Docker Hub, we can run the following within the test directory where the hello-world.sif file is located:
$ singularity exec hello-world.sif /bin/echo Hello World!
FATAL: stat /bin/echo: no such file or directory
Here we see that a container has been started from the hello-world.sif image and the /bin/echo command was not found. The command provided an error and the container has terminated.
Basic exercise: Running a different command within the “hello-world” container
Can you run a container based on the
hello-world.sifimage that replicates the run command?Solution
$ singularity exec hello-world.sif /helloHello from Docker! ...
Running a shell within a container
If you want to open an interactive shell within a container, Singularity provides the singularity shell command. Let’s download a Ubuntu image from Docker Hub:
$ singularity pull ubuntu.sif docker://ubuntu
Again, using the ubuntu.sif image, and within our test directory, we can run a shell within a container from the hello-world image:
$ singularity shell ubuntu.sif
Apptainer> whoami
[<your username>]
Apptainer> ls
hello-world.sif
ubuntu.sif
Apptainer>
As shown above, we have opened a shell in a new container started from the ubuntu.sif image.
Running a shell inside a Singularity container
Q: What do you notice about the output of the above commands entered within the Singularity container shell?
Q: How would some programs behave? Try
lsb_release -aanduname -a.
Use the exit command to exit from the container shell.
Users, files and directories within a Singularity container
The first thing to note is that when you run whoami within the container you should see the username that you are signed in as on the host system when you run the container. For example, if my username is c.username:
$ singularity shell ubuntu.sif
Singularity> whoami
c.username
But hang on! I downloaded the standard, public version of the ubuntu image from Docker Hub. I haven’t customised it in any way. How is it configured with my own user details?!
If you have any familiarity with Linux system administration, you may be aware that in Linux, users and their Unix groups are configured in the /etc/passwd and /etc/group files respectively. In order for the shell within the container to know of my user, the relevant user information needs to be available within these files within the container.
Assuming this feature is enabled on your system, when the container is started, Singularity appends the relevant user and group lines from the host system to the /etc/passwd and /etc/group files within the container [1].
Singularity also binds some directories from the host system where you are running the singularity command into the container that you’re starting. Note that this bind process isn’t copying files into the running container, it is simply making an existing directory on the host system visible and accessible within the container environment. If you write files to this directory within the running container, when the container shuts down, those changes will persist in the relevant location on the host system.
There is a default configuration of which files and directories are bound into the container but ultimate control of how things are set up on the system where you’re running Singularity is determined by the system administrator. As a result, this section provides an overview but you may find that things are a little different on the system that you’re running on.
One directory that is likely to be accessible within a container that you start is your home directory. The mapping of file content and directories from a host system into a Singularity container is illustrated in the example below showing a subset of the directories on the host Linux system and in a Singularity container:
Host system: Singularity container:
------------- ----------------------
/ /
├── bin ├── bin
├── etc ├── etc
│ ├── ... │ ├── ...
│ ├── group ─> user's group added to group file in container ─>│ ├── group
│ └── passwd ──> user info added to passwd file in container ──>│ └── passwd
├── home ├── usr
│ └── c.username ────> home directory made available ──> ─┐ ├── sbin
├── usr in container via bind mount │ ├── home
├── sbin └────────>└── c.username
└── ... └── ...
Questions and exercises: Files in Singularity containers
Q1: What do you notice about the ownership of files in a container started from the ubuntu image? (e.g. take a look at the ownership of files in the root directory (
/))Exercise 1: In this container, try overwriting the
/environmentfile. What do you notice?_If you’re not familiar with shell output redirection please ask.
Exercise 2: In your home directory within the container shell, try and create a simple text file. Is it possible to do this? If so, why? If not, why not?! If you can successfully create a file, what happens to it when you exit the shell and the container shuts down?
Answers
A1: Use the
ls -lcommand to see a detailed file listing including file ownership and permission details. You should see that all the files are owned by you. This looks good - you should be ready to edit something in the exercise that follows…A Ex1: Unfortunately, it’s not so easy, depending on how you modified
/environmentyou probably saw an error similar to the following:Can't open file for writingorRead-only file systemA Ex2: Within your home directory, you should be able to successfully create a file. Since you’re seeing your home directory on the host system which has been bound into the container, when you exit and the container shuts down, the file that you created within the container should still be present when you look at your home directory on the host system.
Using Docker images with Singularity
Singularity can use Docker images as seen earlier, opening up access to a huge number of existing container images available on Docker Hub and other registries.
While Singularity doesn’t support running Docker images directly, it can pull them from Docker Hub and convert them into a suitable format for running via Singularity. When you pull a Docker image, Singularity pulls the slices or layers that make up the Docker image and converts them into a single-file Singularity SIF image.
For example, moving on from the simple Hello World examples that we’ve looked at so far, let’s pull one of the official Docker Python images. We’ll use the image with the tag 3.8.2-slim-buster which has Python 3.8.2 installed on Debian’s Buster (v10) Linux distribution:
$ singularity pull python-3.8.2.sif docker://python:3.8.2-slim-buster
INFO: Converting OCI blobs to SIF format
INFO: Starting build...
Getting image source signatures
Copying blob 54fec2fa59d0 done
Copying blob cd3f35d84cab done
Copying blob a0afc8e92ef0 done
Copying blob 9691f23efdb7 done
Copying blob 6512e60b314b done
Copying config 1c498e093b done
Writing manifest to image destination
Storing signatures
2020/12/13 23:43:09 info unpack layer: sha256:54fec2fa59d0a0de9cd2dec9850b36c43de451f1fd1c0a5bf8f1cf26a61a5da4
2020/12/13 23:43:10 info unpack layer: sha256:cd3f35d84caba5a287676eeaea3d371e1ed5af8c57c33532228a456e0505b2d5
2020/12/13 23:43:10 info unpack layer: sha256:a0afc8e92ef0f5e56ddda03f8af40a4396226443a446e457ab6ed2dcdec62619
2020/12/13 23:43:11 info unpack layer: sha256:9691f23efdb7fd2829d06ad8fb9c8338487c183bb1aefa0d737cece2a612f51b
2020/12/13 23:43:11 info unpack layer: sha256:6512e60b314b980bce8ece057d15292db0f50ca12dbe6dd5752e1e54c64ccca2
INFO: Creating SIF file...
Note how we see singularity saying that it’s “Converting OCI blobs to SIF format”. We then see the layers of the Docker image being downloaded and unpacked and written into a single SIF file. Once the process is complete, we should see the python-3.8.2.sif image file in the current directory.
We can now run a container from this image as we would with any other singularity image.
Running the Python 3.8.2 image that we just pulled from Docker Hub
Try running the Python 3.8.2 image. What happens?
Try running some simple Python statements…
Running the Python 3.8.2 image
$ singularity run python-3.8.2.sifThis should put you straight into a Python interactive shell within the running container:
Python 3.8.2 (default, Apr 23 2020, 14:32:57) [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>>Now try running some simple Python statements:
>>> import math >>> math.pi 3.141592653589793 >>>
In addition to running a container and having it run the default run script, you could also start a container running a shell in case you want to undertake any configuration prior to running Python. This is covered in the following exercise:
Open a shell within a Python container
Try to run a shell within a singularity container based on the
python-3.8.2.sifimage. That is, run a container that opens a shell rather than the default Python interactive console as we saw above. See if you can find more than one way to achieve this.Within the shell, try starting the Python interactive console and running some Python commands.
Solution
Recall from the earlier material that we can use the
singularity shellcommand to open a shell within a container. To open a regular shell within a container based on thepython-3.8.2.sifimage, we can therefore simply run:$ singularity shell python-3.8.2.sifApptainer> echo $SHELL /bin/bash Apptainer> cat /etc/issue Debian GNU/Linux 10 \n \l Apptainer> exit $It is also possible to use the
singularity execcommand to run an executable within a container. We could, therefore, use theexeccommand to run/bin/bash:$ singularity exec python-3.8.2.sif /bin/bashApptainer> echo $SHELL /bin/bashYou can run the Python console from your container shell simply by running the
pythoncommand.
This concludes the second episode and Part I of the Singularity material. Part II contains a further three episodes where we’ll look creating your own images and then more advanced use of containers for running MPI parallel applications and running with Nvidia GPUs.
References
[1] Gregory M. Kurzer, Containers for Science, Reproducibility and Mobility: Singularity P2. Intel HPC Developer Conference, 2017. Available at: https://www.intel.com/content/dam/www/public/us/en/documents/presentation/hpc-containers-singularity-advanced.pdf
Key Points
Singularity caches downloaded images so that an image isn’t downloaded again when it is requested using the
singularity pullcommand.The
singularity execandsingularity shellcommands provide different options for starting containers.Singularity can start a container from a Docker image which can be pulled directly from Docker Hub.