Singularity at Calcula
Introduction
Singularity is a popular containerization solution that allows users to create and run containers for various applications and workflows. This documentation will guide you through the process of using Singularity at Calcula. Singularity provides a convenient way to package your software and its dependencies, ensuring reproducibility and portability across different computing environments.
Installing Singularity
Singularity may already be installed on the Calcula cluster. If not, follow these steps to install Singularity on your account:
1. Log in to Calcula using SSH or the provided remote access method.
2. Check if Singularity is already installed by running the following command:
>> module avail
If a version number is displayed, Singularity is installed, and you can skip the installation step. In this case, to get to module you have to copy your singularity version, for example Singularity/3.11.0, and type:
>> module load singularity/3.11.0
If module command is not found, you can go to the Environment Modules documentation and you will find information about the module there. Currently, only GPI Research Group has module and therefore Singularity available.
If Singularity is not installed, you need administrative (root) privileges to install it. Contact your system administrator or refer to the cluster documentation for instructions on requesting software installations.
3.Once Singularity is installed, you can proceed to the next section.
Overview of the Singularity Interface
Singularity’s command line interface allows you to build and interact with containers transparently. You can run programs inside a container as if they were running on your host system. You can easily redirect IO, use pipes, pass arguments, and access files, sockets, and ports on the host system from within a container.
You can type the help command to see the available options:
>> singularity -h
Where you will find options such as –debug, that allows you to run singularity with debugging messages. The command would be:
>> singularity --debug run library://sylabsed/examples/lolcow
Building Singularity Images
Singularity containers are built from image files that encapsulate the application and its dependencies. You can either build your own images or use pre-built images from trusted sources (verified images). In this documentation we will explain both options:
1. Downloading pre-built Singularity images on Calcula:
You can use the pull and build commands to download pre-built images from an external resource like Docker Hub :
>> singularity pull docker://ubuntu:latest
or the Container Library :
>> singularity pull library://sylabsed/linux/alpine
Please, keep in mind that all of these are examples, and you will need to substitute the files and libraries with your desired ones.
Finally, you can also use the build command to download pre-built images from an external resource. With this command you will have a different image, so you will have to specify a name. In this example, we used lolcow.sif:
>> singularity build ubuntu.sif docker://ubuntu:latest
Keep in mind that Docker Hub is an open library where everyone can upload their container images, so when downloading images, always make sure to do it from a trusted publisher.
Interacting with images
Interacting with images can be done using various commands. Let's take the example of an image named "lolcow_latest.sif".
1. Shell
To create a new shell within the container and interact with it like a virtual machine, you can use the "shell" command:
$ singularity shell myimage.sif Singularity myimage.sif:~>
The change in prompt indicates that you are inside the container.
The "shell" command also works with URIs like "library://", "docker://", and "shub://". It creates a temporary container that disappears when you exit the shell:
$ singularity shell library://sylabsed/examples/lolcow
2. Exec
The "exec" command allows you to execute custom commands within the container by specifying the image file. For example, to run the "cowsay" program inside the "lolcow_latest.sif" container:
$ singularity exec lolcow_latest.sif cowsay "UPC is the best uni!"
_____
< UPC is the best uni! >
-----
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
The "exec" command also works with URIs like "library://", "docker://", and "shub://". It creates a temporary container that executes the command and then disappears:
$ singularity exec library://sylabsed/examples/lolcow cowsay "UPC is the best uni!"
_________________________
< UPC is the best uni! >
-------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
3.Running a container
To run a container, Singularity containers have runscripts, which define the actions to perform when the container is run. You can trigger the runscript using the "run" command or by calling the container as if it were an executable:
$ singularity run lolcow_latest.sif
_____________________________________
/ You have been selected for a secret \
\ mission. /
-------------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
$ ./lolcow_latest.sif
____________________________________
/ Q: What is orange and goes "click, \
\ click?" A: A ball point carrot. /
------------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
The "run" command also works with URIs like "library://", "docker://", and "shub://". It creates a temporary container that runs and then disappears.
$ singularity run library://sylabsed/examples/lolcow
____________________________________
/ Is that really YOU that is reading \
\ this? /
------------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
This documentation is an introduction to working with singularity. To see a more detailed explanation you can visit the official website's documentation
2. Creating images on Calcula from scratch:
Now, instead of using already pre-existing images, we can create our own. Here we will give you the steps to do it:
2.1. Create a Singularity definition file
The Singularity definition file is a text file that describes the steps to build your image. It typically has a '.def' extension. Create a new file and open it in a text editor.
2.2. Define the base image
In the definition file, start by specifying the base image that you want to use. This could be an existing Singularity image or a Docker image. For example, you can use a base image like Ubuntu:
BootStrap: docker From: ubuntu:latest
2.3. Add necessary lines: Inside the definition file, you can specify the packages or dependencies you want to include in the image with the %post section, add files or scripts using the %files section, or even add labels with the %labels section. An example definition file is:
BootStrap: library From: ubuntu:16.04 %post apt-get -y update apt-get -y install fortune cowsay lolcat %environment export LC_ALL=C export PATH=/usr/games:$PATH %runscript fortune | cowsay | lolcat %labels Author GodloveD
Please note that here an overview of the system and the options is given, but to have a more precise and detailed explanation you will need to visit the official website's documentation
2.4. Build the Singularity image
Once you have defined your Singularity definition file, you can build the image using the sudo singularity build command followed by the desired image name and the definition file. For example:
$ sudo singularity build myimage.sif myimage.def
The build command, therefore, is used to create images in both ways. It can either create a new image from another existing image, or create it from scratch using a definition file.
Share: