Singularity Containers- An Introduction

Introduction

Containerization is becoming an important part of high-performance computing environments, offering portability and rapid deployment of applications and workflows previously difficult to accommodate on clusters and supercomputers. Singularity is an increasingly popular implementation of containerization targeted specifically to HPC systems. Importantly, Singularity allows users to effortlessly convert Docker containers into its native format.

Singularity containers can also be used in Slurm job files without the need for any Slurm integration, special job directives, or any other fancy acrobatics! In fact, depending on how the container is built and on the application it encapsulates, users could conceivably employ singularity containers (at the command line or inside a Slurm job) thinking they are simply running a native application.

Why Singulairty Containers for HPC?

• Doesn’t requires any application daemon running on a system.
• Singularity container can be distributed as a file (singularity image file).
• Singularity exectuable can be compiled from source code.
• Can work in HPC systems where shared file system is present. Users can not access/modify data which doesn’t belongs to them.
• No changes required on HPC system or in a resource scheduler.

Install Singularity on Local Machine

Installing Singularity involves different steps depending on your operating system. Below are instructions for Linux and macOS. Please note that Singularity is primarily designed for Linux, and while there is experimental support for macOS, the Linux environment is recommended for optimal performance. Linux Installation:

1. Package Manager (Ubuntu/Debian)

sudo apt-get update
sudo apt-get install -y singularity-container

2. Build from Source:

• Clone the Singularity repository

git clone https://github.com/sylabs/singularity.git

• Change into the Singularity directory

cd singularity

• Build and install Singularity

./mconfig
make -C ./builddir
sudo make -C ./builddir install

Verify Installation: After installation, you can verify that Singularity is installed by running:

singularity --version

Build a Sandbox Singularity Container

Building a Singularity container involves creating a portable environment that encapsulates an application and its dependencies. A sandbox environment allows you to run the application consistently across different computing environments. Here are the general steps to build a Singularity container:

Prerequisites:

1. Install Singularity:
   • Make sure you have Singularity installed on your system. You can follow the installation instructions on the official Singularity website.

Steps to Build a Singularity Container:

1. Create a Definition File:

   • Create a text file with a .def extension. This file will contain the instructions for building the Singularity container. Here’s a simple example:

   Bootstrap: docker
   From: ubuntu:latest
   
   %post
     apt-get update
     apt-get install -y your_dependency_package
   

   • Replace your_dependency_package with the actual package or packages your application depends on.

2. Build the Container:

   • Use the singularity build command to build the container. For example:

   singularity build your_container.sif your_definition_file.def
   

   • This command will create a Singularity Image Format (SIF) file named your_container.sif.

3. Test the Container:

   • Run the container to make sure it works as expected:

   singularity exec your_container.sif your_command
   

   • Replace your_command with the command to execute inside the container.

4. Customize the Container (Optional):

   • You can customize the container further by adding files, environment variables, or configuring the runtime environment. Refer to the Singularity documentation for more options.

5. Share the Container:

   • Once the container is ready, you can share it with others. You can share the SIF file or create a sandbox archive for more portability.

   singularity build --sandbox your_container your_definition_file.def
   

   • This will create a directory named your_container/ containing the container’s filesystem. You can compress and share this directory.

6. Run the Container from the Sandbox:

   • To run the container from the sandbox, use the following command:

   singularity run --sandbox your_container
   

   • This will run the container in the specified sandbox directory.

The Singularity documentation provides more in-depth information and advanced features that you can explore: Singularity Documentation.

Convert Sandbox to Singularity Image File

To convert a Singularity sandbox directory to a Singularity Image File (SIF), you can use the singularity build command with the --sandbox option. Here are the steps:

1. Create a Singularity Sandbox:

   • If you don’t have a Singularity sandbox yet, you can create one using the --sandbox option with the singularity build command:

     singularity build --sandbox your_container your_definition_file.def
     

     Replace your_definition_file.def with your actual Singularity definition file.

2. Convert Sandbox to SIF:

   • Use the singularity build command with the --sandbox option to convert the sandbox to a Singularity Image File (SIF):

     singularity build your_container.sif your_container/
     

     Replace your_container.sif with the desired name for your Singularity Image File and your_container/ with the path to your sandbox directory.

3. Test the SIF:

   • After the conversion, you can test the SIF file by running a command inside the container:

     singularity exec your_container.sif your_command
     

     Replace your_command with the command to execute inside the container.

4. Share or Distribute the SIF:

   • Once you have the SIF file, you can share it with others. The SIF file is a portable format that can be used on systems with Singularity installed.

     You can distribute the SIF file, and others can run it using the singularity run command:

     singularity run your_container.sif
     

That’s it! You’ve successfully converted a Singularity sandbox to a Singularity Image File.