In order to fully understand what enroot is, it is essential to understand containers. A container is a lightweight, portable software package that encapsulates an application and all its dependencies, allowing it to run consistently across different computing environments.
Containers share the host system's operating system kernel but are isolated from each other, ensuring that applications inside them do not interfere with one another. This isolation and consistency make containers ideal for deploying applications in diverse environments, from development to production.
Containers technologies include Docker but also Enroot.
Unlike virtual environments (conda or python’s virtual env), which isolate only the Python dependencies within the same operating system, containers provide full system-level isolation, packaging an entire application along with its runtime, libraries, and system tools.
Using enroot containers in an HPC environment like ours, allows users to have more flexibilities on the features they want to add to their workspace by even accessing inside the container (not on the actual system) as root and apt install desired packages. Of course, you are still able to use your preferred virtual env manager within your container.
In order to built a container you need to perform the following actions:
Being an isolated entity, the container has his own filesystem. This is handful for rapid and isolated testing but not ideal if you want to use your data inside it. In order to see the folders you need to use inside the container you need to mount them. This will allow you not only to work with your data but also to save all the computation you do with them. You can imagine this as a sort of tunnel that allow you to evade the isolation provided by the container. If you do not use this tunnel, the data that you create in a folder only inside the container filesystem will not be saved.
Enroot is a tool which turn traditional container/OS images into unprivileged sandboxes. In traditional container usually the users needed to be root in order to run and use them, which raised security issues. That’s why rootless container emerged and enroot is one of the tools out there which allows you to have achieve this.
Interestingly for our purposes, Enroot has a built-in support for GPU usage.
In a SLURM cluster Enroot management and execution is handled by another useful plugin which is called pyxis. This will handle most of the enroot command for you. However, you would still have to specify how to handle them via specific commands.
Enroot provide facilities to import well known container image format (i.e. Docker). This makes our system compatible with images you can on the most common platform such as docker hub or nvidia ngc.
example: to have a container running ubuntu and with PyTorch and a specific version of CUDA installed? you could download one of these images
In addition to this we also provide some base images with some defaults already installed, you can find that one in: /storage/DSH/personal/base_pytorch.sqsh from any gpu server of the cluster.
base_pytorch.sqsh is the actual image that enroot uses to build the container.