Using Conda/Mamba on Berzelius

• Introduction
• Conda/Mamba on Berzelius
  • Conda
  • Mamba
• Conda/Mamba Environment Relocation

Introduction

Conda is an open-source package management and environment management system primarily used for Python. It allows you to create isolated environments with specific sets of packages and dependencies to avoid conflicts between different projects.

The concepts of Conda, Miniconda, Anaconda, and Mamba are all related to package management and environment management in Python. Let's explore each concept:

• Conda is the core package and environment management tool.
• Miniconda is a minimal Conda installer, allowing you to create custom environments.
• Anaconda is a comprehensive distribution of Conda, focused on data science and scientific computing.
• Mamba is an alternative package manager that can be used as a replacement for the default Conda package manager, providing improved performance.
• Mambaforge is a distribution of Mamba that provides a pre-packaged and ready-to-use environment for Conda users.

The default channel for Conda is anaconda, and Mamba installs packages by default from the channel conda-forge.

We recommend to use Mamba for its improved performance.

Conda/Mamba on Berzelius

Conda

The Anaconda installation is available on Berzelius via the module system, e.g. Anaconda/2021.05-nsc1.

A basic example for creating a Conda environment called myenv with Python 3.8 with the pandas and seaborn packages:

module load Anaconda/2021.05-nsc1
conda create -n myenv python=3.8 pandas seaborn
conda activate myenv

Please see the conda cheatsheet for the most important commands about using Conda.

Mamba

The “solving environment” stage is where Anaconda tries to figure out the best way to install the packages you've requested, considering the other packages you have installed and their dependencies. This process can sometimes take a long time due to complex dependencies, outdated package versions, or network issues. It is highly recommended to use Mamba as an alternative to Conda.

Mamba is a reimplementation of the Conda package manager in C++ offering the following benefits:

• much faster dependency solving
• parallel downloading of repository data and package files

Berzelius makes mamba available via the Mambaforge miniforge distribution. For example, to set up a customized environment pytorch_latest, install PyTorch and run a Python script python_script.py in it:

module load Mambaforge/23.3.1-1-hpc1-bdist
mamba create -n pytorch_latest 
mamba activate pytorch_latest
mamba install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia
python python_script.py

Conda/Mamba Environment Relocation

The default location for Conda/Mamba environments is ~/.conda in your home directory. This location can be problematic since these environments can become very large. Therefore, it is suggested to redirect this directory using a symbolic link to your project directory. Please replace your_proj and username in the example command with your actual project name and username.

mv ~/.conda /proj/your_proj/users/username
ln -s /proj/your_proj/users/username/.conda ~/.conda