Categories
Software Engineering

Isolating Conda Virtual Envs

An amazing shot of the beautiful Butchart Gardens in Brentwood Bay, British Columbia

Rationale

For users and fans of Anaconda, conda provides a useful alternative to traditional virtualenv functionality in Python. However. there are some twists you need to be aware of in order to use it effectively.

One of the features that have tripped me up is the fact that the virtual environment inherits the interpreter, binaries and libraries from the base environment, unless specified otherwise.

Thus if we have the following setup for Python:

$HOME/local/anaconda3/bin/python
$HOME/local/anaconda3/bin/pip
$HOME/local/anaconda3/lib/python3.11/site-packages

for interpreter, pip and modules respectively, then running

conda create -n newenv

will result in this

environment location: /Users/myusername/local/anaconda3/envs/newenv
...

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate newenv
#
# To deactivate an active environment, use
#
#     $ conda deactivate


If we then run conda activate newenv

and then do

 % which python
/Users/myusername/local/anaconda3/bin/python
% which pip
/Users/myusername/local/anaconda3/bin/pip

The result of this is that if we then do

% pip install module

Then module will get installed into

$HOME/local/anaconda3/lib/python3.11/site-packages

rather than

$HOME/local/anaconda3/envs/newenv/lib/python3.11/site-packages

Resolution

To prevent this from happening, always specify a python interpreter when you create a new conda environment :

% conda create -n newenv python=3.11

% conda activate newenv

Next we check to see whether the interpreter is separate from that of the base environment;

% which python
/Users/myusername/local/anaconda3/envs/newenv/bin/python

% which pip
/Users/myusername/local/anaconda3/envs/newenv/bin/pip

Thus when we do a pip install on a module, the module will be installed in the envs/newenv location and true isolation will have been achieved.

References: