Function Repository Resource:

CreatePythonVirtualEnvironment

Source Notebook

Deploy a Python virtual environment

Contributed by: Wolfram Staff

ResourceFunction["CreatePythonVirtualEnvironment"]["dir"]

creates a Python virtual environment with a directory name "dir", using the default Python system.

ResourceFunction["CreatePythonVirtualEnvironment"][spec,"dir"]

creates a Python virtual environment using the Python system specified by spec.

Details

Virtual environments are Python's way of dealing with conflicting package requirements. Each virtual environment has its own Python binary and its own independent set of installed packages.
ResourceFunction["CreatePythonVirtualEnvironment"] creates a new Python system, makes it ready for immediate use and returns a UUID, which is a key to the system in the FindExternalEvaluators["Python"] dataset.
If it fails, ResourceFunction["CreatePythonVirtualEnvironment"] returns a Failure object.
In ResourceFunction["CreatePythonVirtualEnvironment"][spec,"dir"], the Python system specification spec is the same as in StartExternalSession.
spec must refer to a registered evaluator or give an explicit path to a Python executable.
The convention for specifying a directory name "dir" is the same as in CreateDirectory; File["dir"] is also supported.
The repository function DeletePythonVirtualEnvironment deletes the Python system created with ResourceFunction["CreatePythonVirtualEnvironment"].

Examples

Basic Examples (5) 

Create a new virtual environment for the specified Python version:

In[1]:=
id = ResourceFunction[
  "CreatePythonVirtualEnvironment"][<|"System" -> "Python", "Version" -> "3.9.5"|>, "experiments"]
Out[1]=

The environment is registered and ready to use:

In[2]:=
FindExternalEvaluators["Python"][id]
Out[2]=

Start a new Python session in the new environment:

In[3]:=
KeyTake[%, {"System", "Target"}] // Normal
Out[3]=
In[4]:=
session = StartExternalSession[%]
Out[4]=

Evaluate code in the session:

In[5]:=
ExternalEvaluate[session, "2+2"]
Out[5]=

End the session and delete the environment:

In[6]:=
DeleteObject[session]
In[7]:=
ResourceFunction["DeletePythonVirtualEnvironment"][id]
Out[8]=

Scope (3) 

Create a new virtual environment using the default Python system:

In[9]:=
id = ResourceFunction["CreatePythonVirtualEnvironment"][
  "default_python"]
Out[9]=
In[10]:=
FindExternalEvaluators["Python"][id]
Out[10]=

Delete the environment:

In[11]:=
ResourceFunction["DeletePythonVirtualEnvironment"][id]
Out[11]=

Create a new virtual environment based on another Python version:

In[12]:=
id = ResourceFunction[
  "CreatePythonVirtualEnvironment"][<|"System" -> "Python", "Version" -> "3.9.1"|>, "python_env_3.9.1"]
Out[12]=
In[13]:=
FindExternalEvaluators["Python"][id]
Out[13]=

Delete the environment:

In[14]:=
ResourceFunction["DeletePythonVirtualEnvironment"][id]
Out[14]=

Create a virtual environment outside the current working directory:

In[15]:=
id = ResourceFunction[
  "CreatePythonVirtualEnvironment"][<|"System" -> "Python", "Version" -> "3.9.5"|>, FileNameJoin[{$UserBaseDirectory, "Applications", "MyExperiments", "venv"}]]
Out[15]=
In[16]:=
FindExternalEvaluators["Python"][id]["Target"]
Out[16]=

Delete the environment:

In[17]:=
ResourceFunction["DeletePythonVirtualEnvironment"][id]
Out[18]=

Applications (7) 

Virtual environments are perfect for experimenting with a new system version or a new version of a complex package with multiple dependencies without affecting other Python systems. Here is a set of packages installed in the default Python instance:

In[19]:=
ResourceFunction["PythonPackageList"][]
Out[19]=

Create a new virtual environment:

In[20]:=
id = ResourceFunction[
  "CreatePythonVirtualEnvironment"][<|"System" -> "Python", "Version" -> "3.9.5"|>, "python_env_experiments"]
Out[20]=

At the outset, the new environment has only the default package set:

In[21]:=
ResourceFunction["PythonPackageList"][id]
Out[21]=

Install a package with multiple dependencies:

In[22]:=
ResourceFunction["PythonPackageInstall"][id, "tst"]
Out[22]=

The environment now has other packages:

In[23]:=
ResourceFunction["PythonPackageList"][id]
Out[23]=

The default system is not affected:

In[24]:=
ResourceFunction["PythonPackageList"][]
Out[24]=

Delete the environment after the work is done:

In[25]:=
ResourceFunction["DeletePythonVirtualEnvironment"][id]
Out[26]=

Properties and Relations (2) 

The UUID returned by CreatePythonVirtualEnvironment is a key in the dataset of external evaluators given by FindExternalEvaluators for the "Python" system:

In[27]:=
id = ResourceFunction[
  "CreatePythonVirtualEnvironment"][<|"System" -> "Python", "Version" -> "3.9.5"|>, "virtual"]
Out[27]=
In[28]:=
FindExternalEvaluators["Python"][id]
Out[28]=

Delete the environment:

In[29]:=
ResourceFunction["DeletePythonVirtualEnvironment"][id]
Out[30]=

CreatePythonVirtualEnvironment automatically creates intermediate directories:

In[31]:=
dir = FileNameJoin[{"MyExperiments", "1", "2", "MyEnvironment"}]
Out[31]=
In[32]:=
id = ResourceFunction[
  "CreatePythonVirtualEnvironment"][<|"System" -> "Python", "Version" -> "3.9.5"|>, dir]
Out[32]=

DeletePythonVirtualEnvironment, on the other hand, deletes only the directory that contains the environment, leaving the (possibly empty) intermediate directories in place:

In[33]:=
ResourceFunction["DeletePythonVirtualEnvironment"][id]
Out[33]=
In[34]:=
FileNames["*", FileNameJoin[{"MyExperiments", "1", "2"}]]
Out[34]=

Delete the directories to clean up:

In[35]:=
DeleteDirectory["MyExperiments", DeleteContents -> True]

Possible Issues (2) 

If the parent Python system is not registered, the system specification in CreatePythonVirtualEnvironment must refer to the executable file explicitly:

In[36]:=
ResourceFunction[
 "CreatePythonVirtualEnvironment"][<|"System" -> "Python", "Version" :> "3.8.2"|>, "foo"]
Out[36]=
In[37]:=
MemberQ[Lookup[Values[FindExternalEvaluators["Python"] // Normal], "Version"], "3.8.2"]
Out[37]=

Specify the target:

In[38]:=
ResourceFunction[
 "CreatePythonVirtualEnvironment"][<|"System" -> "Python", "Target" :> "/Users/admin/.pyenv/versions/3.8.2/bin/python"|>, "foo"]
Out[38]=
In[39]:=
ResourceFunction["DeletePythonVirtualEnvironment"][%]
Out[39]=

Or the executable:

In[40]:=
ResourceFunction[
 "CreatePythonVirtualEnvironment"][<|"System" -> "Python", "Executable" :> "/Users/admin/.pyenv/versions/3.8.2/bin/python"|>, "foo"]
Out[40]=
In[41]:=
ResourceFunction["DeletePythonVirtualEnvironment"][%]
Out[41]=

CreatePythonVirtualEnvironment is not supported with Python 2:

In[42]:=
ResourceFunction[
 "CreatePythonVirtualEnvironment"][<|"System" -> "Python", "Version" -> "2.7.10"|>, "foo"]
Out[42]=

Version History

  • 1.1.1 – 08 February 2022
  • 1.1.0 – 01 June 2021

Related Resources

License Information