Azure Machine Learning notebooks: ModuleNotFound error

Question

I'm working through a Python exercise using Azure Machine Learning notebooks. I'm unable to import torch even after !pip install torch.

Notebook says Requirement already satisfied, then errors out with:

!pip install torch

import torch

data = torch.tensor(encode(text), dtype=torch.long)
print(data.shape, data.dtype)
print(data[:100])

4 sec

ModuleNotFoundError: No module named 'torch'
Requirement already satisfied: torch in /anaconda/envs/azureml_py38/lib/python3.8/site-packages (1.12.0)
Requirement already satisfied: typing-extensions in /anaconda/envs/azureml_py38/lib/python3.8/site-packages (from torch) (4.4.0)
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
Input In [22], in <cell line: 4>()
      1 # Encode the entire dataset and store it into a torch.Tensor
      3 get_ipython().system('pip install torch')
----> 4 import torch
      5 data = torch.tensor(encode(text), dtype=torch.long)
      6 print(data.shape, data.dtype)

ModuleNotFoundError: No module named 'torch'

I opened up a terminal in Azure ML Studio and tried pip install torch too, same Requirement already satisfied message showed.

How do I get torch (and any other Python modules where this occurs) working in AML notebooks?

Answer 1

I've found creating environments and installing packages through the terminal to be a much more reliable experience than doing it from an AML notebook.

I suggest using one of the provided terminals (either the one available in the compute instance's details, or the one available in JupyterLab) to create a new conda environment which you can customize to your liking.

Something like:

conda create -n my_tutorial python=3.10

conda activate my_tutorial

pip install --user ipykernel

python -m ipykernel install --user --name=my_tutorial

# Do a complete install of PyTorch, take a look at the available versions here https://pytorch.org/get-started/previous-versions/
conda install pytorch=1.13 torchvision=0.14 torchaudio=0.13 pytorch-cuda=11.7 -c pytorch -c nvidia -y

Afterwards, just make sure your notebooks use the my_tutorial kernel and you should be good to go. Whenever you want to pip install something new, just go back to the terminal, activate your kernel, and install the thing, then it should be available in your notebooks as well.

Answer 2

The issue was lined out in the docs here. Don't use !pip within the notebook. Instead use %pip.