T5Tokenizer and T5ForConditionalGeneration producing error in Databricks

1.1k Views Asked by At

I am running my code in a Databricks notebook environment.

Error 1:

To get a tokenizer returned, I am running the following code:

tokenizer = T5Tokenizer.from_pretrained("t5-base")

But I get an error: TypeError: 'NoneType' object is not callable. When I ran this code on Google Colab, I installed the sentencepiece library and restarted the runtime and the error was resolved. I attempted to do the same and then restart the cluster of the Databricks notebook but this error persisted. How can I fix this?

Error 2:

To get a model returned, I am running the following code:

t5_model = T5ForConditionalGeneration.from_pretrained('t5-base')

But I get an error:

ImportError                               Traceback (most recent call last)
<command-3778897587544926> in <module>
      2 from transformers import T5Tokenizer, T5ForConditionalGeneration
      3 import torch
----> 4 t5_model = T5ForConditionalGeneration.from_pretrained('t5-base')

/databricks/python/lib/python3.8/site-packages/transformers/utils/import_utils.py in __getattr__(cls, key)
    786         if key.startswith("_"):
    787             return super().__getattr__(cls, key)
--> 788         requires_backends(cls, cls._backends)
    789 
    790 

/databricks/python/lib/python3.8/site-packages/transformers/utils/import_utils.py in requires_backends(obj, backends)
    774     failed = [msg.format(name) for available, msg in checks if not available()]
    775     if failed:
--> 776         raise ImportError("".join(failed))
    777 
    778 

ImportError: 
T5ForConditionalGeneration requires the PyTorch library but it was not found in your environment. Checkout the instructions on the
installation page: https://pytorch.org/get-started/locally/ and follow the ones that match your environment.

I have installed the torch library using the instructions on the installation page:

pip3 install torch torchvision torchaudio

and imported it as well using import torch. But, this error persists. It ran without any errors in the Colab notebook.

torch: 1.7.1 transformers: 3.3.0 tokenizers: 0.8.1rc2 sentencepiece: 0.1.96

How can I fix these two errors?

0

There are 0 best solutions below