Google's flan-t5 models are not loading on HuggingFaceHub through Langchain

Question

I am trying to replicate the example code provided on Langchain website (link here) but I am getting the following error whether I run it on Google colab or locally:

HfHubHTTPError: 504 Server Error: Gateway Time-out for url: https://huggingface.co/api/models/google/flan-t5-xl

The full code from the website is as follows:

!pip install huggingface_hub > /dev/null
# get a token: https://huggingface.co/docs/api-inference/quicktour#get-your-api-token

from getpass import getpass

HUGGINGFACEHUB_API_TOKEN = getpass()
import os
os.environ["HUGGINGFACEHUB_API_TOKEN"] = HUGGINGFACEHUB_API_TOKEN
from langchain import HuggingFaceHub

repo_id = "google/flan-t5-xl" # See https://huggingface.co/models?pipeline_tag=text-generation&sort=downloads for some other options

llm = HuggingFaceHub(repo_id=repo_id, model_kwargs={"temperature":0, "max_length":64})

the last line is where I am getting the error

llm = HuggingFaceHub(repo_id=repo_id, model_kwargs={"temperature":0, "max_length":64})

I tried it on Google Colab as well as local machine and it throws the same error. I tried hitting the URL https://huggingface.co/api/models/google/flan-t5-xl through a browser and got the same error.

Answer 1

Use the following model: "google/flan-t5-xxl"