TypeError in Python 3.11 when Using BasicModelRunner from llama-cpp-python

Question

I'm currently taking the DeepAI's Finetuning Coursera course and encountered a bug while trying to run one of their demonstrations locally in a Jupyter notebook.

Environment:

• Python version: 3.11
• Required packages:
  • notebook
  • lamini
  • llama-cpp-python (> 0.1.53)

Issue:

When attempting to run the following code:

import os
import lamini

lamini.api_url = os.getenv("POWERML__PRODUCTION__URL")
lamini.api_key = os.getenv("POWERML__PRODUCTION__KEY")

from llama import BasicModelRunner

non_finetuned = BasicModelRunner("meta-llama/Llama-2-7b-hf")

non_finetuned_output = non_finetuned("Tell me how to train my dog to sit")

I receive a TypeError at the last line:

TypeError: can only concatenate str (not "NoneType") to str

This error occurs when I try to pass a string to the non_finetuned BasicModelRunner object.

Answer 1

The environment variables POWERML__PRODUCTION__URL and POWERML__PRODUCTION__KEY were not properly set.