We need an `offload_dir` to dispatch this model according to this `device_map`

287 Views Asked by At

anyone know how to fix this error, heres the python code it keeps giving me the error on.

model = AutoPeftModelForCausalLM.from_pretrained(output_dir, device_map="auto", torch_dtype=torch.bfloat16)

This image is the rror in full. enter image description here

I'm trying to run llama 2 on google colab, more specifically I'm trying to fine tune the model and run it. I'm new to this so I have no idea how you would go about solving this.

0

There are 0 best solutions below