Adding multiple LoRa safetensors to my HuggingFace model in Python

Question

Suppose I use this script to load one fine-tuned model: (example taken from https://towardsdatascience.com/hugging-face-diffusers-can-correctly-load-lora-now-a332501342a3)

import torch
from diffusers import StableDiffusionPipeline
text2img_pipe = StableDiffusionPipeline.from_pretrained(
    "stablediffusionapi/deliberate-v2"
    , torch_dtype = torch.float16
    , safety_checker = None
 ).to("cuda:0")

 lora_path = "<path/to/lora.safetensors>" #only one tensor , not folder
 text2img_pipe.load_lora_weights(lora_path)

This adds one safetensors file. How can I load multiple safetensors? I tried the use_safetensors argument when instantiating the StableDiffusionPipeline, but it is unclear where I should put the safetensors folder I have. I have such an error:

OSError: Could not found the necessary safetensors weights in {'vae/diffusion_pytorch_model.safetensors', 'text_encoder/pytorch_model.bin', 'safety_checker/model.safetensors', 'vae/diffusion_pytorch_model.bin', 'text_encoder/model.safetensors', 'unet/diffusion_pytorch_model.bin', 'safety_checker/pytorch_model.bin', 'unet/diffusion_pytorch_model.safetensors'} (variant=None)

I have also tried to load the weights one after the other, but results suggest that I'm not keeping the previous loaded weights.

Answer 1

Currently you can only use 1 LoRA, in the future Diffusers will create a feature to load multiple LoRAs, currently labeled as WIP https://github.com/huggingface/diffusers/issues/2613