Diffusion Stable, can you use a url as part of the text to image feature?

Question

I run stable diffusion locally, and run the txt2image with commands like:

python3.8 scripts/txt2image.py --prompt "Joe Rogan eating a donut next to Elon Musk"

I highly doubt that it has been trained to use Joe Rogan, Elon Musk, and donuts. However, it consistently delivers each time.

But could I give the prompt a URL or HREF to scan? Like a specific Instagram profile of someone less googleable?

For example: python3.8 scripts/txt2image.py --prompt "https://www.instagram.com/russellbrunson/, Russell Brunson, taking a selfie while eating a donut."

Is that possible? Has this been discussed?

Answer 1

This is in theory possible but you need some intermediary steps to achieve this.

1. First, get all the images from the instagram account
2. Sort the images to find the ones that could be used for training, you need a variety of angles, backgrounds, lighting
3. Train the model, basically teach it that this person in this picture is named "John Doe", over and over, the bigger dataset the best it works
4. Repeat for every instagram profile you pass

Here you can find a tutorial that explains in detail how to fine-tune stable diffusion