The min_diffusion library

A small, minimal library to load a Stable Diffusion pipeline from the HuggingFace model hub.

This library was put together for a series of experiments on Classifier-free Guidance.

Install

pip install min_diffusion

How to use min_diffusion

The library has a single main class MinimalDiffusion.

This class takes three arguments:

  • model_name
  • device
  • dtype

model_name is the string model name on the HuggingFace hub.
device sets the hardware to run on.
dtype is the torch.dtype precision for the torch modules.

# import the library
from min_diffusion.core import MinimalDiffusion

Loading a sample model

Below is an example to load the openjourney model from PromptHero.

The model will be loaded in torch.float16 precision and placed on the GPU.

# set the model to load and its options
model_name = 'prompthero/openjourney'
device     = 'cuda'
dtype      = torch.float16

Creating a MinimalDiffusion with these arguments:

# create the minimal diffusion pipeline
pipeline = MinimalDiffusion(model_name, device, dtype)

Loading the pipeline:

# load the pipeline
pipeline.load();
Enabling default unet attention slicing.

Generating an image

Below is an example text prompt for image generation.

Note the keyword "mdjrny-v4 style" at the start of the prompt. This is how the openjourney model creates images in the style of Midjourney v4.

# text prompt for image generations
prompt = "mdjrny-v4 style a photograph of an astronaut riding a horse"

Calling MinimalDiffusion on the input text prompt

# generate the image
img = pipeline.generate(prompt);
Using the default Classifier-free Guidance.

Here is the generated image:

# view the output image
img

Notes:

The pipeline assumes you have logged in to the HuggingFace hub.