Diffusion models

with Hugging Face Pipelines

Jan Kirenz

🧨 Diffusers

Setup

from diffusers import StableDiffusionPipeline

Basics

  • diffusers: A Python library maintained at 🤗

  • Providing open and responsible access to pre-trained diffusion models.

  • Democratizing the ecosystem of diffusion models by making them easy to use.

Text-to-image with 🧨 diffusers

Note that there are differences between Mac silicon and Windows.

Diffusers (Mac silicon)

model_id = "runwayml/stable-diffusion-v1-5"

pipeline = StableDiffusionPipeline.from_pretrained(model_id)

pipeline = pipeline.to('mps')

# Recommended if your computer has < 64 GB of RAM
pipeline.enable_attention_slicing()

image = pipeline(
    "An astronaut floating through space while plying the guitar").images[0]

image.save("../images/diffusion_astronaut.png")

Output

Diffusers on Windows and Mac Intel

model_id = "runwayml/stable-diffusion-v1-5"

pipeline = StableDiffusionPipeline.from_pretrained(model_id)

pipeline = pipeline.to("cuda")

image = pipeline("An astronaut riding a tiger").images[0]

image.save("image.png")

Resources

Yaramesh, A. (2022). How DALL·E 2 Works. Blog post.

Xiao, Z., Kreis, K. & Arash, A. (2022). Tackling the Generative Learning Trilemma with Denoising Diffusion GANs. International Conference on Learning Representations (ICLR)

What’s next?

Congratulations! You have completed this tutorial 👍

Next, you may want to go back to the lab’s website

Jan Kirenz