Stable Diffusion

Stable Diffusion is a community skill for generating and manipulating images using the Stable Diffusion model family, covering text-to-image generation, image-to-image transformation, inpainting, prompt engineering, and pipeline configuration.

What Is This?

Overview

Stable Diffusion provides patterns for working with latent diffusion models for image generation tasks. It covers pipeline setup using the diffusers library, prompt crafting for desired visual outputs, negative prompt usage, image-to-image transformation with strength control, inpainting masked regions, scheduler selection, and LoRA adapter loading. The skill enables developers to integrate AI image generation into applications using locally hosted models.

Who Should Use This

This skill serves developers integrating image generation into applications, creative professionals automating visual content production workflows, and researchers experimenting with diffusion model capabilities and configurations.

Why Use It?

Problems It Solves

Cloud image generation APIs have per-image costs that compound at scale. Hosted services impose content restrictions that may not match application requirements. Batch processing large numbers of images through APIs is slow due to rate limits. Customizing generation with specific styles requires fine-tuned models that may not be available through cloud providers.

Core Highlights

Local model hosting eliminates per-image API costs and provides full control over generation parameters. Prompt engineering techniques produce more consistent and targeted visual outputs. LoRA adapter support enables style customization without full model fine-tuning. Multiple pipeline types handle text-to-image, image-to-image, and inpainting from the same model checkpoint.

How to Use It?

Basic Usage

from dataclasses import dataclass, field

@dataclass
class GenerationConfig:
    prompt: str
    negative_prompt: str = ""
    width: int = 512
    height: int = 512
    steps: int = 30
    guidance_scale: float = 7.5
    seed: int = -1
    scheduler: str = "euler_a"

class ImageGenerator:
    def __init__(self, model_id: str):
        self.model_id = model_id
        self.lora_adapters: list[str] = []

    def load_lora(self, adapter_path: str, weight: float = 1.0):
        self.lora_adapters.append(
            f"{adapter_path}:{weight}")

    def build_pipeline_args(self, config: GenerationConfig) -> dict:
        args = {
            "prompt": config.prompt,
            "negative_prompt": config.negative_prompt,
            "width": config.width,
            "height": config.height,
            "num_inference_steps": config.steps,
            "guidance_scale": config.guidance_scale,
        }
        if config.seed >= 0:
            args["generator_seed"] = config.seed
        return args

    def generate_batch(self, configs: list[GenerationConfig]) -> list[dict]:
        results = []
        for config in configs:
            args = self.build_pipeline_args(config)
            results.append({"prompt": config.prompt,
                           "args": args, "status": "generated"})
        return results

Real-World Examples

from dataclasses import dataclass

@dataclass
class Img2ImgConfig:
    source_image_path: str
    prompt: str
    strength: float = 0.75
    guidance_scale: float = 7.5
    steps: int = 30

@dataclass
class InpaintConfig:
    source_image_path: str
    mask_image_path: str
    prompt: str
    guidance_scale: float = 7.5
    steps: int = 30

class AdvancedPipeline:
    def __init__(self, model_id: str):
        self.model_id = model_id

    def img2img(self, config: Img2ImgConfig) -> dict:
        return {
            "type": "img2img",
            "source": config.source_image_path,
            "prompt": config.prompt,
            "strength": config.strength,
            "steps": config.steps
        }

    def inpaint(self, config: InpaintConfig) -> dict:
        return {
            "type": "inpaint",
            "source": config.source_image_path,
            "mask": config.mask_image_path,
            "prompt": config.prompt,
            "steps": config.steps
        }

    def prompt_enhance(self, base_prompt: str,
                       style: str = "") -> str:
        quality = "masterpiece, best quality, highly detailed"
        enhanced = f"{base_prompt}, {quality}"
        if style:
            enhanced = f"{enhanced}, {style}"
        return enhanced

Advanced Tips

Use classifier-free guidance scales between 7 and 12 for the best balance of prompt adherence and image quality. Stack multiple LoRA adapters with reduced individual weights to combine style influences. Select schedulers based on the quality-speed tradeoff: Euler Ancestral for variety, DPM++ 2M Karras for quality.

When to Use It?

Use Cases

Generate product visualization images from text descriptions for e-commerce listings. Build a batch image processing pipeline that applies consistent style transformations to a set of source images. Create an inpainting tool that removes or replaces objects in photographs using masked regions.

Related Topics

Diffusers library usage, LoRA adapter training, image prompt engineering, latent diffusion architecture, and ComfyUI workflow design.

Important Notes

Requirements

A GPU with sufficient VRAM for the target model, typically 6 GB or more. The diffusers Python package for pipeline management. Model checkpoint files in the safetensors or diffusers format.

Usage Recommendations

Do: use fixed seeds for reproducible outputs when iterating on prompt refinements. Include negative prompts that exclude common quality issues such as blurry, distorted, and low quality. Test different schedulers to find the best match for the target visual style.

Don't: use maximum guidance scale values that produce oversaturated and artifact-heavy images. Skip the negative prompt which helps steer generation away from undesirable outputs. Generate images at resolutions that differ significantly from the model training resolution without proper upscaling.

Limitations

Generation quality depends heavily on prompt crafting skill and model selection. Fine details like text rendering and precise hand anatomy remain challenging for current models. VRAM requirements limit the maximum resolution achievable on consumer hardware.