Setting Up ComfyUI for Local Image Generation (Complete Guide)

Why ComfyUI?

When Midjourney charges $10-60/month and DALL-E charges per image, running your own image generator sounds expensive. It’s not. A used $150 GPU generates unlimited images, forever, for free.

ComfyUI is the most powerful local image generation interface. It’s node-based — you wire together models, prompts, and post-processing into a visual workflow. It looks intimidating, but once you understand the basics, it’s more flexible than any commercial tool.

ComfyUI vs Alternatives

We use ComfyUI because it’s the most powerful. Once you build a workflow, you can reproduce it endlessly, share it as a JSON file, and automate it via API.

What You Need

Minimum

• GPU with 6GB+ VRAM (8GB recommended)
• 10GB free disk space (for models)
• Python 3.10+
• Git

Recommended

• GPU with 8GB+ VRAM (RTX 3060 Ti or better)
• 20GB+ free disk space (multiple models add up)
• SSD for model storage (loading 4GB models from HDD is slow)

Installation

Linux (Ubuntu/Debian)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

# Clone ComfyUI
git clone https://github.com/comfyanonymous/ComfyUI
cd ComfyUI

# Create virtual environment
python3 -m venv venv
source venv/bin/activate

# Install PyTorch with CUDA support
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124

# Install requirements
pip install -r requirements.txt

# Launch
python main.py

Open http://localhost:8188 in your browser.

Windows

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

# Clone ComfyUI
git clone https://github.com/comfyanonymous/ComfyUI
cd ComfyUI

# Create virtual environment
python -m venv venv
.\venv\Scripts\activate

# Install PyTorch with CUDA
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124

# Install requirements
pip install -r requirements.txt

# Launch
python main.py

Portable Windows Package (Easier)

1. Go to github.com/comfyanonymous/ComfyUI/releases
2. Download the Windows portable package (.7z)
3. Extract and run run_nvidia_gpu.bat
4. Open http://localhost:8188

Downloading Your First Model

ComfyUI needs a model (checkpoint) to generate images. The most popular starting point:

SDXL Base Model

1
2

cd ComfyUI/models/checkpoints/
wget https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors

This is a 6.9GB file. It’s the base SDXL model — good for general image generation.

Better Models for Photorealism

SDXL is good, but the community has fine-tuned it for better results:

Download models to ComfyUI/models/checkpoints/. ComfyUI detects them automatically.

Model Safety Note

Only download models from HuggingFace or CivitAI. Random downloads from unverified sources can contain corrupted weights or malware. Stick to verified, community-rated models.

Your First Image

1. Open http://localhost:8188
2. You’ll see the default workflow (a basic txt2img pipeline)
3. If the workflow is empty, click Load Default in the menu
4. Select your model in the Load Checkpoint node
5. Enter your prompt in the CLIP Text Encode (Prompt) node:

   1	a majestic mountain landscape at sunset, golden hour lighting, ultra detailed, 8k

6. Enter a negative prompt in the negative node:

   1	blurry, low quality, distorted, watermark, text

7. Click Queue Prompt

Your first image generates in 10-30 seconds depending on your GPU. It saves to ComfyUI/output/.

Understanding the Workflow

ComfyUI uses nodes connected by wires. The default workflow has these nodes:

1
2
3
4
5
6
7

Load Checkpoint → CLIP Text Encode (Positive) → ┐
                → CLIP Text Encode (Negative) → ┤
                → VAE Decode ← KSampler ← Empty Latent Image
                                                   ↑
                              (Positive + Negative prompts feed into KSampler)
                                                   
VAE Decode → Save Image

Flow: Model loads → prompts encode → noise sampled → image decoded → saved

You don’t need to understand all of this to use it. The default workflow works. As you learn, you’ll add nodes for upsampling, LoRA loading, and post-processing.

Adding LoRAs (Fine-Tuned Models)

LoRAs are small model additions (50-500MB) that modify the base model’s output — specific art styles, characters, or concepts.

1
2
3

# Download a LoRA to the LoRA folder
cd ComfyUI/models/loras/
# Download from CivitAI or HuggingFace

In ComfyUI:

1. Right-click → Add Node → loaders → Load LoRA
2. Wire it between your Load Checkpoint and CLIP Text Encode nodes
3. Select your LoRA file
4. Set strength (0.7-1.0 is typical)

Optimizing for Low VRAM (6-8GB)

If you’re running on a budget GPU:

Enable VRAM Management

1
2
3

# Launch with low VRAM flags
python main.py --lowvram  # for 6GB cards
python main.py --fp8_e4m3fn  # for 8GB cards with SDXL

Use SD 1.5 Instead of SDXL

SD 1.5 models are 2GB instead of 6.9GB. They generate at 512x512 instead of 1024x1024, but they’re much lighter:

1
2

cd ComfyUI/models/checkpoints/
wget https://huggingface.co/runwayml/stable-diffusion-v1-5/resolve/main/v1-5-pruned-emaonly.safetensors

Generate Smaller, Upscale Later

1. Generate at 512x512 or 768x768
2. Add an Upscale node (Latent Upscale or Image Upscale)
3. Run a second pass with higher resolution

This uses less VRAM than generating at 1024x1024 directly.

Using the API

ComfyUI has a REST API — you can trigger image generation from scripts, n8n, or any application:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

import requests
import json

# Load your workflow JSON
with open('workflow.json') as f:
    workflow = json.load(f)

# Set your prompt
workflow['6']['inputs']['text'] = 'a cat sitting on a windowsill, golden hour'

# Queue the prompt
response = requests.post(
    'http://localhost:8188/prompt',
    json={'prompt': workflow}
)

print(response.json())

This is how we automate image generation in our n8n workflows — the AI writes a prompt, sends it to ComfyUI’s API, and the image lands in a folder ready for use.

Troubleshooting

“CUDA out of memory”

• Use --lowvram flag
• Use a smaller model (SD 1.5 instead of SDXL)
• Reduce resolution (512x512 instead of 1024x1024)
• Close other GPU applications (browser, games)

Model not showing up

• Ensure model is in ComfyUI/models/checkpoints/
• Check file extension (.safetensors or .ckpt)
• Refresh the model list in ComfyUI (right-click node → Refresh)

Slow generation

• Verify GPU is being used (check nvidia-smi during generation)
• SDXL on 8GB VRAM: 15-30 seconds per image is normal
• SD 1.5 on 8GB VRAM: 3-8 seconds per image

Black or garbage images

• Your positive and negative prompts might be swapped
• LoRA strength might be too high (try 0.5)
• Model might be corrupted — re-download

Next Steps

• Self-Hosted Voice AI: Kokoro TTS Setup — Add voice to your stack
• Automate Everything: n8n + Ollama — Wire it all together
• Best GPUs for Local LLMs — Upgrade your hardware

ComfyUI is free and open source. Support the developer on GitHub.