---
name: comfyui-1751726182c5
description: Expert ComfyUI node-based generative AI assistance covering Stable Diffusion, Flux, controlnets, LoRAs, and API automation. Use when building modular visual generation workflows or automating image/video pipelines.
---

# ComfyUI

ComfyUI is a modular, node-based graph execution engine and GUI for Stable Diffusion and generative AI pipelines, offering granular control over every step of diffusion workflows.

## When to Use

- **Advanced Generative Image & Video Pipelines**: Designing node-based workflows for SDXL, Flux.1, and SD3.
- **Modular LoRA & ControlNet Composition**: Layering multi-ControlNet conditions, IP-Adapter, and inpainting nodes.
- **Automated Headless Generation**: Exporting workflow JSONs and executing generation via the ComfyUI REST API / WebSockets.
- **Custom Node Development**: Extending PyTorch pipelines with proprietary filters, conditioning, and model loaders.

## Quick Start

```bash
# Clone and install ComfyUI
git clone https://github.com/comfyanonymous/ComfyUI
cd ComfyUI
pip install -r requirements.txt

# Run server with GPU acceleration
python main.py --listen 127.0.0.1 --port 8188
```

## Core Concepts

### ComfyUI API Execution via Python

Triggering generation workflows programmatically via the REST API:

```python
import json
import urllib.request
import urllib.parse

def queue_prompt(prompt_workflow: dict, server_address="127.0.0.1:8188") -> str:
    p = {"prompt": prompt_workflow}
    data = json.dumps(p).encode('utf-8')
    req = urllib.request.Request(f"http://{server_address}/prompt", data=data)
    req.add_header('Content-Type', 'application/json')

    with urllib.request.urlopen(req) as response:
        res = json.loads(response.read())
        return res['prompt_id']

# Minimal workflow dictionary structure
workflow_example = {
    "3": {
        "class_type": "KSampler",
        "inputs": {
            "seed": 42,
            "steps": 25,
            "cfg": 7.0,
            "sampler_name": "euler_ancestral",
            "scheduler": "normal",
            "denoise": 1.0,
            "model": ["4", 0],
            "positive": ["6", 0],
            "negative": ["7", 0],
            "latent_image": ["5", 0]
        }
    }
}
```

### Developing Custom ComfyUI Extension Nodes

Creating a Python custom node module:

```python
import torch

class ImageBrightnessAdjustNode:
    def __init__(self):
        pass

    @classmethod
    def INPUT_TYPES(cls):
        return {
            "required": {
                "image": ("IMAGE",),
                "factor": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 3.0, "step": 0.05}),
            }
        }

    RETURN_TYPES = ("IMAGE",)
    FUNCTION = "adjust_brightness"
    CATEGORY = "ImageProcessing/Color"

    def adjust_brightness(self, image: torch.Tensor, factor: float):
        # image is a Tensor [batch, height, width, channels] in [0, 1]
        adjusted = torch.clamp(image * factor, 0.0, 1.0)
        return (adjusted,)

NODE_CLASS_MAPPINGS = {
    "ImageBrightnessAdjust": ImageBrightnessAdjustNode
}
NODE_DISPLAY_NAME_MAPPINGS = {
    "ImageBrightnessAdjust": "Adjust Image Brightness"
}
```

### Model Checkpoint & LoRA Weight Stacking

Combining base models with style and character LoRAs:

```text
[Load Checkpoint: flux1-dev.sft]
       │ (MODEL, CLIP, VAE)
       ▼
[LoraLoader: style_hyperrealism.safetensors (0.75)]
       │ (MODEL, CLIP)
       ▼
[CLIPTextEncode (Positive Prompt: "cinematic product shot, studio lighting")]
       │ (CONDITIONING)
       ▼
[KSampler] ──► [VAEDecode] ──► [SaveImage]
```

## Common Patterns

### Headless API Workflow Execution via WebSocket

**Problem**: Executing ComfyUI graph generation workflows programmatically from a backend service.

**Solution**:
POST workflow JSON to the `/prompt` endpoint and monitor generation:

```python
import json
import urllib.request

def queue_prompt(prompt_workflow: dict):
    p = {"prompt": prompt_workflow}
    data = json.dumps(p).encode('utf-8')
    req = urllib.request.Request("http://127.0.0.1:8188/prompt", data=data)
    with urllib.request.urlopen(req) as response:
        return json.loads(response.read())

# prompt_workflow is exported from ComfyUI as 'Save (API Format)'
```

## Best Practices

**Do**:

- Save workflows as API-format JSON (`Save (API Format)`) when building automated backend pipelines.
- Use `torch.cuda.empty_cache()` inside custom nodes processing large latent batches.
- Store model weights (`.safetensors`) on high-speed NVMe drives to minimize checkpoint switching latency.
- Utilize fp8 / nf4 quantized weights for Flux.1 and SD3 when running on consumer GPUs (16GB VRAM or less).

**Don't**:

- Use untrusted third-party custom nodes without reviewing their Python scripts for arbitrary remote execution.
- Bake text watermarks into prompts; use negative embeddings and ControlNet masks.
- Execute long render batches synchronously in UI processes; decouple generation with Redis queues.

## Troubleshooting

| Error                                             | Cause                                                   | Solution                                                      |
| :------------------------------------------------ | :------------------------------------------------------ | :------------------------------------------------------------ |
| `Torch CUDA out of memory`                        | Model checkpoint or latent dimension exceeds GPU VRAM.  | Run ComfyUI with `--lowvram` or `--medvram` flag.             |
| `Node not found in graph`                         | Custom node extension not installed in `custom_nodes/`. | Install missing custom node pack via ComfyUI Manager.         |
| `Checkpoint file missing from models/checkpoints` | Model weights file not placed in models directory.      | Move `.safetensors` model weights into `models/checkpoints/`. |

## References

- [ComfyUI GitHub](https://github.com/comfyanonymous/ComfyUI)


---
Source attribution: G1Joshi
Original project: https://github.com/G1Joshi/Agent-Skills/blob/6a74cf03a8a86dc7d993ead3091b41123dbd0181/skills/ai-ml/comfyui/SKILL.md
License: MIT
License information: https://opensource.org/license/mit
Original upstream instructions; downloaded from Mega AI Hub.

## Upstream license notice

MIT License

Copyright (c) 2026 G1Joshi

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.



# Mega AI Hub resource metadata

Creator / publisher: G1Joshi
Original project: https://github.com/G1Joshi/Agent-Skills/blob/6a74cf03a8a86dc7d993ead3091b41123dbd0181/skills/ai-ml/comfyui/SKILL.md
Attribution basis: Source repository record

Fields of work: Content & Media, Design
Roles: Graphic Designer, UX Designer, Video Producer
Specialties: Design
Tools: Stable Diffusion
Country applicability: Country not specified
Classification basis: Source catalog labels
Instruction SHA-256: 7cace74bc23f633d11f564e0dc41ff9b315c91be37dddea2deba35b2ec40f126