--- name: media-svd description: > Open-source AI video generation with STRICTLY permissive-license models: LTX-Video (Apache 2.0, Lightricks realtime-ish text-to-video), CogVideoX-2B / -5B (Apache 2.0, Tsinghua + Zhipu, 2024), Mochi-1 (Apache 2.0, Genmo), AnimateDiff (Apache 2.0, motion module that animates any diffusion image model), Wan-Video (Apache 2.0, Alibaba 2025 high quality). Text-to-video (t2v), image-to-video (i2v), motion-controlled animation via AnimateDiff, ComfyUI workflow integration. Use when the user asks to generate a video from a text prompt, animate a still image, make a short AI clip, run CogVideoX, use LTX-Video, use Mochi, use Wan-Video, use AnimateDiff, or specifically asks for Apache-licensed / OSI-open / commercial-safe video generation models. Does NOT include Stable Video Diffusion (SVD OpenRAIL-M NC), Sora, Runway, Kling — all proprietary or non-commercial. HunyuanVideo is mentioned but flagged for its commercial cap. argument-hint: "[prompt] [output]" --- # Media SVD (open-source video generation) **Context:** $ARGUMENTS ## Quick start - **Text-to-video:** `scripts/svd.py t2v --model ltx --prompt "..." --duration 5 --fps 24 --out out.mp4` → Step 3. - **Image-to-video:** `scripts/svd.py i2v --model cogvideox --init-image in.png --prompt "..." --duration 5 --out out.mp4` → Step 3. - **AnimateDiff image-model motion:** `scripts/svd.py animate --workflow animatediff.json --out out.mp4` → Step 4. - **Install a model's pip deps:** `scripts/svd.py install ltx` → Step 2. - **Fetch weights (confirm required):** `scripts/svd.py download --model ltx --yes` → Step 2. ## When to use - Generate a short video clip from a text prompt using permissively licensed open-source models only. - Animate an existing still image into motion via image-to-video (i2v). - Use AnimateDiff to add motion to any `media-sd` image model workflow. - Need commercial clearance: all default models are Apache 2.0. - Do NOT use for Stable Video Diffusion (SVD), Sora, Runway Gen-3, Kling — they are either non-commercial or proprietary APIs. HunyuanVideo has a commercial user-size cap (Tencent License) — mention but do not default to. ## Step 1 — Pick a model | Model | License | Type | Typical duration | VRAM (FP16) | Notes | |--------------------|------------|--------------|------------------|------------------|-------| | **LTX-Video** | Apache 2.0 | t2v + i2v | 5 s (up to ~17 s)| 24 GB (12 GB nf4)| Near-realtime on H100; fastest Apache model | | **CogVideoX-5B** | Apache 2.0 | t2v + i2v | 6 s @ 8 fps | 24 GB | Best quality in the 5B class; 2B variant fits 12 GB | | **CogVideoX-2B** | Apache 2.0 | t2v | 6 s | 12 GB | Smaller; good for consumer cards | | **Mochi-1** | Apache 2.0 | t2v | 5.4 s @ 30 fps | 60 GB+ (multi-GPU)| SOTA quality, heavy. Use 8-bit / tensor parallel | | **AnimateDiff** | Apache 2.0 | motion module| 2–4 s | same as image model | Plugs into a `media-sd` image pipeline | | **Wan-Video 2.1** | Apache 2.0 | t2v + i2v | 5 s | 24 GB | Alibaba 2025; strong prompt adherence | | HunyuanVideo | Tencent L. | t2v | 5 s | 60 GB+ | Commercial cap — flag before use; NOT default | Read `references/models.md` for per-model prompt style, sampler/scheduler pairs, and published VBench / human-eval scores. Check environment: ```bash uv run .claude/skills/media-svd/scripts/svd.py check ``` ## Step 2 — Install and download ### Diffusers pipeline (pure Python, no ComfyUI) ```bash uv run .../svd.py install ltx # → pip install diffusers transformers accelerate torch imageio imageio-ffmpeg uv run .../svd.py download --model ltx --yes # → huggingface_hub snapshot_download into HF cache ``` ### ComfyUI (for workflows with AnimateDiff / LoRAs) See the `media-sd` skill for ComfyUI install. Then download video-model nodes via ComfyUI-Manager: - `ComfyUI-LTXVideo` - `ComfyUI-CogVideoXWrapper` - `ComfyUI-MochiWrapper` - `ComfyUI-AnimateDiff-Evolved` - `ComfyUI-WanVideoWrapper` ## Step 3 — Generate ### Text-to-video (t2v) ```bash uv run .../svd.py t2v \ --model ltx \ --prompt "a cat jumping off a sofa, cinematic, 35mm film grain" \ --duration 5 --fps 24 \ --width 768 --height 512 \ --out cat.mp4 --seed 42 ``` LTX-Video wants resolution as a multiple of 32. CogVideoX is fixed to 720x480. The script rounds/validates per-model. ### Image-to-video (i2v) ```bash uv run .../svd.py i2v \ --model cogvideox \ --init-image first_frame.png \ --prompt "the wind blows and the flag starts waving" \ --duration 6 --fps 8 \ --out waving.mp4 ``` The init-image becomes the first frame. Keep the prompt consistent with the image — conflicting prompts ("dog" when the image is a cat) produce artifacted morphs. ### AnimateDiff via ComfyUI ```bash # ComfyUI running on 8188, workflow loaded with AnimateDiff nodes uv run .../svd.py animate \ --workflow animatediff_sana.json \ --out anim.mp4 --server http://127.0.0.1:8188 ``` AnimateDiff adds temporal attention to a frozen image UNet. Inside the ComfyUI workflow, wire `AnimateDiffLoaderGen1` between `ModelLoader` and `KSampler`. Use Apache 2.0 image backbones only (Sana, PixArt-Sigma — see `media-sd`). ## Step 4 — Post-process and mux Typical follow-ups: - Upscale 512p → 1080p: pass to `media-upscale` (Real-ESRGAN). - Interpolate 8 fps → 24/60 fps: pass to `media-interpolate` (RIFE / FILM). - Add audio: `ffmpeg -i video.mp4 -i music.mp3 -c:v copy -c:a aac -shortest final.mp4` (or use `ffmpeg-transcode`). - Concat multiple clips: `ffmpeg-cut-concat`. ## Gotchas - **Stable Video Diffusion (SVD) is NOT in this skill.** Its OpenRAIL-M license is non-commercial for most tiers. If a user says "SVD" they probably mean that model — tell them to use LTX-Video or CogVideoX instead. - **Apache 2.0 is not a free pass on content.** Most Apache models still include an acceptable-use / ethics statement in the model card. These are not license terms (because Apache 2.0 doesn't allow them) but are practical guidelines. Read the HF model card. - **HunyuanVideo has a commercial cap** (same Tencent License 2.0 as HunyuanDiT). Not our default; mention only when the user explicitly asks. - **CogVideoX resolution is fixed**: 720x480 for 2B/5B t2v, 480x720 for 5B i2v. Passing other sizes errors or silently resizes. - **LTX-Video resolution must be a multiple of 32**, duration rounded to multiples of 1/24 s. - **Mochi-1 is 60+ GB VRAM** — effectively multi-GPU only. There's an 8-bit community quant (`genmoai/mochi-1-preview` with `bitsandbytes`) that fits in 40 GB. - **fps control is trickier than it looks.** CogVideoX outputs 8 fps native; upsample later with `media-interpolate`. LTX-Video does 24 fps native. AnimateDiff defaults to 8 fps. - **Diffusers video pipelines return lists of PIL frames**, not an ffmpeg-friendly file. The script writes frames → ffmpeg -i "frame_%04d.png" internally using `imageio_ffmpeg`. - **Prompt length**: CogVideoX uses T5 (up to 226 tokens), LTX uses T5 (120 tokens), Mochi uses T5 (256 tokens). Longer prompts get truncated silently. - **Negative prompts** are not always supported (LTX-Video ignores them). For Mochi and CogVideoX, negative prompts HELP avoid blur/chroma-smearing. - **VRAM paging via `enable_sequential_cpu_offload()`** halves speed but halves VRAM. Use when OOM. - **Seeds drift across diffusers versions** — pin `diffusers>=0.32` for reproducibility. - **AnimateDiff 1.5 vs SDXL** — AnimateDiff-SDXL weights exist but are experimental; the original is trained on SD 1.5 frames and needs an SD-1.5-compatible backbone. We don't expose SD 1.5 in media-sd (CreativeML OpenRAIL-M excluded). Use AnimateDiff-Lightning on a Sana or PixArt backbone, or skip AnimateDiff entirely. ## Examples ### Example 1: LTX-Video cat clip, 5 s @ 24 fps ```bash uv run .../svd.py t2v --model ltx \ --prompt "a ginger cat leaps from a sofa to a bookshelf, slow motion" \ --duration 5 --fps 24 --width 768 --height 512 \ --out cat.mp4 --seed 7 ``` ### Example 2: CogVideoX-5B i2v from a still ```bash uv run .../svd.py i2v --model cogvideox-5b \ --init-image portrait.png \ --prompt "she smiles and turns her head slightly" \ --duration 6 --fps 8 --out smile.mp4 ``` ### Example 3: Low-VRAM CogVideoX-2B on a 12 GB card ```bash uv run .../svd.py t2v --model cogvideox-2b \ --prompt "neon Tokyo street at night, rain, reflections" \ --duration 6 --fps 8 --cpu-offload \ --out tokyo.mp4 ``` ### Example 4: AnimateDiff on a Sana image backbone ```bash # 1. Pre-author the ComfyUI workflow with AnimateDiffLoaderGen1 + Sana backbone # 2. Run it: uv run .../svd.py animate \ --workflow sana_animatediff.json --out anim.mp4 \ --server http://127.0.0.1:8188 ``` ### Example 5: Wan-Video high quality ```bash uv run .../svd.py t2v --model wan \ --prompt "a hot-air balloon drifts over snow-capped mountains at sunrise" \ --duration 5 --fps 24 --width 832 --height 480 \ --out balloon.mp4 ``` ## Troubleshooting ### Error: `ModuleNotFoundError: No module named 'diffusers'` Cause: diffusers stack not installed. Solution: `uv run .../svd.py install ` or `pip install diffusers transformers accelerate torch imageio imageio-ffmpeg`. ### Error: `CUDA out of memory` on Mochi-1 Cause: 60+ GB model doesn't fit. Solution: switch to LTX-Video or CogVideoX-2B, or run Mochi under `bitsandbytes` 8-bit quant with `--cpu-offload` (~40 GB). True FP16 Mochi needs multi-GPU tensor parallel. ### Output is a static image / no motion Cause: i2v with `--strength 0.0` or mismatched prompt, or forgot `--duration`. Solution: set `--duration >= 2`, ensure prompt describes motion ("she walks", "camera pans"), raise strength if using i2v. ### CogVideoX output is blurry Cause: T5 prompt too short, or negative prompt missing. Solution: write a 30–60 word prompt describing subject + motion + camera + style. Add `--negative-prompt "blurry, low quality, distorted"`. ### Frames appear doubled / juddery at 24 fps Cause: native model fps is 8; `--fps 24` upsampled by frame repeat. Solution: do not oversell native fps. Render at native, then pass to `media-interpolate` (RIFE / FILM) for smooth 24/60 fps. ### ComfyUI workflow fails: "AnimateDiffLoaderGen1 missing" Cause: ComfyUI-AnimateDiff-Evolved custom node not installed. Solution: in ComfyUI-Manager, install "AnimateDiff Evolved" by Kosinkadink. Or `git clone https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved custom_nodes/ComfyUI-AnimateDiff-Evolved`. --- Source attribution: damionrashford Original project: https://github.com/damionrashford/media-os/blob/5014c1e8b23fd3e18d49926d9aa147d15a3aa08e/skills/media-svd/SKILL.md License: MIT License information: https://opensource.org/license/mit Original upstream instructions; downloaded from Mega AI Hub. ## Upstream license notice MIT License Copyright (c) 2026 Media OS Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ## Additional upstream notices media-svd/references/LICENSES.md # media-svd — strict license policy Only OSI-open / clearly commercial-safe video weights. ## INCLUDE | Model | License | Commercial? | |--------------------|-------------|-------------| | LTX-Video | Apache 2.0 | Yes | | CogVideoX-2B / -5B | Apache 2.0 | Yes | | Mochi-1 | Apache 2.0 | Yes | | Wan-Video 2.1 | Apache 2.0 | Yes | | AnimateDiff | Apache 2.0 | Yes | ## EXCLUDE | Model | License | Why excluded | |-------------------------------|----------------------------|--------------| | Stable Video Diffusion (SVD) | SVD Community / OpenRAIL-M | NC for most tiers; use restrictions | | Stable Video 1.1 / XT | Stability Community | Not commercial-safe without license | | Zeroscope v2 XL | CreativeML OpenRAIL-M | Use restrictions; not OSI-open | | ModelScope t2v-ms | CC BY-NC 4.0 | Non-commercial | | Sora | Proprietary (OpenAI) | Closed API | | Runway Gen-2 / Gen-3 | Proprietary (Runway) | Closed API | | Kling 1.x / 2.x | Proprietary (Kuaishou) | Closed API | | Luma Dream Machine | Proprietary (Luma) | Closed API | | HunyuanVideo | Tencent License 2.0 | Commercial cap; not default. Mention only. | ## Why HunyuanVideo isn't a default Tencent License 2.0 permits commercial use BUT imposes a monthly-active-user cap (~100M MAU). Practical for most companies but not OSI-open. We do not expose it in the t2v subcommand; if a user explicitly asks for Hunyuan we let them know about the cap and redirect to LTX / CogVideoX. ## Why SVD is excluded despite being called "SVD" Stable Video Diffusion's "Community License" (and older OpenRAIL-M variants) restricts commercial use to low revenue tiers and imposes content-type restrictions. The skill name `media-svd` is retained as a category label (AI video generation); the underlying models are NOT Stable Video Diffusion. ## Verification Check the HF repo's LICENSE file before relying on commercial use: ```bash curl -sL https://huggingface.co//raw/main/LICENSE curl -sL https://huggingface.co//raw/main/README.md | head -50 ``` # Mega AI Hub resource metadata Creator / publisher: damionrashford Original project: https://github.com/damionrashford/media-os/blob/5014c1e8b23fd3e18d49926d9aa147d15a3aa08e/skills/media-svd/SKILL.md Attribution basis: Source repository record Fields of work: Content & Media, Design, Marketing Roles: Content Marketer, Graphic Designer, Influencer Marketer, UX Designer, Video Producer Specialties: Specialty not specified Tools: Comfyui, Huggingface_Hub Country applicability: Country not specified Classification basis: Source catalog labels Instruction SHA-256: 7b7bd62af803633f50db92e1c1cd4e9fb03d574f3f73ef9d6eebbd424a182426