--- name: media-upscale description: > AI super-resolution for images and video beyond ffmpeg's outdated built-in DNN models (SRCNN 2014 / ESPCN 2016 / EDSR 2017). Open-source + commercial-safe only: Real-ESRGAN (BSD-3-Clause, the current default for photos/video), Real-CUGAN (MIT, anime-optimized), SwinIR (Apache 2.0, transformer-based), HAT (Apache 2.0, 2023 SOTA), GFPGAN (Apache 2.0, face restoration), waifu2x-ncnn-vulkan (MIT, classic anime), Upscayl (AGPL-3.0, GUI wrapper around Real-ESRGAN), chaiNNer (GPL-3.0, node pipeline for arbitrary models). Realistic 2x / 4x / 8x SR for photos, film scans, low-res archives, anime, screenshots, game textures. Use when the user asks to upscale an image, super-resolution a video, enhance an old photo, restore a face, upscale anime, rescale game textures, do AI upscaling, or when ffmpeg's built-in SRCNN/ESPCN/EDSR isn't good enough. argument-hint: "[model] [input] [output]" --- # Media Upscale **Context:** $ARGUMENTS ## Quick start - **Photo 2x/4x → upscaled PNG:** Real-ESRGAN → Step 2 recipe A. - **Anime / cel-shaded art:** Real-CUGAN → Step 2 recipe B. - **Restore a face in a blurry photo:** GFPGAN → Step 2 recipe E. - **Upscale a whole video:** Real-ESRGAN frame pipeline → Step 3. - **Batch a folder of images:** `scripts/upscale.py batch` → Step 4. ## When to use - User says "upscale", "super-res", "enhance", "4K-ify", "make it sharper", "AI upscale". - Source is lower resolution than the target medium (SD → HD, HD → 4K, 480p anime → 1080p). - ffmpeg's `dnn_processing` (SRCNN/ESPCN/EDSR) or a plain `scale=...:flags=lanczos` looks soft / blurry on real content. - Face-only restoration of a scanned photo or low-res portrait. - Do NOT use this skill for mere resizing without quality gain — use `ffmpeg-video-filter` (`scale`) instead. - Do NOT use for temporal frame interpolation — that is `media-interpolate`. ## Step 1 — Pick a model | Model | License | Best for | GPU needed? | Notes | |-----------------------|-----------------|-----------------------------------|-------------|---------------------------------------------| | Real-ESRGAN | BSD-3-Clause | Photos, video frames, real-world | Optional | `realesrgan-ncnn-vulkan` — default. 2x/4x. | | Real-CUGAN | MIT | Anime, illustration, cel-art | Optional | `realcugan-ncnn-vulkan` — 2x/3x/4x, denoise | | SwinIR | Apache 2.0 | Fine-texture photo SR | Yes (CUDA) | Transformer, Python inference, slower | | HAT | Apache 2.0 | SOTA photo SR (2023) | Yes (CUDA) | Hybrid Attention Transformer, Python | | GFPGAN | Apache 2.0 | Face restoration in photos | Yes (CUDA) | Face-only; combines with Real-ESRGAN bg | | waifu2x-ncnn-vulkan | MIT | Classic anime, small images | Optional | Original 2015-era anime upscaler | | Upscayl | AGPL-3.0 | GUI for Real-ESRGAN | Optional | Desktop app; script skill stays CLI | | chaiNNer | GPL-3.0 | Node-graph pipeline, ANY SR model | Optional | Run arbitrary PyTorch/ONNX/NCNN SR models | **Decision rules:** 1. Real photos / video frames → **Real-ESRGAN** (`realesr-animevideov3` for cartoons, `realesrgan-x4plus` for photos, `realesrgan-x4plus-anime` for anime stills). 2. Anime or illustration → **Real-CUGAN** (sharper cel edges than Real-ESRGAN), fall back to **waifu2x-ncnn-vulkan** for very small sources. 3. Faces are the focal point (portrait, ID card, old family photo) → **GFPGAN** (then blend with a Real-ESRGAN-upscaled background). 4. Need academic SOTA quality, don't mind Python/CUDA → **HAT** or **SwinIR**. 5. Multi-model pipeline (segment → upscale → denoise → sharpen chain) → **chaiNNer**. **Do NOT use:** CodeFormer (S-Lab License, research-only — not commercial-safe), Topaz Video AI (commercial, paid). This skill covers open-source + commercial-safe only. ## Step 2 — Install the chosen binary / package `scripts/upscale.py install ` prints the exact package-manager command for your platform. It does **not** auto-install. Examples: ```bash # Real-ESRGAN (ncnn-vulkan prebuilt — fastest install, zero Python) # macOS: brew install realesrgan-ncnn-vulkan # if tap available, else download release # or grab a release zip from: https://github.com/xinntao/Real-ESRGAN/releases # Real-CUGAN (ncnn-vulkan) # Download: https://github.com/nihui/realcugan-ncnn-vulkan/releases # waifu2x-ncnn-vulkan brew install waifu2x # macOS; tap: nihui/waifu2x-ncnn-vulkan # GFPGAN (Python) uv pip install gfpgan basicsr realesrgan facexlib # SwinIR / HAT (Python + CUDA) uv pip install torch opencv-python numpy basicsr git clone https://github.com/JingyunLiang/SwinIR # weights via model zoo git clone https://github.com/XPixelGroup/HAT # weights via releases page # chaiNNer # Download from: https://chainner.app/ ``` ### Recipe A — Real-ESRGAN 4x on a photo ```bash realesrgan-ncnn-vulkan \ -i photo.jpg -o photo_4x.png \ -n realesrgan-x4plus -s 4 -f png # Available -n models: realesrgan-x4plus, realesrnet-x4plus, realesr-animevideov3, # realesrgan-x4plus-anime # -s scale: 2, 3, 4 # -t tile: 0 auto, or e.g. 256 to limit VRAM. Set -g for multi-GPU. # -f format: png | jpg | webp ``` Through the driver script: ```bash uv run ${CLAUDE_SKILL_DIR}/scripts/upscale.py image \ --model realesr --scale 4 --in photo.jpg --out photo_4x.png ``` ### Recipe B — Real-CUGAN on anime ```bash realcugan-ncnn-vulkan \ -i anime.png -o anime_4x.png \ -n 0 -s 4 -f png # -n 0 default, -n 3 for heavy denoise, -n -1 conservative; -s 2|3|4. ``` ```bash uv run ${CLAUDE_SKILL_DIR}/scripts/upscale.py image \ --model realcugan --scale 4 --in anime.png --out anime_4x.png ``` ### Recipe C — SwinIR (Python) ```bash python SwinIR/main_test_swinir.py \ --task real_sr --scale 4 --large_model \ --model_path SwinIR/model_zoo/swinir/003_realSR_BSRGAN_DFO_s64w8_SwinIR-L_x4_GAN.pth \ --folder_lq /path/to/input_dir --tile 400 ``` ### Recipe D — HAT (Python) ```bash python HAT/hat/test.py -opt HAT/options/test/HAT_SRx4.yml ``` ### Recipe E — GFPGAN face restoration + Real-ESRGAN background ```bash python GFPGAN/inference_gfpgan.py \ -i inputs/ -o results/ \ -v 1.4 -s 2 --bg_upsampler realesrgan # -v model version: 1.2, 1.3, 1.4; -s output scale factor. # --bg_upsampler realesrgan re-uses Real-ESRGAN for non-face pixels. ``` Driver: ```bash uv run ${CLAUDE_SKILL_DIR}/scripts/upscale.py face-restore \ --model gfpgan --in portrait.jpg --out portrait_restored.png --scale 2 ``` ## Step 3 — Upscale a video (frame pipeline) AI SR models are frame-based. Video workflow: 1. **Extract frames** with ffmpeg (lossless PNGs). 2. **Upscale each frame** with Real-ESRGAN / Real-CUGAN. 3. **Remux** the upscaled frame sequence with the original audio. ```bash uv run ${CLAUDE_SKILL_DIR}/scripts/upscale.py video \ --model realesr --scale 4 --in clip.mp4 --out clip_4x.mp4 \ --model-name realesr-animevideov3 ``` What the script does: ```bash # (1) frames out ffmpeg -i clip.mp4 -qscale:v 1 -qmin 1 -qmax 1 -vsync 0 frames/%08d.png # (2) upscale frames realesrgan-ncnn-vulkan -i frames -o frames_4x -n realesr-animevideov3 -s 4 -f png # (3) remux with original audio, pick an FPS matching the source ffmpeg -framerate -i frames_4x/%08d.png -i clip.mp4 \ -map 0:v -map 1:a? -c:v libx264 -preset slow -crf 17 -pix_fmt yuv420p \ -c:a copy -movflags +faststart clip_4x.mp4 ``` **VRAM control:** add `-t 256` (tile size) to the Real-ESRGAN step if GPU VRAM is tight; 0 = auto. **Native video model choice:** `realesr-animevideov3` is tuned for video and runs faster than the still-image models on frame sequences. ## Step 4 — Batch a folder ```bash uv run ${CLAUDE_SKILL_DIR}/scripts/upscale.py batch \ --model realesr --scale 4 --in-dir photos/ --out-dir photos_4x/ ``` Real-ESRGAN ncnn-vulkan natively accepts `-i -o ` and processes every file in parallel. The wrapper adds logging, dry-run, and skips already-processed outputs. ## Gotchas - **ffmpeg built-in SR is outdated.** `dnn_processing=model=espcn.pb` / EDSR / SRCNN are 2014–2017 research models. On real content they are not competitive with Real-ESRGAN (2021+). Recommend ffmpeg built-ins only as a tiny fallback. - **`-n` model name, not path.** `realesrgan-ncnn-vulkan -n realesrgan-x4plus` expects the model file to live under the binary's `models/` directory (shipped in the release zip). If you get `failed to load model`, pass `-m ` to point at your models. - **Scale factor must match the chosen model.** `realesrgan-x4plus` is 4x-only. Don't pass `-s 2` to a 4x model and expect 2x output — you'll get a downsample of 4x output. - **yuv420p even dimensions.** After a 3x upscale of an odd-width video, libx264 will complain. Add `-vf "scale=trunc(iw/2)*2:trunc(ih/2)*2"` on the remux. - **VRAM.** Real-ESRGAN on a 4K frame wants ~6GB at 4x. Use `-t 256` (tile size) to process 256×256 tiles and stitch — slower but fits in 2GB. - **Face restoration bakes in features** that weren't in the original. GFPGAN will hallucinate plausible detail; don't use it on ID photos you need legally faithful. For cinema/commercial work, only use it when the client has signed off on "AI reconstruction." - **CodeFormer is NOT in this skill.** CodeFormer is S-Lab License research-only. Do not recommend it even though many tutorials online still do. - **Topaz Video AI is NOT in this skill.** It is commercial paid software; outside the open-source + commercial-safe scope. - **RMBG v1.4 is NOT in this skill.** Different project (background removal) — see `media-matte`. Also: v1.4 was NC; only v2.0 (Apache 2.0) is commercial-safe. - **Upscayl is AGPL-3.0.** If you ship code derived from Upscayl's source in a commercial closed product you must release source. Running the bundled app to produce outputs is fine. - **chaiNNer is GPL-3.0.** Same caveat as Upscayl: running it to produce outputs is fine, linking its code into a closed product is not. - **Video frame remux needs the source FPS exactly.** Pull it from `ffprobe` first — don't assume 23.976 / 25 / 29.97. - **8-bit → 10-bit matters.** Upscaling an 8-bit source doesn't add dynamic range. If the source is 10-bit HDR, keep the frame dumps in a 16-bit PNG pipeline or use EXR to preserve the bit depth. ## Examples ### Example 1: 4x an old family photo ```bash uv run ${CLAUDE_SKILL_DIR}/scripts/upscale.py image \ --model realesr --scale 4 --in grandma_1970.jpg --out grandma_4x.png ``` ### Example 2: Upscale a 480p anime clip to 1080p ```bash # 480 * 2.25 ≈ 1080 → use Real-CUGAN 2x, then pad to 1080p if needed. uv run ${CLAUDE_SKILL_DIR}/scripts/upscale.py video \ --model realcugan --scale 2 --in ep01_480p.mp4 --out ep01_960p.mp4 ffmpeg -i ep01_960p.mp4 -vf "scale=-2:1080:flags=lanczos" -c:v libx264 -crf 18 ep01_1080p.mp4 ``` ### Example 3: Restore a face in a scan, keep background ```bash uv run ${CLAUDE_SKILL_DIR}/scripts/upscale.py face-restore \ --model gfpgan --in scan.jpg --out scan_restored.png --scale 2 ``` ### Example 4: Batch 4x a folder of 1000 thumbnails ```bash uv run ${CLAUDE_SKILL_DIR}/scripts/upscale.py batch \ --model realesr --scale 4 --in-dir thumbs/ --out-dir thumbs_4x/ ``` ### Example 5: Check which upscale binaries are on PATH ```bash uv run ${CLAUDE_SKILL_DIR}/scripts/upscale.py check ``` ## Troubleshooting ### Error: `realesrgan-ncnn-vulkan: command not found` Cause: binary not installed. Solution: Download from https://github.com/xinntao/Real-ESRGAN/releases and unzip. Or `brew install realesrgan-ncnn-vulkan`. Or run `uv run .../upscale.py install realesr`. ### Error: `failed to load model: realesrgan-x4plus` Cause: model `.param` / `.bin` files not next to the binary (expected at `/models/`). Solution: The release zip bundles them — always use the full zip. Or pass `-m /path/to/models`. ### Output is blocky / has tile seams Cause: `-t` tile size too small relative to content. Solution: Increase tile (`-t 512` or `-t 0` for auto). Or run without tiling on a GPU with more VRAM. ### HAT / SwinIR inference: `CUDA out of memory` Cause: tile size too large. Solution: Reduce `--tile 400` → `--tile 200`. HAT's test.py reads `tile` from the YML. ### Upscaled video stutters / is desynced Cause: reassembled at wrong FPS, or VFR source. Solution: `ffprobe -select_streams v -show_entries stream=r_frame_rate,avg_frame_rate clip.mp4` — pass that exact rational to `-framerate`. For VFR sources, first normalize with `ffmpeg -vsync cfr`. ### GFPGAN "RuntimeError: CUDA error" on CPU machine Cause: GFPGAN needs a GPU by default. Solution: `python inference_gfpgan.py --device cpu ...` (slow but works). ## Reference docs - Per-model URLs, input res ranges, GPU/CPU support, download commands → `references/models.md`. - Full license table for every model bundled → `references/LICENSES.md`. --- Source attribution: damionrashford Original project: https://github.com/damionrashford/media-os/blob/5014c1e8b23fd3e18d49926d9aa147d15a3aa08e/skills/media-upscale/SKILL.md License: MIT License information: https://opensource.org/license/mit Original upstream instructions; downloaded from Mega AI Hub. ## Upstream license notice MIT License Copyright (c) 2026 Media OS Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ## Additional upstream notices media-upscale/references/LICENSES.md # Licenses for bundled upscale models Every model this skill recommends is open-source AND commercial-safe. Research-only (CodeFormer) and commercial paid (Topaz) tools are explicitly NOT recommended here. | Model | License | Commercial use? | Closed-source-product use? | Upstream | |------------------------|------------------|-----------------|----------------------------|---------------------------------------------------------| | Real-ESRGAN | BSD-3-Clause | Yes | Yes | https://github.com/xinntao/Real-ESRGAN | | realesrgan-ncnn-vulkan | BSD-3-Clause + MIT (ncnn wrapper) | Yes | Yes | https://github.com/xinntao/Real-ESRGAN | | Real-CUGAN | MIT | Yes | Yes | https://github.com/bilibili/ailab/tree/main/Real-CUGAN | | realcugan-ncnn-vulkan | MIT | Yes | Yes | https://github.com/nihui/realcugan-ncnn-vulkan | | SwinIR | Apache-2.0 | Yes | Yes | https://github.com/JingyunLiang/SwinIR | | HAT | Apache-2.0 | Yes | Yes | https://github.com/XPixelGroup/HAT | | GFPGAN | Apache-2.0 | Yes | Yes | https://github.com/TencentARC/GFPGAN | | waifu2x-ncnn-vulkan | MIT | Yes | Yes | https://github.com/nihui/waifu2x-ncnn-vulkan | | Upscayl | AGPL-3.0 | Yes* | No (AGPL copyleft) | https://github.com/upscayl/upscayl | | chaiNNer | GPL-3.0 | Yes* | No (GPL copyleft) | https://github.com/chaiNNer-org/chaiNNer | \* Using Upscayl / chaiNNer *as an app* to generate outputs is fine. Linking their code (or substantially derived code) into your own closed-source product triggers the copyleft obligation to release source under the same license. ## NOT recommended by this skill | Tool | License | Why excluded | |-------------------|----------------------|---------------------------------------------| | CodeFormer | S-Lab License 1.0 | Non-commercial / research-only | | Topaz Video AI | Commercial (paid) | Proprietary; not open-source | ## Model weights vs. code - Real-ESRGAN, Real-CUGAN, SwinIR, HAT, GFPGAN, waifu2x — weights share the same license as the code repository. BSD / MIT / Apache-2.0 — all commercial-safe. - Always re-check the specific release's `LICENSE` file before shipping in a product, in case a future release changes terms. ## Boilerplate attribution (commercial-safe use) Below is minimal attribution for a product that ships outputs generated with these models. Paste into your product's NOTICES / third-party licenses page: This product uses AI super-resolution models: Real-ESRGAN (BSD-3-Clause) https://github.com/xinntao/Real-ESRGAN Real-CUGAN (MIT) https://github.com/bilibili/ailab/tree/main/Real-CUGAN SwinIR (Apache-2.0) https://github.com/JingyunLiang/SwinIR HAT (Apache-2.0) https://github.com/XPixelGroup/HAT GFPGAN (Apache-2.0) https://github.com/TencentARC/GFPGAN waifu2x (MIT) https://github.com/nihui/waifu2x-ncnn-vulkan BSD-3-Clause / Apache-2.0 require keeping the copyright and license text with any distributed binary or source. MIT requires retaining the copyright line. # Mega AI Hub resource metadata Creator / publisher: damionrashford Original project: https://github.com/damionrashford/media-os/blob/5014c1e8b23fd3e18d49926d9aa147d15a3aa08e/skills/media-upscale/SKILL.md Attribution basis: Source repository record Fields of work: Content & Media, Design, Marketing Roles: Content Marketer, Graphic Designer, Product Designer, UX Designer, Video Producer Specialties: API Tools: FFmpeg, Chainner, Gfpgan, Real Cugan, Realesrgan, Swinir, Upscayl, Waifu2X Ncnn Vulkan Country applicability: Country not specified Classification basis: Source catalog labels Instruction SHA-256: ccc3805d4169424b8047c214f3f155d210e25840b23207df78a1f585d569e650