---
name: video-editing-944e2cdf14a3
description: >-
  Non-destructive video editing in Clips — the editsJson model, trim / split /
  cut / speed / blur, transcript-based editing, and ffmpeg.wasm export. Use
  when building the editor UI, adding a new edit operation, or wiring the
  export pipeline.
---

# Video Editing

## When to use

Reach for this skill any time you modify how recordings are edited: new edit operations, the timeline UI, transcript-driven cuts ("remove the ums"), the preview overlay, or the export flow. Editing **must** be non-destructive — the recording blob is immutable after upload.

## Data model touched

- **`recordings.edits_json`** — a JSON column with the edit document. Shape:

  ```json
  {
    "version": 1,
    "trims": [
      { "startMs": 0, "endMs": 3200, "excluded": true },
      { "startMs": 12000, "endMs": 12000, "excluded": false }
    ],
    "blurs": [],
    "thumbnail": null
  }
  ```

- **`recordings.chapters_json`** — JSON array of `{ startMs, title }`.
- **`application_state.editor-draft`** — in-progress editor state the user is previewing (recording id, playhead, preview playback speed, zoom, and edits JSON). Persisted edit operations still write directly to `edits_json` through actions.

## Rules

1. **Non-destructive.** Never re-encode on edit. The original webm/mp4 stays intact at `recordings.video_url`. Edits only change the JSON.
2. **Single source of truth.** The player renders edits at playback time — read `edits_json`, compute the virtual timeline, and skip excluded ranges via `HTMLVideoElement.currentTime` seeks. Do not fork the edit model for the editor vs the player.
3. **Export is explicit.** The user must click Export to render a new file. MP4 export runs in the browser with ffmpeg.wasm; long recordings get a warning and an option to download the original or continue exporting.
4. **Use edit actions.** Agents must use the recording actions below instead of writing `edits_json` directly.

## Timestamp-based editing actions

Timestamp arguments are integer milliseconds on the original video timeline,
even when earlier edits have shortened playback. Convert clock timestamps to
milliseconds before calling an action; for example, `1:23` is `83000` ms.

| Action | Required arguments | Effect |
| --- | --- | --- |
| `trim-recording` | `recordingId`, `startMs`, `endMs` | Excludes the range from playback. `endMs` must be greater than `startMs`; overlapping or adjacent excluded ranges are merged. |
| `split-recording` | `recordingId`, `atMs` | Adds a zero-width split marker for the editor UI. It does not physically split or rewrite the video and does not change playback. |

## Transcript-based editing

Users love editing by text — click a filler word in the transcript, remove it. The editor uses `recording_transcripts.segments_json` to map words to `{ startMs, endMs }` ranges; clicking a segment creates a `cut` edit for that range.

"Remove the filler words" is **not** solved inline — it delegates to the agent. See the `ai-video-tools` skill. The agent analyzes the transcript, proposes a list of cuts, writes them to `editor-draft` for review, and the user one-click approves.

## Export

The editor toolbar calls `exportMp4()` from `app/lib/ffmpeg-export.ts` in the
browser. It loads ffmpeg.wasm on the first export, processes the source video
and kept ranges, then downloads the resulting MP4. There is no server-side
export queue or automatic fallback for long recordings.

When the edited duration exceeds `LONG_EXPORT_THRESHOLD_MS` (10 minutes), the
toolbar warns that browser export may run out of memory and offers **Download
original** or **Export anyway**. Choosing **Export anyway** runs the same
browser export path. Export errors are surfaced to the user; do not report a
failed export as queued or continuing elsewhere.

## Player integration

The player is the canonical renderer for edits. Given `edits_json`, compute a `VirtualTimeline`:

```ts
interface VirtualTimeline {
  // Maps virtual ms -> source ms, skipping cuts/trims.
  toSource(virtualMs: number): number;
  // Total virtual duration after edits applied.
  durationMs: number;
  // Ranges to skip on playback: the player listens for timeupdate and seeks past them.
  excludedRanges: { startMs: number; endMs: number }[];
}
```

Never call `video.currentTime` with a raw segment index — always go through `toSource`.

## Rules

- The edit UI writes to `editor-draft` on every change, and only writes to `edits_json` on Save (so Cmd+Z is cheap and the DB stays clean).
- Never mutate `edits_json` from `db-exec`; use the recording actions above. Use `clear-edits` to reset edits.
- Speed edits compound (`1.5 × 2 = 3x`) — validate the result is in `[0.25, 4]`.
- Blur coordinates are in **source resolution**, not display pixels. Always normalize against `recordings.width` / `recordings.height`.

## Related skills

- `ai-video-tools` — filler-word removal and chapter generation propose edits via the agent.
- `real-time-sync` — when the agent writes `edits_json`, the player must reflect it; `edits_json` is part of the `recordings` row which is already on the sync list.
- `video-sharing` — exports honor the share's `enableDownloads` flag.
- `storing-data` — why edits live in a JSON column rather than a separate table.

---

## Editor implementation reference

### Concrete `editsJson` shape (editor team)

The editor writes and reads this exact shape. Any new edit operation MUST
preserve this shape so the player, editor, and export pipeline stay in sync.

```ts
interface EditsJson {
  version: 1;
  // Ripple-style trim ranges. `excluded:true` ranges are skipped during
  // playback and collapsed during export. `excluded:false` entries where
  // `startMs === endMs` are SPLIT MARKERS used by the editor UI — they
  // never affect playback.
  trims: Array<{ startMs: number; endMs: number; excluded: boolean }>;
  blurs: Array<{
    id: string;
    startMs: number;
    endMs: number;
    x: number; y: number; w: number; h: number; // normalized 0..1
    intensity: number;
  }>;
  thumbnail?:
    | { kind: "url"; value: string /* absolute URL */ }
    | { kind: "frame"; value: string /* timeMs as string */ }
    | { kind: "gif"; value: string /* JSON: { url, startMs, durationMs } */ }
    | null;
  // Provenance for stitched recordings — set by `stitch-recordings`.
  stitchedFrom?: string[];
}
```

Comment `videoTimestampMs`, reaction `videoTimestampMs`, and
`recording_transcripts.segmentsJson` timestamps all refer to **original**
video time. The player converts to edited time; never assume the two are
equal.

### Timestamp helpers

`app/lib/timestamp-mapping.ts` is the single source of truth:

| Helper                                 | Returns                                                                 |
| -------------------------------------- | ----------------------------------------------------------------------- |
| `parseEdits(raw)`                      | A fully-populated `EditsJson` from the DB column (empty on bad input)   |
| `serializeEdits(edits)`                | Stringified JSON ready to write back                                    |
| `getExcludedRanges(edits)`             | Sorted, non-overlapping excluded ranges                                 |
| `getKeptRanges(durationMs, edits)`     | Ordered "kept" ranges in original time — the export pipeline iterates   |
| `originalToEdited(ms, edits)`          | Maps original → edited (playback) timeline                              |
| `editedToOriginal(ms, edits)`          | Maps edited → original (used when seeking the underlying `<video>`)     |
| `effectiveDuration(durationMs, edits)` | Edited duration after excluded ranges are removed                       |
| `isExcluded(ms, edits)`                | True if an original ms falls inside an excluded range                   |
| `mergeExcluded(edits, startMs, endMs)` | Append an excluded range; merges overlapping/adjacent entries           |
| `popLastExcluded(edits)`               | Remove the most-recently-added excluded range (used by `undo-edit`)     |
| `appendSplit(edits, atMs)`             | Append a zero-width split marker                                        |
| `formatMs(ms)`                         | `0:42`, `1:23:04` formatting for timestamps                             |

The player team may have its own copy of some of these. If so, consolidate on
`app/lib/timestamp-mapping.ts` — never let the two drift.

### Editor actions

| Action               | Writes                                   | Purpose                                           |
| -------------------- | ---------------------------------------- | ------------------------------------------------- |
| `set-thumbnail`      | `thumbnailUrl` / `animatedThumbnailUrl` / `editsJson.thumbnail` | Three modes: upload / frame / gif |
| `set-chapters`       | `chaptersJson`                           | Overwrites the chapter array                      |
| `stitch-recordings`  | new `recordings` row                     | Client-side ffmpeg concat + upload + insert       |
| `undo-edit`          | `editsJson.trims`                        | Pop the last excluded range (no redo)             |
| `clear-edits`        | `editsJson`                              | Reset to defaults (chapters/thumbnailUrl kept)    |

Every mutation ends with `writeAppState("refresh-signal", { ts: Date.now() })`.

Chapter writes go through `actions/lib/save-chapters.ts`, which only writes
over the list it was checked against: `set-chapters` needs `expectedChapters`
(as `get-recording-player-data` returns them) or `expectedVersion`, and always
needs `expectedMediaUpdatedAt` from that same response. Pass `expectedCuts`
when times were mapped through cuts. `update-recording --chaptersJson` needs
`expectedChapters` and `expectedMediaUpdatedAt`. A new writer of
`chaptersJson` must check both values, or match the row it read in its update's
WHERE, as `apply-rewind-extension` does. Stored chapters that can't be read
refuse every save (`chapters_unreadable`); `set-chapters --discardUnreadable`
replaces them, and only when the user asked to.

### ffmpeg.wasm usage

`app/lib/ffmpeg-export.ts` lazy-loads `@ffmpeg/ffmpeg` (only fetched on first
Export click; core wasm is ~30MB). Three entry points:

```ts
exportMp4(recording, edits, onProgress)  // kept-range concat → H.264+AAC MP4
exportGif(recording, startMs, durationMs, onProgress)  // animated thumbnail
exportConcat(sources, onProgress)  // stitching N recordings into one MP4
```

**Assumed limits (tested in practice — update if you hit new ceilings):**

- Single-threaded WASM, ~2GB memory ceiling per tab.
- Roughly 10 minutes of 1080p WebM→MP4 is the practical upper bound before
  tabs run out of memory. The editor surfaces a confirm dialog when
  `effectiveDuration(...) > LONG_EXPORT_THRESHOLD_MS` (10 min) and offers
  "Download original" as an escape hatch.
- Both the trim/concat paths and the `exportConcat` (for stitching) re-encode
  to H.264+AAC so the output plays everywhere.

### Stitching

Decision: **client-side ffmpeg concat** rather than a virtual playlist that
plays N sources sequentially. Rationale:

- A real combined MP4 plays in share links, embeds, iframes, and any mobile
  browser without special-case player logic.
- The virtual-playlist approach couples the player to the stitched structure
  and breaks sharing / thumbnails / comments (which are keyed off
  `recording_id`).

Flow from the UI:

1. `stitch-manager.tsx` collects the ordered list of source recordings.
2. `exportConcat()` fetches each `videoUrl` and concatenates with ffmpeg.wasm.
3. We upload the resulting blob via `uploadFile()` (or fall back to a data
   URL when no provider is configured — dev mode).
4. We call `stitch-recordings` with the uploaded URL + total duration; the
   action inserts a new `recordings` row with `status: "ready"` and
   `editsJson.stitchedFrom` set to the source IDs for provenance.

### Waveform peak caching

`computePeaks()` decodes the video's audio track via the Web Audio API and
downsamples to a 2000-point peaks array. Peaks are cached in
`application_state` under `waveform-<recordingId>` so remounts don't
recompute. The editor reads the cache first and falls back to computing only
when the key is missing or corrupted.

### Timeline filmstrip

The trim track shows video frames behind the waveform. Two paths, and the order
matters:

1. **Sprite (preferred).** `generate-filmstrip` runs one ffmpeg pass
   (`fps` → `scale` → `pad` → `tile`) into a single JPEG grid, uploads it, and
   stores the URL plus grid geometry on the recording. The editor renders cells
   with CSS `background-position` — one cached image request, no video decoding
   in the browser.
2. **Browser extraction (fallback).** `extractFilmstripThumbnails()` seeks a
   detached `<video>` once per frame. Only used when there is no sprite, which
   means hosts without ffmpeg and local/dev media the server cannot fetch.

Three traps this design exists to avoid:

- **Pass the proxied URL.** Frame extraction must use `getWaveformMediaUrl()`,
  never `recording.videoUrl` — reading pixels back out of a cross-origin video
  taints the canvas and `toDataURL` throws, so provider media silently yields no
  filmstrip at all.
- **Both paths sample cell midpoints,** not `0 … duration` endpoints. The strip
  renders N equal cells, so cell `i` must show the middle of the slot it
  occupies or every thumbnail sits up to half a cell from the time beneath it.
- **Cell count comes from the geometry,** `trackWidth / (height × aspect)`, not
  from the sprite's frame count. Rendering all 40 frames across an unzoomed
  track makes each cell portrait, and `object-cover` then shows a narrow centre
  slice of each frame instead of a recognisable thumbnail.

The filmstrip never replaces the waveform — peaks still draw on top of a scrim,
because frames say nothing about the audio.

### Keyboard shortcuts (editor scope)

- **Space** — play / pause (even while focused in the editor area)
- **Cmd/Ctrl+Z** — undo the last trim (no redo stack)
- **I / O** — mark in / out of the current trim selection
- **Hold R and drag on the timeline** — select an exact range; use Cut
  Selection or Delete to remove it. A normal click still selects and seeks to a
  section, and dragging a section edge still changes that edge.
- **S** — split at playhead (via `split-button.tsx`)
- **Delete / Backspace** — while the transcript editor has a selection: trim
  the selected text's timestamp range

### File map

```
actions/
  trim-recording.ts   split-recording.ts   set-thumbnail.ts
  set-chapters.ts     stitch-recordings.ts
  undo-edit.ts        clear-edits.ts
app/lib/
  timestamp-mapping.ts   waveform-peaks.ts   ffmpeg-export.ts
app/components/editor/
  editor-layout.tsx       editor-toolbar.tsx
  waveform.tsx            trim-handles.tsx      timeline.tsx
  transcript-editor.tsx   chapters-editor.tsx
  thumbnail-picker.tsx    stitch-manager.tsx    split-button.tsx
```

When the recording route enters edit mode, render `<EditorLayout recordingId={id} />` — it wires the toolbar, video preview, transcript editor, waveform, trim handles, timeline, and chapters sidebar. The toolbar includes a Loom-style preview-speed dropdown next to the playhead time; it changes `video.playbackRate` for trimming/review only and writes `playbackSpeed` into `editor-draft` so the agent can see how the user is previewing. The dialogs for thumbnail picking and stitching are mounted inside the layout and toggled by the toolbar.


---
Source attribution: BuilderIO
Original project: https://github.com/BuilderIO/agent-native/blob/29ecbd7c72ab69d8128b679f086cbf42f5b5b41f/templates/clips/.agents/skills/video-editing/SKILL.md
License: MIT
License information: https://opensource.org/license/mit
Original upstream instructions; downloaded from Mega AI Hub.

## Upstream license notice

MIT License

Copyright (c) 2026 Builder.io, Inc.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.



# Mega AI Hub resource metadata

Creator / publisher: BuilderIO
Original project: https://github.com/BuilderIO/agent-native/blob/29ecbd7c72ab69d8128b679f086cbf42f5b5b41f/templates/clips/.agents/skills/video-editing/SKILL.md
Attribution basis: Source repository record

Fields of work: Content & Media, Marketing
Roles: Video Producer, Content Marketer, Social Media Manager
Specialties: Specialty not specified
Tools: FFmpeg, Clips
Country applicability: Country not specified
Classification basis: Source category, role and specialization labels
Instruction SHA-256: 000c25a91e173758e35b08a1ca2f42be2280cc71abd254290728ecb1e69b97f4