Embedded Video Pip Smooth Playback
Documentsembedded-video-pip-smooth-playbackEmbedded Video Pip Smooth Playback is an Agent Skill for documents published in the inclusionAI/AWorld repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.
Prevent stutter and frozen frames when embedding a child video inside a parent in code-driven pipelines (Remotion, After Effects scripting, FFmpeg filter graphs). Explains why sparse keyframes break frame-accurate seek during per-frame export, and how re-encoding with H.264 all-intra GOP (-g 1) and yuv420p makes every frame independently decodable. Includes FFmpeg command, parameter notes, file-size tradeoffs, and a reusable rule for any seek-heavy programmatic video workflow.
How to use this skill
Bring this guide into your coding agent with a prompt tailored to the tool you use.
- Open your project in Codex.
- Copy and paste the prompt below.
- Review files and risks before approving any action.
Inspect the Agent Skill at https://github.com/inclusionAI/AWorld/blob/HEAD/aworld-skills/embedded_video/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/embedded-video-pip-smooth-playback, but wait for my approval before writing files or running scripts.
Copying this prompt does not install or run the skill. Review third-party files before use.
1. Problem scenario
When you build videos with code-driven renderers (e.g. Remotion, AE scripts, complex FFmpeg filter graphs), you often need picture-in-picture: one main composition with another video embedded inside it.
Typical symptom: In the exported file, motion on the main layer (translation, scale, etc.) looks smooth, but the embedded clip stutters badly, drops frames, or even freezes for long stretches.
2. Root cause: sparse keyframes
Modern codecs (H.264/H.265) save space by storing full pictures only at scene cuts or every few seconds (keyframes / I-frames). Frames in between (P-frames / B-frames) only store differences from neighbors.
Engines like Remotion export frame by frame. To render frame N, the embedded clip must seek to the matching timestamp.
If the embedded file has almost no keyframes (e.g. one I-frame at the start of a 10 s clip), the decoder often has to decode from frame 0 forward to reach frame N. That leads to:
- Very slow seeks: Decoding takes so long that the renderer times out and grabs a frame before the decode finishes.
- Repeated frames: The decoder cannot keep up, so several consecutive captures show the same old image—stutter in the final output.
3. Fix: all-intra encoding (every frame a keyframe)
Idea: Re-encode the embedded asset so every frame is a keyframe. Then any seek returns a full picture immediately, with no long chains of dependent frames.
Steps
Step 1: Re-encode with FFmpeg
Run:
ffmpeg -i input.mp4 -c:v libx264 -g 1 -pix_fmt yuv420p output_keyframes.mp4
Parameters:
| Flag | Meaning |
|---|---|
-i input.mp4 | Source clip you embed. |
-c:v libx264 | H.264 for broad compatibility with web and renderers. |
-g 1 | Critical: GOP size 1 → one keyframe per frame. |
-pix_fmt yuv420p | Common 8-bit 4:2:0 layout for players and pipelines. |
output_keyframes.mp4 | Output used as the fixed asset. |
(All-intra files are often several times larger than the original. That is usually fine for an intermediate asset used only during rendering.)
Step 2: Point your project at the new file
Replace paths so the composition uses output_keyframes.mp4 instead of the old input.mp4.
Step 3: Re-render
Export again; embedded playback should track smoothly with the main timeline.
4. Rules of thumb
Rule: Any asset that must be seeked frame-accurately from code should be pre-converted to all-intra (-g 1) before use.
This applies broadly in programmatic video: PiP, reverse playback, scrubber-driven playback, etc. If embedded video looks choppy, check keyframe spacing first and re-encode if needed.