OpenParable
Skip to content
← Back to skills

Wan Multitalk

Apps & Automation
wan-multitalk

Wan Multitalk is an Agent Skill for apps & automation published in the artokun/comfyui-mcp repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Build WAN MultiTalk audio-driven talking-avatar / lip-sync video workflows — MeiGen-AI MultiTalk on WAN 2.1 14B I2V via kijai WanVideoWrapper (portrait + audio → lip-synced video)

View on GitHub ↗
QUICK START

How to use this skill

Bring this guide into your coding agent with a prompt tailored to the tool you use.

  1. Open your project in Codex.
  2. Copy and paste the prompt below.
  3. Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/artokun/comfyui-mcp/blob/HEAD/plugin/skills/wan-multitalk/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/wan-multitalk, but wait for my approval before writing files or running scripts.

Copying this prompt does not install or run the skill. Review third-party files before use.

WAN MultiTalk — Audio-Driven Talking Avatar

Overview

MultiTalk (MeiGen-AI) drives a still portrait's lip-sync and head motion from an audio track. It runs on WAN 2.1 14B Image-to-Video via kijai's ComfyUI-WanVideoWrapper: Wav2Vec speech embeddings condition the WAN sampler so the mouth/expression follow the speech, while the lightx2v step-distill LoRA keeps it to a few sampling steps.

Use it for talking heads, dubbing, and single-speaker avatar clips (~10s at 480p). It is distinct from wan-animate (pose/motion-driven character animation) — this is audio → lip-sync, not reference-video motion transfer.

Pack: wan-multitalk (480p, ~10s). Higher-res/longer variants exist in the source bundle (720p, long-context) as VRAM/duration knobs on the same graph.

Pipeline (node graph)

LoadImage (portrait) ─┐
LoadAudio ─ AudioSeparation ─ AudioCrop ─ DownloadAndLoadWav2VecModel ─ MultiTalkWav2VecEmbeds ─┐
                                                                                                 ▼
WanVideoModelLoader (WAN 2.1 14B I2V GGUF) ─ MultiTalkModelLoader ─ WanVideoLoraSelect (lightx2v)
   + LoadWanVideoT5TextEncoder (umt5) + WanVideoTextEncode + WanVideoClipVisionEncode (clip_vision_h)
   + WanVideoVAELoader ──────────────────────────────────────────────────────────────────────────┘
                                                     ▼
                       WanVideoImageToVideoMultiTalk ─ WanVideoSampler ─ WanVideoDecode ─ VHS_VideoCombine

Key nodes (all kijai WanVideoWrapper unless noted):

  • DownloadAndLoadWav2VecModel — auto-downloads the Wav2Vec speech model on first run (no manifest entry needed).
  • MultiTalkWav2VecEmbeds — turns the (separated, cropped) speech into the embeddings that steer the mouth/expression.
  • MultiTalkModelLoader + WanVideoImageToVideoMultiTalk — the MultiTalk head on top of the WAN I2V model.
  • AudioSeparation — isolate the voice from music/noise before embedding (cleaner lip-sync). AudioCrop — trim to the segment you want to animate.
  • ImageResizeKJv2 (KJNodes), VHS_VideoCombine (VideoHelperSuite) — resize + mux to mp4.

Models

FileLoaderFolder
Wan2.1_14b_Image_to_Video_480p_GGUF_Q8.ggufWanVideoModelLoaderdiffusion_models/
WanVideo_2_1_Multitalk_14B_fp32.safetensorsMultiTalkModelLoaderdiffusion_models/
umt5_xxl_fp8_e4m3fn_scaled.safetensorsLoadWanVideoT5TextEncodertext_encoders/
Wan2_1_VAE_bf16.safetensorsWanVideoVAELoadervae/
clip_vision_h.safetensorsCLIPVisionLoaderclip_vision/
Wan21_I2V_14B_lightx2v_cfg_step_distill_lora_rank64_fixed.safetensorsWanVideoLoraSelectloras/

Sources: kijai Kijai/WanVideo_comfy, MeiGen-AI MeiGen-AI/MeiGen-MultiTalk, GGUF city96/Wan2.1-I2V-14B-480P-gguf. See packs/wan-multitalk/manifest.yaml (some URLs are best-effort — verify per mirror). Wav2Vec auto-downloads.

Inputs & key parameters

  • Portrait (LoadImage): front-facing, clear face, neutral-ish expression works best. Resized by ImageResizeKJv2 to the target (480p).
  • Audio (LoadAudio): the speech track. AudioSeparation isolates the voice; AudioCrop selects the segment (drives clip length).
  • Steps: low (the lightx2v distill LoRA is why — typically ~4–8). Raising steps rarely helps and costs time.
  • BlockSwap (WanVideoBlockSwap): trade VRAM for speed — increase blocks swapped to CPU on lower-VRAM cards.

VRAM tiers (from the source bundle's variants)

TargetApprox VRAMLever
480p 10s~8–12 GBbase
480p low-VRAM~6–8.4 GBmore BlockSwap, GGUF quant, lower quality
720p 10s~11–16 GBhigher res

Pair with the VRAM launch-flags guidance (see troubleshooting): --use-sage-attention

  • appropriate --*vram mode; MultiTalk benefits from --reserve-vram headroom for the Wav2Vec + VAE round-trips.

Gotchas

  • Audio must be voice-isolated for good lip-sync — skipping AudioSeparation on a music-heavy track makes the mouth chase the wrong signal.
  • One speaker. This graph is single-speaker; multi-speaker MultiTalk needs the multi-embed variant (not in this pack).
  • Wav2Vec first run downloads a model — the first render is slower.
  • If lips look under-driven, check the MultiTalk embeds are actually wired into WanVideoImageToVideoMultiTalk (not bypassed), and that the audio isn't silent after AudioCrop.