AI generating a few seconds of video is no longer news. But keeping the same character consistent across several minutes — face unchanged, clothes not drifting, voice not wavering — that's the hard bone AI video generation really needs to gnaw on.

Today, two open-source frameworks delivered their respective answers: VideoClaw from Harbin Institute of Technology in collaboration with Alibaba, and JoyAI-Echo from JD. Both target long-video consistency, but their technical routes are completely different.

The Problem: Why Long Videos Are So Hard

Long-video generation is fundamentally not a "stretch the timeline" problem — it's a continuous storytelling problem across shots, scenes, and actions:

• Character drift: after multiple shot changes, the face changes, the clothes change
• Voice drift: the speaker's voice is inconsistent across segments
• Narrative breaks: scene transitions become logically incoherent
• Error accumulation: the model's deviations grow larger over long temporal sequences

VideoClaw: A Multi-Agent "Digital Film Crew"

VideoClaw comes from Professor Zhang Min's team at Harbin Institute of Technology in collaboration with Alibaba. Its core idea is to break long-video generation into a multi-agent collaborative pipeline.

• GitHub: https://github.com/HITsz-TMG/VideoClaw
• Star count: 1.3K+
• Related projects: ComfyUI-Copilot (5.2K Star), Pixelle-Video (20.8K Star)

Core Architecture

The user only needs to feed in a spark of inspiration or a story synopsis, and the system orchestrates a LLM-driven "digital film crew" to complete, in sequence:

1. Script expansion
2. Character and scene design
3. Storyboard planning
4. Keyframe composition
5. Segmented video generation
6. Audio synthesis and final assembly

Unlike black-box video generation, VideoClaw pauses after the script, character/scene, and storyboard stages to show intermediate artifacts, letting creators step in and make changes at key checkpoints.

The Script Supervisor Library: The Key to Long-Range Consistency

VideoClaw introduces a state library that functions like a film set's "script supervisor," distilling character relationships, spatial positions, scene storyboards, and version information into structured assets. Later generation stages pull reference constraints from this state library.

This means VideoClaw supports unlimited story continuation — video extends segment after segment, plot conflicts escalate naturally, and character interactions build on what's already happened.

VLM Closed-Loop Quality Inspection

VideoClaw embeds vision-language models (VLMs) into the generation pipeline, triggering a review once images, keyframes, and video clips are generated: comparing whether the visuals match the script's settings and checking whether characters, scenes, and narrative logic have drifted. If a candidate version misses the quality threshold, it outputs a diagnostic report and triggers backtracking and regeneration.

Installation

VideoClaw supports quick installation across Linux / Mac / Windows, provides a WebUI, and can also be integrated into communication tools like WeChat and Feishu.

JoyAI-Echo: Long-Video Generation Driven by Cross-Modal Memory

JoyAI-Echo comes from JD, and its core idea is to give the model a "memory bank" so it doesn't forget what characters look and sound like while generating long videos.

Core Technology

Cross-modal audio-visual memory bank

The system records not just what characters look like but also the speaker's voice, binding the two together. When a character first appears, visual features and voice features are extracted and written into the memory bank; every subsequent shot pulls references from it.

The memory bank isn't infinitely expansive — it keeps the key shots from the story's opening plus the most recently generated shots, balancing efficiency and consistency.

Memory-driven post-training: 7.5x faster inference

The post-training pipeline has three steps:

1. SFT (supervised fine-tuning): learning high-quality audio-video generation
2. RLHF (reinforcement learning from human feedback): optimizing character consistency, visual quality, and audio-visual sync
3. DMD (Distribution Matching Distillation): compressing the large model's capabilities into an efficient inference model

DMD optimization alone delivers roughly a 7.5x inference speedup.

Lightweight real-time super-resolution

Outputs high-definition footage while preserving generation efficiency, suited to scenarios with quality demands like digital humans and brand marketing.

Benchmark Data

• Speech accuracy: 0.8646
• User preference: 59.4% ~ 81.7%
• Cross-shot consistency: leads the industry across the board

Head-to-Head Comparison

Real-World Use Cases

VideoClaw Cases

• Derivative fan content: rewrote the ending of "A Love Letter to Grandma" — Musheng returns home and spends the rest of his life with Shurou
• Realistic short drama: a 6-episode drama about a laid-off programmer rebuilding their life through entrepreneurship, with additional continuation supported
• Sci-fi comic drama: a 5-episode comic drama based on Liu Cixin's "The Village Teacher"

How to Choose

• Need full control over the video creation process (script, storyboard, and character design all open to human intervention): pick VideoClaw
• Need high-consistency audio-video content (no face swaps, no voice glitches): pick JoyAI-Echo
• Try both: they're both open source, so you can run comparative tests in your own scenarios