RhymeFlow: Open-Sourcing a 1.8x Speedup for Video Generation

·Toolin Editorial Team

Tsinghua University has open-sourced RhymeFlow, a video generation acceleration framework that speeds inference on DiT models like Wan 2.1 and CogVideoX by 1.5x-1.8x with no retraining required, at near-lossless quality — 62.5% of users couldn't tell the difference.

RhymeFlow: Open-Sourcing a 1.8x Speedup for Video Generation

Tsinghua University and GigaAI have jointly open-sourced RhymeFlow, a completely training-free video generation acceleration framework. It requires no model retraining and applies "inter-frame asynchronous scheduling" directly at inference time, boosting inference speed on DiT video models like Wan 2.1 and CogVideoX by 1.5x to 1.8x.

In a double-blind user study with 82 participants, 62.5% could not distinguish RhymeFlow's output from the original model's.

The Problem It Solves

Today's mainstream DiT video models (Wan 2.1, CogVideoX, Sora) share one pain point: generating a single 81-frame 720p clip takes nearly 17 minutes on a single A800 GPU.

Existing acceleration methods (sparse attention, KV caching, quantization) optimize the compute within each step. But nobody has touched a more fundamental issue — every frame is treated equally, so even adjacent frames with nearly identical content still run through the full 50-step denoising process.

RhymeFlow's core insight: a video's semantics and motion are continuous, keyframes determine the global structure, and non-keyframe trajectories are highly predictable. Given that, why not let different frames take different paths?

Image

Three Core Modules

1. Content-aware keyframe selection

Rather than naive uniform sampling, it uses latent-space semantic similarity to automatically identify keyframes containing scene cuts or abrupt object motion. These frames get full compute resources, ensuring the video's structural integrity and semantic accuracy.

2. Progressive asynchronous denoising schedule

Keyframes update at every step, while non-keyframes skip steps on a schedule that varies by noise stage:

  • Warm-up stage (first 15 steps): all frames denoise in sync to lay the foundation of the global composition
  • High-noise stage (structure-sensitive): non-keyframes update every 2 steps
  • Low-noise stage (detail refinement): non-keyframes update every 3 steps
  • Sync points: all frames periodically reconverge to calibrate non-keyframe trajectories and prevent error accumulation

3. Latent trajectory projection

Once non-keyframes skip steps, the missing intermediate states break the temporal consistency of 3D attention. RhymeFlow uses a linear projection module with negligible compute cost to precisely predict the intermediate latents from the two known states before and after — effectively drawing a smooth motion trajectory for each non-keyframe.

Image

Benchmark Results

Results on mainstream open-source models:

Against SOTA methods:

  • On Wan 2.1: RhymeFlow's PSNR is 1.84 higher than SAP's and its SSIM 0.053 higher, at comparable speed
  • On CogVideoX: a 1.78x speedup while retaining 98.6% subject consistency
  • Stacked with SAP: the speedup rises further to 1.93x, with better quality than SAP alone

Image

82-person double-blind user study:

  • 53.7% of users rated RhymeFlow's temporal coherence above SVG
  • 74.4% of users preferred RhymeFlow over SAP
  • Against the original model, 62.5% of users could not tell the difference — no statistically significant gap

Who It's For

RhymeFlow fits these scenarios:

  • Batch video generation with open-source DiT video models like Wan 2.1 and CogVideoX
  • Cutting video generation inference costs (GPU time roughly halved)
  • Quality-sensitive workloads that can tolerate an extremely slight quality loss

Where to Get It

The framework is fully open source, requires no model retraining, and plugs directly into existing DiT inference pipelines.

Related articles

DeepXiv: A CLI Tool That Lets AI Agents Directly Consume 200 Million Papers
AI Tutorials

DeepXiv: A CLI Tool That Lets AI Agents Directly Consume 200 Million Papers

DeepXiv is an open-source CLI tool that turns 200 million+ open papers into data interfaces Agents can call, supporting search, progressive reading, trending-topic tracking, and deep research.

Toolin Editorial Team
The Six Core Components of a Coding Agent, Fully Explained
AI Tutorials

The Six Core Components of a Coding Agent, Fully Explained

A breakdown of the core architecture of AI coding agents like Claude Code and Codex CLI, covering repository context, prompt caching, tool calling, context slimming, session memory, and subagent delegation.

Toolin Editorial Team
Graphify: Turn Any Folder into a Knowledge Graph with One Command
AI Products

Graphify: Turn Any Folder into a Knowledge Graph with One Command

A zero-configuration, all-modal, locally run open-source knowledge graph tool that cuts token consumption by 71.5x, needs no vector database, and installs with one pip command.

Toolin Editorial Team
Feishu, DingTalk, and WeCom All Ship CLIs: How to Use Agent-Native Tools
AI Tutorials

Feishu, DingTalk, and WeCom All Ship CLIs: How to Use Agent-Native Tools

Feishu, DingTalk, and WeCom have each rolled out command-line tools in quick succession, and Karpathy is loudly championing the CLI revival. This article shows you how to use CLIs to let AI Agents operate enterprise software directly.

Toolin Editorial Team
The caveman Plugin: Making Claude Code Cut the Fluff and Save Tokens
AI Products

The caveman Plugin: Making Claude Code Cut the Fluff and Save Tokens

The caveman plugin that blew up on GitHub with 20K stars saves tokens by streamlining AI output style. It supports Claude Code and Codex, with three compression levels to switch between as needed.

Toolin Editorial Team
Entire CLI: The Open-Source Tool That Makes AI-Generated Code Traceable
AI Products

Entire CLI: The Open-Source Tool That Makes AI-Generated Code Traceable

Entire CLI is an open-source tool from the company founded by GitHub's former CEO. Its Checkpoints technology automatically captures AI coding context so that every commit carries the full reasoning process, solving the pain point of AI code that is hard to trace.

Toolin Editorial Team