Behind the boom of the diffusion model ecosystem, a persistent headache remains: the model ecosystem is far too "fragmented". You finally get a new model running, then want to add a control feature only to find the code structure is completely incompatible; try to stack multiple control conditions, and the plugins fight each other under the hood. Diffusion Templates exists to unify this mess.

It is a plugin framework designed specifically for diffusion models, letting tools like ControlNet, LoRA, and IP-Adapter combine freely like building blocks without conflicting with one another.

What Is Diffusion Templates

Diffusion Templates is a full-featured plugin framework for diffusion models. Each plugin is called a Template, and it can guide a base model to generate specific visual content. The core design of the framework is the Template Cache -- a medium format for model capabilities (such as KV-Cache), through which a Template passes control information to the base model.

The code has now been fully open-sourced under the DiffSynth-Studio project, along with 11 open-source Template models trained on FLUX.2-klein-base-4B.

Project URL: https://github.com/modelscope/DiffSynth-Studio

Framework Structure

The framework consists of four modules:

• Template Input: the model's input, with fields decided by each Template itself, giving developers maximum freedom
• Template Model: the template model, loadable from the ModelScope model hub or a local path
• Template Cache: the model's output, also a subset of the base model Pipeline's input parameters
• Template Pipeline: the module that schedules multiple Template models, responsible for loading and integrating their outputs

The 11 Open-Source Templates at a Glance

The templates below are trained on FLUX.2-klein-base-4B and are compatible with that model's LoRA and fine-tuned versions:

Template Stacking in Action

The biggest advantage of Templates is that they stack freely, like building blocks. Take the same photo of a cat:

• Template 1 (image editing): change the visual style to anime
• Template 2 (inpainting): keep the surrounding environment unchanged
• Template 3 (brightness adjustment): raise the brightness a notch

Or take a blurry cat photo and stack the super resolution and sharpness boost Templates:

Two models trained toward different objectives render in concert, noticeably improving image quality.

Related Resources

• Project homepage: https://modelscope.github.io/diffusion-templates-web/
• Open-source code: https://github.com/modelscope/DiffSynth-Studio
• Technical report: https://arxiv.org/abs/2604.24351
• Model collection (11 models): https://modelscope.cn/collections/DiffSynth-Studio/KleinBase4B-Templates
• Datasets (17 datasets): https://modelscope.cn/collections/DiffSynth-Studio/ImagePulseV2

Who It Is For

• AI image creators: stop worrying about plugin conflicts and combine control conditions freely
• Model developers: the framework lowers the barrier to training controllable-generation models, so you can build new Templates quickly
• Product teams: a standardized plugin mechanism makes model integration simple and reliable