CosyVoice2

CosyVoice is an open-source multilingual large speech generation model from Alibaba's FunAudioLLM team, offering full-stack capabilities from inference to training and deployment. Built on a large language model architecture, it achieves high-quality text-to-speech synthesis through supervised semantic token techniques. The latest CosyVoice 3.0 expands training data from 10,000 hours to 1 million hours and grows model parameters from 500 million to 1.5 billion, significantly improving content consistency, speaker similarity, and prosodic naturalness.

Key Features

• Zero-shot voice cloning: Quickly replicates a voice from just 3-10 seconds of raw audio, including details like prosody and emotion
• Broad language support: Covers 9 major languages (Chinese, English, Japanese, Korean, German, Spanish, French, Italian, Russian) and 18+ Chinese dialects (Cantonese, Minnan, Sichuanese, Northeastern, etc.)
• Cross-lingual speech synthesis: Generates speech in one language using a speaker's voice from another
• Natural language control: Instruction text controls language, dialect, emotion, speaking rate, volume, and other parameters
• Streaming speech synthesis: Supports bidirectional streaming with first-packet synthesis latency as low as 150 milliseconds
• Pronunciation fine-tuning: Fine-grained pronunciation control via Chinese pinyin and English CMU phonemes

Use Cases

In speech translation, CosyVoice preserves the speaker's voice while producing cross-lingual output; in audiobook and interactive podcast production, creators can quickly generate multi-character, multi-emotion speech content; in intelligent customer service and human-computer interaction, enterprises can use its ultra-low latency for fluid real-time voice conversations.

Technical Advantages

CosyVoice 2.0 scored 5.53 MOS (Mean Opinion Score) in evaluations, close to the 5.52 of commercial models and a significant improvement over version 1.0's 5.4. On the Seed-TTS hard test set, CosyVoice 2.0 achieved the lowest character error rate, excelling at tongue twisters, polyphonic characters, and rare characters. Pronunciation error rates dropped 30-50% versus 1.0, and it supports TensorRT-LLM accelerated inference for a 4x performance gain. The project has 18.4k stars on GitHub, remains actively maintained, and released its latest version in December 2025.