VoxCPM2: Tokenizer-Free Multilingual Text-to-Speech

VoxCPM2 is OpenBMB's tokenizer-free, diffusion autoregressive text-to-speech model. It speaks 30 languages, outputs 48 kHz audio, and was trained on over 2 million hours of multilingual speech data. No language tag is needed: you pass text in any supported language directly, and the model infers appropriate prosody and expressiveness from the content.

Beyond plain synthesis, VoxCPM2 can design a new voice from a written description, clone a voice from a short reference clip, and reproduce a speaker in fine detail when given the clip's transcript as well.

Key Features

• 30 languages - Arabic, Burmese, Chinese, Danish, Dutch, English, Finnish, French, German, Greek, Hebrew, Hindi, Indonesian, Italian, Japanese, Khmer, Korean, Lao, Malay, Norwegian, Polish, Portuguese, Russian, Spanish, Swahili, Swedish, Tagalog, Thai, Turkish, and Vietnamese, plus Chinese dialects including Sichuanese, Cantonese, Wu, and Northeastern Mandarin
• Voice design - generate a novel voice from a natural-language description alone (gender, age, tone, emotion, pace) by putting the description in parentheses at the start of the text, with no reference audio required
• Controllable cloning - clone any voice from a short clip, with optional style guidance to steer emotion, pace, and expression while preserving timbre
• Ultimate cloning - provide reference audio together with its transcript for audio-continuation cloning that carries over the speaker's vocal nuances
• 48 kHz output - accepts 16 kHz reference audio and produces 48 kHz speech through the built-in super-resolution of its AudioVAE V2, with no external upsampler
• Streaming - supports real-time streaming generation

Architecture

VoxCPM2 is built on the MiniCPM-4 backbone and works without a discrete audio tokenizer. Its pipeline runs a local encoder, a base language model, a residual language model, and a local diffusion transformer (LocEnc, TSLM, RALM, LocDiT) autoregressively over continuous speech representations, and AudioVAE V2 decodes the result into the waveform. The language model runs at a token rate of 6.25Hz.

Serving on Vast

This template serves VoxCPM2 with vLLM-Omni through the OpenAI-compatible audio speech endpoint, returning 48 kHz mono audio as WAV or as streaming PCM. Send plain text for zero-shot speech, start the text with a description in parentheses to design a voice, or include reference audio to clone a speaker. The bundled Model UI offers standard synthesis and reference-audio cloning.

Use Cases

• Multilingual narration, dubbing, and localization across 30 languages
• Character and persona voices created from a text description for games, audio drama, and assistants
• Voice cloning for consistent narrators and branded voices from short recordings
• Audiobook, podcast, and accessibility narration at 48 kHz
• Fine-tuning a custom voice, which the model supports with full fine-tuning or LoRA from as little as 5 to 10 minutes of audio

Model weights are available on HuggingFace at openbmb/VoxCPM2.