- We introduce MiniMax-Speech, an autoregressive Transformer-based Text-to-Speech (TTS) model that generates
- high-quality
- speech. A key innovation is our learnable speaker encoder, which extracts timbre features from a reference audio
- without
- requiring its transcription. This enables MiniMax-Speech to produce highly expressive speech with timbre
- consistent with
- the reference in a zero-shot manner, while also supporting one-shot voice cloning with exceptionally high
- similarity to
- the reference voice. In addition, the overall quality of the synthesized audio is enhanced through the proposed
- Flow-VAE. Our model supports 32 languages and demonstrates excellent performance across multiple objective and
- subjective evaluations metrics. Notably, it achieves state-of-the-art (SOTA) results on objective voice cloning
- metrics
- (Word Error Rate and Speaker Similarity) and has secured the top position on the public TTS Arena leaderboard.
- Another
- key strength of MiniMax-Speech, granted by the robust and disentangled representations from the speaker encoder,
- is its
- extensibility without modifying the base model, enabling various applications such as: arbitrary voice emotion
- control
- via LoRA; text to voice (T2V) by synthesizing timbre features directly from text description; and professional
- voice
- cloning (PVC) by fine-tuning timbre features with additional data.
-
-
-
-
-
-
-
-
-
Architecture Overview
-
-
-
- An overview of the architecture of MiniMax-Speech.
-
-
-
-
-
-
Expressiveness Demonstrations
-
Showcase with High Versatility
-
-
-
-
-
Description
-
Source Audio
-
Generated Audio
-
-
-
- A Compelling and Persuasive Speaker Voice
-
-
-
-
-
-
-
-
-
-
- A Clear and Explanatory Voice with Broad Emotional Dynamics Across Different Texts
-
-
-
-
-
-
-
-
-
-
- Another Explanatory Voice with Supernatural Prosody,
- Featuring Distinct Ethnic and Age Characteristics
-
-
-
-
-
-
-
-
-
-
- A Warm and Magnetic Voice that Brings Comfort
-
-
-
-
-
-
-
-
-
-
- An ASMR Whispering Voice with Generated Breathing and Sound Effects
-
-
-
-
-
-
-
-
-
-
- A Robotic Voice with Rich Bass Resonance and Spatial Presence
-
-
-
-
-
-
-
-
-
-
- A Sardonic Mature Female Voice
-
-
-
-
-
-
-
-
-
-
-
-
-
Showcase with Multiple Generation Attempts, Post-Processing
- Audio Effects and Added Sound Effects
-
-
-
-
-
Description
-
Generated Audio
-
-
-
- A Husky Male Voice: From Soft Murmur to Excitement to Anger, then to Whispers
-
-
-
-
-
-
-
- An Angry Female Voice: From Soft Murmur to Rage to Reminiscence, then to Weeping
-
-
-
-
-
-
-
-
-
-
-
-
Zero-Shot vs. One-Shot Demonstrations
-
- ZeroShot maintains speaker identity while generating more natural emotions, pauses, and other expressive
- features based
- on the text content, whereas OneShot adheres more strictly to the speaker characteristics (prosody, speech
- rate,
- emotions, etc.). For details of Zero-Shot and One-Shot, refer to the technical report.
-
-
- Capable of Generating
- Relatively Calmer Emotions
- while Preserving Voice
- Identity
-
-
-
- Consistently Reproducing the
- Angry Emotion from Prompt
- in Every Utterance
-
-
-
-
-
-
-
-
-
-
- Would you believe what happened at the
- grocery store today? My goodness! The
- avocados were on sale - half price! Half
- price! I bought twenty of them!
-
-
-
- Effectively follows textual cues
- for both longer and shorter
- inter-sentence pauses
-
-
-
- Better reproduces the
- exaggerated high pitch
- characteristic of anime voices
-
-
-
-
-
-
-
-
-
-
- Oh my gosh, like, I literally can't believe
- what just happened! Um, so basically, I was,
- you know, just sitting there in class,
- right? And then, ugh, this totally weird
- thing happened - like, seriously weird! Wait,
- wait... Should I even be talking about this?
- Ugh, whatever.
-
-
-
- Effectively follows textual cues
- for both longer and shorter
- inter-sentence pauses
-
-
-
- Better reproduces the
- exaggerated high pitch
- characteristic of anime voices
-
-
-
-
-
-
-
-
-
-
-
-
Multilingual and Cross-Lingual Capabilities
- Demonstrations
-
Speech-02-HD maintains high naturalness in less common languages while demonstrating significant advantages
- in
- Standard
- Chinese pronunciation accuracy.
Speech-02-HD has superior performance in zero-shot cross-lingual scenarios.
-
-
-
-
-
Original Language
-
Source Audio
-
Mixed Language
-
Text
-
MiniMax Speech_02_HD
-
ElevenLabs Multilingual_v2
-
OpenAI TTS_1_HD (*not cloned voice)
-
-
-
English
-
-
-
-
English + Mandarin
-
- Kiddo! Come come come, 学如逆水行舟，不进则退。
- I see you're using AI tools already - so smart!
- But eh, cannot just rely on tools only lah!
- The future belongs to those who can work alongside AI,
- not those scared of it.
-
- The people said, 桂林's scenery is the first under heaven.
- Yet in my opinion, 阳朔 scenery is better than 桂林。
- 群峰倒影山浮水，无水无山不入神。
-
-
-
-
-
-
-
-
-
-
-
-
-
English
-
-
-
-
English + Spanish
-
- Mi abuelita always told me "el que persevera, alcanza".
- If you persevere, you'll achieve your dreams!
- Guess what! They choose me to play the lead role in our BIG show!
-
-
-
-
-
-
-
-
-
-
-
-
-
Japanese
-
-
-
-
Japanese + Korean
-
- 最近の天気予報によりますと、今週末は桜の開花に最適
- な気温になる予定です。
- 東京都内の各公園では花見客で賑わうことが予想されますが、
- 서울에서도 벚꽃이 피기 시작했다고 하네요.
- 이번 주말에는 여의도 공원에서 벚꽃 축제가 열린다고 하니
- 많은 분들이 찾아오실 것 같습니다.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
*Although OpenAI currently does not support voice cloning functionality, we still wish to conduct comparative
- listening
- tests with its excellent naturalness as a reference.
-
-
-
-
Flow-VAE vs. VAE Comparison
-
Flow-VAE is less likely to produce the following instabilities.
-
-
-
-
-
Source Audio
-
Flow-VAE
-
VAE
-
Differences
-
-
-
-
-
-
-
-
-
-
-
-
- Flow-VAE reproduces more continuous
- and natural reverberation
-
- VAE produces electronic-sounding
- artifacts at the beginning
-
-
-
-
-
-
-
-
-
Professional Voice Clone (PVC) Demonstrations
-
For more complex dialectal accents and tonal characteristics, PVC can reproduce these features while
- maintaining high
- naturalness based on the text content.
-
-
-
-
-
Source Audio
-
Zero-Shot
-
PVC
-
Differences
-
-
-
-
-
-
-
-
-
-
-
-
- Like the ZeroShot version, the PVC
- version has rising sentence-final intonation,
- but distinctively sustains this
- elevated pitch instead of the typical
- pitch declination found in common
- declarative sentences
-
-
-
-
-
-
-
-
-
-
-
-
-
- With more materials, the model not only
- reproduces the speaker's voice characteristics
- but also accurately captures more
- dialectal features
-
- English-speaking female voice, sounding relatively young,
- with a sweet and pleasant tone. Speaking at a moderate pace
- with a touch of energy, similar to someone narrating a
- beauty/makeup tutorial video. The overall atmosphere is
- relaxed and cheerful.
-
-
- Hi everyone! Today I'll be sharing a soft, romantic
- makeup look that's perfect for dates. Many of you have
- been asking how to apply this eyeshadow naturally - the
- key is using gentle techniques. Let's go through the
- steps together...
-
-
-
-
-
-
-
- English-speaking middle-aged male voice, slightly husky,
- speaking at a moderate-to-slow pace with a deep tone. Like
- someone telling an old story, conveying a nostalgic feeling,
- with a relaxed and composed manner of speaking.
-
-
- That was back in the late 1970s. I remember when our
- village first got electricity - everyone was so excited.
- In theevenings, people would bring their stools and
- gather under the big banyan tree by the village committee
- office to watch movies projected on the wall. Even now,
- thinking back to those moments still fills me with warmth.
-
-
-
-
-
-
-
-
-
-
-
-
Comparison of voice naturalness
- with the previous generation products
-
The new model demonstrates significant advantages in naturalness compared to the previous version.
-
Source Audio for Radiant_Girl
-
-
DEMO
-
-
-
-
-
Text
-
MiniMax Speech_02_HD
-
Microsoft Azure TTS
-
AWS Polly
-
-
-
- I sat alone in the empty room, staring at the old photographs,
- wondering how everything could change so quickly,
- how a lifetime of memories could fade away just like that.
-
-
-
-
-
-
-
-
-
-
-
-
-
- The moment I held my acceptance letter, my heart burst with joy -
- all those sleepless nights finally paid off, and I couldn't stop
- dancing around the room, calling everyone I knew to share this amazing news!
-
-