is that able to train on Chinese dataset?

[lucasjinreal]

is that able to train on Chinese dataset?

[jaywalnut310]

Definitely, yes! But you may need any text-to-phone converter such as Phonemizer to convert Chinese text into phonemes.
This model gets phonemes as input rather than characters.

[lucasjinreal]

@jaywalnut310 I probably would try Biaobei data for Chinese, I am totally newbie in tts though. Let me have a deep look. what would be phonemes like in Chinese?

[LG-SS]

@jaywalnut310 I probably would try Biaobei data for Chinese, I am totally newbie in tts though. Let me have a deep look. what would be phonemes like in Chinese?
The phonemes in Chinese are initials and finals with tone, for example, "ni2 hao3" can be converted into "n i2 h ao3"

[LG-SS]

Definitely, yes! But you may need any text-to-phone converter such as Phonemizer to convert Chinese text into phonemes.
This model gets phonemes as input rather than characters.

Curiously, has the paper been released publically? I have not retrieved it on arxiv or Google Scholar currently.

[jaywalnut310]

@LG-SS Now the paper is available: https://arxiv.org/abs/2106.06103

Definitely, yes! But you may need any text-to-phone converter such as Phonemizer to convert Chinese text into phonemes.
This model gets phonemes as input rather than characters.

Curiously, has the paper been released publically? I have not retrieved it on arxiv or Google Scholar currently.

[lucasjinreal]

@jaywalnut310 Hi, may I ask one last question, how's the latency compare with tacotron2 (I mean e2e lantency, tacotron2 may also need a vcoder which count in), is vits faster in terms of predict same length sentence, or slow? How much?

[leminhnguyen]

@jaywalnut310 This model is autoregressive or non autoregressive ?

[jaywalnut310]

@jaywalnut310 Hi, may I ask one last question, how's the latency compare with tacotron2 (I mean e2e lantency, tacotron2 may also need a vcoder which count in), is vits faster in terms of predict same length sentence, or slow? How much?

Hi @jinfagang. In a previous work, Glow-TTS, a synthesis speed test between Tacotron 2 and Glow-TTS was reported. As the synthesis speed of VITS is faster than that of Glow-TTS + HiFi-GAN(vocoder), it would be much faster than that of Tacotron 2 + HiFi-GAN(vocoder).

[jaywalnut310]

@jaywalnut310 This model is autoregressive or non autoregressive ?

Hi @leminhnguyen, this model is non autoregressive.

[leminhnguyen]

@jaywalnut310 Thanks you, I have some questions.

1. How about controllability ?
2. We can change the duration, energy or pitch ?
3. In the paper, you mentioned FastSpeech2 in related work. Did you try to compare the speed between Fastspeech2 and VITS ?

[lucasjinreal]

@jaywalnut310 I listened the sample audio from vits, it's much more better and natural than tactron2, so it's better and faster, more valuable to have a try. Do u guys have a Chinese pretrained model BTW?

[jaywalnut310]

@leminhnguyen Well, VITS provides controllability to some extent. You can control and change the duration manually. You can control and change the energy and pitch by manipulating the latent representation (z in our code), but you cannot predict how much the energy and pitch changed beforehand. and I only compared with open-sourced official implementations (unfortunately FastSpeech 2 is not), in quality.

@jinfagang Thank you :). I haven't trained on Chinese dataset, but it would be great if someone try and share it later.

[lucasjinreal]

@jaywalnut310 I can train on BIAOBEI dataset which is a opensource Chinese dataset. But can u tell me which way should I organise it?

[TaoTeCha]

Well, VITS provides controllability to some extent. You can control and change the duration manually. You can control and change the energy and pitch by manipulating the latent representation (z in our code), but you cannot predict how much the energy and pitch changed beforehand. and I only compared with open-sourced official implementations (unfortunately FastSpeech 2 is not), in quality.

@jaywalnut310 I am only familiar with Tacotron, and have not yet used a model with variability. What parameters should I change in inference the inference code to change duration or pitch? Or are you saying this needs to be done during training?

[WadoodAbdul]

@jinfagang @TaoTeCha Have you trained a Chinese model successfully? Also are you planning to open source the model?