From ca809df71f94ce713eb3d44017ec34ae50021813 Mon Sep 17 00:00:00 2001 From: zhaokexin01 Date: Thu, 5 Mar 2020 08:47:07 +0800 Subject: [PATCH 1/5] Update README.md --- README.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index ff46a3b..f32b730 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,9 @@ # Parakeet -Parakeet aims to provide a flexible, efficient and state-of-the-art text-to-speech toolkit for the open-source community. It is built on Paddle Fluid dynamic graph, with the support of many influential TTS models proposed by [Baidu Research](http://research.baidu.com) and other academic institutions. +Parakeet aims to provide a flexible, efficient and state-of-the-art text-to-speech toolkit for the open-source community. It is built on PaddlePaddle Fluid dynamic graph and includes many influential TTS models proposed by [Baidu Research](http://research.baidu.com) and other research groups. + +In particular, it features the latest [WaveFlow] (https://arxiv.org/abs/1912.01219) model proposed by Baidu Research. WaveFlow is a small-footprint generative flow for raw audio, which is directly trained with maximum likelihood. It generates high-fidelity speech as WaveNet, while synthesizing serval orders of magnitude faster as it only requires a few sequential steps to generate very long waveforms. Furthermore, it can significantly reduce the likelihood gap that has existed between autoregressive models and flow-based models for efficient synthesis. Finally, our small-footprint WaveFlow has +only 5.9M parameters, which is 15 times smaller than WaveGlow. It can generate 22.05 kHz high-fidelity audio around 40 times faster than real-time on a V100 GPU without engineered inference kernels.

@@ -8,7 +11,7 @@ Parakeet aims to provide a flexible, efficient and state-of-the-art text-to-spee ### Setup -Make sure the library `libsndfile1` installed, e.g., on Ubuntu +Make sure the library `libsndfile1` is installed, e.g., on Ubuntu. ```bash sudo apt-get install libsndfile1 @@ -16,7 +19,7 @@ sudo apt-get install libsndfile1 ### Install PaddlePaddle -See [install](https://www.paddlepaddle.org.cn/install/quick) for more details. This repo requires paddlepaddle's version is above 1.7. +See [install](https://www.paddlepaddle.org.cn/install/quick) for more details. This repo requires paddlepaddle 1.7 or above. ### Install Parakeet From 61fc5d4cd1e0c41fc991ca6f44aea2bbea15925e Mon Sep 17 00:00:00 2001 From: zhaokexin01 Date: Thu, 5 Mar 2020 09:46:03 +0800 Subject: [PATCH 2/5] Update README.md --- README.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index f32b730..9b993d0 100644 --- a/README.md +++ b/README.md @@ -2,8 +2,11 @@ Parakeet aims to provide a flexible, efficient and state-of-the-art text-to-speech toolkit for the open-source community. It is built on PaddlePaddle Fluid dynamic graph and includes many influential TTS models proposed by [Baidu Research](http://research.baidu.com) and other research groups. -In particular, it features the latest [WaveFlow] (https://arxiv.org/abs/1912.01219) model proposed by Baidu Research. WaveFlow is a small-footprint generative flow for raw audio, which is directly trained with maximum likelihood. It generates high-fidelity speech as WaveNet, while synthesizing serval orders of magnitude faster as it only requires a few sequential steps to generate very long waveforms. Furthermore, it can significantly reduce the likelihood gap that has existed between autoregressive models and flow-based models for efficient synthesis. Finally, our small-footprint WaveFlow has -only 5.9M parameters, which is 15 times smaller than WaveGlow. It can generate 22.05 kHz high-fidelity audio around 40 times faster than real-time on a V100 GPU without engineered inference kernels. +In particular, it features the latest [WaveFlow] (https://arxiv.org/abs/1912.01219) model proposed by Baidu Research. +- WaveFlow can synthesize 22.05 kHz high-fidelity speech around 40x faster than real-time on a Nvidia V100 GPU without engineered inference kernels, which is faster than WaveGlow and serveral orders of magnitude faster than WaveNet. +- WaveFlow is a small-footprint flow-based model for raw audio. It has only 5.9M parameters, which is 15x smalller than WaveGlow (87.9M) and comparable to WaveNet (4.6M). +- WaveFlow is directly trained with maximum likelihood without probability density distillation and auxiliary losses as used in Parallel WaveNet and ClariNet, which simplifies the training pipeline and reduces the cost of development. +

From e26b0c104e9dc3c6a56ecc11d6c5b784c423c24e Mon Sep 17 00:00:00 2001 From: zhaokexin01 Date: Thu, 5 Mar 2020 09:51:58 +0800 Subject: [PATCH 3/5] Update README.md --- examples/clarinet/README.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/examples/clarinet/README.md b/examples/clarinet/README.md index 6046327..58bca99 100644 --- a/examples/clarinet/README.md +++ b/examples/clarinet/README.md @@ -1,6 +1,6 @@ # Clarinet -Paddle implementation of clarinet in dynamic graph, a convolutional network based vocoder. The implementation is based on the paper [ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech](arxiv.org/abs/1807.07281). +PaddlePaddle dynamic graph implementation of ClariNet, a convolutional network based vocoder. The implementation is based on the paper [ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech](arxiv.org/abs/1807.07281). ## Dataset @@ -30,7 +30,7 @@ Train the model using train.py, follow the usage displayed by `python train.py - usage: train.py [-h] [--config CONFIG] [--device DEVICE] [--output OUTPUT] [--data DATA] [--resume RESUME] [--wavenet WAVENET] -train a clarinet model with LJspeech and a trained wavenet model. +train a ClariNet model with LJspeech and a trained WaveNet model. optional arguments: -h, --help show this help message and exit @@ -54,12 +54,12 @@ optional arguments: ``` 5. `--device` is the device (gpu id) to use for training. `-1` means CPU. -6. `--wavenet` is the path of the wavenet checkpoint to load. if you do not specify `--resume`, then this must be provided. +6. `--wavenet` is the path of the wavenet checkpoint to load. If you do not specify `--resume`, then this must be provided. -Before you start training a clarinet model, you should have trained a wavenet model with single gaussian as output distribution. Make sure the config for teacher matches that for the trained model. +Before you start training a ClariNet model, you should have trained a WaveNet model with single Gaussian output distribution. Make sure the config of the teacher model matches that of the trained model. -example script: +Example script: ```bash python train.py --config=./configs/clarinet_ljspeech.yaml --data=./LJSpeech-1.1/ --output=experiment --device=0 --conditioner=wavenet_checkpoint/conditioner --conditioner=wavenet_checkpoint/teacher @@ -77,7 +77,7 @@ tensorboard --logdir=. usage: synthesis.py [-h] [--config CONFIG] [--device DEVICE] [--data DATA] checkpoint output -train a clarinet model with LJspeech and a trained wavenet model. +train a ClariNet model with LJspeech and a trained WaveNet model. positional arguments: checkpoint checkpoint to load from. @@ -96,7 +96,7 @@ optional arguments: 4. `output_path` is the directory to save results. The output path contains the generated audio files (`*.wav`). 5. `--device` is the device (gpu id) to use for training. `-1` means CPU. -example script: +Example script: ```bash python synthesis.py --config=./configs/wavenet_single_gaussian.yaml --data=./LJSpeech-1.1/ --device=0 experiment/checkpoints/step_500000 generated From 786b2206aa617e23aa490783db55836a404fe30a Mon Sep 17 00:00:00 2001 From: zhaokexin01 Date: Thu, 5 Mar 2020 10:02:13 +0800 Subject: [PATCH 4/5] Update README.md --- examples/deepvoice3/README.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/examples/deepvoice3/README.md b/examples/deepvoice3/README.md index 0138414..b03634b 100644 --- a/examples/deepvoice3/README.md +++ b/examples/deepvoice3/README.md @@ -1,8 +1,8 @@ -# Deepvoice 3 +# Deep Voice 3 -Paddle implementation of deepvoice 3 in dynamic graph, a convolutional network based text-to-speech synthesis model. The implementation is based on [Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning](https://arxiv.org/abs/1710.07654). +PaddlePaddle dynamic graph implementation of Deep Voice 3, a convolutional network based text-to-speech generative model. The implementation is based on [Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning](https://arxiv.org/abs/1710.07654). -We implement Deepvoice 3 in paddle fluid with dynamic graph, which is convenient for flexible network architectures. +We implement Deep Voice 3 using Paddle Fluid with dynamic graph, which is convenient for building flexible network architectures. ## Dataset @@ -15,9 +15,9 @@ tar xjvf LJSpeech-1.1.tar.bz2 ## Model Architecture -![DeepVoice3 model architecture](./images/model_architecture.png) +![Deep Voice 3 model architecture](./images/model_architecture.png) -The model consists of an encoder, a decoder and a converter (and a speaker embedding for multispeaker models). The encoder, together with the decoder forms the seq2seq part of the model, and the converter forms the postnet part. +The model consists of an encoder, a decoder and a converter (and a speaker embedding for multispeaker models). The encoder and the decoder together form the seq2seq part of the model, and the converter forms the postnet part. ## Project Structure @@ -37,7 +37,7 @@ Train the model using train.py, follow the usage displayed by `python train.py - ```text usage: train.py [-h] [-c CONFIG] [-s DATA] [-r RESUME] [-o OUTPUT] [-g DEVICE] -Train a deepvoice 3 model with LJSpeech dataset. +Train a Deep Voice 3 model with LJSpeech dataset. optional arguments: -h, --help show this help message and exit @@ -55,7 +55,7 @@ optional arguments: 1. `--config` is the configuration file to use. The provided `ljspeech.yaml` can be used directly. And you can change some values in the configuration file and train the model with a different config. 2. `--data` is the path of the LJSpeech dataset, the extracted folder from the downloaded archive (the folder which contains metadata.txt). 3. `--resume` is the path of the checkpoint. If it is provided, the model would load the checkpoint before trainig. -4. `--output` is the directory to save results, all result are saved in this directory. The structure of the output directory is shown below. +4. `--output` is the directory to save results, all results are saved in this directory. The structure of the output directory is shown below. ```text ├── checkpoints # checkpoint @@ -69,7 +69,7 @@ optional arguments: 5. `--device` is the device (gpu id) to use for training. `-1` means CPU. -example script: +Example script: ```bash python train.py --config=./ljspeech.yaml --data=./LJSpeech-1.1/ --output=experiment --device=0 @@ -86,7 +86,7 @@ tensorboard --logdir=. ```text usage: synthesis.py [-h] [-c CONFIG] [-g DEVICE] checkpoint text output_path -Synthsize waveform with a checkpoint. +Synthsize waveform from a checkpoint. positional arguments: checkpoint checkpoint to load. @@ -107,7 +107,7 @@ optional arguments: 4. `output_path` is the directory to save results. The output path contains the generated audio files (`*.wav`) and attention plots (*.png) for each sentence. 5. `--device` is the device (gpu id) to use for training. `-1` means CPU. -example script: +Example script: ```bash python synthesis.py --config=./ljspeech.yaml --device=0 experiment/checkpoints/model_step_005000000 sentences.txt generated From b1c64eecdc512fdf495779777a1a80b3845401bd Mon Sep 17 00:00:00 2001 From: zhaokexin01 Date: Thu, 5 Mar 2020 10:40:42 +0800 Subject: [PATCH 5/5] Update README.md --- README.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 9b993d0..a9e969f 100644 --- a/README.md +++ b/README.md @@ -2,16 +2,15 @@ Parakeet aims to provide a flexible, efficient and state-of-the-art text-to-speech toolkit for the open-source community. It is built on PaddlePaddle Fluid dynamic graph and includes many influential TTS models proposed by [Baidu Research](http://research.baidu.com) and other research groups. +
+
+
+ In particular, it features the latest [WaveFlow] (https://arxiv.org/abs/1912.01219) model proposed by Baidu Research. - WaveFlow can synthesize 22.05 kHz high-fidelity speech around 40x faster than real-time on a Nvidia V100 GPU without engineered inference kernels, which is faster than WaveGlow and serveral orders of magnitude faster than WaveNet. - WaveFlow is a small-footprint flow-based model for raw audio. It has only 5.9M parameters, which is 15x smalller than WaveGlow (87.9M) and comparable to WaveNet (4.6M). - WaveFlow is directly trained with maximum likelihood without probability density distillation and auxiliary losses as used in Parallel WaveNet and ClariNet, which simplifies the training pipeline and reduces the cost of development. - -
-
-
- ### Setup Make sure the library `libsndfile1` is installed, e.g., on Ubuntu.