diff --git a/doc/source/advanced.rst b/doc/source/advanced.rst new file mode 100644 index 0000000..dccb98e --- /dev/null +++ b/doc/source/advanced.rst @@ -0,0 +1,31 @@ +====================== +Advanced Usage +====================== + +This sections covers how to extend parakeet by implementing you own models and +experiments. Guidelines on implementation are also elaborated. + +Model +------------- + +As a common practice with paddlepaddle, models are implemented as subclasse +of ``paddle.nn.Layer``. More complicated models, it is recommended to split +the model into different components. + +For a encoder-decoder model, it is natural to split it into the encoder and +the decoder. For a model composed of several similar layers, it is natural to +extract the sublayer as a seperate layer. + +There are two common ways to define a model which consists of several modules. + +#. + + +Data +------------- + +Config +------------- + +Experiment +-------------- \ No newline at end of file diff --git a/doc/source/conf.py b/doc/source/conf.py index dd4a270..3b652e4 100644 --- a/doc/source/conf.py +++ b/doc/source/conf.py @@ -48,6 +48,7 @@ extensions = [ "sphinx_rtd_theme", 'sphinx.ext.mathjax', 'numpydoc', + 'sphinx.ext.autosummary', ] # Add any paths that contain templates here, relative to this directory. @@ -63,8 +64,10 @@ exclude_patterns = [] # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. # + html_theme = "sphinx_rtd_theme" + # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". diff --git a/doc/source/design.rst b/doc/source/design.rst new file mode 100644 index 0000000..6e7fb11 --- /dev/null +++ b/doc/source/design.rst @@ -0,0 +1,4 @@ +============================== +Design of Parakeet +============================== + diff --git a/doc/source/index.rst b/doc/source/index.rst index 389f291..08eb577 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -3,14 +3,43 @@ You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. -Welcome to parakeet's documentation! +Parakeet ==================================== +``parakeet`` is a deep learning based text-to-speech toolkit built upon ``paddlepaddle`` framework. It aims to provide a flexible, efficient and state-of-the-art text-to-speech toolkit for the open-source community. It includes many influential TTS models proposed by `Baidu Research `_ and other research groups. + +``parakeet`` mainly consists of components below. + +#. Implementation of models and commonly used neural network layers. +#. Dataset abstraction and common data preprocessing pipelines. +#. Ready-to-run experiments. + .. toctree:: - :maxdepth: 2 - :caption: Contents: + :caption: Getting started + :maxdepth: 1 + install + tutorials +.. toctree:: + :caption: Design of Parakeet + :maxdepth: 1 + + advanced + design + +.. toctree:: + :caption: Documentation + :maxdepth: 1 + + parakeet.audio + parakeet.data + parakeet.datasets + parakeet.frontend + parakeet.modules + parakeet.models + parakeet.training + parakeet.utils Indices and tables ================== diff --git a/doc/source/install.rst b/doc/source/install.rst new file mode 100644 index 0000000..3758ef7 --- /dev/null +++ b/doc/source/install.rst @@ -0,0 +1,85 @@ +============= +Installation +============= + + +Install PaddlePaddle +------------------- +Parakeet requires PaddlePaddle as its backend. Not that 2.0rc or newer versions +of paddle is required. + +Since paddlepaddle has multiple packages depending on the device (cpu or gpu) +and the dependency libraries, it is recommended to install a proper package of +paddlepaddle with respect to the device and dependency library versons via +pip. + +Installing paddlepaddle with conda or build paddlepaddle from source is also +supported. Please refer to `PaddlePaddle installation `_ for more details. + +Example instruction to install paddlepaddle via pip is listed below. + +**PaddlePaddle with gpu** + +.. code-block:: bash + + python -m pip install paddlepaddle-gpu==2.0.0rc1.post101 -f https://paddlepaddle.org.cn/whl/stable.html + python -m pip install paddlepaddle-gpu==2.0.0rc1.post100 -f https://paddlepaddle.org.cn/whl/stable.html + + +**PaddlePaddle with cpu** + +.. code-block:: bash + + python -m pip install paddlepaddle==2.0.0rc1 -i https://mirror.baidu.com/pypi/simple + + +Install libsndfile +------------------- + +Experimemts in parakeet often involve audio and spectrum processing, thus +``librosa`` and ``soundfile`` are required. ``soundfile`` requires a extra +C library ``libsndfile``, which is not always handled by pip. + +For windows and mac users, ``libsndfile`` is also installed when Installing +``soundfile`` via pip, but for linux users, installing ``libsndfile`` via +system package manager is required. Example commands for popular distributions +are listed below. + +.. code-block:: + + # ubuntu, debian + sudo apt-get install libsndfile1 + + # centos, fedora, + sudo yum install libsndfile + + # openSUSE + sudo zypper in libsndfile + +For any problem with installtion of soundfile, please refer to +`SoundFile `_. + +Insrall Parakeet +------------------ + +There are two ways to install parakeet according to the purpose of using it. + +#. If you want to run experiments provided by parakeet or add new models and +experiments, it is recommended to clone the project from github +(`Parakeet `_), and install it in +editable mode. + + .. code-block:: bash + + git clone https://github.com/PaddlePaddle/Parakeet + cd Parakeet + pip install -e . + + +#. If you only need to use the models for inference by parakeet, install from +pypi is recommended。 + + .. code-block:: bash + + pip install paddle-parakeet + diff --git a/doc/source/tutorials.rst b/doc/source/tutorials.rst new file mode 100644 index 0000000..99e7fcb --- /dev/null +++ b/doc/source/tutorials.rst @@ -0,0 +1,65 @@ +=========== +Tutorials +=========== + +Basic Usage +------------------- + +Pretrained models are provided in a archive. Extract it to get a folder like this:: + + checkpoint_name/ + ├──config.yaml + └──step-310000.pdparams + +The ``config.yaml`` stores the config used to train the model, the ``step-N.pdparams`` is the parameter file, where N is the steps it has been trained. + +The example code below shows how to use the models for prediction. + +text to spectrogram +^^^^^^^^^^^^^^^^^^^^^^ + +The code below show how to use a transformer_tts model. After loading the pretrained model, use ``model.predict(sentence)`` to generate spectrogram (in numpy.ndarray format), which can be further used to synthesize waveflow. + +>>> import parakeet +>>> from parakeet.frontend import English +>>> from parakeet.models import TransformerTTS +>>> from pathlib import Path +>>> import yacs + +>>> # load the pretrained model +>>> frontend = English() +>>> checkpoint_dir = Path("transformer_tts_pretrained") +>>> config = yacs.config.CfgNode.load_cfg(str(checkpoint_dir / "config.yaml")) +>>> checkpoint_path = str(checkpoint_dir / "step-310000") +>>> model = TransformerTTS.from_pretrained( +>>> frontend, config, checkpoint_path) +>>> model.eval() + +>>> # text to spectrogram +>>> sentence = "Printing, in the only sense with which we are at present concerned, differs from most if not from all the arts and crafts represented in the Exhibition" +>>> outputs = model.predict(sentence, verbose=args.verbose) +>>> mel_output = outputs["mel_output"] + +vocoder +^^^^^^^^^^ + +Like the example above, after loading the pretrained ConditionalWaveFlow model, call ``model.predict(mel)`` to synthesize waveflow (in numpy.ndarray format). + +>>> import soundfile as df +>>> from parakeet.models import ConditionalWaveFlow + +>>> # load the pretrained model +>>> checkpoint_dir = Path("waveflow_pretrained") +>>> config = yacs.config.CfgNode.load_cfg(str(checkpoint_dir / "config.yaml")) +>>> checkpoint_path = str(checkpoint_dir / "step-2000000") +>>> vocoder = ConditionalWaveFlow.from_pretrained(config, checkpoint_path) +>>> vocoder.eval() + +>>> # synthesize +>>> audio = vocoder.predict(mel_output) +>>> sf.write(audio_path, audio, config.data.sample_rate) + +For more details on how to use the model, please refer the documentation. + + + diff --git a/docs/installation_cn.md b/docs/install.rst similarity index 52% rename from docs/installation_cn.md rename to docs/install.rst index 030b721..4f6191e 100644 --- a/docs/installation_cn.md +++ b/docs/install.rst @@ -1,57 +1,63 @@ -# 安装 - -[TOC] +============= +安装 +============= -## 安装 PaddlePaddle - -Parakeet 以 PaddlePaddle 作为其后端,因此依赖 PaddlePaddle,值得说明的是 Parakeet 要求 2.0 及以上版本的 PaddlePaddle。你可以通过 pip 安装。如果需要安装支持 gpu 版本的 PaddlePaddle,需要根据环境中的 cuda 和 cudnn 的版本来选择 wheel 包的版本。使用 conda 安装以及源码编译安装的方式请参考 [PaddlePaddle 快速安装](https://www.paddlepaddle.org.cn/install/quick/zh/2.0rc-linux-pip). +安装 PaddlePaddle +------------------- +Parakeet 以 PaddlePaddle 作为其后端,因此依赖 PaddlePaddle,值得说明的是 Parakeet 要求 2.0 及以上版本的 PaddlePaddle。你可以通过 pip 安装。如果需要安装支持 gpu 版本的 PaddlePaddle,需要根据环境中的 cuda 和 cudnn 的版本来选择 wheel 包的版本。使用 conda 安装以及源码编译安装的方式请参考 `PaddlePaddle 快速安装 `_. **gpu 版 PaddlePaddle** -```bash -python -m pip install paddlepaddle-gpu==2.0.0rc0.post101 -f https://paddlepaddle.org.cn/whl/stable.html -python -m pip install paddlepaddle-gpu==2.0.0rc0.post100 -f https://paddlepaddle.org.cn/whl/stable.html -``` +.. code-block:: bash + + python -m pip install paddlepaddle-gpu==2.0.0rc1.post101 -f https://paddlepaddle.org.cn/whl/stable.html + python -m pip install paddlepaddle-gpu==2.0.0rc1.post100 -f https://paddlepaddle.org.cn/whl/stable.html + **cpu 版 PaddlePaddle** -```bash -python -m pip install paddlepaddle==2.0.0rc0 -i https://mirror.baidu.com/pypi/simple -``` +.. code-block:: bash -## 安装 libsndfile + python -m pip install paddlepaddle==2.0.0rc1 -i https://mirror.baidu.com/pypi/simple -因为 Parakeet 的实验中常常会需要用到和音频处理,以及频谱处理相关的功能,所以我们依赖 librosa 和 soundfile 进行音频处理。而 librosa 和 soundfile 依赖一个 C 的库 libsndfile, 因为这不是 python 的包,对于 windows 用户和 mac 用户,使用 pip 安装 soundfile 的时候,libsndfile 也会被安装。如果遇到问题也可以参考 [SoundFile](https://pypi.org/project/SoundFile). + +安装 libsndfile +------------------- + +因为 Parakeet 的实验中常常会需要用到和音频处理,以及频谱处理相关的功能,所以我们依赖 librosa 和 soundfile 进行音频处理。而 librosa 和 soundfile 依赖一个 C 的库 libsndfile, 因为这不是 python 的包,对于 windows 用户和 mac 用户,使用 pip 安装 soundfile 的时候,libsndfile 也会被安装。如果遇到问题也可以参考 `SoundFile `_. 对于 linux 用户,需要使用系统的包管理器安装这个包,常见发行版上的命令参考如下。 -```bash -# ubuntu, debian -sudo apt-get install libsndfile1 +.. code-block:: -# centos, fedora, -sudo yum install libsndfile + # ubuntu, debian + sudo apt-get install libsndfile1 -# openSUSE -sudo zypper in libsndfile -``` + # centos, fedora, + sudo yum install libsndfile -## 安装 Parakeet + # openSUSE + sudo zypper in libsndfile +安装 Parakeet +------------------ + 我们提供两种方式来使用 Parakeet. -1. 需要运行 Parakeet 自带的实验代码,或者希望进行二次开发的用户,可以先从 github 克隆本工程,cd 仅工程目录,并进行可编辑式安装(不会被复制到 site-packages, 而且对工程的修改会立即生效,不需要重新安装),之后就可以使用了。 +#. 需要运行 Parakeet 自带的实验代码,或者希望进行二次开发的用户,可以先从 github 克隆本工程,cd 仅工程目录,并进行可编辑式安装(不会被复制到 site-packages, 而且对工程的修改会立即生效,不需要重新安装),之后就可以使用了。 - ```bash - # -e 表示可编辑式安装 - pip install -e . - ``` + .. code-block:: bash -2. 仅需要使用我们提供的训练好的模型进行预测,那么也可以直接安装 pypi 上的 wheel 包的版本。 + # -e 表示可编辑式安装 + pip install -e . + + +#. 仅需要使用我们提供的训练好的模型进行预测,那么也可以直接安装 pypi 上的 wheel 包的版本。 + + .. code-block:: bash + + pip install paddle-parakeet - ```bash - pip install paddle-parakeet - ``` diff --git a/images/logo-small.png b/images/logo-small.png new file mode 100644 index 0000000..fa2b7f3 Binary files /dev/null and b/images/logo-small.png differ diff --git a/parakeet/data/__init__.py b/parakeet/data/__init__.py index 26a6c7f..31304ab 100644 --- a/parakeet/data/__init__.py +++ b/parakeet/data/__init__.py @@ -1,3 +1,5 @@ +"""Parakeet's infrastructure for data processing. +""" # Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License");