From 17e5df1de3301896e669e478e55abaca02d0389d Mon Sep 17 00:00:00 2001 From: LDOUBLEV Date: Sun, 20 Sep 2020 02:19:21 +0800 Subject: [PATCH] mv en.doc to ./doc_en --- doc/doc_ch/benchmark_en.md | 56 ----------------------------- doc/doc_en/benchmark_en.md | 74 ++++++++++++++++++++++++-------------- 2 files changed, 47 insertions(+), 83 deletions(-) delete mode 100644 doc/doc_ch/benchmark_en.md diff --git a/doc/doc_ch/benchmark_en.md b/doc/doc_ch/benchmark_en.md deleted file mode 100644 index 1e8c31df..00000000 --- a/doc/doc_ch/benchmark_en.md +++ /dev/null @@ -1,56 +0,0 @@ -# BENCHMARK - -This document gives the performance of the series models for Chinese and English recognition. - -## TEST DATA - -We collected 300 images for different real application scenarios to evaluate the overall OCR system, including contract samples, license plates, nameplates, train tickets, test sheets, forms, certificates, street view images, business cards, digital meter, etc. The following figure shows some images of the test set. - -
- -
- -## MEASUREMENT - -Explanation: -- v1.0 indicates DB+CRNN models without the strategies. v1.1 indicates the PP-OCR models with the strategies and the direction classify. slim_v1.1 indicates the PP-OCR models with prunner or quantization. - -- The long size of the input for the text detector is 960. - -- The evaluation time-consuming stage is the complete stage from image input to result output, including image pre-processing and post-processing. - -- ```Intel Xeon 6148``` is the server-side CPU model. Intel MKL-DNN is used in the test to accelerate the CPU prediction speed. - -- ```Snapdragon 855``` is a mobile processing platform model. - -Compares the model size and F-score: - -| Model Name | Model Size
of the
Whole System\(M\) | Model Size
of the Text
Detector\(M\) | Model Size
of the Direction
Classifier\(M\) | Model Size
of the Text
Recognizer \(M\) | F\-score | -|:-:|:-:|:-:|:-:|:-:|:-:| -| ch\_ppocr\_mobile\_v1\.1 | 8\.1 | 2\.6 | 0\.9 | 4\.6 | 0\.5193 | -| ch\_ppocr\_server\_v1\.1 | 155\.1 | 47\.2 | 0\.9 | 107 | 0\.5414 | -| ch\_ppocr\_mobile\_v1\.0 | 8\.6 | 4\.1 | \- | 4\.5 | 0\.393 | -| ch\_ppocr\_server\_v1\.0 | 203\.8 | 98\.5 | \- | 105\.3 | 0\.4436 | - -Compares the time-consuming on T4 GPU (ms): - -| Model Name | Overall | Text Detector | Direction Classifier | Text Recognizer | -|:-:|:-:|:-:|:-:|:-:| -| ch\_ppocr\_mobile\_v1\.1 | 137 | 35 | 24 | 78 | -| ch\_ppocr\_server\_v1\.1 | 204 | 39 | 25 | 140 | -| ch\_ppocr\_mobile\_v1\.0 | 117 | 41 | \- | 76 | -| ch\_ppocr\_server\_v1\.0 | 199 | 52 | \- | 147 | - -Compares the time-consuming on CPU (ms): - -| Model Name | Overall | Text Detector | Direction Classifier | Text Recognizer | -|:-:|:-:|:-:|:-:|:-:| -| ch\_ppocr\_mobile\_v1\.1 | 421 | 164 | 51 | 206 | -| ch\_ppocr\_mobile\_v1\.0 | 398 | 219 | \- | 179 | - -Compares the model size, F-score, the time-consuming on SD 855 of between the slim models and the original models: - -| Model Name | Model Size
of the
Whole System\(M\) | Model Size
of the Text
Detector\(M\) | Model Size
of the Direction
Classifier\(M\) | Model Size
of the Text
Recognizer \(M\) | F\-score | SD 855
\(ms\) | -|:-:|:-:|:-:|:-:|:-:|:-:|:-:| -| ch\_ppocr\_mobile\_v1\.1 | 8\.1 | 2\.6 | 0\.9 | 4\.6 | 0\.5193 | 306 | -| ch\_ppocr\_mobile\_slim\_v1\.1 | 3\.5 | 1\.4 | 0\.5 | 1\.6 | 0\.521 | 268 | diff --git a/doc/doc_en/benchmark_en.md b/doc/doc_en/benchmark_en.md index 9e2dadb1..1e8c31df 100644 --- a/doc/doc_en/benchmark_en.md +++ b/doc/doc_en/benchmark_en.md @@ -1,36 +1,56 @@ # BENCHMARK -This document gives the prediction time-consuming benchmark of PaddleOCR Ultra Lightweight Chinese Model (8.6M) on each platform. +This document gives the performance of the series models for Chinese and English recognition. ## TEST DATA -* 500 images were randomly sampled from the Chinese public data set [ICDAR2017-RCTW](https://github.com/PaddlePaddle/PaddleOCR/blob/develop/doc/doc_ch/datasets.md#ICDAR2017-RCTW-17). - Most of the pictures in the set were collected in the wild through mobile phone cameras. - Some are screenshots. - These pictures show various scenes, including street scenes, posters, menus, indoor scenes and screenshots of mobile applications. + +We collected 300 images for different real application scenarios to evaluate the overall OCR system, including contract samples, license plates, nameplates, train tickets, test sheets, forms, certificates, street view images, business cards, digital meter, etc. The following figure shows some images of the test set. + +
+ +
## MEASUREMENT -The predicted time-consuming indicators on the four platforms are as follows: - -| Long size(px) | T4(s) | V100(s) | Intel Xeon 6148(s) | Snapdragon 855(s) | -| :---------: | :-----: | :-------: | :------------------: | :-----------------: | -| 960 | 0.092 | 0.057 | 0.319 | 0.354 | -| 640 | 0.067 | 0.045 | 0.198 | 0.236 | -| 480 | 0.057 | 0.043 | 0.151 | 0.175 | Explanation: -* The evaluation time-consuming stage is the complete stage from image input to result output, including image -pre-processing and post-processing. -* ```Intel Xeon 6148``` is the server-side CPU model. Intel MKL-DNN is used in the test to accelerate the CPU prediction speed. -To use this operation, you need to: - * Update to the latest version of PaddlePaddle: https://www.paddlepaddle.org.cn/documentation/docs/zh/install/Tables.html#whl-dev - Please select the corresponding mkl version wheel package according to the CUDA version and Python version of your environment, - for example, CUDA10, Python3.7 environment, you should: +- v1.0 indicates DB+CRNN models without the strategies. v1.1 indicates the PP-OCR models with the strategies and the direction classify. slim_v1.1 indicates the PP-OCR models with prunner or quantization. - ``` - # Obtain the installation package - wget https://paddle-wheel.bj.bcebos.com/0.0.0-gpu-cuda10-cudnn7-mkl/paddlepaddle_gpu-0.0.0-cp37-cp37m-linux_x86_64.whl - # Installation - pip3.7 install paddlepaddle_gpu-0.0.0-cp37-cp37m-linux_x86_64.whl - ``` - * Use parameters ```--enable_mkldnn True``` to turn on the acceleration switch when making predictions -* ```Snapdragon 855``` is a mobile processing platform model. +- The long size of the input for the text detector is 960. + +- The evaluation time-consuming stage is the complete stage from image input to result output, including image pre-processing and post-processing. + +- ```Intel Xeon 6148``` is the server-side CPU model. Intel MKL-DNN is used in the test to accelerate the CPU prediction speed. + +- ```Snapdragon 855``` is a mobile processing platform model. + +Compares the model size and F-score: + +| Model Name | Model Size
of the
Whole System\(M\) | Model Size
of the Text
Detector\(M\) | Model Size
of the Direction
Classifier\(M\) | Model Size
of the Text
Recognizer \(M\) | F\-score | +|:-:|:-:|:-:|:-:|:-:|:-:| +| ch\_ppocr\_mobile\_v1\.1 | 8\.1 | 2\.6 | 0\.9 | 4\.6 | 0\.5193 | +| ch\_ppocr\_server\_v1\.1 | 155\.1 | 47\.2 | 0\.9 | 107 | 0\.5414 | +| ch\_ppocr\_mobile\_v1\.0 | 8\.6 | 4\.1 | \- | 4\.5 | 0\.393 | +| ch\_ppocr\_server\_v1\.0 | 203\.8 | 98\.5 | \- | 105\.3 | 0\.4436 | + +Compares the time-consuming on T4 GPU (ms): + +| Model Name | Overall | Text Detector | Direction Classifier | Text Recognizer | +|:-:|:-:|:-:|:-:|:-:| +| ch\_ppocr\_mobile\_v1\.1 | 137 | 35 | 24 | 78 | +| ch\_ppocr\_server\_v1\.1 | 204 | 39 | 25 | 140 | +| ch\_ppocr\_mobile\_v1\.0 | 117 | 41 | \- | 76 | +| ch\_ppocr\_server\_v1\.0 | 199 | 52 | \- | 147 | + +Compares the time-consuming on CPU (ms): + +| Model Name | Overall | Text Detector | Direction Classifier | Text Recognizer | +|:-:|:-:|:-:|:-:|:-:| +| ch\_ppocr\_mobile\_v1\.1 | 421 | 164 | 51 | 206 | +| ch\_ppocr\_mobile\_v1\.0 | 398 | 219 | \- | 179 | + +Compares the model size, F-score, the time-consuming on SD 855 of between the slim models and the original models: + +| Model Name | Model Size
of the
Whole System\(M\) | Model Size
of the Text
Detector\(M\) | Model Size
of the Direction
Classifier\(M\) | Model Size
of the Text
Recognizer \(M\) | F\-score | SD 855
\(ms\) | +|:-:|:-:|:-:|:-:|:-:|:-:|:-:| +| ch\_ppocr\_mobile\_v1\.1 | 8\.1 | 2\.6 | 0\.9 | 4\.6 | 0\.5193 | 306 | +| ch\_ppocr\_mobile\_slim\_v1\.1 | 3\.5 | 1\.4 | 0\.5 | 1\.6 | 0\.521 | 268 |