diff --git a/APP_Framework/Applications/knowing_app/helmet_detect/README.md b/APP_Framework/Applications/knowing_app/helmet_detect/README.md new file mode 100644 index 00000000..006427ed --- /dev/null +++ b/APP_Framework/Applications/knowing_app/helmet_detect/README.md @@ -0,0 +1,167 @@ +# Helmet detection demo + +### A helmet and head without helmet object detection task demo. Running MobileNet-yolo on K210-based edge devices. + +--- + +## Training + +### Enviroment preparation + +Model generated by [aXeleRate](https://forgeplus.trustie.net/projects/yangtuo250/aXeleRate) and converted to kmodel by [nncase](https://github.com/kendryte/nncase/tree/v0.1.0-rc5). + +```shell +# master branch for MobileNetv1-yolov2 and unstable branch to test MobileNetv1(v2)-yolov2(v3) +git clone https://git.trustie.net/yangtuo250/aXeleRate.git (-b unstable) +cd aXeleRate +pip install -r requirments.txt && pip install -e . +``` + +### training config setting + +Example [config](https://forgeplus.trustie.net/projects/yangtuo250/aXeleRate/tree/master/configs/detector.json), some hyper-parameters: + +- architecture: backbone, MobileNet7_5 for default, MobileNet1_0(α = 1.0) and above cannot run on K210 because of OOM on feature map in master branch. For unstable branch MobileNetV2_1_0 is OK. + +- input_size: fixed model input size, single integer for height equals to width, otherwise a list([height, width]). +- anchors: yolov2 anchor(for master) or anchor scaled to 1.0(for unstable), can be generate by [darknet](https://github.com/AlexeyAB/darknet). +- labels: labels of all classes. +- train(valid)_image(annot)_folder: path of images and annoations for training and validation. +- saved_folder: path for trainig result storage(models, checkpoints, logs ...). + +Mine config for unstable: +```json +{ + "model": { + "type": "Detector", + "architecture": "MobileNetV2_1_0", + "input_size": [ + 224, + 320 + ], + "anchors": [ + [ + [ + 0.1043, + 0.1560 + ], + [ + 0.0839, + 0.3036 + ], + [ + 0.1109, + 0.3923 + ], + [ + 0.1378, + 0.5244 + ], + [ + 0.2049, + 0.6673 + ] + ] + ], + "labels": [ + "human" + ], + "obj_thresh": 0.5, + "iou_thresh": 0.45, + "coord_scale": 1.0, + "class_scale": 0.0, + "object_scale": 5.0, + "no_object_scale": 3.0 + }, + "weights": { + "full": "", + "backend": "" + }, + "train": { + "actual_epoch": 2000, + "train_image_folder": "mydata/human/Images/train", + "train_annot_folder": "mydata/human/Annotations/train", + "train_times": 2, + "valid_image_folder": "mydata/human/Images/val", + "valid_annot_folder": "mydata/human/Annotations/val", + "valid_times": 1, + "valid_metric": "precision", + "batch_size": 32, + "learning_rate": 2e-5, + "saved_folder": "mydata/human/results", + "first_trainable_layer": "", + "augmentation": true, + "is_only_detect": false, + "validation_freq": 5, + "quantize": false, + "class_weights": [1.0] + }, + "converter": { + "type": [ + "k210" + ] + } +} +``` + +*(For more detailed config usage, please refer to original aXeleRate repo.)* + +### data preparation + +Please refer to [VOC format](https://towardsdatascience.com/coco-data-format-for-object-detection-a4c5eaf518c5), path as config above. + +### train it! + +```shell +python -m aXeleRate.train -c PATH_TO_YOUR_CONFIG +``` + +### model convert + +Please refer to [nncase repo](https://github.com/kendryte/nncase/tree/v0.1.0-rc5). + +--- + +## Deployment + +### compile and burn + +Use `(scons --)menuconfig` in bsp folder *(Ubiquitous/RT_Thread/bsp/k210)*, open: + +- More Drivers --> ov2640 driver +- Board Drivers Config --> Enable LCD on SPI0 +- Board Drivers Config --> Enable SDCARD (spi1(ss0)) +- Board Drivers Config --> Enable DVP(camera) +- RT-Thread Components --> POSIX layer and C standard library --> Enable pthreads APIs +- APP_Framework --> Framework --> support knowing framework --> kpu model postprocessing --> yolov2 region layer +- APP_Framework --> Applications --> knowing app --> enable apps/helmet detect + +`scons -j(n)` to compile and burn in by *kflash*. + +### json config and kmodel + +Copy json config for deployment o SD card */kmodel*. Example config file is *helmet.json* in this directory. Something to be modified: + +- net_input_size: same as *input_size* in training config file, but array only. +- net_output_shape: final feature map size, can be found in **nncase** output. +- sensor_output_size: image height and width from camera, same as *dvp_set_image_size(Ubiquitous/RT_Thread/bsp/k210/base-drivers/drv_dvp.c L70(71))*. +- kmodel_size: kmodel size shown in file system. +- anchors: same as *anchor* in training config file(multi-dimention anchors flatten to 1 dim). +- labels: same as *label* in training config file. +- obj_thresh: array, object threshold of each label. +- nms_thresh: NMS threshold of boxes. + +Copy final kmodel to SD card */kmodel* either. + +--- + +## Run + +In serial terminal, `helmet_detect` to start a detection thread, `helmet_detect_delete` to stop it. Detection results can be found in output. + +--- + +## TODO + +- [ ] Fix LCD real-time result display. +- [ ] Test more object detection backbone and algorithm(like yolox).