[FFmpeg-devel] [PATCH V8 3/3] lavfi: add filter dnn_detect for object detection

[Guo, Yejun]

> -----Original Message-----
> From: Guo, Yejun <yejun.guo at intel.com>
> Sent: 2021年4月13日 13:14
> To: ffmpeg-devel at ffmpeg.org
> Cc: Guo, Yejun <yejun.guo at intel.com>
> Subject: [PATCH V8 3/3] lavfi: add filter dnn_detect for object detection
> 
> Below are the example steps to do object detection:
> 
> 1. download and install l_openvino_toolkit_p_2021.1.110.tgz from
> https://software.intel.com/content/www/us/en/develop/tools/openvino-to
> olkit/download.html
>   or, we can get source code (tag 2021.1), build and install.
> 2. export LD_LIBRARY_PATH with openvino settings, for example:
> .../deployment_tools/inference_engine/lib/intel64/:.../deployment_tools/i
> nference_engine/external/tbb/lib/
> 3. rebuild ffmpeg from source code with configure option:
> --enable-libopenvino
> --extra-cflags='-I.../deployment_tools/inference_engine/include/'
> --extra-ldflags='-L.../deployment_tools/inference_engine/lib/intel64'
> 4. download model files and test image
> wget
> https://github.com/guoyejun/ffmpeg_dnn/raw/main/models/openvino/20
> 21.1/face-detection-adas-0001.bin
> wget
> https://github.com/guoyejun/ffmpeg_dnn/raw/main/models/openvino/20
> 21.1/face-detection-adas-0001.xml
> wget
> https://github.com/guoyejun/ffmpeg_dnn/raw/main/models/openvino/20
> 21.1/face-detection-adas-0001.label
> wget https://github.com/guoyejun/ffmpeg_dnn/raw/main/images/cici.jpg
> 5. run ffmpeg with:
> ./ffmpeg -i cici.jpg -vf
> dnn_detect=dnn_backend=openvino:model=face-detection-adas-0001.xml:
> input=data:output=detection_out:confidence=0.6:labels=face-detection-ad
> as-0001.label,showinfo -f null -
> 
> We'll see the detect result as below:
> [Parsed_showinfo_1 @ 0x560c21ecbe40]   side data - detection bounding
> boxes:
> [Parsed_showinfo_1 @ 0x560c21ecbe40] source:
> face-detection-adas-0001.xml
> [Parsed_showinfo_1 @ 0x560c21ecbe40] index: 0,  region: (1005, 813) ->
> (1086, 905), label: face, confidence: 10000/10000.
> [Parsed_showinfo_1 @ 0x560c21ecbe40] index: 1,  region: (888, 839) ->
> (967, 926), label: face, confidence: 6917/10000.
> 
> There are two faces detected with confidence 100% and 69.17%.
> 
> Signed-off-by: Guo, Yejun <yejun.guo at intel.com>
> ---
>  configure                   |   1 +
>  doc/filters.texi            |  40 ++++
>  libavfilter/Makefile        |   1 +
>  libavfilter/allfilters.c    |   1 +
>  libavfilter/vf_dnn_detect.c | 421
> ++++++++++++++++++++++++++++++++++++

will push the patch set tomorrow if there's no objection.