[FFmpeg-devel] [PATCH 4/4] libavfilter: add filter dnn_detect for object detection

[Guo, Yejun]

> -----Original Message-----
> From: Guo, Yejun <yejun.guo at intel.com>
> Sent: 2021年2月19日 15:00
> To: ffmpeg-devel at ffmpeg.org
> Cc: Guo, Yejun <yejun.guo at intel.com>
> Subject: [PATCH 4/4] libavfilter: add filter dnn_detect for object detection
> 
> Below are the example steps to do object detection:
> 
> 1. download and install l_openvino_toolkit_p_2021.1.110.tgz from
> https://software.intel.com/content/www/us/en/develop/tools/openvino-toolk
> it/download.html
>   or, we can get source code (tag 2021.1), build and install.
> 2. export LD_LIBRARY_PATH with openvino settings, for example:
> .../deployment_tools/inference_engine/lib/intel64/:.../deployment_tools/infer
> ence_engine/external/tbb/lib/
> 3. rebuild ffmpeg from source code with configure option:
> --enable-libopenvino
> --extra-cflags='-I.../deployment_tools/inference_engine/include/'
> --extra-ldflags='-L.../deployment_tools/inference_engine/lib/intel64'
> 4. download model files and test image
> wget
> https://github.com/guoyejun/ffmpeg_dnn/raw/main/models/openvino/2021.
> 1/face-detection-adas-0001.bin
> wget
> https://github.com/guoyejun/ffmpeg_dnn/raw/main/models/openvino/2021.
> 1/face-detection-adas-0001.xml
> wget
> https://github.com/guoyejun/ffmpeg_dnn/raw/main/models/openvino/2021.
> 1/face-detection-adas-0001.label
> wget https://github.com/guoyejun/ffmpeg_dnn/raw/main/images/cici.jpg
> 5. run ffmpeg with:
> ./ffmpeg -i cici.jpg -vf
> dnn_detect=dnn_backend=openvino:model=face-detection-adas-0001.xml:inp
> ut=data:output=detection_out:confidence=0.6:labels=face-detection-adas-000
> 1.label,showinfo -f null -
> 
> We'll see the detect result as below:
> [Parsed_showinfo_1 @ 0x55817e43ce40]   side data - bounding boxes:
> [Parsed_showinfo_1 @ 0x55817e43ce40] source:
> face-detection-adas-0001.xml
> [Parsed_showinfo_1 @ 0x55817e43ce40] index: 0, region: (1005, 813) ->
> (1086, 905), label: face, confidence: 10000/10000.
> [Parsed_showinfo_1 @ 0x55817e43ce40] index: 1, region: (888, 839) -> (967,
> 926), label: face, confidence: 6917/10000.
> 
> There are two faces detected with confidence 100% and 69.17%.
> 
> Signed-off-by: Guo, Yejun <yejun.guo at intel.com>
> ---
>  configure                              |   1 +
>  doc/filters.texi                       |  40 +++
>  libavfilter/Makefile                   |   1 +
>  libavfilter/allfilters.c               |   1 +
>  libavfilter/dnn/dnn_backend_openvino.c |  12 +
>  libavfilter/dnn_filter_common.c        |   7 +
>  libavfilter/dnn_filter_common.h        |   1 +
>  libavfilter/dnn_interface.h            |   6 +-
>  libavfilter/vf_dnn_detect.c            | 425
> +++++++++++++++++++++++++
>  9 files changed, 492 insertions(+), 2 deletions(-)  create mode 100644
> libavfilter/vf_dnn_detect.c
> 
> +static int read_detect_label_file(AVFilterContext *context) {
> +    int line_len;
> +    FILE *file;
> +    DnnDetectContext *ctx = context->priv;
> +
> +    file = av_fopen_utf8(ctx->labels_filename, "r");
> +    if (!file){
> +        av_log(context, AV_LOG_ERROR, "failed to open file %s\n",
> ctx->labels_filename);
> +        return AVERROR(EINVAL);
> +    }
> +
> +    while (!feof(file)) {
> +        char *label;
> +        char buf[256];
> +        fgets(buf, 256, file);

A build warning is caught by patchwork, I could not see the warning log,
and my local machine did not report the warning, I think the warning
should be that the return value of fgets is not checked, will fix and send out V2,
thanks.