[FFmpeg-devel] [PATCH V2 6/6] lavfi/dnn_classify: add filter dnn_classify for classification based on detection bounding boxes

[Guo, Yejun]

> -----Original Message-----
> From: Guo, Yejun <yejun.guo at intel.com>
> Sent: 2021年4月29日 21:37
> To: ffmpeg-devel at ffmpeg.org
> Cc: Guo, Yejun <yejun.guo at intel.com>
> Subject: [PATCH V2 6/6] lavfi/dnn_classify: add filter dnn_classify for
> classification based on detection bounding boxes
> 
> classification is done on every detection bounding box in frame's side data,
> which are the results of object detection (filter dnn_detect).
> 
> Please refer to commit log of dnn_detect for the material for detection,
> and see below for classification.
> 
> - download material for classifcation:
> wget
> https://github.com/guoyejun/ffmpeg_dnn/raw/main/models/openvino/202
> 1.1/emotions-recognition-retail-0003.bin
> wget
> https://github.com/guoyejun/ffmpeg_dnn/raw/main/models/openvino/202
> 1.1/emotions-recognition-retail-0003.xml
> wget
> https://github.com/guoyejun/ffmpeg_dnn/raw/main/models/openvino/202
> 1.1/emotions-recognition-retail-0003.label
> 
> - run command as:
> ./ffmpeg -i cici.jpg -vf
> dnn_detect=dnn_backend=openvino:model=face-detection-adas-0001.xml:in
> put=data:output=detection_out:confidence=0.6:labels=face-detection-adas-0
> 001.label,dnn_classify=dnn_backend=openvino:model=emotions-recognition-
> retail-0003.xml:input=data:output=prob_emotion:confidence=0.3:labels=em
> otions-recognition-retail-0003.label:target=face,showinfo -f null -
> 
> We'll see the detect&classify result as below:
> [Parsed_showinfo_2 @ 0x55b7d25e77c0]   side data - detection bounding
> boxes:
> [Parsed_showinfo_2 @ 0x55b7d25e77c0] source:
> face-detection-adas-0001.xml, emotions-recognition-retail-0003.xml
> [Parsed_showinfo_2 @ 0x55b7d25e77c0] index: 0,  region: (1005, 813) ->
> (1086, 905), label: face, confidence: 10000/10000.
> [Parsed_showinfo_2 @ 0x55b7d25e77c0]            classify:  label:
> happy, confidence: 6757/10000.
> [Parsed_showinfo_2 @ 0x55b7d25e77c0] index: 1,  region: (888, 839) ->
> (967, 926), label: face, confidence: 6917/10000.
> [Parsed_showinfo_2 @ 0x55b7d25e77c0]            classify:  label: anger,
> confidence: 4320/10000.
> 
> Signed-off-by: Guo, Yejun <yejun.guo at intel.com>
> ---
> the main change of V2 in this patch set is to rebase with latest code
> by resolving the conflicts.
> 
>  configure                     |   1 +
>  doc/filters.texi              |  39 ++++
>  libavfilter/Makefile          |   1 +
>  libavfilter/allfilters.c      |   1 +
>  libavfilter/vf_dnn_classify.c | 330
> ++++++++++++++++++++++++++++++++++
>  5 files changed, 372 insertions(+)
>  create mode 100644 libavfilter/vf_dnn_classify.c
> 
will push tomorrow if there's no objection, thanks.