[FFmpeg-devel] [PATCH WIP 0/9] Refactor DNN

[Zhao Zhili]

> On Apr 28, 2024, at 18:34, Guo, Yejun <yejun.guo-at-intel.com at ffmpeg.org> wrote:
> 
>> -----Original Message-----
>> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org <mailto:ffmpeg-devel-bounces at ffmpeg.org>> On Behalf Of Zhao
>> Zhili
>> Sent: Sunday, April 28, 2024 12:42 AM
>> To: ffmpeg-devel at ffmpeg.org <mailto:ffmpeg-devel at ffmpeg.org>
>> Cc: Zhao Zhili <zhilizhao at tencent.com <mailto:zhilizhao at tencent.com>>
>> Subject: [FFmpeg-devel] [PATCH WIP 0/9] Refactor DNN
>> 
>> From: Zhao Zhili <zhilizhao at tencent.com>
>> 
>> During the refactor progress, I have found some serious issues, which is not
>> resolved by the patchset:
>> 
>> 1. Tensorflow backend is broken.
>> 
>> I think it doesn't work since 2021 at least. For example, it destroy a thread and
>> create a new thread for each frame, and it destroy an invalid thread at the first
>> frame:
> 
> It works from the day that code is merged, till today. It is by design to keep the
> code simplicity by using the feature that pthread_join accepts a parameter that
> is not a joinable thread.
> 
> Please share more info if you experienced a real case that it does not work.

It will abort if ASSERT_LEVEL > 1.

#define ASSERT_PTHREAD_ABORT(func, ret) do {                            \
    char errbuf[AV_ERROR_MAX_STRING_SIZE] = "";                         \
    av_log(NULL, AV_LOG_FATAL, AV_STRINGIFY(func)                       \
           " failed with error: %s\n",                                  \
           av_make_error_string(errbuf, AV_ERROR_MAX_STRING_SIZE,       \
                                AVERROR(ret)));                         \
    abort();                                                            \
} while (0)

I think the check is there just to prevent call pthread_join(0, &ptr) by accident, so we
shouldn’t do that on purpose.

>> 
>> 
>>    pthread_join(async_module->thread_id, &status);
>>    if (status == DNN_ASYNC_FAIL) {
>>        av_log(ctx, AV_LOG_ERROR, "Unable to start inference as previous
>> inference failed.\n");
>>        return DNN_GENERIC_ERROR;
>>    }
>>    ret = pthread_create(&async_module->thread_id, NULL,
>> async_thread_routine, async_module);
>> 
>> 
>> 2. Openvino V1 doesn't compile. It doesn't compile and no one complains, I
>> think it's a hint to just keep the code for V2.
> 
> In plan, and patch is welcome.
> 
>> 
>> 3. Error handling. It's easy to crash with incorrect command line arguments.
> 
> Thanks, will review your patchset one by one, it may take some time.
> 
>> 
>> I don't have enough test case. Please share your test case and help on test.
>> 
>> Zhao Zhili (9):
>>  avfilter/dnn: Refactor DNN parameter configuration system
>>  avfilter/dnn_backend_openvino: Fix free context at random place
>>  avfilter/dnn_backend_openvino: simplify memory allocation
>>  avfilter/dnn_backend_tf: Remove one level of indentation
>>  avfilter/dnn_backend_tf: Fix free context at random place
>>  avfilter/dnn_backend_tf: Simplify memory allocation
>>  avfilter/dnn_backend_torch: Simplify memory allocation
>>  avfilter/dnn: Remove a level of dereference
>>  avfilter/dnn: Use dnn_backend_info_list to search for dnn module
>> 
>> libavfilter/dnn/dnn_backend_common.h   |  13 +-
>> libavfilter/dnn/dnn_backend_openvino.c | 210 ++++++++++---------------
>> libavfilter/dnn/dnn_backend_tf.c       | 194 ++++++++++-------------
>> libavfilter/dnn/dnn_backend_torch.cpp  | 112 +++++--------
>> libavfilter/dnn/dnn_interface.c        | 107 ++++++++++---
>> libavfilter/dnn_filter_common.c        |  38 ++++-
>> libavfilter/dnn_filter_common.h        |  37 ++---
>> libavfilter/dnn_interface.h            |  73 +++++++--
>> libavfilter/vf_derain.c                |   5 +-
>> libavfilter/vf_dnn_classify.c          |   3 +-
>> libavfilter/vf_dnn_detect.c            |   3 +-
>> libavfilter/vf_dnn_processing.c        |   3 +-
>> libavfilter/vf_sr.c                    |   5 +-
>> 13 files changed, 428 insertions(+), 375 deletions(-)
>> 
>> --
>> 2.34.1
>> 
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel at ffmpeg.org <mailto:ffmpeg-devel at ffmpeg.org>
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> 
>> To unsubscribe, visit link above, or email ffmpeg-devel-request at ffmpeg.org <mailto:ffmpeg-devel-request at ffmpeg.org>
>> with subject "unsubscribe".
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org <mailto:ffmpeg-devel at ffmpeg.org>
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org <mailto:ffmpeg-devel-request at ffmpeg.org> with subject "unsubscribe".