[FFmpeg-devel] [GSOC] [PATCH] TensorFlow backend introduction for DNN module

[Guo, Yejun]

Did you try to build ffmpeg with TENSORFLOW_BACKEND enabled, and run it without TF library?  This case is possible when an end user install pre-built package on a machine without TF library.

In function init, the logic is to fall back to cpu path (DNN_NATIVE) if unable to load tensorflow backend. While in function ff_get_dnn_module, it has no chance to 'return NULL'.

static av_cold int init(AVFilterContext* context)
{
    SRCNNContext* srcnn_context = context->priv;

#ifdef TENSORFLOW_BACKEND
    srcnn_context->dnn_module = ff_get_dnn_module(DNN_TF);
    if (!srcnn_context->dnn_module){
        av_log(context, AV_LOG_INFO, "could not load tensorflow backend, using native backend instead\n");
        srcnn_context->dnn_module = ff_get_dnn_module(DNN_NATIVE);
    }
#else
    srcnn_context->dnn_module = ff_get_dnn_module(DNN_NATIVE);
#endif



DNNModule* ff_get_dnn_module(DNNBackendType backend_type)
{
...
    case DNN_TF:
    #ifdef TENSORFLOW_BACKEND
        //add a runtime check here, possible?
        dnn_module->load_model = &ff_dnn_load_model_tf;
        dnn_module->load_default_model = &ff_dnn_load_default_model_tf;
        dnn_module->execute_model = &ff_dnn_execute_model_tf;
        dnn_module->free_model = &ff_dnn_free_model_tf;
    #else
        av_freep(dnn_module);
        return NULL;
    #endif



Thanks
Yejun (郭叶军)

-----Original Message-----
From: ffmpeg-devel [mailto:ffmpeg-devel-bounces at ffmpeg.org] On Behalf Of Sergey Lavrushkin
Sent: Thursday, May 31, 2018 11:01 PM
To: FFmpeg development discussions and patches <ffmpeg-devel at ffmpeg.org>
Cc: Pedro Arthur <bygrandao at gmail.com>
Subject: [FFmpeg-devel] [GSOC] [PATCH] TensorFlow backend introduction for DNN module

Hello,

This patch introduces TensorFlow backend for DNN inference module.
This backend uses TensorFlow binary models and requires from model to have the operation named 'x' as an input operation and the operation named 'y' as an output operation. Models are executed using libtensorflow.