[FFmpeg-devel] [PATCH] lavc/nvenc: enable nvenc encoder instance reuse after draining

[Timo Rothenpieler]

On 07.06.2018 06:38, Pavel Koshevoy wrote:
> ---
>  libavcodec/nvenc.c | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c
> index b4186c0bec..8928eacc70 100644
> --- a/libavcodec/nvenc.c
> +++ b/libavcodec/nvenc.c
> @@ -2181,6 +2181,12 @@ int ff_nvenc_receive_packet(AVCodecContext *avctx, AVPacket *pkt)
>  
>          av_fifo_generic_write(ctx->unused_surface_queue, &tmp_out_surf, sizeof(tmp_out_surf), NULL);
>      } else if (ctx->encoder_flushing) {
> +        /* reset to initial state so the encoder can be re-used */
> +        ctx->encoder_flushing = 0;
> +        ctx->first_packet_output = 0;
> +        ctx->initial_pts[0] = AV_NOPTS_VALUE;
> +        ctx->initial_pts[1] = AV_NOPTS_VALUE;
> +        av_fifo_reset(ctx->timestamp_list);
>          return AVERROR_EOF;
>      } else {
>          return AVERROR(EAGAIN);
> 

I'm not sure if the send/receive API intends for an encoder to be
re-used after is has entered EOF state.

If an API user were to rely on getting EOF here repeatedly, and suddenly
getting EAGAIN after a single EOF, it might mess things up.

The only way I'd see to make this work is to introduce another flag, so
it stays in EOF state until more input is given. And I'm not even sure
if that is true to the API.