[FFmpeg-devel] FFmpeg A/V filtering questions

[Stefano Sabatini]

On date Monday 2012-02-13 16:15:12 +0100, Clément Bœsch encoded:
> Hi,
> 
> I have a few questions about this part of the code in ffmpeg.c (around L2140):
> 
>     avcodec_get_frame_defaults(ist->filtered_frame);
>     filtered_frame = ist->filtered_frame;
>     *filtered_frame= *decoded_frame; //for me_threshold
>     if (ost->picref) {
>         avfilter_fill_frame_from_video_buffer_ref(filtered_frame, ost->picref);
>         filtered_frame->pts = av_rescale_q(ost->picref->pts, ist_pts_tb, AV_TIME_BASE_Q);
>     }
>     if (ost->picref->video && !ost->frame_aspect_ratio)
>         ost->st->codec->sample_aspect_ratio = ost->picref->video->sample_aspect_ratio;
>     do_video_out(output_files[ost->file_index].ctx, ost, ist, filtered_frame);
> 

> 1) how is *filtered_frame = *decoded_frame (only?) related to me_threshold?
> 
> 2) what is the point of getting the frame defaults if they are overwritten just
>    after that (first 3 lines)?
> 
> 3) ost->picref can't be changed in the "if (ost->picref)" scope, so it looks
>    like to me that "if (ost->picref->video)" will leads to a sigsegv if
>    ost->picref was NULL in the first place. Since no such bug exists at
>    the moment (AFAIK), couldn't we just drop the "if (ost->picref)" block?

Michael already replied to these ones.
 
> 4) why is there an exception for the sample aspect ratio since
>    avfilter_fill_frame_from_video_buffer_ref() will actually do that
>    copy?

    if (ost->picref->video && !ost->frame_aspect_ratio)
        ost->st->codec->sample_aspect_ratio = ost->picref->video->sample_aspect_ratio;

This is copying the sample_aspect_ratio to the stream context if it is
not defined in ost, ost->picref->video->sample_aspect_ratio is
equivalent to filtered_frame->sample_aspect_ratio, also maybe the code
should be moved to do_video_out(), note that I tried to keep the
setting logic unchanged when I introduced
avfilter_fill_frame_from_video_buffer_ref().

> 5) what is the audio equivalent for
>    avfilter_fill_frame_from_video_buffer_ref()? According to
>    libavdevice/lavfi.c, it seems to only need the data memcpy() and a few
>    props copy.

The code in lavfi.c predates the audio-in-AVFrame changes, the right
solution would consist now into adding
avfilter_fill_frame_from_audio_buffer_ref() *or* even better
generalize the function:
avfilter_fill_frame_from_video_buffer_ref -> avfilter_fill_frame_from_buffer_ref

> 6) libavdevice/lavfi.c seems to have a similar code to
>    avfilter_fill_frame_from_video_buffer_ref(); why is it handled differently
>    than in ffmpeg.c?

Different data paths.

ffmpeg.c:transcode_video():
get a buffer ref from the filterchain, convert it to an AVFrame, pass
it to do_video_out(), do_video_out() performs more magic on the frame,
passes it to the encoder and to write_frame() (which performs muxing)

lavfi.c/lavfi_read_packet()
get the buffer ref from the sink buffer, and assemblate a raw video
packet directly from the video data

> If anyone can answer a few questions, that would be welcome :)

HTH
-- 
FFmpeg = Fierce and Fantastic Minimalistic Practical Exuberant Guide