[FFmpeg-devel] [crop support for matroska demuxer, V3 1/3] libavcodec: Add crop related fields to structure AVCodecContext and AVCodecParameters.

[James Almer]

On 10/11/2022 9:43 AM, Timo Rothenpieler wrote:
> 
> 
> On 07/10/2022 16:59, OvchinnikovDmitrii wrote:
>> ---
>>   libavcodec/avcodec.h       | 35 +++++++++++++++++++++++++++++++++++
>>   libavcodec/codec_par.h     |  8 ++++++++
>>   libavcodec/options_table.h |  4 ++++
>>   3 files changed, 47 insertions(+)
>>
>> diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
>> index 7365eb5cc0..d28a7cc022 100644
>> --- a/libavcodec/avcodec.h
>> +++ b/libavcodec/avcodec.h
>> @@ -380,6 +380,19 @@ typedef struct RcOverride{
>>    */
>>   #define AV_GET_ENCODE_BUFFER_FLAG_REF (1 << 0)
>> +/**
>> +* Video decoding only. Certain container support cropping, meaning that
>> +* only a sub-rectangle of the decoded frame is intended for display.
>> +* Certain codec supports cropping as well.This option controls how
>> +* cropping is handled by libavcodec when  container cropping and
>> +* codec cropping exist.
>> +*/
>> +enum CONTAINER_CROPPING_POLICY_TYPE {
>> +    FF_CONTAINER_CROPPING_IGNORE = 0,
>> +    FF_CONTAINER_CROPPING_ADDITION,
>> +    FF_CONTAINER_CROPPING_OVERWRITE
>> +};
>> +
>>   struct AVCodecInternal;
>>   /**
>> @@ -2057,6 +2070,28 @@ typedef struct AVCodecContext {
>>        *             The decoder can then override during decoding as 
>> needed.
>>        */
>>       AVChannelLayout ch_layout;
>> +
>> +    /* When set to 1 (the default), libavcodec will apply container 
>> cropping
>> +     * to codec cropping additionally.
>> +     *
>> +     * When set to 2, libavcodec will use container cropping to 
>> overwrite
>> +     * codec cropping (the final cropping uses container cropping 
>> parameters)
>> +     *
>> +     * When set to 0, libavcodec will ignore container cropping 
>> parameters
>> +     * (the final cropping uses codec cropping parameters)
>> +     *
>> +     * This field works with "apply_cropping". Only if apply_cropping 
>> is 1, this
>> +     * field works
>> +     */
>> +    enum CONTAINER_CROPPING_POLICY_TYPE container_apply_cropping;
>> +
>> +    /**
>> +     * The cropping parameters from container.
>> +     */
>> +    int container_crop_top;
>> +    int container_crop_left;
>> +    int container_crop_bottom;
>> +    int container_crop_right;
>>   } AVCodecContext;
>>   /**
>> diff --git a/libavcodec/codec_par.h b/libavcodec/codec_par.h
>> index 7660791a12..12d8ceb521 100644
>> --- a/libavcodec/codec_par.h
>> +++ b/libavcodec/codec_par.h
>> @@ -210,6 +210,14 @@ typedef struct AVCodecParameters {
>>        * Audio only. The channel layout and number of channels.
>>        */
>>       AVChannelLayout ch_layout;
>> +
>> +    /**
>> +     * The cropping parameters from container.
>> +     */
>> +    int container_crop_top;
>> +    int container_crop_left;
>> +    int container_crop_bottom;
>> +    int container_crop_right;
>>   } AVCodecParameters;
> 
> I really don't like seeing this in avcodec.
> Why does an encoder or decoder need to care about this, given it's 
> container level information?
> 
> Do you plan on implementing cropping support into all the encoders?
> This should be handled via an avfilter, which might get the cropping 
> info from the demuxer via site data or something.

There's logic in decode.c to apply bitstream level cropping (like in 
h264). This could be extended to also apply container level cropping 
exported as packet side data, like you suggested.

How are we handling stream side data, for that matter? Apparently 
injected only on the first output packet, which is then converted into 
frame side data in lavc, and only done by ffplay.c?