[FFmpeg-devel] Adding ICC profile support to MOV decode/encode

[James Almer]

On 9/18/2019 7:27 AM, Hello Vectronic wrote:
> 
> 
>> On 18 Sep 2019, at 11:09, Hendrik Leppkes <h.leppkes at gmail.com> wrote:
>>
>> On Wed, Sep 18, 2019 at 11:37 AM Hello Vectronic
>> <hello.vectronic at gmail.com> wrote:
>>>
>>> Hello,
>>>
>>> I need to implement support for reading and writing ICC profiles which can be stored in MOV/MP4 sample descriptor colour information.
>>>
>>> The relevant extract from the ISO standard is:
>>>
>>> class ColourInformationBox extends Box('colr'){
>>>    unsigned int(32) colour_type;
>>>    if (colour_type == 'nclx')  /* on-screen colours */
>>>    {
>>>        unsigned int(16) colour_primaries;
>>>        unsigned int(16) transfer_characteristics;
>>>        unsigned int(16) matrix_coefficients;
>>>        unsigned int(1)  full_range_flag;
>>>        unsigned int(7)  reserved = 0;
>>>    }
>>>    else if (colour_type == 'rICC')
>>>    {
>>>        ICC_profile;    // restricted ICC profile
>>>    }
>>>    else if (colour_type == 'prof')
>>>    {
>>>        ICC_profile;    // unrestricted ICC profile
>>>    }
>>> }
>>>
>>> At the moment the code only supports nclc/nclx colour type in:
>>>
>>> libavformat/mov.c  => mov_read_colr()
>>> libavformat/moveenc.c =>  mov_write_colr_tag()
>>>
>>> Support for ICC profile is implemented on a per frame basis in AVFrameSideDataType.AV_FRAME_DATA_ICC_PROFILE. This is used by the PNG, WEBP and MJPEG codec implementations. The ICC profile in this scenario is treated as an opaque octet buffer. I don't believe this is relevant to my use case as the colour information within the MOV/MP4 sample descriptor relates to the entire stream and is not used on a per-frame basis.
>>>
>>> My thinking is to implement support for ICC profile in a similar way that color_range, color_primaries etc. are currently handled:
>>>
>>> 1. Store the ICC profile as an opaque octet buffer stored in an AVCodecParameters struct value:
>>>
>>> uint8_t *icc_profile_data;
>>> int     icc_profile_size;
>>>
>>
>> Something like this has no place in AVCodecParameters, which is
>> intentionally kept simple and easy. It should be stream-global
>> side-data, similar to the frame side data, just on a stream level,
>> which is already a concept we have, so it should be easy to fit it in
>> there.
>>
>> - Hendrik
>>
> 
> 
> 
> Thanks for the advice. 
> 
> So it seems I would:
> 
> 1. add a new enum value AVPacketSideDataType.AV_PKT_DATA_ICC_PROFILE
> 
> 2. use av_stream_add_side_data() when reading from the MOV/MP4 in libavformat/mov.c => mov_read_colr()
> 
> 3. use av_stream_get_side_data() when writing to the MOV/MP4 in libavformat/moveenc.c => mov_write_colr_tag()
> 
> And I think that is it..?

You may also need to add the new packet side data type to the list in
libavcodec/decode.c:ff_decode_frame_props(), so the icc profiles are
attached to decoded frames. But otherwise, yes, that should be about it.