[FFmpeg-devel] [PATCH v3 1/5] avutil: add ambient viewing environment metadata side data

[Andreas Rheinhardt]

Lynne:
> 5 Apr 2022, 17:32 by andreas.rheinhardt at outlook.com:
> 
>> lance.lmwang at gmail.com:
>>
>>> From: Limin Wang <lance.lmwang at gmail.com>
>>>
>>> Signed-off-by: Limin Wang <lance.lmwang at gmail.com>
>>> ---
>>>  libavutil/frame.c                      |  1 +
>>>  libavutil/frame.h                      |  6 +++++
>>>  libavutil/mastering_display_metadata.c | 23 +++++++++++++++++
>>>  libavutil/mastering_display_metadata.h | 45 ++++++++++++++++++++++++++++++++++
>>>  libavutil/version.h                    |  2 +-
>>>  5 files changed, 76 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/libavutil/frame.c b/libavutil/frame.c
>>> index fbb869f..8882da2 100644
>>> --- a/libavutil/frame.c
>>> +++ b/libavutil/frame.c
>>> @@ -815,6 +815,7 @@ const char *av_frame_side_data_name(enum AVFrameSideDataType type)
>>>  case AV_FRAME_DATA_DETECTION_BBOXES:            return "Bounding boxes for object detection and classification";
>>>  case AV_FRAME_DATA_DOVI_RPU_BUFFER:             return "Dolby Vision RPU Data";
>>>  case AV_FRAME_DATA_DOVI_METADATA:               return "Dolby Vision Metadata";
>>> +    case AV_FRAME_DATA_AMBIENT_VIEWING_ENV:         return "Ambient Viewing Environment";
>>>  }
>>>  return NULL;
>>>  }
>>> diff --git a/libavutil/frame.h b/libavutil/frame.h
>>> index 33fac20..92413c9 100644
>>> --- a/libavutil/frame.h
>>> +++ b/libavutil/frame.h
>>> @@ -209,6 +209,12 @@ enum AVFrameSideDataType {
>>>  * volume transform - CUVA 005.1-2021.
>>>  */
>>>  AV_FRAME_DATA_DYNAMIC_HDR_VIVID,
>>> +
>>> +    /**
>>> +     * ambient viewing environment for a video frame, as described by
>>> +     * the AVAmbientViewingEnv
>>> +     */
>>> +    AV_FRAME_DATA_AMBIENT_VIEWING_ENV,
>>>  };
>>>  
>>>  enum AVActiveFormatDescription {
>>> diff --git a/libavutil/mastering_display_metadata.c b/libavutil/mastering_display_metadata.c
>>> index 6069347..f094eab 100644
>>> --- a/libavutil/mastering_display_metadata.c
>>> +++ b/libavutil/mastering_display_metadata.c
>>> @@ -64,3 +64,26 @@ AVContentLightMetadata *av_content_light_metadata_create_side_data(AVFrame *fram
>>>  
>>>  return (AVContentLightMetadata *)side_data->data;
>>>  }
>>> +
>>> +AVAmbientViewingEnv *av_ambient_viewing_env_alloc(size_t *size)
>>> +{
>>> +    AVAmbientViewingEnv *metadata = av_mallocz(sizeof(*metadata));
>>> +
>>> +    if (size)
>>> +        *size = sizeof(*metadata);
>>> +
>>> +    return metadata;
>>> +}
>>> +
>>> +AVAmbientViewingEnv *av_ambient_viewing_env_create_side_data(AVFrame *frame)
>>> +{
>>> +    AVFrameSideData *side_data = av_frame_new_side_data(frame,
>>> +            AV_FRAME_DATA_AMBIENT_VIEWING_ENV,
>>> +            sizeof(AVAmbientViewingEnv));
>>> +    if (!side_data)
>>> +        return NULL;
>>> +
>>> +    memset(side_data->data, 0, sizeof(AVAmbientViewingEnv));
>>> +
>>> +    return (AVAmbientViewingEnv *)side_data->data;
>>> +}
>>> diff --git a/libavutil/mastering_display_metadata.h b/libavutil/mastering_display_metadata.h
>>> index c23b07c..c1ba659 100644
>>> --- a/libavutil/mastering_display_metadata.h
>>> +++ b/libavutil/mastering_display_metadata.h
>>> @@ -125,4 +125,49 @@ AVContentLightMetadata *av_content_light_metadata_alloc(size_t *size);
>>>  */
>>>  AVContentLightMetadata *av_content_light_metadata_create_side_data(AVFrame *frame);
>>>  
>>> +/**
>>> + * The characteristics of the nominal ambient viewing environment for
>>> + * the display of the associated video content.
>>> + * To be used as payload of a AVFrameSideData the appropriate type.
>>> + *
>>> + * @note The struct should be allocated with av_ambient_viewing_env_alloc()
>>> + *       and its size is not a part of the public ABI.
>>> + */
>>> +typedef struct AVAmbientViewingEnv {
>>> +    /**
>>> +     * specifies the environmental illuminance of the ambient viewing
>>> +     * environment in units of 0.0001 lux.
>>> +     * ambient_illuminance shall not be equal to 0.
>>> +     */
>>> +    uint32_t ambient_illuminance;
>>> +    /**
>>> +     * specify the normalized x and y chromaticity coordinates, respectively,
>>> +     * of the environmental ambient light in the nominal viewing environment,
>>> +     * according to the CIE 1931 definition of x and y as specified in ISO
>>> +     * 11664-1 (see also ISO 11664-3 and CIE 15), in normalized increments of
>>> +     * 0.00002. The values of ambient_light_x and ambient_light_y shall be in
>>> +     * the range of 0 to 50000
>>> +     */
>>> +    uint16_t ambient_light_x;
>>> +    uint16_t ambient_light_y;
>>> +} AVAmbientViewingEnv;
>>> +
>>> +/**
>>> + * Allocate an AVAmbientViewingEnv structure and set its fields to
>>> + * default values. The resulting struct can be freed using av_freep().
>>> + *
>>> + * @return An AVAmbientViewingEnv filled with default values or NULL
>>> + *         on failure.
>>> + */
>>> +AVAmbientViewingEnv *av_ambient_viewing_env_alloc(size_t *size);
>>> +
>>> +/**
>>> + * Allocate a complete AVAmbientViewingEnv and add it to the frame.
>>> + *
>>> + * @param frame The frame which side data is added to.
>>> + *
>>> + * @return The AVAmbientViewingEnv structure to be filled by caller.
>>> + */
>>> +AVAmbientViewingEnv *av_ambient_viewing_env_create_side_data(AVFrame *frame);
>>>
>>
>> All this boilerplate code for the various side-data--types makes me
>> wonder whether they should not be replaced by something like
>> void *av_frame_side_data_alloc(enum AVFrameSideDataType, size_t *size)
>> and a corresponding av_frame_create_side_data(AVFrame *frame, enum
>> AVFrameSideDataType).
>> These functions would return an error in case it doesn't make sense to
>> create a side data this way (because there is no struct associated to
>> this side data type or because the size of this struct depends upon
>> other parameters, too).
>>
> 
> Why is this side data type allocated, rather than what used to be
> the standard, making the side data struct size const? Copy-paste?
> 
> Are new fields expected to be added in the future? HDR mastering
> metadata is also allocated, but we've never added any fields since
> it was merged.

With "const" you mean make the size part of the ABI? Well, given that
side-data is always allocated, it doesn't really hurt to make the size
of these fields not part of the ABI.
(There are cases where this is not so: sizeof(AVBufferRef) not being
part of the public ABI means that every av_buffer_ref() has an
allocation and therefore needs to be checked. No addition to AVBufferRef
has ever been made.)

- Andreas