[FFmpeg-devel] [PATCH] Add A53 Closed Captions to MPEG header if they are available.

[John P Poet]

On Wed, Jul 5, 2017 at 3:30 PM Marton Balint <cus at passwd.hu> wrote:

>
> On Fri, 9 Jun 2017, Devin Heitmueller wrote:
>
> > Hello Marton,
> >
> > On Tue, Jun 6, 2017 at 5:45 PM, Marton Balint <cus at passwd.hu> wrote:
> >
> >> As far as I remember multiple side data of the same type is not
> something we
> >> wanted to support. Why do you need it? Can't a single
> AV_FRAME_DATA_A53_CC
> >> side data packet contain many CC entries?
> >
> > Could you please expand on where this was discussed?  Is there any
> > design documentation for side data infrastructure that's been
> > introduced into ffmpeg?  Is there some list of other known design
> > constraints/limitations?
> >
> > While I agree it would be great to simply say that you should never
> > have multiple side data items of the same type, I'm not sure how
> > realistic that is.  It would be helpful if I could better understand
> > the rationale in that thinking.
>
> I guess there are two reasons for prohibiting same type side data:
>
> - existing API implicitly gave the impression that is is not supported
>
> Av_frame_get_side_data gives you no possibility to retrieve multiple side
> data of the same type. API users might assumed that the reason for that is
> because it is not possible/permitted.
>
> - consistency with packet/stream side data types
>
> For stream side data is is already explicitly disallowed, API would be
> inconsistent if multiple side data entries of the same types were possible
> for one kind of side data (frame), but not for the other (stream).
>
> > I'm starting a rather large project which will likely stretch the
> > design for side data in order to support a variety of other ancillary
> > data protocols (e.g. SCTE-104, SMPTE 2038, etc), and it would be great
> > to better understand where the constraints are (so I can either plan
> > to address those issues, or if too significant then choose a different
> > multimedia framework to work with before making a significant
> > investment building out a bunch of features in ffmpeg).
>
> Yeah, you definitely have to think about this, depending on your actual
> goals libavcodec/format might not be the best choice for complex ancillary
> data handling.
>
> It seems multiple side data of the same type is not going to pick up
> support from other developers, so you have to come up with something else
> to move forward.
>

Hi Marton,

If I understand what you and others are saying, AVFrameSideDataType should
not be used to hold "MPEG user data", such as CEA-708, as described here
https://www.atsc.org/wp-content/uploads/2015/03/a_53-Part-4-2009.pdf in
section 6.2.

What I am unclear about, is the proper way to handle MPEG user data.  Are
you saying FFmpeg has no desire to handle this type of data, and that I
need to look to another project?  Or, are you just saying I need to define
a new structure (AVFrameUserDataType or AVFrameMPEGUserDataType) for this
type of data?

Thank you,

John