[FFmpeg-devel] [PATCH] ffmpeg: set user-set rotation for encoded streams too

[Jan Ekström]

On Sun, May 15, 2022 at 2:58 PM Gyan Doshi <ffmpeg at gyani.pro> wrote:
>
>
>
> On 2022-05-15 04:29 pm, Jan Ekström wrote:
> > On Thu, May 12, 2022 at 11:19 AM Anton Khirnov <anton at khirnov.net> wrote:
> >> Quoting Gyan Doshi (2022-05-12 09:01:31)
> >>> A substitute option for metadata rotate would accept a single rotation
> >>> value.
> >>> If adding an option for matrix, we should allow flips too.
> >> I see no reason not to have both. You can have a simple option for just
> >> the rotation and an advanced option for specifying full matrix.
> >>
> > FYI I had thought about arguments-to-side data earlier, and as I saw
> > this thread I decided to make a proof-of-concept that has one layer
> > which constructs an AVBufferRef of the side data itself based on an
> > AVDictionary (constructor, basically), and then another which then
> > goes over type-arguments sort of dictionary and attaches things (to an
> > AVFrame in this case). AVPacket and AVFrame side data is slightly
> > different, so I guess you'd have to split one layer which generates an
> > AVBufferRef, and then that'd be utilized (discarded or not) by the
> > utilizing function.
> >
> > I do wonder if AVOptions wouldn't be better for something like this,
> > but at least it lets you play around with the idea. My initial API
> > user ended up being the setparams filter.
> >
> > Link to PoC branch:
> > https://github.com/jeeb/ffmpeg/commits/arguments_to_side_data
> >
> > Usage example:
> > ffmpeg -v verbose -i INPUT -map 0:v:0 -vf
> > setparams=side_data=displaymatrix=angle=34.34,showinfo -vframes 1 -f
> > null -
>
> The metadata hack uses avutil/display API to set the matrix and then
> attach it to the *stream* as side data  A filter will force encoding.
> Since this is usability/display info for the consumer , it should remain
> available for streamcopy.

Yes, that is why I noted that there probably needs to be a separation
of APIs between the AVBufferRef and the "final" one. For re-encoding a
video filter would lead to the first AVFrame having it, and then that
could be utilized in the encoder initialization - just like my changes
pushing the video encoding later enabled color space information to be
passed on. For stream copy, it should be set either to the AVPacket(s)
or AVStream, yes. Anton talked about a bit stream filter but I have
not looked into that side of things.

This was just a proof-of-concept so people can see how string
arguments to side data might look, since IMHO we are badly lacking
such an interface (while the C APIs might be OK, we are clearly
lacking stuff to do string based object generation, which is more or
less required for CLI etc usage).

Jan