[FFmpeg-devel] [RFC] Moving ffprobe's writers to lavu

[Nicolas George]

Jan Ekström (12020-04-11):
> I think this all depends on how much of a use there is for this sort of thing.
> 
> Just looking at the case of wanting to output data from a filter, I
> would default to the opinion that it would make sense to have these
> writers (or at least some of them) common to the fftools tooling. Then
> common code in the API clients could be written that would utilize the
> metadata or side data that these filters generate.

The way I see it, code common to fftools should be specific to the kind
of applications the fftools are: command-line applications with a lot of
options.

Writing JSON is not specific to command-line applications with a lot of
options. Therefore, it belongs in lavu rather than fftools. That also
means it should be designed more cleanly, which is a good thing.

> Of course, (unfortunately and without actually looking) I would bet
> right now there would be not so much normalization with these filters'
> output, but it would make much more sense to have a common way of
> pushing data out with the frames (be it side data or metadata), which
> then could be pushed out in a required format by the API client. And
> then an example could be added under doc/examples to show API users
> how to output data produced by filters.

What I didn't write in my first mail is that I have a plan. And what you
just described is more or less the third step in it.

The first step is to have clean code to produce formatted output. This
is what I am discussing now.

The second step, see below.

> If you have a feeling there's a specific need for this to be available
> through the library to all components, I would like to hear the
> reasoning. My default would be that the API client should handle
> dumping the data exported by filters into wherever the API client
> requires (memory, file, JSON, XML).

My reasoning is that some formatters are common enough that its logical
to have them in lavu. Just an example: using lavf's HTTP client to POST
a JSON request; if lavu does not have a JSON formatter, new code or
another library is needed. Which is a waste, since there already is
JSON code in FFmpeg.

Note that, on a closer look, not all writers code from ffprobe deserves
to be imported to lavu. The flat/compact/default outputs are way too
specific to ffprobe, and the structure in sections too.

But at the very least JSON and XML serialization APIs would make a lot
of sense.

Also worth noting: writing a low-level JSON or XML serialization API is
actually very easy. It is just a matter of escaping special characters
and printing delimiters. And the bit about where the output should go, I
already solved it with my AVWriter proposition.

What is more work is to write a parser. I had started on a limited XML
parser, but it was too early, it was not ripe in my mind, and I ended up
bikeshedding myself on the actual design.

The other thing that is harder is to write a good high-level API for
serializing structure data to JSON or XML. And the thing is, the reason
I am proposing it now is: I have cracked it, I have a good solution for
our kind of use.

This is the second step of my plan: a lightweight polymorphic
introspection system.

It sounds scary, but it is not. What it means is that you write a small
bit of code that describes a structure: “there is a field quantizer
which is an integer; there is a field noise which is a float; there is a
field name which is a string”. And then you pass around a pointer to
that description, and the JSON or XML code uses it to serialize the
structure.

The first benefit of this system is that if you have n kinds of
structures and m different serialization formats, you only need n+m
code, not n×m.

The next benefit is that it will allow type-safe introspection. Like
what we do with av_opt_set_something(), but not limited to a few types
and more extensible.

And later, it could allow for reliable de-serialization, with limited
need for escaping.

I know this may seem like a castle on the sand, but we have to realize:
incremental changes lead us to local potential pits: suboptimal, not
really convenient APIs that we keep because it would be a lot of work to
redesign them.

Some times, we need annealing.

The move to refcounted frames was one. The move to use AVFrame in
libavfilter, Anton's “evil plan” was another. My rewrite of lavfi's
internals to be non-recursive was yet another.

Well, I say we need annealing for the way we handle structured data.

And I say I can do it, in a way that does not disrupt the project much.
You can trust me on this or ask for details.

Regards,

-- 
  Nicolas George
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20200414/76c5ff0a/attachment.sig>