[FFmpeg-devel] [PATCH] libavcodec/zmbvenc: Add support for RGB formats

[Tomas Härdin]

ons 2019-03-13 klockan 17:46 +0000 skrev Matthew Fearnley:
> > > > On 12 Mar 2019, at 11:46, Tomas Härdin <tjoppen at acc.umu.se> wrote:
> > 
> > tis 2019-03-12 klockan 10:27 +0000 skrev Matthew Fearnley:
> > > > 
> > > It occurs to me that adding sub-8bpp has some implications:
> > > 
> > > My current understanding (I could be wrong) is that FFmpeg tends to
> > > detect the pix_fmt based on the first frame. If FFmpeg detects the
> > > first frame as e.g. PAL4, and chooses that as its output, that means
> > > the rest of the video will have to be encodable as PAL4, otherwise it
> > > (obviously) won’t be encoded properly.
> > > 
> > > So adding a PAL4 format puts a new constraint on encoders (inside and
> > > outside FFmpeg) to not encode frames in a way that looks like PAL4,
> > > unless the whole video will be encodable that way.
> > 
> > Yes, FFmpeg will probe the initial format of the video and audio.
> > Nothing says these are constant. There are FATE samples specifically
> > for files that change resolution. Since ZMBV is a DOS capture codec,
> > and DOS programs frequently change resolution and colordepth, this is
> > indeed something we have to think about. Example: DOS boots into mode
> > 3h, 80x25 16-color text. A recording may start in this mode, then
> > switch to mode 13h (320x200 256 colors graphics), if I understand the
> > format correctly.
> 
> DOSBox actually avoids this issue by outputting to a new file
> whenever it detects a change in colour depth, dimensions or FPS. So
> in practice, these remain constant over a single video.

Ah, convenient

> I’ve tried stitching two AVI files together (palette + RGB, same
> dimensions), and found that if a palette-capable format like ‘png’ is
> chosen, all of the resulting AVI file is encoded with a palette, and
> RGB sections are dithered.  That possibly suggests FFmpeg doesn’t
> like changing format halfway through videos, for AVI at least?

It probably implies AVI can't change palette, or that FFmpeg's support
for changing palettes is spotty at best

> > > Thanks for writing the email to the DOSBox crew.
> > > If they choose a channel order, then we have good grounds for fixing
> > > the encoder (if need be), and implementing the decoder in the same
> > > way.
> > > It occurs to me that they might (in theory) also want to specify 2/4
> > > byte alignment on RGB, like with the MVs.  My gut says there’d be
> > > very little benefit though, and it would only be seen with strange
> > > video / block widths.
> > > 
> > > It also occurs to me this will may warrant a version bump in the
> > > format, to give an easy error case for decoders that don’t expect it.
> > > Particularly if our decoder has to redefine its channel order.
> > 
> > Are there any decoders besides ours and dosbox's?
> 
> I don’t know of any public implementations.
> 
> (That said, I have written a stand-alone tool that encodes/decodes
> ZMBV, but I’ve not published it anywhere. It’s based heavily on the
> DOSBox implementation.)
> > It also turns out the creator of this codec is Harekiet, who hangs
> > out
> > in #revision on IRCnet
> 
> Ah ok, do you know him?
> It sounds like he’s not concerned too much about what direction we
> take the format in. But I guess anything we implement may not get
> made official unless DOSBox adds decoding support.

I suspect he'd be more than happy handing the ZMBV reins to you. Since
24-bit mode isn't official yet I'd say there's room to make
improvements to the format itself, via that flag field. There's also
that unused bit in every MV, which together with the other bit gives
you the possibility of two extra modes beyond copy and XOR. 1-byte MVs
might also be something to investigate, especially for smaller
blocksizes (say 4x4)

> By the way, I’m happy for this patch to be committed as-is (possibly
> without the extra note on unsupported bit depths, if that causes any
> issues). Any new additions outside the existing spec would warrant a
> new patch I think.

Oh yeah, We've gotten a bit sidetracked with this 24-bit business :)
I'll push it once FATE passes. Should be an hour or so.

/Tomas