[Libav-user] av_read_frame gets stuck in a loop after an "invalid" video frame

[GOSDEN Peter]

Good afternoon all,

As this is my first post, and I'm trying my hardest to keep it simple and keep
to your standards - let me know if I get anything wrong, especially
terminology!

I've been troubleshooting a problem with our video playout application that
uses avformat-52.dll from a scarily old drop of code taken out of svn a year
ago. The problem is reproducible with avformat-53.dll from FFmpeg 0.8.

I'm posting here, rather than raising a bug report since we've done the donkey
work; have got to the bottom of the problem and have a dirty fix. If a ticket
needs raising in trac, I'll put one in once I've got some feedback from you
all. Should this fix go into the mainstream ffmpeg distribution? If so then
I can supply the patch and I should be able to make some test media available.
I'm sure there are "better" fixes for the issue.

I've an mxf file with I-frame only MPEG2 content, generated by another
company's editing software. The same problem happens in a noticeable number of
files, so it's not a one-off issue. There are legacy files building up with
the same problem over time, so ideally, I want to allow our playout
application to deal with these files. Playout of these files stalls at a
specific point in the file. It's always an edit point that causes this, but
not all edit points cause the issue.

What happens at the edit point, is that we get an "invalid" bit of data. The
frame in question has a sequence extension where "low delay" is not set (byte
9, bit 7 according to http://dvd.sourceforge.net/dvdinfo/mpeghdrs.html#ext).
According to arcane lore, the sequence extension should not vary like this.
Cerify reports "In repeated sequence extensions, all data elements must have
the same values as in the first sequence_extension()". But in these files, it
does. And although we will obviously make the editing software manufacturer
aware, there is still media out there in the wild that we need to play.

The reason that the playout stalls is that our call to av_read_frame doesn't
return promptly. Our app gets reset by some of our other software by
virtue of the playout stopping unexpectedly before it ever returns. In fact,
having got to the bottom of the problem, I think it would eventually return
having parsed the entire video file, but I haven't had reason to test that.

We have hit an "unexpected combination" of AVFMT_FLAG_GENPTS and the logic in
mpegvideo_extract_headers, compute_pkt_fields and av_read_frame. We use
AVFMT_FLAG_GENPTS since we need it for other scenarios. av_read_frame is
therefore trying to ensure it generates pts and dts for each packet. Until
this point, compute_pkt_fields has been setting dts and pts for each frame to
be the stream's current dts. Bog standard "low delay" I-frame playout.

When the frame marked "not low delay" appears, the codec context field
"avctx->has_b_frames" is set to 1 in mpegvideo_extract_headers. In turn, this
causes compute_pkt_fields to set the "presentation_delayed" flag, implying
long GOP logic is being used. compute_pkt_fields only sets pkt->dts. pkt->pts
will be set later, in av_read_frame when the next I- or P- frame is found.
When av_read_frame is processing the frame, it invokes its own long GOP logic.
It needs to find the next I- or P- frame, so it calls for the next packets to
be decoded until it finds them. Unfortunately it never does. For all of the
rest of the packets in the stream, "low delay" is set correctly, so
compute_pkt_fields sets every subsequent I-frame to have dts and pts set to be
the stream's current dts. The long GOP logic in av_read_frame (we are still
processing the bad frame, remember) sees these as b-frames and will not use
them to provide the correct pts.

We found it difficult to figure out a robust patch for this. Obviously, the
file is at fault. However I want the code to keep running despite that. The
cause of the problem in code seemed to me to be a couple of assumptions. I'm
not sure that "not low delay" is exactly the same as "has b-frames", which is
driving the logic in compute_pkt_fields. I really don't know the code, or the
MPEG standard well enough to know if there is a better test available to say
"use long GOP logic". Secondly, av_read_frame implies that "if a packet has
pts == dts it is a b-frame". Of course, that's always true for long GOP. My
preference is to make the test against a flag in the packet that says "this
is a b-frame". Of course there is no such flag in the packet. The parser
context has a flag for that transiently, which only applies to the last decoded
packet. There may be many reasons why the packet is the wrong level to store
that information anyway.

Eventually we came up with a patch that works within the constraints of the
current code, as well as our limited understanding of it. Let me know if
you're interested in fixing this in the main stream and I'll get back to you.

Best Regards,
Pete Gosden
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ffmpeg.org/pipermail/libav-user/attachments/20110701/96c35535/attachment.html>