[FFmpeg-devel] [PATCH] avformat/utils: only call h264 decoder private function if h264 decoder is in use

[Michael Niedermayer]

On Sun, Sep 18, 2016 at 01:46:07PM +0200, Timo Rothenpieler wrote:
> Fixes a crash when decoding with for example h264_cuvid, as
> avpriv_h264_has_num_reorder_frames assumes the AVCodecContext->priv_data
> to be a H264Context.
> ---
>  libavformat/utils.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Relevant IRC discussion:

<michaelni> BtbN, JEEB why is st->internal->avctx not the libavcodec software decoder ?
<BtbN> because it's the cuvid h264 decoder.
<michaelni> I mean if this is the hw decoder you would be running the hw decoder twice
<michaelni> once in the application and once in libavformat
<michaelni> is that intended ?
<BtbN> hm?
<michaelni> or am i misunderstanding?
<BtbN> the cuvid h264 decoder is completely inside of lavc, completely independend from the outside
<michaelni> sure but libavformat opens a decoder to get some information about a stream, if thats the hw decoder then there would be 2 as the user app would also have a decoder independat of libavformat
<BtbN> It indeed seems to open it twice
<BtbN> But I don't see anything I could possibly do about that from inside of the cuvid decoder, and it also works just fine.
<michaelni> not from the inside but the code setting up st->internal->avctx should maybe not open a hw decoder
<michaelni> or if we want it to do that then indeed the avpriv_h264_has_num_reorder_frames() call is garbage
<BtbN> Forcing it to use the software decoder for the initial parsing seems like the better solution. As it returns way more information about the stream.

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

it is not once nor twice but times without number that the same ideas make
their appearance in the world. -- Aristotle
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20160918/44c555c1/attachment.sig>