[Libav-user] avcodec_decode_video2 missing the first 8 frames
cyril poulet
cyril.poulet at centraliens.net
Wed Jul 23 00:28:43 CEST 2014
For your first question, it seems that some info about the stream (in
stream->codec->extradata) needs to be passed on to the avCodecContext, so
these three lines do exactly that (but I never used it personnally, so I'm
not sure)
For your second question, it seems that you did not see that packetToSend.data
is a pointer and is not increased but moved,but it also seems that there is
a small mistake : when updating packet, offsetInData must be reset to 0
The code is to take in its entirety :
- offsetInData is the amount of data in packet already processed by
avcodec_decode_video2
: if you look at the beginning of the loop, the structure Packet is only
updated when all its data has been processed. However, there is no guaranty
that avcodec_decode_video2 will use all data in the Packet. What the whole
structure means is :
- process data from the packet through a secondary structure packetToSend,
return amount of data processed
- if the amount of data processed is >= to the amount of data in Packet,
update packet
- otherwise, copy in packetToSend data not yet processed (at address
packet.packet.data
+ offsetInData and of size packet.packet.size - offsetInData) and process
this data
Repeat last step until an update of Packet becomes necessary (and reset
offsetInData to 0)
The data in packetToSend is thus re-addressed at each step, not increased.
2014-07-22 22:40 GMT+02:00 Matt Orlando <morlando616 at gmail.com>:
> So this tutorial looks like it's covering 2 things that I don't do within
> my plugin, which is great news since the answer is somewhere within!
> Unfortunately, I'm a bit confused with some things from the tutorial.
>
>
> Can anyone explain this to me, and/or perhaps provide a more legible code
> example (c++ / c#)?
>
> // we need to make a copy of videoStream->codec->extradata and give it to the context
> // make sure that this vector exists as long as the avVideoCodec exists
> std::vector codecContextExtraData(stream->codec->extradata, stream->codec->extradata + stream->codec->extradata_size);
> avVideoCodec->extradata = reinterpret_cast(codecContextExtraData.data());
> avVideoCodec->extradata_size = codecContextExtraData.size();
>
> The other part I don't understand is what he's doing with the offsetInData
> variable. I'm almost definitely misunderstanding his usage, but it looks
> as though he's increasing packet.packet.Data by this ever-growing
> offsetInData. So hypothetically, if he had a 3 hour movie, that
> offsetInData would be enormous! How could his packet.packet.Data array be
> large enough to shift the pointer by that amount? I also don't understand
> why he's only reading (through the reset function of his packet) if that
> offsetInData is larger than the packet size.
>
> Both of these things confuse me...and both of these things I need to
> implement. If anyone can explain any/all of this in as much detail as
> possible, I'd be forever grateful. I've got a terrible deadline to hit and
> I simply need this to work. Thank you sincerely for the help, folks!
>
> -Matt
>
>
>
> On Tue, Jul 22, 2014 at 2:59 PM, Matt Orlando <morlando616 at gmail.com>
> wrote:
>
>> Thank you very much, I'll take some time and look at this tutorial.
>>
>>
>> On Tue, Jul 22, 2014 at 11:22 AM, cyril poulet <
>> cyril.poulet at centraliens.net> wrote:
>>
>>> I think "*For video, the packet contains exactly one frame.*" is not
>>> true for all codecs, but is for most of the usual ones.
>>> try this one :
>>> http://blog.tomaka17.com/2012/03/libavcodeclibavformat-tutorial/
>>> It may be that avcodec_decode_video2 recognizes that the first packet
>>> data is header and discards it as irrelevant for image decoding...
>>>
>>>
>>> 2014-07-22 17:14 GMT+02:00 Matt Orlando <morlando616 at gmail.com>:
>>>
>>> The documentation
>>>> <https://www.ffmpeg.org/doxygen/trunk/group__lavf__decoding.html#ga4fdb3084415a82e3810de6ee60e46a61> says
>>>> "*For video, the packet contains exactly one frame.*" I used this
>>>> <http://dranger.com/ffmpeg/> tutorial to understand the flow of what
>>>> needs to be done. Some of the calls were deprecated so a few google
>>>> searches got me the replacements. What appears to be happening is those
>>>> lost frames are just gone...the data isn't being accumulated. Is it
>>>> possible my av_free_packet call is in a bad spot?
>>>>
>>>>
>>>> On Tue, Jul 22, 2014 at 10:58 AM, cyril poulet <
>>>> cyril.poulet at centraliens.net> wrote:
>>>>
>>>>> what av_read_frame does is take some data and fill your packet with
>>>>> it. There is no guaranty that a full frame is in the packet, or even image
>>>>> data.
>>>>> avcodec_decode_video2 is the only way to know if you have decoded an
>>>>> image : you feed it your successive packets, it accumulates data, and tells
>>>>> you when the data allowed him to successfully decode an image (through
>>>>> frameFinished)
>>>>> You should look online for basic examples of how to decode safely
>>>>> vids, there is in particular a tutorial which introduces the structure
>>>>> SafePacket (first version has been updated for latest versions of the libs)
>>>>>
>>>>>
>>>>> 2014-07-22 16:49 GMT+02:00 Matt Orlando <morlando616 at gmail.com>:
>>>>>
>>>>> Thanks for the response. That's very interesting, though I wonder if
>>>>>> it could actually be header related. Based on the videos, I know for sure
>>>>>> there are exactly 64 frames in the first movie, for instance. My
>>>>>> m_FrameRead is only increased if avcodec_decode_video2 says it was able to
>>>>>> get a picture. My thinking is that if it was a header that was read, that
>>>>>> function wouldn't set frameFinished to "true". So av_read_frame hits the
>>>>>> EOF when my m_FrameRead counter is only 56.
>>>>>>
>>>>>> Are you saying that perhaps the av_read_frame is reading a bunch of
>>>>>> data (header + frames) and my avcodec_decode_video2 says there's no picture
>>>>>> because there's header data in there. Then a few frames later (or maybe
>>>>>> next frame), av_read_frame reads more data, now including the 9th 'frame'
>>>>>> and finally avcodec_decode_video2 decodes and I increment my counter (which
>>>>>> would be now 1 when I'd want it to be 9).
>>>>>>
>>>>>> Thanks for the help!
>>>>>> -Matt
>>>>>>
>>>>>>
>>>>>> On Tue, Jul 22, 2014 at 8:30 AM, cyril poulet <
>>>>>> cyril.poulet at centraliens.net> wrote:
>>>>>>
>>>>>>> according to the doc, av_read_packet does not check wether the data
>>>>>>> in the decoded frame is a valid image or not.
>>>>>>> As it is always the 8 first "frames" missing, I would assume it is
>>>>>>> the header that gets decoded first.
>>>>>>> Have you noticed missing images when comparing the first real
>>>>>>> decoded image and your stream in a player ?
>>>>>>>
>>>>>>>
>>>>>>> 2014-07-21 18:37 GMT+02:00 Matt Orlando <morlando616 at gmail.com>:
>>>>>>>
>>>>>>>> Hi folks,
>>>>>>>>
>>>>>>>> I'm having some trouble with the decoding of my h.264 movies. I've
>>>>>>>> tested with different videos, one 64 frames long and another 52 frames
>>>>>>>> long. In both cases, the first 8 frames (exactly 8) weren't decoding. In
>>>>>>>> other words, the call to avcodec_decode_video2 returns 0 for the
>>>>>>>> got_picture variable. I track the frames read and received for our system
>>>>>>>> and I never get notice that all the frames were read because my counter
>>>>>>>> doesn't increase for the first 8...however all of the subsequent calls to
>>>>>>>> av_read_frame return EOF. To prevent any confusion (or perhaps just to add
>>>>>>>> some context), I need the raw data from each decoded frame since I don't
>>>>>>>> treat them as a frame to a movie but rather a frame of x,y,z positional
>>>>>>>> data.
>>>>>>>>
>>>>>>>> I work in Unity but made a plugin to communicate with the libraries
>>>>>>>> to read and decode the h.264 movies. This is our update function:
>>>>>>>>
>>>>>>>> bool H264Stream::Update(int desiredFrame)
>>>>>>>> {
>>>>>>>> if( desiredFrame <= m_FrameRead )
>>>>>>>> return true;
>>>>>>>>
>>>>>>>> int frameFinished = 0;
>>>>>>>>
>>>>>>>> if( av_read_frame( m_AVFormat, &m_Packet ) >= 0 )
>>>>>>>> {
>>>>>>>> if( m_Packet.stream_index == m_AVStream->index )
>>>>>>>> {
>>>>>>>> av_frame_unref( m_AVFrame );
>>>>>>>> avcodec_decode_video2( m_AVContext, m_AVFrame,
>>>>>>>> &frameFinished, &m_Packet );
>>>>>>>>
>>>>>>>> if( frameFinished != 0 )
>>>>>>>> {
>>>>>>>> sws_scale( m_SwsContext, m_AVFrame->data,
>>>>>>>> m_AVFrame->linesize, 0, m_AVContext->height, m_Picture->data,
>>>>>>>> m_Picture->linesize );
>>>>>>>>
>>>>>>>> m_FrameRead++;
>>>>>>>>
>>>>>>>> av_free_packet( &m_Packet );
>>>>>>>> }
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>> return m_FrameRead >= desiredFrame;
>>>>>>>> }
>>>>>>>>
>>>>>>>>
>>>>>>>> Oh, quick note, this is my first time using these libraries so if
>>>>>>>> there's something blatantly wrong with how I'm doing this, please let me
>>>>>>>> know. :) Just keep in mind, it's the full frame of 'raw' data I use
>>>>>>>> (post-decode) that I pass back into my Unity c# scripts.
>>>>>>>>
>>>>>>>> Thanks for the help everyone!
>>>>>>>> -Matt
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Libav-user mailing list
>>>>>>>> Libav-user at ffmpeg.org
>>>>>>>> http://ffmpeg.org/mailman/listinfo/libav-user
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Libav-user mailing list
>>>>>>> Libav-user at ffmpeg.org
>>>>>>> http://ffmpeg.org/mailman/listinfo/libav-user
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Libav-user mailing list
>>>>>> Libav-user at ffmpeg.org
>>>>>> http://ffmpeg.org/mailman/listinfo/libav-user
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Libav-user mailing list
>>>>> Libav-user at ffmpeg.org
>>>>> http://ffmpeg.org/mailman/listinfo/libav-user
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Libav-user mailing list
>>>> Libav-user at ffmpeg.org
>>>> http://ffmpeg.org/mailman/listinfo/libav-user
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Libav-user mailing list
>>> Libav-user at ffmpeg.org
>>> http://ffmpeg.org/mailman/listinfo/libav-user
>>>
>>>
>>
>
> _______________________________________________
> Libav-user mailing list
> Libav-user at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/libav-user
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://ffmpeg.org/pipermail/libav-user/attachments/20140723/90547de7/attachment.html>
More information about the Libav-user
mailing list