[Libav-user] avcodec_decode_video2 missing the first 8 frames
Matt Orlando
morlando616 at gmail.com
Tue Jul 22 22:40:22 CEST 2014
So this tutorial looks like it's covering 2 things that I don't do within
my plugin, which is great news since the answer is somewhere within!
Unfortunately, I'm a bit confused with some things from the tutorial.
Can anyone explain this to me, and/or perhaps provide a more legible code
example (c++ / c#)?
// we need to make a copy of videoStream->codec->extradata and give it
to the context
// make sure that this vector exists as long as the avVideoCodec exists
std::vector codecContextExtraData(stream->codec->extradata,
stream->codec->extradata + stream->codec->extradata_size);
avVideoCodec->extradata = reinterpret_cast(codecContextExtraData.data());
avVideoCodec->extradata_size = codecContextExtraData.size();
The other part I don't understand is what he's doing with the offsetInData
variable. I'm almost definitely misunderstanding his usage, but it looks
as though he's increasing packet.packet.Data by this ever-growing
offsetInData. So hypothetically, if he had a 3 hour movie, that
offsetInData would be enormous! How could his packet.packet.Data array be
large enough to shift the pointer by that amount? I also don't understand
why he's only reading (through the reset function of his packet) if that
offsetInData is larger than the packet size.
Both of these things confuse me...and both of these things I need to
implement. If anyone can explain any/all of this in as much detail as
possible, I'd be forever grateful. I've got a terrible deadline to hit and
I simply need this to work. Thank you sincerely for the help, folks!
-Matt
On Tue, Jul 22, 2014 at 2:59 PM, Matt Orlando <morlando616 at gmail.com> wrote:
> Thank you very much, I'll take some time and look at this tutorial.
>
>
> On Tue, Jul 22, 2014 at 11:22 AM, cyril poulet <
> cyril.poulet at centraliens.net> wrote:
>
>> I think "*For video, the packet contains exactly one frame.*" is not
>> true for all codecs, but is for most of the usual ones.
>> try this one :
>> http://blog.tomaka17.com/2012/03/libavcodeclibavformat-tutorial/
>> It may be that avcodec_decode_video2 recognizes that the first packet
>> data is header and discards it as irrelevant for image decoding...
>>
>>
>> 2014-07-22 17:14 GMT+02:00 Matt Orlando <morlando616 at gmail.com>:
>>
>> The documentation
>>> <https://www.ffmpeg.org/doxygen/trunk/group__lavf__decoding.html#ga4fdb3084415a82e3810de6ee60e46a61> says
>>> "*For video, the packet contains exactly one frame.*" I used this
>>> <http://dranger.com/ffmpeg/> tutorial to understand the flow of what
>>> needs to be done. Some of the calls were deprecated so a few google
>>> searches got me the replacements. What appears to be happening is those
>>> lost frames are just gone...the data isn't being accumulated. Is it
>>> possible my av_free_packet call is in a bad spot?
>>>
>>>
>>> On Tue, Jul 22, 2014 at 10:58 AM, cyril poulet <
>>> cyril.poulet at centraliens.net> wrote:
>>>
>>>> what av_read_frame does is take some data and fill your packet with it.
>>>> There is no guaranty that a full frame is in the packet, or even image data.
>>>> avcodec_decode_video2 is the only way to know if you have decoded an
>>>> image : you feed it your successive packets, it accumulates data, and tells
>>>> you when the data allowed him to successfully decode an image (through
>>>> frameFinished)
>>>> You should look online for basic examples of how to decode safely vids,
>>>> there is in particular a tutorial which introduces the structure SafePacket
>>>> (first version has been updated for latest versions of the libs)
>>>>
>>>>
>>>> 2014-07-22 16:49 GMT+02:00 Matt Orlando <morlando616 at gmail.com>:
>>>>
>>>> Thanks for the response. That's very interesting, though I wonder if
>>>>> it could actually be header related. Based on the videos, I know for sure
>>>>> there are exactly 64 frames in the first movie, for instance. My
>>>>> m_FrameRead is only increased if avcodec_decode_video2 says it was able to
>>>>> get a picture. My thinking is that if it was a header that was read, that
>>>>> function wouldn't set frameFinished to "true". So av_read_frame hits the
>>>>> EOF when my m_FrameRead counter is only 56.
>>>>>
>>>>> Are you saying that perhaps the av_read_frame is reading a bunch of
>>>>> data (header + frames) and my avcodec_decode_video2 says there's no picture
>>>>> because there's header data in there. Then a few frames later (or maybe
>>>>> next frame), av_read_frame reads more data, now including the 9th 'frame'
>>>>> and finally avcodec_decode_video2 decodes and I increment my counter (which
>>>>> would be now 1 when I'd want it to be 9).
>>>>>
>>>>> Thanks for the help!
>>>>> -Matt
>>>>>
>>>>>
>>>>> On Tue, Jul 22, 2014 at 8:30 AM, cyril poulet <
>>>>> cyril.poulet at centraliens.net> wrote:
>>>>>
>>>>>> according to the doc, av_read_packet does not check wether the data
>>>>>> in the decoded frame is a valid image or not.
>>>>>> As it is always the 8 first "frames" missing, I would assume it is
>>>>>> the header that gets decoded first.
>>>>>> Have you noticed missing images when comparing the first real decoded
>>>>>> image and your stream in a player ?
>>>>>>
>>>>>>
>>>>>> 2014-07-21 18:37 GMT+02:00 Matt Orlando <morlando616 at gmail.com>:
>>>>>>
>>>>>>> Hi folks,
>>>>>>>
>>>>>>> I'm having some trouble with the decoding of my h.264 movies. I've
>>>>>>> tested with different videos, one 64 frames long and another 52 frames
>>>>>>> long. In both cases, the first 8 frames (exactly 8) weren't decoding. In
>>>>>>> other words, the call to avcodec_decode_video2 returns 0 for the
>>>>>>> got_picture variable. I track the frames read and received for our system
>>>>>>> and I never get notice that all the frames were read because my counter
>>>>>>> doesn't increase for the first 8...however all of the subsequent calls to
>>>>>>> av_read_frame return EOF. To prevent any confusion (or perhaps just to add
>>>>>>> some context), I need the raw data from each decoded frame since I don't
>>>>>>> treat them as a frame to a movie but rather a frame of x,y,z positional
>>>>>>> data.
>>>>>>>
>>>>>>> I work in Unity but made a plugin to communicate with the libraries
>>>>>>> to read and decode the h.264 movies. This is our update function:
>>>>>>>
>>>>>>> bool H264Stream::Update(int desiredFrame)
>>>>>>> {
>>>>>>> if( desiredFrame <= m_FrameRead )
>>>>>>> return true;
>>>>>>>
>>>>>>> int frameFinished = 0;
>>>>>>>
>>>>>>> if( av_read_frame( m_AVFormat, &m_Packet ) >= 0 )
>>>>>>> {
>>>>>>> if( m_Packet.stream_index == m_AVStream->index )
>>>>>>> {
>>>>>>> av_frame_unref( m_AVFrame );
>>>>>>> avcodec_decode_video2( m_AVContext, m_AVFrame,
>>>>>>> &frameFinished, &m_Packet );
>>>>>>>
>>>>>>> if( frameFinished != 0 )
>>>>>>> {
>>>>>>> sws_scale( m_SwsContext, m_AVFrame->data,
>>>>>>> m_AVFrame->linesize, 0, m_AVContext->height, m_Picture->data,
>>>>>>> m_Picture->linesize );
>>>>>>>
>>>>>>> m_FrameRead++;
>>>>>>>
>>>>>>> av_free_packet( &m_Packet );
>>>>>>> }
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> return m_FrameRead >= desiredFrame;
>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>> Oh, quick note, this is my first time using these libraries so if
>>>>>>> there's something blatantly wrong with how I'm doing this, please let me
>>>>>>> know. :) Just keep in mind, it's the full frame of 'raw' data I use
>>>>>>> (post-decode) that I pass back into my Unity c# scripts.
>>>>>>>
>>>>>>> Thanks for the help everyone!
>>>>>>> -Matt
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Libav-user mailing list
>>>>>>> Libav-user at ffmpeg.org
>>>>>>> http://ffmpeg.org/mailman/listinfo/libav-user
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Libav-user mailing list
>>>>>> Libav-user at ffmpeg.org
>>>>>> http://ffmpeg.org/mailman/listinfo/libav-user
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Libav-user mailing list
>>>>> Libav-user at ffmpeg.org
>>>>> http://ffmpeg.org/mailman/listinfo/libav-user
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Libav-user mailing list
>>>> Libav-user at ffmpeg.org
>>>> http://ffmpeg.org/mailman/listinfo/libav-user
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Libav-user mailing list
>>> Libav-user at ffmpeg.org
>>> http://ffmpeg.org/mailman/listinfo/libav-user
>>>
>>>
>>
>> _______________________________________________
>> Libav-user mailing list
>> Libav-user at ffmpeg.org
>> http://ffmpeg.org/mailman/listinfo/libav-user
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://ffmpeg.org/pipermail/libav-user/attachments/20140722/05308657/attachment.html>
More information about the Libav-user
mailing list