[Libav-user] avcodec_decode_video2 missing the first 8 frames
Matt Orlando
morlando616 at gmail.com
Wed Jul 23 00:39:49 CEST 2014
Thank you so much Cyril, I'll take a look at this first thing in the
morning, with a fresh mind, and see what I can do with it!
On Tue, Jul 22, 2014 at 6:28 PM, cyril poulet <cyril.poulet at centraliens.net>
wrote:
> For your first question, it seems that some info about the stream (in
> stream->codec->extradata) needs to be passed on to the avCodecContext, so
> these three lines do exactly that (but I never used it personnally, so
> I'm not sure)
> For your second question, it seems that you did not see that packetToSend.data
> is a pointer and is not increased but moved,but it also seems that there is
> a small mistake : when updating packet, offsetInData must be reset to 0
> The code is to take in its entirety :
> - offsetInData is the amount of data in packet already processed by avcodec_decode_video2
> : if you look at the beginning of the loop, the structure Packet is only
> updated when all its data has been processed. However, there is no guaranty
> that avcodec_decode_video2 will use all data in the Packet. What the
> whole structure means is :
> - process data from the packet through a secondary structure packetToSend,
> return amount of data processed
> - if the amount of data processed is >= to the amount of data in Packet,
> update packet
> - otherwise, copy in packetToSend data not yet processed (at address packet.packet.data
> + offsetInData and of size packet.packet.size - offsetInData) and process
> this data
> Repeat last step until an update of Packet becomes necessary (and reset
> offsetInData to 0)
>
> The data in packetToSend is thus re-addressed at each step, not increased.
>
>
> 2014-07-22 22:40 GMT+02:00 Matt Orlando <morlando616 at gmail.com>:
>
> So this tutorial looks like it's covering 2 things that I don't do within
>> my plugin, which is great news since the answer is somewhere within!
>> Unfortunately, I'm a bit confused with some things from the tutorial.
>>
>>
>> Can anyone explain this to me, and/or perhaps provide a more legible code
>> example (c++ / c#)?
>>
>> // we need to make a copy of videoStream->codec->extradata and give it to the context
>> // make sure that this vector exists as long as the avVideoCodec exists
>> std::vector codecContextExtraData(stream->codec->extradata, stream->codec->extradata + stream->codec->extradata_size);
>> avVideoCodec->extradata = reinterpret_cast(codecContextExtraData.data());
>> avVideoCodec->extradata_size = codecContextExtraData.size();
>>
>> The other part I don't understand is what he's doing with the
>> offsetInData variable. I'm almost definitely misunderstanding his usage,
>> but it looks as though he's increasing packet.packet.Data by this
>> ever-growing offsetInData. So hypothetically, if he had a 3 hour movie,
>> that offsetInData would be enormous! How could his packet.packet.Data
>> array be large enough to shift the pointer by that amount? I also don't
>> understand why he's only reading (through the reset function of his packet)
>> if that offsetInData is larger than the packet size.
>>
>> Both of these things confuse me...and both of these things I need to
>> implement. If anyone can explain any/all of this in as much detail as
>> possible, I'd be forever grateful. I've got a terrible deadline to hit and
>> I simply need this to work. Thank you sincerely for the help, folks!
>>
>> -Matt
>>
>>
>>
>> On Tue, Jul 22, 2014 at 2:59 PM, Matt Orlando <morlando616 at gmail.com>
>> wrote:
>>
>>> Thank you very much, I'll take some time and look at this tutorial.
>>>
>>>
>>> On Tue, Jul 22, 2014 at 11:22 AM, cyril poulet <
>>> cyril.poulet at centraliens.net> wrote:
>>>
>>>> I think "*For video, the packet contains exactly one frame.*" is not
>>>> true for all codecs, but is for most of the usual ones.
>>>> try this one :
>>>> http://blog.tomaka17.com/2012/03/libavcodeclibavformat-tutorial/
>>>> It may be that avcodec_decode_video2 recognizes that the first packet
>>>> data is header and discards it as irrelevant for image decoding...
>>>>
>>>>
>>>> 2014-07-22 17:14 GMT+02:00 Matt Orlando <morlando616 at gmail.com>:
>>>>
>>>> The documentation
>>>>> <https://www.ffmpeg.org/doxygen/trunk/group__lavf__decoding.html#ga4fdb3084415a82e3810de6ee60e46a61> says
>>>>> "*For video, the packet contains exactly one frame.*" I used this
>>>>> <http://dranger.com/ffmpeg/> tutorial to understand the flow of what
>>>>> needs to be done. Some of the calls were deprecated so a few google
>>>>> searches got me the replacements. What appears to be happening is those
>>>>> lost frames are just gone...the data isn't being accumulated. Is it
>>>>> possible my av_free_packet call is in a bad spot?
>>>>>
>>>>>
>>>>> On Tue, Jul 22, 2014 at 10:58 AM, cyril poulet <
>>>>> cyril.poulet at centraliens.net> wrote:
>>>>>
>>>>>> what av_read_frame does is take some data and fill your packet with
>>>>>> it. There is no guaranty that a full frame is in the packet, or even image
>>>>>> data.
>>>>>> avcodec_decode_video2 is the only way to know if you have decoded an
>>>>>> image : you feed it your successive packets, it accumulates data, and tells
>>>>>> you when the data allowed him to successfully decode an image (through
>>>>>> frameFinished)
>>>>>> You should look online for basic examples of how to decode safely
>>>>>> vids, there is in particular a tutorial which introduces the structure
>>>>>> SafePacket (first version has been updated for latest versions of the libs)
>>>>>>
>>>>>>
>>>>>> 2014-07-22 16:49 GMT+02:00 Matt Orlando <morlando616 at gmail.com>:
>>>>>>
>>>>>> Thanks for the response. That's very interesting, though I wonder if
>>>>>>> it could actually be header related. Based on the videos, I know for sure
>>>>>>> there are exactly 64 frames in the first movie, for instance. My
>>>>>>> m_FrameRead is only increased if avcodec_decode_video2 says it was able to
>>>>>>> get a picture. My thinking is that if it was a header that was read, that
>>>>>>> function wouldn't set frameFinished to "true". So av_read_frame hits the
>>>>>>> EOF when my m_FrameRead counter is only 56.
>>>>>>>
>>>>>>> Are you saying that perhaps the av_read_frame is reading a bunch of
>>>>>>> data (header + frames) and my avcodec_decode_video2 says there's no picture
>>>>>>> because there's header data in there. Then a few frames later (or maybe
>>>>>>> next frame), av_read_frame reads more data, now including the 9th 'frame'
>>>>>>> and finally avcodec_decode_video2 decodes and I increment my counter (which
>>>>>>> would be now 1 when I'd want it to be 9).
>>>>>>>
>>>>>>> Thanks for the help!
>>>>>>> -Matt
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Jul 22, 2014 at 8:30 AM, cyril poulet <
>>>>>>> cyril.poulet at centraliens.net> wrote:
>>>>>>>
>>>>>>>> according to the doc, av_read_packet does not check wether the data
>>>>>>>> in the decoded frame is a valid image or not.
>>>>>>>> As it is always the 8 first "frames" missing, I would assume it is
>>>>>>>> the header that gets decoded first.
>>>>>>>> Have you noticed missing images when comparing the first real
>>>>>>>> decoded image and your stream in a player ?
>>>>>>>>
>>>>>>>>
>>>>>>>> 2014-07-21 18:37 GMT+02:00 Matt Orlando <morlando616 at gmail.com>:
>>>>>>>>
>>>>>>>>> Hi folks,
>>>>>>>>>
>>>>>>>>> I'm having some trouble with the decoding of my h.264 movies.
>>>>>>>>> I've tested with different videos, one 64 frames long and another 52
>>>>>>>>> frames long. In both cases, the first 8 frames (exactly 8) weren't
>>>>>>>>> decoding. In other words, the call to avcodec_decode_video2 returns 0 for
>>>>>>>>> the got_picture variable. I track the frames read and received for our
>>>>>>>>> system and I never get notice that all the frames were read because my
>>>>>>>>> counter doesn't increase for the first 8...however all of the subsequent
>>>>>>>>> calls to av_read_frame return EOF. To prevent any confusion (or perhaps
>>>>>>>>> just to add some context), I need the raw data from each decoded frame
>>>>>>>>> since I don't treat them as a frame to a movie but rather a frame of x,y,z
>>>>>>>>> positional data.
>>>>>>>>>
>>>>>>>>> I work in Unity but made a plugin to communicate with the
>>>>>>>>> libraries to read and decode the h.264 movies. This is our update function:
>>>>>>>>>
>>>>>>>>> bool H264Stream::Update(int desiredFrame)
>>>>>>>>> {
>>>>>>>>> if( desiredFrame <= m_FrameRead )
>>>>>>>>> return true;
>>>>>>>>>
>>>>>>>>> int frameFinished = 0;
>>>>>>>>>
>>>>>>>>> if( av_read_frame( m_AVFormat, &m_Packet ) >= 0 )
>>>>>>>>> {
>>>>>>>>> if( m_Packet.stream_index == m_AVStream->index )
>>>>>>>>> {
>>>>>>>>> av_frame_unref( m_AVFrame );
>>>>>>>>> avcodec_decode_video2( m_AVContext, m_AVFrame,
>>>>>>>>> &frameFinished, &m_Packet );
>>>>>>>>>
>>>>>>>>> if( frameFinished != 0 )
>>>>>>>>> {
>>>>>>>>> sws_scale( m_SwsContext, m_AVFrame->data,
>>>>>>>>> m_AVFrame->linesize, 0, m_AVContext->height, m_Picture->data,
>>>>>>>>> m_Picture->linesize );
>>>>>>>>>
>>>>>>>>> m_FrameRead++;
>>>>>>>>>
>>>>>>>>> av_free_packet( &m_Packet );
>>>>>>>>> }
>>>>>>>>> }
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> return m_FrameRead >= desiredFrame;
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Oh, quick note, this is my first time using these libraries so if
>>>>>>>>> there's something blatantly wrong with how I'm doing this, please let me
>>>>>>>>> know. :) Just keep in mind, it's the full frame of 'raw' data I use
>>>>>>>>> (post-decode) that I pass back into my Unity c# scripts.
>>>>>>>>>
>>>>>>>>> Thanks for the help everyone!
>>>>>>>>> -Matt
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Libav-user mailing list
>>>>>>>>> Libav-user at ffmpeg.org
>>>>>>>>> http://ffmpeg.org/mailman/listinfo/libav-user
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Libav-user mailing list
>>>>>>>> Libav-user at ffmpeg.org
>>>>>>>> http://ffmpeg.org/mailman/listinfo/libav-user
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Libav-user mailing list
>>>>>>> Libav-user at ffmpeg.org
>>>>>>> http://ffmpeg.org/mailman/listinfo/libav-user
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Libav-user mailing list
>>>>>> Libav-user at ffmpeg.org
>>>>>> http://ffmpeg.org/mailman/listinfo/libav-user
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Libav-user mailing list
>>>>> Libav-user at ffmpeg.org
>>>>> http://ffmpeg.org/mailman/listinfo/libav-user
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Libav-user mailing list
>>>> Libav-user at ffmpeg.org
>>>> http://ffmpeg.org/mailman/listinfo/libav-user
>>>>
>>>>
>>>
>>
>> _______________________________________________
>> Libav-user mailing list
>> Libav-user at ffmpeg.org
>> http://ffmpeg.org/mailman/listinfo/libav-user
>>
>>
>
> _______________________________________________
> Libav-user mailing list
> Libav-user at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/libav-user
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://ffmpeg.org/pipermail/libav-user/attachments/20140722/50dc6a19/attachment.html>
More information about the Libav-user
mailing list