[FFmpeg-devel] Google Summer of Code participation
Thilo Borgmann
thilo.borgmann
Mon Mar 30 16:23:02 CEST 2009
Michael Niedermayer schrieb:
> On Mon, Mar 30, 2009 at 12:33:01AM +0200, Thilo Borgmann wrote:
>
>> Vitor Sessak schrieb:
>>
>>> Thilo Borgmann wrote:
>>>
>>>> Kostya schrieb:
>>>>
>>>>> On Sat, Mar 28, 2009 at 12:20:32PM +0100, Thilo Borgmann wrote:
>>>>> [...]
>>>>>
>>>>>
>>>>>> no. 12: "16-bit Interplay Video Decoder"
>>>>>> Sounds interesting and as there is a working 8-bit decoder, wich throws
>>>>>> some errors if operating on the 16-bit demo file, there seems to be a
>>>>>> good starting point. Unfortunately, the section about 16-bit opcodes is
>>>>>> far from useful, if opcodes would have to be changed, this task becomes
>>>>>> very difficult...
>>>>>> http://wiki.multimedia.cx/index.php?title=Interplay_Video
>>>>>>
>>>>>> -> The suggestion in the wiki is to ask the development community
>>>>>> (you!) about a good start-off with that problem. So I do: Anyone who
>>>>>> can help on this? Anyone with more information about the 16-bit mode?
>>>>>> (Already inspected the code here... sounds like altering the #define'd
>>>>>> functions but how to know if it is 8-bit palette mode or 16-bit
>>>>>> whatsoever mode?)
>>>>>>
>>>>>>
>>>>> I suppose, it's only Google who knows more about this format.
>>>>> Maybe some container parameters should give demuxer a hint on whether
>>>>> this
>>>>> is 16-bit mode or not.
>>>>>
>>>>>
>>>>>> no. 23: "CorePNG Decoder"
>>>>>> Well, the descriptions sound quite easy, BUT the current svn version of
>>>>>> ffmpeg (as well as my very old one) say that it "could not find codec
>>>>>> parameters". Thus, I suppose there is no existing PNG1 decoder wich
>>>>>> decodes RGB I-Frame video? Is there a PNG1 coded RGB I-Frame demo
>>>>>> video? So this task seems to me like implementing a whole decoder on
>>>>>> top of the png image decoder?
>>>>>> http://samples.mplayerhq.hu/V-codecs/PNG1/
>>>>>>
>>>>>> -> Seems to be a not that difficult task but to have a complex
>>>>>> start-off.
>>>>>>
>>>>>>
>>>>>> Anyone has an idea about these?
>>>>>>
>>>>>>
>>>>> I've looked at it once. If you use PNG decoder for that frames you'll
>>>>> see
>>>>> good picture for the first frame and many colourful dots on the latter.
>>>>> That's
>>>>> because its developer(s) decided to store latter frames as a difference
>>>>> to
>>>>> the key frame and you have to use demuxer keyframe flag to distinguish
>>>>> between
>>>>> both.
>>>>>
>>>>> Download http://samples.mplayerhq.hu/V-codecs/PNG1/corepng.avi
>>>>> run
>>>>> ffmpeg -i corepng.avi -f image2 -vcodec copy %04d.png
>>>>> and see that for yourself.
>>>>>
>>>>> P.S. We of FFmpeg welcome creativity since it's not that rare when the
>>>>> only
>>>>> information about codec is located in libavcodec/*.c
>>>>>
>>>>>
>>>> Ok next problem arises. I'm currently trying to distinguish the I and P
>>>> frames to be decoded.
>>>> As you mentioned, I tried to check the keyframe flag, that comes from
>>>> 'outside'.
>>>> For that, I "av_log"ed the "key_frame" and "pict_type" attributes:
>>>>
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> static int decode_frame(AVCodecContext *avctx,
>>>> void *data, int *data_size,
>>>> const uint8_t *buf, int buf_size)
>>>> {
>>>> PNGDecContext * const s = avctx->priv_data;
>>>> AVFrame *picture = data;
>>>> AVFrame * const p= &s->picture;
>>>> #ifdef DEBUG
>>>> av_log(avctx, AV_LOG_DEBUG, "data->key_frame: %i, data->pict_type: %d\n",
>>>> picture->key_frame, picture->pict_type);
>>>> av_log(avctx, AV_LOG_DEBUG, "s->picture->key_frame: %i,
>>>> s->picture->pict_type: %d\n",
>>>> p->key_frame, p->pict_type);
>>>> #endif
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> (source listing is highly mailing list dependent - tell me how you like
>>>> it if this is a pain in your eyes)
>>>>
>>>> As these debug outputs are telling that every frame is a key frame, which
>>>> most of them are not, the question is if I'm looking at the wrong
>>>> attributes or if this information is not parsed at all?
>>>>
>>> There are three possibilities:
>>>
>>> 1- This information is coded somewhere in the AVI format (and thus it is
>>> the work of the AVI demuxer to read it and set the parameters).
>>>
>>> 2- This information is in some reserved bits of the PNG stream (and so it
>>> is in pngdec.c that it should be read).
>>>
>>> 3- This information is nowhere and the codec follow some stupid convention
>>> like one frame every X are key-frames.
>>>
>>> I suggest you to look in the available documentation/source code to know
>>> where this information is supposed to be. I find (1) unlikely since ffmpeg
>>> AVI demuxer is not so buggy, and it would set picture->key_frame
>>> correctly...
>>>
>>>
>> Found something here. I think, the avi dumexer does its job and reads the
>> keyframe flags. I tested this by logging the PKT_FLAG_KEY or'ed into the
>> 'flags' attribute of the 'AVPacket' structure (libavcodec/avidec.c:
>> avi_read_packet)
>> This detected a keyframe at the 1st, 7th, 8th, 21st and 22nd frames of the
>> stream, which makes sense as far as I can tell.
>>
>> Ok, but unfortunately, the AVPacket structure is not passed completely to
>> the decoder (libavcodec/pngdec.c: decode_frame()), just the attributes
>> 'data' and 'size'. (attribute 'flags' hold the keyframe information, as I
>> said above). These attributes are passed at the top level - ffplay.c:
>> video_thread().
>>
>> Next to this, an 'AVCodecContext' is passed to the decoder function. I've
>> already tried to get the keyframe information out of them but they declare
>> al frames to be keyframes. (see above for that)
>> As far as I understand the code now, this AVCodecContext is created once
>> for each stream and therefore can not carry the information per frame.
>> (This ist right, isn't it?)
>>
>>
>
>
>> Well, the keyframe flag is not coded in the PNG buffer either. So I do not
>> see a chance to get the keyframe flag from the packages read by the demuxer
>> into the decoder function. At least, not without changes at top level,
>> which means at ffplay.c which is not an option in my eyes.
>>
> [...]
>
>> So, I wonder how other codecs get told about a keyframe they have to
>> decode...
>>
>
> other codecs store the keyframe info in the frames passed to the decoder.
> btw, where is the source for corepng? i mean not a win32 self extracting
> thing
>
>
It can be selected during the codec installation. Comes as a rar-archive
afterwards. Attached it for convenience. The part that selects keyframes
in that can be found in the vfw/VFWhandler.cpp in the functions:
DecompressXXXFrame. Stored in the flags as ICDECOMPRESS_NOTKEYFRAME. As
far as my search revealed, this is supported by the AVI file functions
of video for windows.
>
>
>> As you mentioned, the demuxer detects the keyframes but the
>> picture->key_frame is not set. I checked this by my first attempt in
>> reading ist value (was always set to 'keyframe') and by looking into the
>> avidec.c (which is the avi demuxer, or am I completely wrong about that?),
>> where the key_frame attribute is never touched.
>>
>> Though, what would be the preferred way to carry this information from the
>> package-struct to the decoder function? (I hope I just missed a clue
>> somewhere but I looked through the code the whole day...)
>>
>
> If it really is not stored in the frames there are only 3 ways
>
> 1. change every muxer and demuxer that allows corepng to add&remove a
> byte specifying the keyframe and fix pngdec/enc to handle that byte
> 2. add a avcodec_decode_video2() that takes a AVPacket and update
> ffmpeg.c and ffplay.c (this was planned since a long time and is welcome)
> 3. try I & P and compare which is closer to the previous frame
> this if course is not 100% but easy
>
>
1) Well for the decoding it might not make a difference in my eyes, but
when it would come to encoding, this would not comply to the original
CorePNG anymore, would it?
3) Ok, but we want to do it as good as possible. For now, there are
other Ideas left to try.
2) Sounds good, will try to do this. Unfortunately, there will be no
progress today as I've got other work to do. I will start with that
tomorrow.
TB
-------------- next part --------------
A non-text attachment was scrubbed...
Name: CorePNG-src.rar
Type: application/x-rar
Size: 87755 bytes
Desc: not available
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20090330/078f8047/attachment.bin>
More information about the ffmpeg-devel
mailing list