December 2003 - MPlayer-G2-dev - lists.mplayerhq.hu

vo3
by Ivan Kalvachev 02 Jan '04

02 Jan '04

Hi, here is some of my ideas, i'm afraid that there already too late to be implemented, as dalias is coding him pipeline system, while i have not finished the drafts already, but feel free to send coments... Ivan Kalvachev iive

3 6

VP layer progress
by D Richard Felker III 01 Jan '04

01 Jan '04

In the interest of promoting discussion, I'd like to open up my in-progress VP code for public viewing. Keep in mind this is under constant revision, and probably doesn't compile, and would earn me crates full of cola if it were actually used in some sort of release. :)))) http://brightrain.aerifal.cx/~dalias/vp-in-progress/ I would recommend looking mainly at the API docs, vp.h, vf_pullup.c (essentially finished), and vd_ffmpeg.c (almost finished). If you're interested in the internals, you could look at vp.c too. The vo2 wrapper is mostly incomplete, and might change completely anyway. Please send comments (or questions, if you don't understand) to the list. Naturally the code by itself doesn't tell you what you can _do_ with it. Rich

3 7

framer API, demuxer chaining
by Arpi 01 Jan '04

01 Jan '04

Hi, I'm introducing a new thing for g2, called Framer. It's a layer between the demuxer and the VP layer (decoder/filter/vo/encoder). The goal of this thing to frame raw data to frames/packets. Why? Some container formats store audio/video data in raw form. Ie. the container has no information about the frame/packet boundaries, it just keeps the streams separated. Some examples: - raw mp3, raw ac3 - raw mpeg-es, raw h263, raw .264 - raw mjpeg - audio and video stream of mpeg-ps In these cases, we cannot ask the demuxer to 'give me a video frame', as it doesn't know how long is it or even where does it starts. Some other containers cleanly separates a/v packets, for example: - avi - rm - asf - mov In these, you can read a demuxer packet, and it will contain a single video or audio frame. (ok, in case of avi and mov, audio is tricky) Also, some codecs can handle frames only (most win32 deocders), some can handle raw data only (libmpeg2), and some can handle both (libavcodec). This is not new anyway, in mplayer-g1 it was present, and implemented in libmpdemux/video.c, in a very messy, ugly way. I thought i can skip it in g2, by doing raw parsig inside the demuxers, and/or the codecs, but it is also ugly and sometimes redundant code, so the old idea came back. Also, when we do streamcopy to a framed container from a raw one (for example, mpeg-ps to avi) we need to parse the stream anyway, without actual decoding. So placing framer into decoder is bad idea. I think it could be implemented as the top object of the new VP layer. ie it would replace current codecs, and codecs would be just filters. The other way to go, is implementing it as independent layer. (codecs as conversion filters is not a new idea, it could help in cases of raw rgb video / pcm audio, hardware decoders (ac3/mpeg passthrough), framecopy encoding, transcoding (think of an div3->mpeg4 one) etc). Since i don't know the new VP thing yet, i cantd ecide which one is the beter, but i hope it's cleaner to put into VP. Yet another thing: demuxer chaining. (we've also discussed it on irc, ie its messy and automatic chaining should be solved somehow without ugly hacks) So i've found the solution today: add yet another stream type to demuxer layer, besides audio, video and subtitles: the muxedstream type. In case of multi-layer muxed streams (like ogg-in-avi, rawdv-in-avi, mpegps-in-mov, mpegps-in-mpegts etc) the top level demuxer (in case of ogg-in-avi the avi demuxer) could export the 2nd level demuxed steram as a new stream, with type muxedstream, and set format to the 2ndlevel demuxer name (if known, anyway it should be known...). So, in case of ogg-in-avi, teh avi demuxer opens the stram, and exports stream#1=video(format=div3) and stream#2=muxed(format=ogg), the the caller of demux_open (the main app) will see that we have muxed substreams, and it can create new virtual stream (type=ds) from it and call demux_open again with this stream as input. This way we can avoid callong demux_open and such from demuxers, and still have the whole mess simple. Also, if user doesnt want the muxed stream (for example, -nosound for ogg-in-avi) we dont need to work with that stream. A'rpi / Astral & ESP-team -- Developer of MPlayer G2, the Movie Framework for all - http://www.MPlayerHQ.hu

5 5

Recommendations for DEMUXER layer
by D Richard Felker III 30 Dec '03

30 Dec '03

I've been reading some demuxer code to figure out how pts is computed for various demuxers, in order to understand how it needs to be handled by the new video (and eventually new audio!) layer. In the process, I've come up with a few recommendations for changes. 1. Some demuxers, such as AVI, seek into the middle of an audio chunk without understanding audio packet boundaries at all (because the container format sucks too much to distinguish packets), forcing the decoder to recover. This also means (a) the demuxer will output a broken packet, which is bad if you just want to remux without using any codecs, and (b) pts is no longer exact, only approximate, which IMO sucks really bad. My recommendation would be to _always_ seek to a boundary the demuxer understands. That way you have exact pts, and no broken packets for the decoder or muxer to deal with. The demuxer can skip video frames up to the next keyframe (the point you were trying to seek to) and the audio pipeline can skip the audio _after_ decoding it so that it can keep track of the exact number of samples. (Since audio decoding is very fast, this should not impact performance when seeking.) 2. After seeking, demuxers call resync_audio_stream, which depends on there being an audio decoder! I found this problem a long time ago while adding seeking support to mencoder: it was crashing with -oac copy! It's bad because it makes the demuxer layer dependent on the codec layer. My recommendation is to eliminate resync_audio_stream, and instead just report a discontinuity the next time the demuxer stream is read. That way the codec, if one exists, can decide what to do when it reads from the demuxer, without having to use a callback from the demuxer layer to the codec. Also, resync should become unnecessary for most codecs if my above seeking recommendation is implemented. Please comment on whether you think these changes are acceptible. Rich

4 13

Limitations in vo2 api :(
by D Richard Felker III 25 Dec '03

25 Dec '03

Hey, Arpi. Libvo2 is your baby, so I need some help overcoming limitations in libvo2 so that I can write the vp wrapper for it. The problem is that the vo driver just exports an array of buffers without any information about which ones are in use internally and expects the wrapper to manage that information. This is _somewhat_ possible, except that the wrapper doesn't have enough info to go on. For example, vo_mga with triple buffering has three buffer states: unused, displayed, and pending-display. Since a maximum of only four buffers are available (and currently only 3 are allowed!!), we need to make sure we make very good use of them! I believe it's possible to tell from the hardware (dunno whether mga_vid module implements this tho) whether the pending-display buffer has actually been displayed yet. Knowing this is important since it means we can reuse the previously displayed buffer. Without this knowledge, we're either forced to give up direct rendering (and maybe slices too), or else we run the risk of _tearing_ and out-of-order display if the cpu is too fast! Consider the following example: A: displayed B: pending-display C: free The codec decodes a P-frame, slice-rendering it into buffer C, then needs to decode a B-frame to display before it. Provided the cpu isn't too fast and buffer B has already been displayed, we can put the B-frame in buffer A. But if buffer A is still visible, very bad things will happen! There are even worse things that can happen if the movie fps is greater than the video refresh rate, but anyone running the player in that situation is stupid to begin with... The above example can be solved by having the wrapper layer remember the last 2 buffers it showed, and refusing to use those. However, this will disable direct rendering of B-frames unless you have 4 buffers available. So, what kind of solution would I like?? I'm not really sure. :( I tend to think that the vp layer should call the vo layer's get_buffer and release_buffer every time it wants to use buffers, rather than just getting them all at the beginning. This way, the vo driver would be free to decide when it can or can't give you buffers. But right now, the vo drivers are NOT written to support that sort of use. For example, the x11 driver allocates an image when you call get_buffer, and frees it when you call release_buffer -- very inefficient! Please comment on what you'd like to do. IMO this isn't a fundamental design issue, but it will probably require changes to the vp/vo wrapper interface to be made at some point. Rich

4 30

Information about video coding
by Attila Kinali 24 Dec '03

24 Dec '03

Heyo, As i keep asking more and more questions about video coding the last few days, i'd like to ask whether there are any good resources (webpages, papers, books) about video coding, especialy in practical applications not only academic stuff. Thanks Attila Kinali -- egp ist vergleichbar mit einem ikea bausatz fuer flugzeugtraeger -- reeler in +kaosu

4 7

Re: [MPlayer-G2-dev] basic g2 vp (video pipeline) code
by D Richard Felker III 23 Dec '03

23 Dec '03

[resent because mphq lost it] [cut to keep from digging up an old thread with mostly outdated info] In-Reply-To: <200311021858.hA2IwT9B025362(a)mail.mplayerhq.hu> On Sun, Nov 02, 2003 at 07:58:29PM +0100, Arpi wrote: > > Maybe stride should be negotiated at config-time? That would help > > nope, its impossiblee imho in many cases, and adds yet another > limitation... I decided to negotiate stride at config-time anyway. :))))) So now it's time to address these issues... > dont forget that config() happens befor ethe first frame is passed, > so in case of 3rd party codecs, or anything special, you wont know > the wanted stride before processing first frame >From what I can tell, there are two cases: 1. The codec is rendering into its own internal buffer, which we'll export. In that case, we can wait to call config until it decodes the first frame, and then we're safe! In fact, in this case the only stride restrictions will be that the output filter has to accept the stride that the codec exports. 2. The codec is using a buffer provided by MPlayer, either automatic or direct rendering. So we already had to allocate an image before decoding the first frame! And thus we must have already known (somehow!) what stride we wanted. Actually there's a slices case too, but we can handle that just like with vd_ffmpeg. If there are any problems with this, we can put stride restrictions in the codecs.conf data. In fact, one is already there. The old "flip" flag is dead, replaced by negative stride. This makes it so every MPlayer-native filter can support flipped images, without having to write special support for them. So, I don't see any obstacles to config-time stride decision. Now I just have to design a system of negotiation to decide where to put the scale/expand/etc. filters to maximize performance... Rich

3 3

RCS for G2
by Attila Kinali 22 Dec '03

22 Dec '03

Heyo People, I think it's time to put G2 onto a RCS. Currently there are a few developments going on, afaik Arpi, Rich and Alex are working on G2 but very little of their work is seen because there is no place they can put it. IMHO it would also help to attract other developers to review the already existing code and to contribute on the current development. So, the question is what shall we use, currently there are a few different systems around: cvs, bk, svn, tla, darcs. cvs has some known problems (which we currently workaround) and limited abilities in mirroring or distributed development. bk has proven its quality in the kernel development, but its biggest disadvantage is its non-free license that'll force some developers out of the project. svn seems to solve most problems with cvs, is free, allows simple mirroring, branching and distributed development. The client usage is quite similar to cvs which helps in getting used to it (didnt need more than 15m for me) But its server is quite bloated imho. tla has some nice concepts and aims in the same direction as svn, but it is still unfinished and has IMHO some design problems that can leed to problems in the future. i can't say much about darcs as i just heard about it yesterday and didn't had but a short look at the docs. It seems to be similar to svn and tla. Currently i vote for using svn as it seems to be superior to all others and it's quit mature. I'm also using it for my personal projects for a few weeks and didnt run in any problems so far. Attila Kinali -- egp ist vergleichbar mit einem ikea bausatz fuer flugzeugtraeger -- reeler in +kaosu

7 7

PTS and AVI files, bleh!
by D Richard Felker III 20 Dec '03

20 Dec '03

Could someone who understands it (probably this means Arpi, sorry to keep asking you questions!) explain to me how one obtains PTS from AVI files? I'm reading the G2 source, but it's very confusing. I'm working on interfacing codecs with the VP layer, and trying to write the code for handling the PTS from the demuxers... Rich

2 4

G1 limitations
by Joey Parrish 18 Dec '03

18 Dec '03

Hello, My thoughts on G1 limitations... Well, the biggest ones to me are the lack of certain runtime features. For example, switching audio/video/subtitle tracks on the fly, or seeking in DVD's by title/chapter. Another great thing would be DVD menu support. I wrote a hack to switch DVD audio and subtitle at runtime, but was told that this will never be CVS because G1 can't reinit audio codecs. Luckily, none of my DVDs have audio tracks in different codecs. :) I also want to design a clean native windows GUI for MPlayer, but digging through G1's mplayer.c is very overwhelming. G2 makes this easier as well. I would like to change -vo gif89a into an encoder for MEncoder. It makes much more sense there. Why would I want to transcode from one format to another in MPlayer's vo? Because I believe MEncoder is too much of a hassle to add a new container format. Hopefully G2 will provide a solution to this too. I don't think that any of my complaints on G1 are new or have not been discussed already about G2. It's just basically a list of everything I ever tried to do in G1 but couldn't figure out, as well as everything I did in G1 but was told "This will never be in CVS, because there's no right way to do it," or "Wait for G2." :) Oh, and one more thing on my G2 wishlist. Please provide complete documentation for every API very early on. (As soon as they are mostly stable, perhaps.) It will be extremely encouraging to potential developers. --Joey -- "Living in the complex world of the future is somewhat like having bees live in your head. But, there they are."

2 1