Hey devs! It's far from finished, but here's a glimpse at the new video pipeline layer for mplayer-g2. The main functions of interest are vp_pull_image, vp_get_image, and vp_release_image. vp_config and vp_insert_filter are somewhat relevant to the dreaded auto-insertion hack too. :) Please read and make comments. I have no idea if the code actually compiles but it should be roughly correct. Some key points: Image dimensions: You don't pick the image size when requesting an image like you did in G1. Instead it's implicit from how config() was called. Along with this change, there's a much clearer distinction between actual picture dimensions and allocated (stride). Thus filters will be sure to have the correct values for width/height so they don't end up processing extra junk/blackspace outside the image (possibly causing them to behave wrong!). Aspect: Use sample aspect ratio rather than display aspect ratio! This means we don't have to constantly recompute d_width/d_height (making bad estimates and introducing error) every time we crop/expand/etc. Most filters can just pass sample aspect through unchanged, or making simple changes such as doubling the sample width or height (the only notable exception is scale). Monitor aspect can get figured in at the _end_ of the chain (where it belongs) instead of the beginning. Stride restrictions: Greatly simplified, and the flag names are actually consistent with what they mean now. From vp.h: // Note: Stride restrictions are for compatibility with legacy codecs ONLY! // ALL filters and vo MUST accept any stride as input! // Aligned stride is always implicitly preferred when not forbidden!! PTS: We now use a discrete time base fraction instead of floats. Filters are free to modify the time base if they wish. This should prevent error accumulation during lengthy encodes and help mencoder automatically select a good time base for the output file. Decoders should select the correct natural timebase (msec for asf, 60000/1001 for ntsc mpeg, 50/1 for pal mpeg, fps for fixed-fps formats, etc.). DR and slices: "Slices" are replaced with an indirect buffer type, indirect meaning that you can't write directly to it, only through the draw_indirect() function. Both DR and indirect rendering (slices) are accomplished by requesting special buffer types from vp_get_image. Rather than falling back to automatic (allocated) buffer, vp_get_image will _fail_ if DR/indirect buffer is not available! This is to give the source filter/codec more flexibility in deciding which buffer type to use. (For example, it may be better to fallback to export instead of automatic buffer if DR fails.) Unlike in G1, indirect rendering does not have to be done in "slices", but any region can be drawn via draw_indirect(). The auto-insertion hack: ... :)) It should be pretty self-explanatory. Basically, when a new filter is auto-inserted at vp_config() time, the link between the two filters is split in two. The part connected to the dest filter remains in place, and a new "sister" link is created to remain with the source filter. Then scale (or whatever you like) is inserted in between. When execution gets back to vp_pull_image, it sees that the link it's processing has a sister, so it stuffs the image back to the sister and clears the sister field, then calls pull_image on the link again, this time getting an image from the newly inserted conversion filter. It's hard to explain so RTFS!! :) Comments are welcome, but please make an attempt to understand the code before just flaming! Rich
Hi,
Hey devs! It's far from finished, but here's a glimpse at the new video pipeline layer for mplayer-g2. The main functions of interest are vp_pull_image, vp_get_image, and vp_release_image. vp_config and vp_insert_filter are somewhat relevant to the dreaded auto-insertion hack too. :)
ok, i'm answering yoru question on irc: <dalias> in g2, any object to requiring that all filters/vo be able to accept images with any stride? YES. it limits g2 vf layer to use fully controlled filter code only. ie you cannot call 3rd party code which doesnt accept any kind of stride, be it to-mpeg transcoder, codecs (remember, video encoders are connected to vf layer anyway, but it can be workarounded by ugly hacks), external filters and so on. it should be handled, at least by auto-inserting expand filter. A'rpi / Astral & ESP-team -- Developer of MPlayer G2, the Movie Framework for all - http://www.MPlayerHQ.hu
On Sun, Nov 02, 2003 at 01:17:10PM +0100, Arpi wrote:
Hi,
Hey devs! It's far from finished, but here's a glimpse at the new video pipeline layer for mplayer-g2. The main functions of interest are vp_pull_image, vp_get_image, and vp_release_image. vp_config and vp_insert_filter are somewhat relevant to the dreaded auto-insertion hack too. :)
ok, i'm answering yoru question on irc:
<dalias> in g2, any object to requiring that all filters/vo be able to accept images with any stride?
YES. it limits g2 vf layer to use fully controlled filter code only. ie you cannot call 3rd party code which doesnt accept any kind of stride, be it to-mpeg transcoder, codecs (remember, video encoders are connected to vf layer anyway, but it can be workarounded by ugly hacks), external filters and so on.
it should be handled, at least by auto-inserting expand filter.
OK. IMO there are a couple solutions. One, like you said, is loading expand. I'm not sure exactly how this works. Does it expand the image up to stride by adding a small border, or copy the image to a new buffer with stride = width*bpp? I assume the latter since the former isn't always possible (if stride[0] != stride[1]<<csh, etc.). Part of the problem with auto-inserting expand is that it has to be done at config time, when you don't yet know what stride will be. Maybe stride should be negotiated at config-time? That would help codecs/filters that don't like stride changing from one frame to another, but it also might interfere with DR1 of B frames for IPB codecs. It would also add a lot of complexity to the config process. So should we do like G1 and auto-load expand if the destination vf/ve has restrictions on which strides it will accept? Will expand behave well and just do nothing in the case where the strides already meet the dest's requirements? Rich
Hi,
<dalias> in g2, any object to requiring that all filters/vo be able to accept images with any stride?
YES. it limits g2 vf layer to use fully controlled filter code only. ie you cannot call 3rd party code which doesnt accept any kind of stride, be it to-mpeg transcoder, codecs (remember, video encoders are connected to vf layer anyway, but it can be workarounded by ugly hacks), external filters and so on.
it should be handled, at least by auto-inserting expand filter.
OK. IMO there are a couple solutions. One, like you said, is loading expand. I'm not sure exactly how this works. Does it expand the image up to stride by adding a small border, or copy the image to a new buffer with stride = width*bpp? I assume the latter since the former
the later, of course
isn't always possible (if stride[0] != stride[1]<<csh, etc.).
Part of the problem with auto-inserting expand is that it has to be done at config time, when you don't yet know what stride will be.
yes
Maybe stride should be negotiated at config-time? That would help
nope, its impossiblee imho in many cases, and adds yet another limitation... dont forget that config() happens befor ethe first frame is passed, so in case of 3rd party codecs, or anything special, you wont know the wanted stride before processing first frame
So should we do like G1 and auto-load expand if the destination vf/ve has restrictions on which strides it will accept? Will expand behave well and just do nothing in the case where the strides already meet the dest's requirements?
it depends on implementation of new vf_expand... A'rpi / Astral & ESP-team -- Developer of MPlayer G2, the Movie Framework for all - http://www.MPlayerHQ.hu
On Sun, Nov 02, 2003 at 02:09:21AM -0500, D Richard Felker III wrote:
mp_image_t *vp_pull_image(vp_link_t *link, int drop) { mp_image_t *img = link->img_redir; if (img) { link->img_redir = NULL; return img; } again: img = link->src->pull_image(link, drop); if (link->sister) { link->sister->img_redir = img; link->sister = NULL; goto again; } return img; }
One other feature worth mentioning, the drop flag to vp_pull_image: If you intend to ignore the contents of a frame (for -framedrop, a/v sync in fixed-fps output, or whatever reason), pass the drop flag when calling vp_pull_image. The filter is _still_ required to return a valid mpi for timing purposes, but the actual contents of the image can be meaningless. Filters are recommended but not required to propagate the drop flag back through the chain...and of course if it gets all the way back to the codec and the next frame is a B frame, the codec can skip decoding. Optional ideas: require filters to propagate drop flag if its value is greater than 1? Include a way to signal hard framedrop? Rich
Hi, any progress on this lately? -- Alex Beregszaszi <alex@fsn.hu> (MPlayer Core Developer -- http://www.mplayerhq.hu/)
On Tue, Dec 09, 2003 at 10:58:03AM +0100, Alex Beregszaszi wrote:
Hi,
any progress on this lately?
Ask again in about a week. :) Rich
participants (3)
-
Alex Beregszaszi -
Arpi -
D Richard Felker III