From andrej at lucky.net Wed Apr 14 13:22:16 2004 From: andrej at lucky.net (Andriy N. Gritsenko) Date: Wed, 14 Apr 2004 14:22:16 +0300 Subject: [MPlayer-G2-dev] Yet another video API. :) Message-ID: <20040414112216.GA27461@lucky.net> Hi! So after some discussions and rethinking whole thing, I've made this. Big thanks for Rich and Ivan - this thing was partially based on Rich's VP api, and many things from Ivan's letter (in December 2003) were very useful. Images and buffers must be managed differently. My goals was: 1) have as little API as possible 2) use direct rendering as much as possible 3) use copying as little as possible 4) do memory allocations as little as possible 5) processing slices the same way as full frames (to be possible process slices even from decoder to vo without commiting to frame) 6) avoid pushing frames at all So I mean I wanted to do all faster and with less memory. Rich asked me on IRC about how much API we need. Here is full list of APIs which we need (AFAIK): 1) modules/config API (common); 2) stream API (file, network, etc.); 3) demuxer API; 4) video API; 5) audio API; 6) OSD/subtitles/input control API; 7) muxer API. So with these APIs we will have those (sub)layers: 1) core layer (config parser, module finder/loader) - modules/config API; 2) input stream layer (file, cache, network, etc.) - stream API; 3) demuxer layer - from stream API to demuxer API; 4) video decoder layer - from demuxer API to video API; 5) audio decoder layer - from demuxer API to audio API; 6) subtitles layer - from stream or demuxer API to OSD/subtitles API; 7) video layer - video API (chains); 8) audio layer - audio API (chains); 9) video output layer - video API (end); 10) audio output layer - audio API (end); 11) video encoder layer - from video API to muxer API; 12) audio encoder layer - from audio API to muxer API; 13) muxer layer - from muxer API to stream API; Each sublayer has own specifics and own number of APIs so don't have to be messed with another. Frames that are pulled from layers 2...8 must be pushed to layers 9...13 by application to do A/V sync. Files in attachment contain video API (for using by layers 4,7,9,11, and may be used by layer 6 too). Some minor parts are still not developed yet, I'm still in doubt about them, see my comments there. :) About future of G2... I think it's good to migrate from G1 to G2 and it may be good to migrate step by step, i.e.: a) create new development branch of MPlayer (1.1 for example); b) implement modules/config API to it; c) create new demuxer layer with new demuxer API; d) develop new video layer with new video API and so on. Each step will take some time but it's much better than rewrite all at once. With best wishes. Andriy. P.S. Don't be mad at my poor English, please. :) P.P.S. All above is IMHO. :) -------------- next part -------------- Video pipeline Video pipeline consists of a collection of decoders, filters, output devices, and encoders (vd/vf/vo/ve), collectively referred to as nodes. All can either serve as a source of images, or a destination, and some (filters) can serve as both. The nodes of the pipeline are connected by links. A link maintains the negotiated parameters for passing images between nodes: image format, dimensions, aspect, strides, etc., and manages the buffers that will pass between its endpoints. Most filters have exactly one input and one output link, but the possibility is allowed for a filter to have multiple inputs or multiple outputs. Images The link serves as the broker for images that will pass between its source and destination nodes. All image structures (mp_image_t) are associated with a link, and may only be used in the context of that link. A filter cannot take a source image it receives and pass the same image along as output to other parts of the pipeline! However, via EXPORT and DIRECT type images, buffers can be passed on in either direction in the chain. Knowing how to make use of them allows codec and filter authors to eliminate unnecessary copying of images, maximizing performance. Images & locking One of the most fundamental parts of the G2 video pipeline layer is the system of buffer handling. Each image has a reference count, or number of locks held on it. When an image is obtained from vp_get_image, it initially has one lock. Subsequent locks can be placed by either the source or destination node associated with the link. Once the lock count reaches zero, the image is no longer valid and must not be used. When this happens, the owner of the image (if any) will be notified. By means of locking, codecs may keep any number of reference frames needed for decoding predicted frames, and temporal filters may keep their input frames without having to make copies. Image buffer types AUTO(matic): Allocated and managed by the vp layer. Automatically uses the negotiated strides for the link. No owner. DIRECT(rendering): Buffer pointers are filled in by the destination node's get_buffer function. The destination node becomes the owner, and will be informed via release_buffer when the reference count reaches zero. EXPORT: Buffer pointers are filled in by the source node after requesting the image. Keep in mind that they need not point to a codec-internal buffer. They might point to the buffers from an AUTO-type image earlier in the pipeline, or (in some very fancy multiple-vo setups :) a DIRECT-type buffer from another branch of the pipeline. The source node which obtains the EXPORT-type buffer becomes its owner and will be informed when it's released. [Note that both EXPORT and DIRECT buffers are very useful for avoiding excess copying between filters. EXPORT should not be thought of as a backwards-compatibility type for old codecs, because it can do a lot more than just that!] DUMMY: used for (intentionally!) dropped frames. No buffers whatsoever, only pts and perhaps some other metainformation. May be used for indication of EOS (end of stream) - ??? Image buffer flags: READABLE: The buffer may be read (without incurring penalty). PRESERVE: The source node expects the destination node not to clobber buffer contents as long as the buffer is valid. (It rarely makes sense for a source to keep an image locked unless the preserve flag is set!) REUSABLE: The source node may clobber the buffer contents when rendering subsequent frames, so the destination cannot expect it to remain valid. (It rarely makes sense for a destination to keep an image locked if the reusable flag is set!) INDIRECT: The image buffer is not directly addressible, and may only be drawn via slices. Image flags: MF_SLICE: Set if image isn't full image but slice instead. Filters who don't accept slices will anyway get a full frame because video layer will commit all slices to frame for it. MF_FINAL: Set if image is last slice of a frame so all slices before it may be committed to full frame. MF_DROP: Set if decoder or filter may find that: a) there is yet one frame before pts; b) that frame wants to be dropped. So it marks the frame as may be dropped and any filter except those who desires all frames may just ignore it. MF_DECODE_ORDER: Set by decoder to indicate that frame was pulled in decode order. MF_NOT_IN_ORDER: Set by decoder to indicate that frame was pulled in decode order and is out of order. For example in standard MPEG sequence 1 4 2 3 7 5 6 if frames are pulled in decode order then frames 4 and 7 will get that flag. Filters API has number of functions. Each function should be not called directly but only via vp_* API calls. Most common rule is "if you get an image buffer then you _MUST_ return it to owner", i.e. each vp_get_image() or vp_pull_image() must be followed by either vp_release_image() either returned from function. It is also available to call vp_keep_image() to save you a copy of image or vp_export_image() to save you an image descriptor but that keeped image also must be followed by one of these functions later. I hope that rule is simple. If you follow that rule then you'll never get your image lost or get memory leak. So, image that returned by one of: vp_get_image() vp_keep_image() vp_pull_image() vp_export_image() must be consumed by one of: vp_release_image() vp_show_image() or by return from pull_image or get_image functions. Basic filters API should implement: mp_image_t *pull_image(vp_link_t *link, long long pts, int mode, int flags); Called when one of the filter's output links desires a frame. If the filter or decoder can provide one, it should return an image. Mode may be one of: VPP_MODE_FRAME: Get nearest frame before pts VPP_MODE_KEYFRAME: Get nearest keyframe before pts VPP_MODE_SLICE: Get next slice for current pts if VPP_SEEK is not set. Otherwise reposition stream and get first slice of nearest frame before pts (may have weird effects!) (Note: If some filter between decoder and caller of vp_pull_image() doesn't support slices then work as in VPP_MODE_FRAME) Value of flags is interpreting as follows: VPP_DECODE_ORDER: Get frame in decoder order if supported. Slices will be pulled always in decode order. (Note: If some filter between decoder and caller of vp_pull_image() doesn't accept decoder order then that flag will be seized) VPP_CANSKIP: Set if decoder may skip frames up to nearest frame before pts if there are any. It does the effect of '-hardframedrop' switch and may corrupt the image. VPP_CANDROP: Set if decoder or filter may set MF_DROP flag in image. VPP_SEEK: If that flag is set and pts points to the same frame as on previous call then decoder must just give out the same frame again (regetting frame). Otherwise if that flag is set then reposition the stream to near pts and get the frame. This function is main for filter functionality so it's most complex and important. But it's a good thing that video layer will make it simple as much as possible. When decoder or filter already has an image which meet request then it will return it, else it will get next image from it's source(s). If it need a buffer for processing it may get it from any link. If decoder requested for full frame but have only slice it may return slice, then video layer will render slice into frame itself. If filter may work with frame it got from source without additional buffer then it should process frame and then return it. For example, filter which just counts valid frames may be such simple: mp_image_t *pull_image(vp_link_t *link, long long pts, int mode, int flags) { mp_image_t *img; if ((img = vp_pull_image(link->src->in, pts, mode, flags)) == NULL) return NULL; link->src->priv->count++; return img; } Decoder behavior: 1) if VPP_SEEK is set: a) previous image pts is equal to requested pts: return previous image; b) else reposition source and return a frame with MF_NOT_IN_ORDER flag; 2) next image pts is above of requested pts (previous frame continues): return NULL; 3) next image pts is below or equal of requested pts and above of pts of previous request: a) image damaged: release buffer and return NULL; b) else process image; 4) next image pts is below of pts of previous request (there is extra frame in queue): a) VPP_CANSKIP is set: don't process it but get next frame from source until frame's pts is above of pts of previous request; b) VPP_CANDROP is set: process a frame but set MF_DROP flag; c) else just process a frame. Filter behavior: 1) VPP_CANDROP is unset: process image as usual; 2) MF_DROP is set: if filter wants to process the image then process it and export it; 3) VPP_CANDROP is set but MF_DROP is unset: process image as usual and may set MF_DROP in resulting image. Decoder may return slice even if VPP_MODE_SLICE is unset. In that case video layer will render that slice for next filter in pipeline itself. VO and encoders API should implement: int process_image(vp_node_t *node, mp_image_t *img); ??????? Advanced API for filters to implement: int open(vp_node_t *node); As with all modules in the G2 framework, the open function is called to initialize a module instance after loading the module. If open() is implemented then it may perform some initialization tasks. Returns 0 on failure or 1 on success. void close(vp_node_t *node); Free all resources allocated by the filter and return to pre-open state. This includes closing all links in *xin and *xout lists, if filter handles ones. int query_format(vp_link_t *link, fmtdesc_t *fmt); Report whether the filter can accept images in the specified format. In deciding how to respond, the filter should consider both (a) whether it can process images in the specified format, and (b) whether subsequent filters will be able to process its output, if it uses the requested input format. The return value should be: 0, if the format causes no conflicts 1, if this filter chain cannot process the format A filter that does not implement query_format implicitly accepts any format except compressed and special types! int config(vp_link_t *link, int w, int h, int samp_w, int samp_h, fmtdesc_t *fmt); ??????? void reset(vp_node_t *node, config_data_t *cfg); If implemented then reset function will apply new settings to current filter instance. It should not perform any dropping data or unlinking any link. int get_buffer(vp_link_t *link, mp_image_t *img); Used for direct rendering and/or slices, the get_buffer function may be requested by one of link's filter to provide a buffer to render into. Requested buffer may be DIRECT or EXPORT type. If the filter can provide a buffer matching the requested buffer memory flags then the filter should setup the buffer pointers in the image structure passed to it, and store any private data it needs to keep track of the buffer. If a suitable buffer is not available, get_buffer should return 0 to indicate failure. Otherwise it should return 1 for success. If get_buffer isn't implemented then video layer will reexport buffer from next filter in pipeline. void release_buffer(vp_node_t *self, bufdesc_t *buf); When a buffer's lock count reaches zero, its owner's release_buffer function is called. The image type may be EXPORT or DIRECT. int request_link(vp_link_t *link, int src); That function designed for filters that may (and/or must) have more than one input or output stream. If implemented then it must do some link management (sorting, etc.) at vp_node_t structure. If src is 0 then request_link was called to add output for node link->src. If src is 1 then request_link was called to add source to node link->dst. Returns 0 if no more links supported or 1 on success. If request_link not implemented then video layer assumes filter may have only one source and output links and do that management itself. int close_link(vp_link_t *link, int src); That function designed for filters that may (and/or must) have more than one input or output stream. If implemented then it must do some link management (sorting, etc.) at vp_node_t structure. If src is 0 then close_link was called to remove output for node link->src. If src is 1 then close_link was called to remove source from node link->dst. Returns 0 if no such link removed or 1 on success. If close_link not implemented then video layer assumes filter may have only one source and output links and do that management itself. /* main vp API - use it instead of direct access to vp_node_s */ vp_node_t *vp_open(module_info_t *module, config_data_t *cfg); As with all modules in the G2 framework, the open function is called to initialize a module instance after loading the module. It should setup all node structure setup, including startup settings, and may perform other initialization tasks. Function may return NULL in case of failure of initialization. void vp_close(vp_node_t *self); Free all resources allocated by the filter, destroy all links and return to pre-open state. This includes converting any locked images to AUTO type. vp_link_t *vp_request_link(vp_node_t *dst, vp_node_t *src); Try to link output of node src and input of node dst together. Returns newly created link or NULL on failure. void vp_close_link(vp_link_t *link); Destroy link and release all images that belong to it. void vp_reset(vp_node_t *node, config_data_t *cfg); Apply new settings to current filter instance. mp_image_t *vp_pull_image(vp_link_t *link, long long pts, int mode, int flags); Really does the same as pull_image filter function, see description above. But functionality is extended: vp_pull_image() does export implicitly so returned image always belongs to link; it also compose slices into frame if decoder returns slice instead of frame. int vp_query_format(vp_link_t *head, fmtdesc_t *fmt, int force); Report whether the filter chain can accept images in the specified format. If there was a problem and force is set then function tries to resolve it by inserting transcoding filter into conflicting link. The return value should be: 0, if the format causes no conflicts >0, if this chain cannot process the format int vp_config(vp_link_t *link, int w, int h, int samp_w, int samp_h, fmtdesc_t *fmt); ??????? mp_image_t *vp_get_image(vp_link_t *link, int type, int flags, int x, int y, int w, int h); Used for direct rendering and/or slices, the vp_get_image function requests one of link's filter to provide a buffer for the caller to render into. See description above about type and flags for buffer. If a suitable buffer is not available then vp_get_image() should return NULL. Otherwise it should return pointer to image structure. void vp_release_image(mp_image_t *img); Decrement reference count for image. If no references left then image and buffer should be freed. int vp_show_image(vp_node_t *node, mp_image_t *img); ??????? /* additional vp API */ mp_image_t *vp_keep_image(mp_image_t *img); This should be used by filters that want to keep their input images beyond subsequent requests, which could clobber the input if it lies in a reusable buffer. mp_image_t *vp_export_image(vp_link_t *link, mp_image_t *orig); Export current image to link, i.e. create new image descriptor belonging to link from orig descriptor and return new descriptor. That function helps when you want preserve your descriptor and buffer for later. Note that buffer may be overwritten by following filters so if you want keep buffer contents too then you should use vp_keep_image() instead. So if your pull_image function has: return img; then you'll lost that img descriptor but if instead it'll have: return vp_export_image(link, img); then your img will be preserved for you. void vp_select_stride(int *stride, stride_rest_t *r, fmtdesc_t *fmt); ??????? -------------- next part -------------- #include "cfg.h" // filter capabilities/limitations #define VP_CAPS_ACCEPT_SLICES (1<<16) /* if filter accepts slices */ #define VP_CAPS_NO_DECODE_ORDER (1<<17) /* if filter doesn't accept decode order */ typedef struct vp_node_s vp_node_t; typedef struct vp_priv_s vp_priv_t; #include "imgfmt.h" // BUFFER DESCRIPTOR // Stores layout of the actual buffers and nothing else. // Often this can rereferenced when reexporting or re-dr'ing. // Owner is node which allocated a buffer, if there is any // Buffer management types #define BUF_TYPE_DUMMY 0 // no buffers needed (dropped frame or eos) #define BUF_TYPE_AUTO 1 // auto-managed by link layer (internal purpose?) #define BUF_TYPE_EXPORT 2 // exported from source to dest #define BUF_TYPE_DIRECT 3 // exported from dest to source (DR) //#define BUF_TYPE_INDIRECT 4 // indirect rendering via slices, no pointers // Memory restrictions #define BUF_MEM_READABLE 0x1 // buffer is in readable memory #define BUF_MEM_PRESERVE 0x2 // dest will not clobber buffer contents #define BUF_MEM_REUSABLE 0x4 // source can reuse & re-return buffer >1 times typedef struct { int type; int flags; vp_node_t *owner; /* who allocated this buffer */ fmtdesc_t *fmt; int lock; /* reference count */ int w[MAX_PLANES]; int h[MAX_PLANES]; int stride[MAX_PLANES]; /* size of pixel */ int size[MAX_PLANES]; unsigned char *planes[MAX_PLANES]; unsigned char *palette; void *priv; // private data for owner } bufdesc_t; // META-FRAME // Describes the properties of an image which pertain to it being // a frame of a movie. Often this can be copied as-is when only // filtering the contents of the image. #define MF_SLICE (1<<0) /* it's not full frame but slice */ #define MF_FINAL (1<<1) /* it's last slice or EOF */ #define MF_DROP (1<<2) /* set to 1 if frame is out of pts */ #define MF_DECODE_ORDER (1<<3) /* if frame is in decode order */ #define MF_NOT_IN_ORDER (1<<4) /* if frame isn't in display order */ #define MF_EOS (1<<5) /* set by decoder when no images left */ typedef struct { long long pts; // absolute pts of this frame, -1 if unknown int rel_pts; // pts relative to previous frame (required) int duration; // duration of frame, -1 if unknown int color_type; int pict_type; int field_based; int field_flags; int x, y; // disposition of slice int w, h; // size of slice int flags; } imagedesc_t; // MPLAYER IMAGE // This is the main image structure used for passing frames between // nodes of the pipeline. Each image is always associated with a link. // The elements of mp_image_t are divided into substructs according // to whether they are properties of the frame the image represents, // or properties of the buffer in which it is stored. This allows one // or the other to easily be copied without modification. typedef struct { vp_link_t *link; imagedesc_t mf; bufdesc_t *buf; // pointer to buffer for image bufdesc_t bb; // builted-in buffer, may be unused } mp_image_t; // STRIDE RESTRICTIONS #define STRIDE_ALIGN 0x01 #define STRIDE_EXACT 0x02 #define STRIDE_STATIC 0x04 #define STRIDE_POSITIVE 0x08 #define STRIDE_NEGATIVE 0x10 #define STRIDE_COMMON 0x20 #define STRIDE_COMMON_UV 0x40 #define STRIDE_COMMON_PLANE 0x80 typedef struct { int flags; int align[MAX_PLANES]; int stride[MAX_PLANES]; } stride_rest_t; typedef struct { vp_node_t *src, *dst; fmtdesc_t *fmt; int w, h; int samp_w, samp_h; long long last_pts; stride_rest_t sr; } vp_link_t; typedef struct { // Main vp api int (*open)(vp_node_t *node); void (*close)(vp_node_t *node); void (*reset)(vp_node_t *node, config_data_t *cfg); // if called for dest node then src==0, otherwise - for src node int (*request_link)(vp_link_t *link, int src); int (*close_link)(vp_link_t *link, int src); mp_image_t *(*pull_image)(vp_link_t *link, long long pts, int mode, int flags); int (*query_format)(vp_link_t *link, fmtdesc_t *fmt); int (*config)(vp_link_t *link, int w, int h, int samp_w, int samp_h, fmtdesc_t *fmt); // Direct rendering int (*get_buffer)(vp_link_t *link, mp_image_t *img); void (*release_buffer)(vp_node_t *self, bufdesc_t *buf); // Slices // void (*attach_slices)(vp_link_t *link, mp_image_t *img); // void (*detach_slices)(vp_node_t *self, mp_image_t *img); // void (*draw_slice)(vp_link_t *link, mp_image_t *img, // unsigned char **src, int *stride, int x, int y, int w, int h); // void (*commit_slice)(vp_link_t *link, mp_image_t *img, int x, int y, int w, int h); // VO API int (*process_image)(vp_node_t *node, mp_image_t *img); } vp_functions_t; struct vp_node_s { module_info_t *info; config_data_t *cfg; vp_functions_t *func; // Links, primary & extra vp_link_t *in, *out; vp_link_t **xin, **xout; // must be handled only by filter! mp_image_t *pending; // Private data for this node of the pipeline vp_priv_t *priv; }; /* flags for vp_pull_image() */ #define VPP_MODE_FRAME 0 /* get nearest frame */ #define VPP_MODE_KEYFRAME 1 /* get nearest keyframe */ #define VPP_MODE_SLICE 2 /* get next slice */ #define VPP_DECODE_ORDER (1<<0) /* get frame in decoder order */ #define VPP_CANSKIP (1<<1) /* decoder can skip frame (-hardframedrop) */ #define VPP_CANDROP (1<<2) /* filter/decoder may set MF_DROP */ #define VPP_SEEK (1<<3) /* seek or reget the same frame */ /* main vp API - use it instead of direct access of vp_node_s */ vp_node_t *vp_open(module_info_t *module, config_data_t *cfg); void vp_close(vp_node_t *self); void vp_reset(vp_node_t *node, config_data_t *cfg); vp_link_t *vp_request_link(vp_node_t *dst, vp_node_t *src); void vp_close_link(vp_link_t *link); mp_image_t *vp_pull_image(vp_link_t *link, long long pts, int mode, int flags); int vp_query_format(vp_link_t *head, fmtdesc_t *fmt, int force); int vp_config(vp_link_t *link, int w, int h, int samp_w, int samp_h, fmtdesc_t *fmt); mp_image_t *vp_get_image(vp_link_t *link, int type, int flags, int x, int y, int w, int h); void vp_release_image(mp_image_t *img); int vp_show_image(vp_node_t *node, mp_image_t *img); /* additional vp API */ mp_image_t *vp_keep_image(mp_image_t *img); mp_image_t *vp_export_image(vp_link_t *link, mp_image_t *orig); void vp_select_stride(int *stride, stride_rest_t *r, fmtdesc_t *fmt); //int vp_attach_slices(vp_link_t *link, mp_image_t *img); //void vp_draw_slice(vp_link_t *link, mp_image_t *img, unsigned char **src, int *stride, int x, int y, int w, int h); //void vp_commit_slice(vp_link_t *link, mp_image_t *src, int x, int y, int w, int h); -------------- next part -------------- #include #include #include "cfg.h" #include "vp.h" #define IMAGE_POOL_SIZE 32 /* minimal pool size */ typedef struct image_pool { mp_image_t pool[IMAGE_POOL_SIZE]; struct image_pool *next; } image_pool_t; static image_pool_t *image_pool = NULL; static mp_image_t *vp_allocate_image(void) { mp_image_t **pool = &image_pool; int i; #define image(x) (*pool)->pool[x] while(*pool) { for (i = 0; i < IMAGE_POOL_SIZE; i++) { if (!image(x).link && !image(x).pp.lock) return &image(x); // we check both ->link and pp.lock now since: // link==NULL and pp.lock!=0 when buffer still used by another link // link!=NULL and pp.lock==0 when this is exported image } pool = &(*pool)->next; } /* we cannot do realloc here, all references (i.e. all images) will be invalid after realloc :( */ *pool = calloc(1, sizeof(image_pool_t)); return &image(0); #undef image } vp_link_t *vp_request_link(vp_node_t *dst, vp_node_t *src) { vp_link_t *link = malloc(sizeof(vp_link_t)); if ((!dst->func->request_link && dst->in) || (!src->func->request_link && src->out)) return NULL; // src or dst is already linked link = malloc(sizeof(vp_link_t)); link->src = src; link->dst = dst; link->fmt = NULL; link->last_pts = -1; memset(&link->sr, 0, sizeof(stride_rest_t)); if (dst->func->request_link) // try to add link to destination node { if (!dst->func->request_link(link, 1)) { free(link); return NULL; } } else dst->in = link; if (src->func->request_link) // try to add link to source node { if (!src->func->request_link(link, 0)) { if (dst->func->close_link) dst->func->close_link(link, 1); else dst->in = NULL; free(link); return NULL; } } else src->out = link; return link; } void vp_close_link(vp_link_t *link) { mp_image_pool **pool = &image_pool; /* close the link */ if (src->func->close_link) src->func->close_link(link, 0); else src->out = NULL; if (dst->func->close_link) dst->func->close_link(link, 1); else dst->in = NULL; /* release all images owned by link */ while (*pool) { for (i = 0; i < IMAGE_POOL_SIZE; i++) if ((*pool)->pool[x].link == link) vp_release_image(&(*pool)->pool[x]); pool = &(*pool)->next; } /* release memory */ free(link); } vp_node_t *vp_open(module_info_t *module, config_data_t *cfg) { vp_node_t *node; /* create an instance */ node = malloc(vp_node_t); node->info = module; node->func = (vp_function_t *)module->entry; node->cfg = mpconfig_get_cfg(module, cfg); node->in = node->out = NULL; node->xin = node->xout = NULL; node->pending = NULL; node->priv = NULL; /* setting up the instance */ if (node->func->open && !node->func->open(node)) { mpconfig_release_cfg(module, node->cfg); free(node); return NULL; } return node; } void vp_reset(vp_node_t *node, config_data_t *cfg) { if (!cfg) return; // nothing to do? if (node->func->reset) // let filter do it nice node->func->reset(node, cfg); else // it cannot do it so just replace config data { mpconfig_release_cfg(node->info, node->cfg); node->cfg = mpconfig_get_cfg(node->info, cfg); } } void vp_close(vp_node_t *node) { int i; mp_image_pool **pool = &image_pool; bufdesc_t *buf; unsigned char *planes[MAX_PLANES]; int size; /* does it has buffers in pool? if so then, sorry, replace them with AUTO */ while (*pool) { for (i = 0; i < IMAGE_POOL_SIZE; i++) { buf = &(*pool)->pool[x].bb; if (buf->lock && buf->owner == node) { size = 0; for (i = 0; i < buf->fmt->num_planes; i++) size += buf->size[i]; planes[0] = memalign(64, size); for (i = 1; i < buf->fmt->num_planes; i++) planes[buf->fmt->order[i]] = planes[i-1] + buf->size[i-1]; memcpy_pic_planes(planes, buf->planes, buf->w, buf->h, buf->stride, buf->stride, buf->fmt); if (node->release_buffer) node->release_buffer(node, buf); memcpy(&buf->planes, &planes, MAX_PLANES*sizeof(unsigned char *)); buf->owner = NULL; buf->type = BUF_TYPE_AUTO; } } pool = &(*pool)->next; } if (node->func->close) node->func->close(node); // it handles xin and xout lists too! if (node->in) vp_close_link(node->in); if (node->out) vp_close_link(node->out); mpconfig_release_cfg(node->info, node->cfg); free(node); } #if 0 ////int vp_config(vp_link_t *link, int w, int h, int samp_w, int samp_h, fmtdesc_t *fmt); int vp_config(vp_link_t *link, fmtdesc_t *fmt, int w, int h, int samp_w, int samp_h, fmtdesc *fmt) { vp_node_t *src = link->src, *dest = link->dest; if (!dest->query_format(link, fmt)) { vp_insert_filter(link, vp_open_filter(vp_find_filter("scale"), NULL)); dest = link->dest; } if (!dest->config(link, fmt, w, h, samp_w, samp_h, time_n, time_d)) return 0; link->fmt = fmt; link->w = w; link->h = h; link->samp_w = samp_w; link->samp_h = samp_h; link->time_n = time_n; link->time_d = time_d; return 1; } // Before calling, the stride restrictions must already be checked for // consistency! Otherwise the results are undefined! The stride array // should be initialized with width or desired stride. ////void vp_select_stride(int *stride, stride_rest_t *r, fmtdesc_t *fmt); void vp_select_stride(int *stride, stride_rest_t *r, fmtdesc_t *fmt) { int s[MAX_PLANES]; if (r->flags & STRIDE_EXACT) { for (i=0; inum_planes; i++) stride[i] = r->stride[i]; return; } for (i=0; inum_planes; i++) s[i] = abs(stride[i]); for (i=0; inum_planes; i++) { if (r->flags & STRIDE_EXACT) s[i] = r->stride[i]; if (r->flags & STRIDE_ALIGN && s[i]%r->align[i]) s[i] += r->align[i] - s[i]%r->align[i]; if (r->flags & STRIDE_COMMON_STRIDE) s[i] = s[0] >> fmt->ssx[i]; } for (i=0; inum_planes; i++) { if ( !(r->flags & STRIDE_POSITIVE) && (stride[i] < 0 || (r->flags & STRIDE_NEGATIVE)) ) stride[i] = -s[i]; else stride[i] = s[i]; } } #endif mp_image_t *vp_get_image(vp_link_t *link, int type, int flags, int x, int y, int w, int h) { fmtdesc_t *fmt = link->fmt; vp_link_t *dstlink = link; mp_image_t *img; bufdesc_t *buf; int size; /* check for get_buffer and try to get thru chain */ if (type == BUF_TYPE_DIRECT) while (!link->dst->get_buffer) { if (!link->dst->out) return NULL; link = link->dst->out; } else if (type == BUF_TYPE_EXPORT) while (!link->src->get_buffer) { if (!link->src->in) return NULL; link = link->src->in; } /* some initial setup for buffer */ img = vp_allocate_image(); buf = img->buf = &img->bb; img->mf.x = x; img->mf.y = y; img->mf.w = w; img->mf.h = h; img->mf.pts = -1; img->mf.duration = -1; buf->type = type; buf->flags = flags; buf->fmt = fmt; for (i = 0; i < fmt->num_planes; i++) { buf->w[i] = w >> fmt->ssx[i]; buf->h[i] = h >> fmt->ssy[i]; buf->stride[i] = link->stride[i]; buf->size[i] = buf->h[i]*abs(buf->stride[i]); } switch (type) { case BUF_AUTO: size = 0; for (i = 0; i < fmt->num_planes; i++) size += buf->size[i]; buf->planes[0] = memalign(64, size); // order planes in "common_plane" layout for (i = 1; i < fmt->num_planes; i++) buf->planes[fmt->order[i]] = buf->planes[i-1] + buf->size[i-1]; for (i = 0; i < fmt->num_planes; i++) memset(buf->planes[i], fmt->black[i], buf->size[i]); case BUF_DUMMY: buf->owner = NULL; break; case BUF_EXPORT: buf->owner = link->src; if (!link->src->func->get_buffer(dstlink, img)) return NULL; break; case BUF_DIRECT: buf->owner = link->dst; if (!link->dst->func->get_buffer(dstlink, img)) return NULL; break; default: // unknown image type! return NULL; } img->link = dstlink; buf->lock++; return img; } // This should be used by filters that want to keep their input images // beyond subsequent requests, which could clobber the input if it // lies in a reusable buffer. mp_image_t *vp_keep_image(mp_image_t *img) { mp_image_t *copy; img->buf->lock++; if (!(img->buf->flags & BUF_MEM_REUSABLE)) return img; copy = vp_get_image(img->link, BUF_TYPE_AUTO, 0, img->mf.x, img->mf.y, img->mf.w, img->mf.h); memcpy_pic_planes(copy->buf->planes, img->buf->planes, img->mf.w, img->mf.h, copy->buf->stride, img->buf->stride, img->link->fmt); vp_release_image(img); return copy; } mp_image_t *vp_export_image(vp_link_t *link, mp_image_t *orig) { mp_image_t *img = vp_allocate_image(); if (orig->buf->fmt != link->fmt) // ugly filter!!! { // cannot just copy between formats!!! mp_error("frame transferred to another format, dropped!"); return NULL; } img->buf = orig->buf; // buffer is the same img->buf->lock++; // but mark it used one more time img->link = link; memcpy(&img->mf, &orig->mf, sizeof(imagedesc_t)); // just copy it return img; } void vp_release_image(mp_image_t *img) { bufdesc_t *buf = img->buf; img->link = NULL; // it's unused now if (--buf->lock) return; if (buf->owner) // BUF_TYPE_EXPORT and BUF_TYPE_DIRECT { if (buf->owner->func->release_buffer) // hmm, it doesn't worry to deallocate? buf->owner->func->release_buffer(buf->owner, buf); buf->owner = NULL; return; } if (buf->type != BUF_TYPE_AUTO) return; free(buf->planes[0]); // deallocate memory } int vp_query_format(vp_link_t *link, fmtdesc_t *fmt, int force) { if (!link || !link->dst) return 0; // dead end, assume it's ok if (!link->dst->func->query_format) // assume it accepts any format return vp_query_format(link->dst->out, fmt, force); if (!link->dst->func->query_format(link, fmt)) return 0; if (!force) return 1; ??????? // try auto-insert format converting filter before link->dst } ////int vp_show_image(vp_node_t *node, mp_image_t *img); #if 0 static int vp_attach_slices(vp_link_t *link, mp_image_t *img) { vp_node_t *dest = link->dest; mp_image_t *img2; if (img->buf.type != BUF_TYPE_AUTO && img->buf.type != BUF_TYPE_EXPORT) return 0; if (dest->attach_slices) return dest->attach_slices(link, img); img2 = vp_get_image(link, BUF_TYPE_INDIRECT, 0); } static void vp_draw_slice(vp_link_t *link, mp_image_t *img, unsigned char **src, int *stride, int x, int y, int w, int h) { if (img->buf.type == BUF_TYPE_DIRECT) memcpy_subpic_planes(img->buf.planes, src, x, y, 0, 0, w, h, img->buf.stride, stride, img->fmt); if (link->dest->draw_slice) link->dest->draw_slice(link, img, src, stride, x, y, w, h); } static void vp_commit_slice(vp_link_t *link, mp_image_t *src, int x, int y, int w, int h) { if (src->target->buf.type == BUF_TYPE_DIRECT) memcpy_subpic_planes(src->target->buf.planes, src->buf.planes, x, y, 0, 0, w, h, src->target->buf.stride, src->buf.stride, dest->fmt); if (link->dest->commit_slice) link->dest->commit_slice(link, src, x, y, w, h); else if (link->dest->draw_slice) { int i; unsigned char *src2[MAX_PLANES]; for (i=0; ifmt->num_planes; i++) src2[i] = src->buf.planes[i] + src->buf.stride[i]*(y>>link->fmt->ssy[i]) + ((link->fmt->bpp[i]*(x>>link->fmt->ssx[i]))>>3); link->dest->draw_slice(link, dest, src2, src->buf.stride, x, y, w, h); } } #endif mp_image_t *vp_pull_image(vp_link_t *link, long long pts, int mode, int flags) { mp_image_t *img; /* getting the same image again? may be recursion? avoid it! */ if (pts == link->last_pts && !(flags & VPP_SEEK)) return NULL; /* I think it's impossible but... */ if (!link->src || !link->src->func->pull_image) return NULL; /* erasure for flags in dependence of filter capabilities */ if (link->dst && !(link->dst->info->flags & VP_CAPS_ACCEPT_SLICES)) mode &= ~VPP_MODE_SLICE; if (link->dst && (link->dst->info->flags & VP_CAPS_NO_DECODE_ORDER)) flags &= ~VPP_DECODE_ORDER; /* call function and return if no image ready yet */ if ((img = link->src->func->pull_image(link, pts, mode, flags)) == NULL) return NULL; /* if it's a slice but slice wasn't requested then commit seq. slices */ if ((img->mf.flags & MF_SLICE) && mode != VPP_MODE_SLICE) { mp_image_t *img2; if (!(img2 = vp_get_image(link, BUF_TYPE_DIRECT, 0, 0, 0, link->w, link->h))) img2 = vp_get_image(link, BUF_TYPE_AUTO, 0, 0, 0, link->w, link->h); while (img) { /* commit slice - I'm afraid it's never possible to do it through filter chain so let's just copy it */ memcpy_subpic_planes(img2->buf->planes, img->buf->planes, x, y, 0, 0, w, h, img2->buf->stride, img->buf->stride, link->fmt); vp_release_image(img); // line below is valid until multithreaded if ((img->mf.flags & MF_SLICE) && !(img->mf.flags & MF_FINAL)) img = link->src->func->pull_image(link, pts, mode, flags); else break; } img = img2; } /* if image was completed then check some fields and return it */ if (img->buf->fmt != link->fmt) // ugly filter!!! { // cannot just copy between formats!!! vp_release_image(img); mp_error("frame transferred to another format, dropped!"); return NULL; } link->last_pts = img->mf.pts; img->link = link; return img; } From dalias at aerifal.cx Wed Apr 14 15:25:08 2004 From: dalias at aerifal.cx (D Richard Felker III) Date: Wed, 14 Apr 2004 09:25:08 -0400 Subject: [MPlayer-G2-dev] Yet another video API. :) In-Reply-To: <20040414112216.GA27461@lucky.net> References: <20040414112216.GA27461@lucky.net> Message-ID: <20040414132507.GC250@brightrain.aerifal.cx> On Wed, Apr 14, 2004 at 02:22:16PM +0300, Andriy N. Gritsenko wrote: > Image flags: > > MF_SLICE: Set if image isn't full image but slice instead. Filters who > don't accept slices will anyway get a full frame because video layer > will commit all slices to frame for it. > > MF_FINAL: Set if image is last slice of a frame so all slices before it > may be committed to full frame. You fail to address what "all slices before it" means. This is the big question. > MF_DROP: Set if decoder or filter may find that: a) there is yet one > frame before pts; b) that frame wants to be dropped. So it marks > the frame as may be dropped and any filter except those who desires > all frames may just ignore it. This is meaningless. Read my drop spec. Dropping works the other way around. The source of a frame only "drops" it if it's damaged beyond repair and can't be decoded. If the destination is dropping it, it doesn't need a signal from the source to do so. It already knows. > MF_DECODE_ORDER: Set by decoder to indicate that frame was pulled in > decode order. This is also nonsense. > MF_NOT_IN_ORDER: Set by decoder to indicate that frame was pulled in > decode order and is out of order. For example in standard MPEG sequence > 1 4 2 3 7 5 6 if frames are pulled in decode order then frames 4 and 7 > will get that flag. More nonsense. Learn how slices/dr work. Something like this is not necessary or useful! > Filters API has number of functions. Each function should be not called > directly but only via vp_* API calls. > > Most common rule is "if you get an image buffer then you _MUST_ return > it to owner", i.e. each vp_get_image() or vp_pull_image() must be > followed by either vp_release_image() either returned from function. ROTFL! This makes buffer management useless. > It is also available to call vp_keep_image() to save you a copy of > image or vp_export_image() to save you an image descriptor but that > keeped image also must be followed by one of these functions later. > I hope that rule is simple. If you follow that rule then you'll never > get your image lost or get memory leak. This seems to contradict what you said above. > Called when one of the filter's output links desires a frame. If the > filter or decoder can provide one, it should return an image. Mode may > be one of: > > VPP_MODE_FRAME: Get nearest frame before pts > > VPP_MODE_KEYFRAME: Get nearest keyframe before pts This is all stupid. These are seek commands, not the way you get a frame. When requesting a frame, you DO NOT KNOW what pts you want. You just get frames in sequence. If you don't want some of them, throw them away!! > VPP_MODE_SLICE: Get next slice for current pts if VPP_SEEK is not set. > Otherwise reposition stream and get first slice of nearest frame before > pts (may have weird effects!) Sounds dumb. > VPP_DECODE_ORDER: Get frame in decoder order if supported. Slices will > be pulled always in decode order. > (Note: If some filter between decoder and caller of vp_pull_image() > doesn't accept decoder order then that flag will be seized) Useless. RTFM slices/dr! > VPP_CANSKIP: Set if decoder may skip frames up to nearest frame before > pts if there are any. It does the effect of '-hardframedrop' switch and > may corrupt the image. > > VPP_CANDROP: Set if decoder or filter may set MF_DROP flag in image. > > VPP_SEEK: If that flag is set and pts points to the same frame as on > previous call then decoder must just give out the same frame again > (regetting frame). Otherwise if that flag is set then reposition the > stream to near pts and get the frame. WTF. What you have submitted is NOT a new design. It's my design, with lots of stupid crap added on top of it. It doesn't fix any of the design flaws in my spec; it just adds a lot more. If you want to propose a new spec, fine. But don't rehash old stuff, bloat it with useless "features" that were already addressed if you had taken the time to read the old spec, and call it something new... Rich From dalias at aerifal.cx Wed Apr 14 18:09:30 2004 From: dalias at aerifal.cx (D Richard Felker III) Date: Wed, 14 Apr 2004 12:09:30 -0400 Subject: [MPlayer-G2-dev] Yet another video API. :) In-Reply-To: <20040414112216.GA27461@lucky.net> References: <20040414112216.GA27461@lucky.net> Message-ID: <20040414160929.GG250@brightrain.aerifal.cx> On Wed, Apr 14, 2004 at 02:22:16PM +0300, Andriy N. Gritsenko wrote: > Hi! > > So after some discussions and rethinking whole thing, I've made this. BTW, one more thing. There's no need to C&P the entire old proposal and all my code (unmodified!) into one huge email that takes forever to read. Just writing your proposed changes would be sufficient. Rich From mr at ramendik.ru Sat Apr 17 20:35:24 2004 From: mr at ramendik.ru (Mikhail Ramendik) Date: Sat, 17 Apr 2004 22:35:24 +0400 Subject: [MPlayer-G2-dev] An open, standard video/audio API? Message-ID: <1082226924.3729.75.camel@localhost.localdomain> Hello, (SUMMARY. A general-purpose standard API for all multimedia processing, including codecs, filters, demuxing etc., would be very useful in GNU/Linux, and resolve many existing problems, including licensing. The MPlayer team is the best source for a standard that could be widely adopted. I would volunteer to do tech writing work on the spec). (DISCLAIMER. The following may need a sanity check; i.e. I may have lost contact with reality. If so please tell me where). I'm a user of mplayer, and I like it a big darn lot, so thanks people. (I know it's not exactly the right list for this but I had to put it in my first message to an mplayer devel list). I'm also a big fan of Open Standards. Free Software too - but while Free Software would totally rule in an ideal world, Open Standards totally rule *now*, they make the Internet run. They also encourage the development of Free Software, and minimize the effects of non-free software. And the one thing that I feel is missing in [GNU/]Linux, while present (although crippled) in the Non-Free OS, is a clearly defined standard API for all multimedia stuff. A Free Software masterpiece, VirtualDub, is made possible by DirectShow. A clear (hmm, theoretically) interface to all possible codecs, the AVI file format, and some other stuff is just already there. Yes, it's stinky (the 4GB limit, and of course the lack of freedom). But GNU/Linux has none whatsoever. Video4Linux2 contains some Codec API, but who uses it? As a result, a GNU/Linux program that wants to use codecs, filters, etc, has to care about interaction with every library out there. Look at transcode, which had to actually include a copy of ffmpeg. (And no, one can not just forget everything else and go ffmpeg only). >From the discussion I saw here, it seems that while the MPLayer G2 core could be a sort of "clearing center" for multimedia processing under GNU/Linux, it will still have a pretty specific API. My question: why not aim for a standard, generic, published, well-described multimedia processing API, one that would (because of its technical benefits and open nature) far surpass DirectShow? It should work across machines (not to speak of working across very different programs). X-Windows might provide an example. And yes, I would support sockets as the main tool, and TCP sockets where possible. Among other things, this will allow interoperation of different machines, as already done in X. Just imagine: one machine to grab the source and apply pre-procesing filters, another to do the coding, and a third to stream the content, all communicating over the Open Multimedia Protocol. This would be the *easy* scenario; hard ones would include load balancing. In most cases of course the API would work on the same machine - but still provide nice separation for different tasks, so loved by the Unix world. A codec, a filter, a muxer or demuxer, a player would all communicate via socket streams. Each one easily replaceable, without micro-management from the player (common today). A user has installed a cool hardware MPEG5 decoder. He previously used to run MPlayer G2 for playing MPEG5 files; his brother ran XMMS G3, and used Transcode to convert MPEG5 to Theora for watching on his Zaurus handheld. Now the just replaces the primary system codec for MPEG5. Neither MPlayer nor XMMS nor Transcode will ever notice (unless they were configured to run a soecific codec by name, overriding system settings). Among other matters, an open socket-based standards might resolve the recent licensing issue. The problem with hardware player developers, as it seems to be, is that they just can't release all of the source to their firmware, because of licensing matters with other companies. But what if the components they use will all interact via a standard socket interface? In this way, as the GPL FAQ exlicitly says, they can use GPL-ed and non-GPL modules together! So they can use some modules from MPlayer, AND be forced to release any changes to these modules, AND be able to pay MPlayer developers for enhancing them. But if their proprietary modules use the standard protocol, they'll be able to keep them - and honor any licensing requirement with any third company. It's stricter than the LGPL. They will be forced to adhere to the protocol and use sockets, rather than access system calls in a particular library. In fact, any DRM they install will be easy to circumvent - the output will go through the standard stream protocol, so go intercept it, and whoops! we're done. If this standard is adopted widely enough, the DRM-mongers will either have to stop using all newer versions of Linux video tools altogether, or see their data intercepted and their DRM broken by the nearest hacker who knows how to read a manual for a standard. (They still have the option of preventing physical access inside the hardware, so that one will be unable to do this interception; but one can't fix that with software licenses. That would work with 100% GPL too, just keep the key values [data not code!] in a chip inside). MPlayer is widely accepted as the best multimedia player under GNU/Linux; ffmpeg/libavcodec, as the best codec library; MEncoder and transcode are the two commonly used processing solutions. Therefore, I think the MPlayer team would be the best people for developing and promoting this standard. Now for a bit of personal stuff. I am a tech writer with over 5 years of experience behind me. English is not my native language, but I am employed as a tech writer in English by a major international company. For money, I write development docs for internal-use software (it's not distributed outside of the company) - and they like the work I do. I would be willing to take up the writing of the spec. Not the technical side of the development - I don't know *nearly* enough to do that. But I can glean information from programmer-speak, list discussions, etc., and convert it into an organized, readable document. And that's what an open standard spec should be, if it's to be widely adopted. (Ultimately, why not make it an RFC?) But of course, this is useful only if the developers here are willing to go that way. To develop an open standard for interaction between multimedia processing components. And to base the development of MPlayer G2 on this standard, not an internal API. This could be a crucial step for development of GNU/Linux as a user OS. This may be an insane raving by a misguided guy. ANyway, replies will be very much appreciated :) Yours, Mikhail Ramendik Moscow, Russia From diego at biurrun.de Sat Apr 17 23:11:40 2004 From: diego at biurrun.de (Diego Biurrun) Date: Sat, 17 Apr 2004 23:11:40 +0200 Subject: [MPlayer-G2-dev] An open, standard video/audio API? In-Reply-To: <1082226924.3729.75.camel@localhost.localdomain> References: <1082226924.3729.75.camel@localhost.localdomain> Message-ID: <16513.40332.236256.251292@biurrun.de> Mikhail Ramendik writes: > I would be willing to take up the writing of the spec. Not the technical > side of the development - I don't know *nearly* enough to do that. But I > can glean information from programmer-speak, list discussions, etc., and > convert it into an organized, readable document. And that's what an open > standard spec should be, if it's to be widely adopted. (Ultimately, why > not make it an RFC?) This is not exactly a direct answer, but also relevant in this area: We are in the midst of developing an open, liberally licensed and patent unencumbered container format. It is called nut. The spec is available in the MPlayer source tree in DOCS/tech/mpcf.txt or here: http://www1.mplayerhq.hu/cgi-bin/cvsweb.cgi/main/DOCS/tech/mpcf.txt?rev=1.47&content-type=text/x-cvsweb-markup The spec is not yet completely finished but it could sure use the hand of a professional technical writer once it is or maybe earlier. I assume that your help will be most welcome. A free container format is an important part of a free multimedia infrastructure, just like free codecs and maybe a standard API. Regards Diego - local technical writer and documentation maintainer From mr at ramendik.ru Sun Apr 18 00:28:36 2004 From: mr at ramendik.ru (Mikhail Ramendik) Date: Sun, 18 Apr 2004 02:28:36 +0400 Subject: [MPlayer-G2-dev] An open, standard video/audio API? In-Reply-To: <16513.40332.236256.251292@biurrun.de> References: <1082226924.3729.75.camel@localhost.localdomain> <16513.40332.236256.251292@biurrun.de> Message-ID: <1082240916.3729.85.camel@localhost.localdomain> Hello, Diego Biurrun wrote: > This is not exactly a direct answer, but also relevant in this area: > > We are in the midst of developing an open, liberally licensed and > patent unencumbered container format. It is called nut. Great. If it won't have the dreaded 4 GB limit, I'm willing to get in :) Which list was used as the main developer discussion forum? To get "into" this I should probably review the archives. > The spec is not yet completely finished but it could sure use the hand > of a professional technical writer once it is or maybe earlier. I > assume that your help will be most welcome. A free container format > is an important part of a free multimedia infrastructure, just like > free codecs and maybe a standard API. Agreed. Actually a standard container format and a standard API fit together quite well. The way I usually go about tech writing is: gather info that's available, structure it into the document, go get the missing info (questions to developers, testers, etc), structure it into the document, repeat until all info is there :) Then go through it a couple of times for readability, and go give it to as many readers as possible. Including developers, but mostly to those unfamiliar with the thing. Gather responses from developers (telling me where I got it wrong). Gather responses from the other readers. Fix the problems. At this point it's at least a good beta :) Here, a draft exists already. So I should take care to understand what it means, from the archives mostly; and if I don't understand, ask questions. Then I should fix it so that these questions, and some others I can think of, no longer arise :) I've reviewed it quickly and currently don't understand much of it. I need to see the archives before I can ask meaningful questions. Yours, Mikhail Ramendik From ivan at cacad.com Sat Apr 17 23:34:11 2004 From: ivan at cacad.com (Ivan Kalvachev) Date: Sun, 18 Apr 2004 00:34:11 +0300 (EEST) Subject: [MPlayer-G2-dev] An open, standard video/audio API? In-Reply-To: <1082226924.3729.75.camel@localhost.localdomain> References: <1082226924.3729.75.camel@localhost.localdomain> Message-ID: <1632.212.116.154.222.1082237651.squirrel@mail.cacad.com> Mikhail Ramendik said: > Hello, > > (SUMMARY. A general-purpose standard API for all multimedia > processing, including codecs, filters, demuxing etc., would > be very useful in GNU/Linux, and resolve many existing problems, > including licensing. The MPlayer team is the best source for > a standard that could be widely adopted. I would volunteer to > do tech writing work on the spec). Actually as somebody already have said there is an common api. All players under linux use ffmpeg as primary codec monkey. And the documentation of ffmpeg is smaller than the sample files. If you look for alternative of DirectShow, you may take a look of GStreamer project. It is generally what are you trying to achieve. I don't know details of it. But in general I could say that if we try to use common standart/interface we should make it simple. If we make it simple it won't have best available speed. All complications about direct rendering, slice rendering, out of order rendering is to get maximum speed. Anyway I would be very happy to have an profesional tech writer on our side :) Best Regards Ivan Kalvachev iive From mr at ramendik.ru Sun Apr 18 00:58:12 2004 From: mr at ramendik.ru (Mikhail Ramendik) Date: Sun, 18 Apr 2004 02:58:12 +0400 Subject: [MPlayer-G2-dev] An open, standard video/audio API? In-Reply-To: <1632.212.116.154.222.1082237651.squirrel@mail.cacad.com> References: <1082226924.3729.75.camel@localhost.localdomain> <1632.212.116.154.222.1082237651.squirrel@mail.cacad.com> Message-ID: <1082242692.3729.100.camel@localhost.localdomain> Hello, Ivan Kalvachev wrote: > > (SUMMARY. A general-purpose standard API for all multimedia > > processing, including codecs, filters, demuxing etc., would > > be very useful in GNU/Linux, and resolve many existing problems, > > including licensing. The MPlayer team is the best source for > > a standard that could be widely adopted. I would volunteer to > > do tech writing work on the spec). > > Actually as somebody already have said there is an common api. > All players under linux use ffmpeg as primary codec monkey. > And the documentation of ffmpeg is smaller than the sample files. ffmpeg is not yet a standard. And it does not support all things needed. For example, it seems to support DV video but not DV audio (strange, but true - it's better at the video than libdv, just no audio!) > If you look for alternative of DirectShow, you may take a look of > GStreamer project. It is generally what are you trying to achieve. Not really. Well, perhaps it is an alternative - but it assumes too much "windoze-like" stuff. And, Gnome is really overkill outside of explicit GUI works. I would really support a socket-based interface. > I don't know details of it. But in general I could say that > if we try to use common standart/interface we should make it simple. > If we make it simple it won't have best available speed. The Unix people have managed to keep stuff simple and yet *very* efficient. An interface that is simple and yet efficient would be a true work of art. Supported by the well-known MPlayer team, it could stand a chance of becoming the standard. Not only ffmpeg, but other codecs could support it as well - and imagine availability of transcode's big set of filters for use outside of transcode, via this API! > All complications about direct rendering, slice rendering, > out of order rendering is to get maximum speed. I think that, given careful analysis, these things could be fit into a simple logic. Two streams in each direction, maximum. It's only as impossible as playing divx on a K6-200... (I mean, that too was impossible before ffmpeg/mplayer). > Anyway I would be very happy to have an profesional tech writer > on our side :) Warning: I'm somewhat limited on the time front. At present, I probably won't take up things like detailed GUI descriptions - they take quite a lot of time and are useful for just one application. But to work on open standards specs is somewhat different: takes less drudgery, and the effect is (in case of success) quite significant. Yours, Mikhail Ramendik From diego at biurrun.de Sun Apr 18 01:03:42 2004 From: diego at biurrun.de (Diego Biurrun) Date: Sun, 18 Apr 2004 01:03:42 +0200 Subject: [MPlayer-G2-dev] An open, standard video/audio API? In-Reply-To: <1082240916.3729.85.camel@localhost.localdomain> References: <1082226924.3729.75.camel@localhost.localdomain> <16513.40332.236256.251292@biurrun.de> <1082240916.3729.85.camel@localhost.localdomain> Message-ID: <16513.47054.641426.798909@biurrun.de> Mikhail Ramendik writes: > Diego Biurrun wrote: > > This is not exactly a direct answer, but also relevant in this area: > > > > We are in the midst of developing an open, liberally licensed and > > patent unencumbered container format. It is called nut. > > Great. If it won't have the dreaded 4 GB limit, I'm willing to get in :) Fear not, it doesn't :-) > Which list was used as the main developer discussion forum? To get > "into" this I should probably review the archives. Look at the archives of the mplayer-dev-eng mailing list. Diego From diego at biurrun.de Sun Apr 18 01:30:13 2004 From: diego at biurrun.de (Diego Biurrun) Date: Sun, 18 Apr 2004 01:30:13 +0200 Subject: [MPlayer-G2-dev] An open, standard video/audio API? In-Reply-To: <1082226924.3729.75.camel@localhost.localdomain> References: <1082226924.3729.75.camel@localhost.localdomain> Message-ID: <16513.48645.33383.309982@biurrun.de> Mikhail Ramendik writes: > Now for a bit of personal stuff. I am a tech writer with over 5 years of > experience behind me. English is not my native language, but I am > employed as a tech writer in English by a major international company. > For money, I write development docs for internal-use software (it's not > distributed outside of the company) - and they like the work I do. Before I forget: If you are interested in helping out, I sure could use a hand with the documentation maintenance, every bit helps as I am quite busy these days. The Russian translation could also use some love ;-) Regards Diego From dalias at aerifal.cx Mon Apr 19 22:36:09 2004 From: dalias at aerifal.cx (D Richard Felker III) Date: Mon, 19 Apr 2004 16:36:09 -0400 Subject: [MPlayer-G2-dev] An open, standard video/audio API? In-Reply-To: <1082242692.3729.100.camel@localhost.localdomain> References: <1082226924.3729.75.camel@localhost.localdomain> <1632.212.116.154.222.1082237651.squirrel@mail.cacad.com> <1082242692.3729.100.camel@localhost.localdomain> Message-ID: <20040419203608.GU250@brightrain.aerifal.cx> On Sun, Apr 18, 2004 at 02:58:12AM +0400, Mikhail Ramendik wrote: > Hello, > > Ivan Kalvachev wrote: > > > > (SUMMARY. A general-purpose standard API for all multimedia > > > processing, including codecs, filters, demuxing etc., would > > > be very useful in GNU/Linux, and resolve many existing problems, > > > including licensing. The MPlayer team is the best source for > > > a standard that could be widely adopted. I would volunteer to > > > do tech writing work on the spec). > > > > Actually as somebody already have said there is an common api. > > All players under linux use ffmpeg as primary codec monkey. > > And the documentation of ffmpeg is smaller than the sample files. > > ffmpeg is not yet a standard. And it does not support all things needed. > For example, it seems to support DV video but not DV audio (strange, but > true - it's better at the video than libdv, just no audio!) We're talking about standard API, not the codec selection. It's not really relevant that ffmpeg doesn't have DV audio, since it can be added. > > If you look for alternative of DirectShow, you may take a look of > > GStreamer project. It is generally what are you trying to achieve. > > Not really. Well, perhaps it is an alternative - but it assumes too much > "windoze-like" stuff. And, Gnome is really overkill outside of explicit > GUI works. GStreamer uses GNOME? Uhg... > I would really support a socket-based interface. This is totally ignorant. (Read slow) Why does there need to be any IPC? Everything can take place in one process. > > I don't know details of it. But in general I could say that > > if we try to use common standart/interface we should make it simple. > > If we make it simple it won't have best available speed. > > The Unix people have managed to keep stuff simple and yet *very* > efficient. Actually not. It's just that efficiency doesn't matter for processing text... > An interface that is simple and yet efficient would be a true work of > art. Supported by the well-known MPlayer team, it could stand a chance > of becoming the standard. Not only ffmpeg, but other codecs could > support it as well - and imagine availability of transcode's big set of > filters for use outside of transcode, via this API! We're largely against a standard API because there's nothing to gain from it, and all the people who want it want to make an API that sucks. > > All complications about direct rendering, slice rendering, > > out of order rendering is to get maximum speed. > > I think that, given careful analysis, these things could be fit into a > simple logic. Two streams in each direction, maximum. Huh? I don't understand what you mean. > It's only as impossible as playing divx on a K6-200... (I mean, that too > was impossible before ffmpeg/mplayer). It's impossible now too except for crappy low-res stuff. Rich From dalias at aerifal.cx Mon Apr 19 22:46:22 2004 From: dalias at aerifal.cx (D Richard Felker III) Date: Mon, 19 Apr 2004 16:46:22 -0400 Subject: [MPlayer-G2-dev] An open, standard video/audio API? In-Reply-To: <1082226924.3729.75.camel@localhost.localdomain> References: <1082226924.3729.75.camel@localhost.localdomain> Message-ID: <20040419204622.GV250@brightrain.aerifal.cx> On Sat, Apr 17, 2004 at 10:35:24PM +0400, Mikhail Ramendik wrote: > And the one thing that I feel is missing in [GNU/]Linux, while present > (although crippled) in the Non-Free OS, is a clearly defined standard > API for all multimedia stuff. Why should there be such a thing? I've seen no good argument except "windows has one" and that's stupid. Windows also has BSOD. > A Free Software masterpiece, VirtualDub, is made possible by DirectShow. No. VirtualDub is NOT a free software masterpiece. It's very buggy and ugly. I don't think any serious release groups use it, and for good reason. > A clear (hmm, theoretically) interface to all possible codecs, the AVI > file format, and some other stuff is just already there. Yes, it's > stinky (the 4GB limit, and of course the lack of freedom). And the slowness? And the lack of good interoperation? Why do windows users have to rip .vobs to their hd before they can encode?? > But GNU/Linux has none whatsoever. Video4Linux2 contains some Codec API, > but who uses it? Uhg, v4l is stupid and should have absolutely nothing to do with codecs. > As a result, a GNU/Linux program that wants to use codecs, filters, etc, > has to care about interaction with every library out there. Look at What is such a program? Either a movie player, editor, or transcoder. If you have a specialized app, say maybe a game, in which you want to include video, you only need support for the particular codecs you'll be using for your movies, not everything out there. And if all you want is a movie player, you have MPlayer. What use is a bloated API? > From the discussion I saw here, it seems that while the MPLayer G2 core > could be a sort of "clearing center" for multimedia processing under > GNU/Linux, it will still have a pretty specific API. > > My question: why not aim for a standard, generic, published, > well-described multimedia processing API, one that would (because of its > technical benefits and open nature) far surpass DirectShow? Because it will be poorly designed and slow. > It should work across machines (not to speak of working across very > different programs). X-Windows might provide an example. And yes, I > would support sockets as the main tool, and TCP sockets where possible. > Among other things, this will allow interoperation of different > machines, as already done in X. This is totally idiotic. You don't transport uncompressed video over a network. > Just imagine: one machine to grab the source and apply pre-procesing > filters, another to do the coding, and a third to stream the content, > all communicating over the Open Multimedia Protocol. This would be the > *easy* scenario; hard ones would include load balancing. ROTFL!!! You really need to learn something before writing a proposal. Your design here is much less efficient than processing on one computer, not more efficient. > In most cases of course the API would work on the same machine - but > still provide nice separation for different tasks, so loved by the Unix > world. A codec, a filter, a muxer or demuxer, a player would all > communicate via socket streams. Each one easily replaceable, without > micro-management from the player (common today). Now I need a 3GHz athlon to play a movie? NO THANKS!!!!!!!!!!!!! > A user has installed a cool hardware MPEG5 decoder. He > previously used to run MPlayer G2 for playing MPEG5 files; his brother > ran XMMS G3, and used Transcode to convert MPEG5 to Theora for watching > on his Zaurus handheld. Theora? Very bad idea, and it suggests you're just full of buzzwords rather than knowledge. Theora is (a) bad compression, (b) bad implementation, and (c) NOT ANY MORE PATENT-FREE THAN MPEG1/2/4! > Among other matters, an open socket-based standards might resolve the > recent licensing issue. > > The problem with hardware player developers, as it seems to be, is that > they just can't release all of the source to their firmware, because of > licensing matters with other companies. > > But what if the components they use will all interact via a standard > socket interface? In this way, as the GPL FAQ exlicitly says, they can > use GPL-ed and non-GPL modules together! HAHAHAHAHHA! OK, I'm sorry, now you're a total idiot. This is exactly what we DO NOT WANT!!!!!!!! > Now for a bit of personal stuff. I am a tech writer with over 5 years of > experience behind me. A tech writer with no understanding of the tech. How typical... > But of course, this is useful only if the developers here are willing to > go that way. To develop an open standard for interaction between > multimedia processing components. And to base the development of MPlayer > G2 on this standard, not an internal API. I'm not willing. > This could be a crucial step for development of GNU/Linux as a user OS. > This may be an insane raving by a misguided guy. ANyway, replies will be > very much appreciated :) Yes, very misguided. Rich From dalias at aerifal.cx Mon Apr 19 23:16:54 2004 From: dalias at aerifal.cx (D Richard Felker III) Date: Mon, 19 Apr 2004 17:16:54 -0400 Subject: [MPlayer-G2-dev] An open, standard video/audio API? In-Reply-To: <1082226924.3729.75.camel@localhost.localdomain> References: <1082226924.3729.75.camel@localhost.localdomain> Message-ID: <20040419211653.GW250@brightrain.aerifal.cx> On Sat, Apr 17, 2004 at 10:35:24PM +0400, Mikhail Ramendik wrote: > I would be willing to take up the writing of the spec. Not the technical > side of the development - I don't know *nearly* enough to do that. But I > can glean information from programmer-speak, list discussions, etc., and > convert it into an organized, readable document. And that's what an open > standard spec should be, if it's to be widely adopted. (Ultimately, why > not make it an RFC?) A couple other notes... I'd be happy to have you write a formal spec for NUT once it's complete, or to write up the spec of our API. But myself and the others I'm working with won't be working on an API that's slow, that's socket/IPC-based, or that encourages or allows proprietary software to take advantage of our code. Rich From mr at ramendik.ru Tue Apr 20 00:14:43 2004 From: mr at ramendik.ru (Mikhail Ramendik) Date: Tue, 20 Apr 2004 02:14:43 +0400 Subject: [MPlayer-G2-dev] An open, standard video/audio API? In-Reply-To: <20040419203608.GU250@brightrain.aerifal.cx> References: <1082226924.3729.75.camel@localhost.localdomain> <1632.212.116.154.222.1082237651.squirrel@mail.cacad.com> <1082242692.3729.100.camel@localhost.localdomain> <20040419203608.GU250@brightrain.aerifal.cx> Message-ID: <1082412883.1983.11.camel@localhost.localdomain> Hello, D Richard Felker III wrote: > > ffmpeg is not yet a standard. And it does not support all things needed. > > For example, it seems to support DV video but not DV audio (strange, but > > true - it's better at the video than libdv, just no audio!) > > We're talking about standard API, not the codec selection. It's not > really relevant that ffmpeg doesn't have DV audio, since it can be > added. If we call it the standard, we exclude all codecs that are not supported in ffmpeg. DV audio is just one example. > > > If you look for alternative of DirectShow, you may take a look of > > > GStreamer project. It is generally what are you trying to achieve. > > > > Not really. Well, perhaps it is an alternative - but it assumes too much > > "windoze-like" stuff. And, Gnome is really overkill outside of explicit > > GUI works. > > GStreamer uses GNOME? Uhg... I meant to write GLib. (Actually I just mustrust object orientation outside of GUIs.) > > I would really support a socket-based interface. > > This is totally ignorant. (Read slow) Why does there need to be any > IPC? Everything can take place in one process. Why does it have to be *limited* to all-in-one-process? This limits us to one PC, in some cases to one CPU. A socket-based interface does not have to be the only one. It may be a wrapper for exchange of standard data structures. > > An interface that is simple and yet efficient would be a true work of > > art. Supported by the well-known MPlayer team, it could stand a chance > > of becoming the standard. Not only ffmpeg, but other codecs could > > support it as well - and imagine availability of transcode's big set of > > filters for use outside of transcode, via this API! > > We're largely against a standard API because there's nothing to gain > from it, and all the people who want it want to make an API that > sucks. How about availability for specialized apps? For example, non-linear video editing. Or even what VirtualDub does -- easy transcoding and subtitling. Why does avidemux have to talk to many libraries? And then there's streaming video both ways (in and out). Yours, Mikhail Ramendik From dalias at aerifal.cx Tue Apr 20 05:39:15 2004 From: dalias at aerifal.cx (D Richard Felker III) Date: Mon, 19 Apr 2004 23:39:15 -0400 Subject: [MPlayer-G2-dev] An open, standard video/audio API? In-Reply-To: <1082412883.1983.11.camel@localhost.localdomain> References: <1082226924.3729.75.camel@localhost.localdomain> <1632.212.116.154.222.1082237651.squirrel@mail.cacad.com> <1082242692.3729.100.camel@localhost.localdomain> <20040419203608.GU250@brightrain.aerifal.cx> <1082412883.1983.11.camel@localhost.localdomain> Message-ID: <20040420033915.GK250@brightrain.aerifal.cx> On Tue, Apr 20, 2004 at 02:14:43AM +0400, Mikhail Ramendik wrote: > > > I would really support a socket-based interface. > > > > This is totally ignorant. (Read slow) Why does there need to be any > > IPC? Everything can take place in one process. > > Why does it have to be *limited* to all-in-one-process? This limits us > to one PC, in some cases to one CPU. Our API designs can run on one cpu or multiple cpus. But there's simply no way to spread them out across multiple systems without shared memory. It's many many orders of magnitude too slow to be useful. Every time data has to be processed/copied, that's 20 megs/second of bandwidth, assuming full DVD resolution. You simply CANNOT be wasting time on additional copies. Copying over a network is totally useless unless you have gigabit ethernet, and even then the cpu load of all that network traffic will be so high that it will negate any benefit you get of processing on multiple computers. > A socket-based interface does not have to be the only one. It may be a > wrapper for exchange of standard data structures. I failed to mention before that sockets creates a whole new layer of trouble you don't even want to deal with. If you use unix sockets, it's unix-specific and not portable to windows and such. Using tcp sockets is totally out of the question since it requires an authentication layer with advanced key verification, encryption, etc. We will NOT go through the hell of ensuring that something like that is secure against cryptographic attack and not vulnerable to exploits in the implementation, when it's not even useful. Network functionality does not belong at this level. You can build other bloatware layers on top of the basic API to put this dangerous, slow, useless code in, if you really insist. > > > An interface that is simple and yet efficient would be a true work of > > > art. Supported by the well-known MPlayer team, it could stand a chance > > > of becoming the standard. Not only ffmpeg, but other codecs could > > > support it as well - and imagine availability of transcode's big set of > > > filters for use outside of transcode, via this API! BTW, I'm curious...since when does transcode have lots of useful filters? Last I checked it was quite limited, and some of the filters were just ports from MPlayer. > > We're largely against a standard API because there's nothing to gain > > from it, and all the people who want it want to make an API that > > sucks. > > How about availability for specialized apps? For example, non-linear > video editing. Or even what VirtualDub does -- easy transcoding and > subtitling. Why does avidemux have to talk to many libraries? The design we're working on covers all of these things. Your design _only_ covers slow nonlinear video editing apps. It's prohibitively slow for playback or transcoding. > And then there's streaming video both ways (in and out). There's no reason this needs to be connected to the API for processing it. Transporting data and processing it are two separate tasks. Rich From gabor at z10n.net Tue Apr 20 09:46:01 2004 From: gabor at z10n.net (gabor) Date: Tue, 20 Apr 2004 09:46:01 +0200 Subject: [MPlayer-G2-dev] An open, standard video/audio API? In-Reply-To: <20040419203608.GU250@brightrain.aerifal.cx> References: <1082226924.3729.75.camel@localhost.localdomain> <1632.212.116.154.222.1082237651.squirrel@mail.cacad.com> <1082242692.3729.100.camel@localhost.localdomain> <20040419203608.GU250@brightrain.aerifal.cx> Message-ID: <1082447160.2817.11.camel@dubb> On Mon, 2004-04-19 at 22:36, D Richard Felker III wrote: > > > If you look for alternative of DirectShow, you may take a look of > > > GStreamer project. It is generally what are you trying to achieve. > > > > Not really. Well, perhaps it is an alternative - but it assumes too much > > "windoze-like" stuff. And, Gnome is really overkill outside of explicit > > GUI works. > > GStreamer uses GNOME? Uhg... no. they only use glib....which is a LOT less than gnome (not even gtk). for example there is a good change that gstreamer will be used in kde4 (which is qt..) gabor From gabor at z10n.net Tue Apr 20 09:53:28 2004 From: gabor at z10n.net (gabor) Date: Tue, 20 Apr 2004 09:53:28 +0200 Subject: [MPlayer-G2-dev] An open, standard video/audio API? In-Reply-To: <1082447160.2817.11.camel@dubb> References: <1082226924.3729.75.camel@localhost.localdomain> <1632.212.116.154.222.1082237651.squirrel@mail.cacad.com> <1082242692.3729.100.camel@localhost.localdomain> <20040419203608.GU250@brightrain.aerifal.cx> <1082447160.2817.11.camel@dubb> Message-ID: <1082447607.2817.13.camel@dubb> On Tue, 2004-04-20 at 09:46, gabor wrote: > On Mon, 2004-04-19 at 22:36, D Richard Felker III wrote: > > > > If you look for alternative of DirectShow, you may take a look of > > > > GStreamer project. It is generally what are you trying to achieve. > > > > > > Not really. Well, perhaps it is an alternative - but it assumes too much > > > "windoze-like" stuff. And, Gnome is really overkill outside of explicit > > > GUI works. > > > > GStreamer uses GNOME? Uhg... > > no. > > they only use glib....which is a LOT less than gnome (not even gtk). > for example there is a good change that gstreamer will be used in kde4 > (which is qt..) change => chance. gabor From mr at ramendik.ru Tue Apr 20 10:18:57 2004 From: mr at ramendik.ru (Mikhail Ramendik) Date: Tue, 20 Apr 2004 12:18:57 +0400 Subject: [MPlayer-G2-dev] An open, standard video/audio API? In-Reply-To: <20040420033915.GK250@brightrain.aerifal.cx> References: <1082226924.3729.75.camel@localhost.localdomain> <1632.212.116.154.222.1082237651.squirrel@mail.cacad.com> <1082242692.3729.100.camel@localhost.localdomain> <20040419203608.GU250@brightrain.aerifal.cx> <1082412883.1983.11.camel@localhost.localdomain> <20040420033915.GK250@brightrain.aerifal.cx> Message-ID: <1082449136.1948.36.camel@localhost.localdomain> Hello, First: Rich, thanks for the sanity checks. While I disagree with some of your points, others (notably the fact that IPC can not be required if all one wants is a movie player) are valid. And some amount of heat belongs in a kitchen anyway. D Richard Felker III wrote: > > > This is totally ignorant. (Read slow) Why does there need to be any > > > IPC? Everything can take place in one process. > > > > Why does it have to be *limited* to all-in-one-process? This limits us > > to one PC, in some cases to one CPU. > > Our API designs can run on one cpu or multiple cpus. But there's > simply no way to spread them out across multiple systems without > shared memory. It's many many orders of magnitude too slow to be > useful. Every time data has to be processed/copied, that's 20 > megs/second of bandwidth, assuming full DVD resolution. Since when did this become VERY big, when we are talking of something other than a home PC watching movies? A database cluster would handle much more than that. And I was thinking of a setting not unlike a database cluster. There are CPUs for which a Gigabit Ethernet is not a big load. Then there are other ways of communication, like FC/AL. I definitely do agree now that sockets/IPC should not be the only way. (Got some offlist comments with a good explanation of how this should work, but I'm not sure I can quote them). > > A socket-based interface does not have to be the only one. It may be a > > wrapper for exchange of standard data structures. > > I failed to mention before that sockets creates a whole new layer of > trouble you don't even want to deal with. If you use unix sockets, > it's unix-specific and not portable to windows and such. Using tcp > sockets is totally out of the question since it requires an > authentication layer with advanced key verification, encryption, etc. > We will NOT go through the hell of ensuring that something like that > is secure against cryptographic attack and not vulnerable to exploits > in the implementation, when it's not even useful. Network > functionality does not belong at this level. You can build other > bloatware layers on top of the basic API to put this dangerous, slow, > useless code in, if you really insist. Actually, if the API allows for *easy* building of a stream wrapper (said streams may then go through any IPC of choice, from mere pipes to TCP sockets), it will be useful enough for my taste. > > > > An interface that is simple and yet efficient would be a true work of > > > > art. Supported by the well-known MPlayer team, it could stand a chance > > > > of becoming the standard. Not only ffmpeg, but other codecs could > > > > support it as well - and imagine availability of transcode's big set of > > > > filters for use outside of transcode, via this API! > > BTW, I'm curious...since when does transcode have lots of useful > filters? Last I checked it was quite limited, and some of the filters > were just ports from MPlayer. Either we're not on the same page on what constitutes a useful filter, or my copy of MPlayer is too old, or I got lost in the MPlayer manpage despite reading the relevant part several times. Here are the filters in transcode that I think are useful, and that I can't find in MPlayer (actually, the relevant comparison is with MEncoder): 32detect - autodetect 3:2 pulldown and interlacing cshift - chroma lag shifter decimate - NTSC decimation in pure software logoaway - removal of an image/logo from the movie preview - preview while encoding smartdeinter/smartyuv - "smart" deinterlacing, only deinterlaces motion areas and keeps the full resoultion in others Yours, Mikhail Ramendik From dalias at aerifal.cx Tue Apr 20 15:50:42 2004 From: dalias at aerifal.cx (D Richard Felker III) Date: Tue, 20 Apr 2004 09:50:42 -0400 Subject: [MPlayer-G2-dev] An open, standard video/audio API? In-Reply-To: <1082449136.1948.36.camel@localhost.localdomain> References: <1082226924.3729.75.camel@localhost.localdomain> <1632.212.116.154.222.1082237651.squirrel@mail.cacad.com> <1082242692.3729.100.camel@localhost.localdomain> <20040419203608.GU250@brightrain.aerifal.cx> <1082412883.1983.11.camel@localhost.localdomain> <20040420033915.GK250@brightrain.aerifal.cx> <1082449136.1948.36.camel@localhost.localdomain> Message-ID: <20040420135042.GO250@brightrain.aerifal.cx> On Tue, Apr 20, 2004 at 12:18:57PM +0400, Mikhail Ramendik wrote: > Actually, if the API allows for *easy* building of a stream wrapper > (said streams may then go through any IPC of choice, from mere pipes to > TCP sockets), it will be useful enough for my taste. How could an API make this difficult? I don't understand that. Just make a filter that outputs over the IPC mechanism of your choice, or gets its input from the same... > Here are the filters in transcode that I think are useful, and that I > can't find in MPlayer (actually, the relevant comparison is with > MEncoder): > > 32detect - autodetect 3:2 pulldown and interlacing This is probably the port of my old bad vf_detc. Maybe with some enhancements. But mplayer has much better filters for doing it now. If you really need autodetection between the two, there's vf_filmdint, but from my experience autodetection is always a bad idea -- you'll invariably end up incorrectly deinterlacing frames that are actually telecined. > cshift - chroma lag shifter What is chroma lag? > decimate - NTSC decimation in pure software What is NTSC decimation? > logoaway - removal of an image/logo from the movie This is useful, yes. If it's not a stupid implementation like the one in mplayer. > preview - preview while encoding Useful but IMO it's a bit different than a filter. > smartdeinter/smartyuv - "smart" deinterlacing, only deinterlaces motion > areas and keeps the full resoultion in others Similar to pp=md. Rich From mr at ramendik.ru Tue Apr 20 17:15:56 2004 From: mr at ramendik.ru (Mikhail Ramendik) Date: Tue, 20 Apr 2004 19:15:56 +0400 Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: <20040420135042.GO250@brightrain.aerifal.cx> References: <1082226924.3729.75.camel@localhost.localdomain> <1632.212.116.154.222.1082237651.squirrel@mail.cacad.com> <1082242692.3729.100.camel@localhost.localdomain> <20040419203608.GU250@brightrain.aerifal.cx> <1082412883.1983.11.camel@localhost.localdomain> <20040420033915.GK250@brightrain.aerifal.cx> <1082449136.1948.36.camel@localhost.localdomain> <20040420135042.GO250@brightrain.aerifal.cx> Message-ID: <1082474155.1922.33.camel@localhost.localdomain> Hello, D Richard Felker III wrote: > > cshift - chroma lag shifter > > What is chroma lag? The chrominance component shifted to the left or right compared to luminance. This seems to happen in some broadcast situations. Granted this is not *widely* necessary. > > decimate - NTSC decimation in pure software > > What is NTSC decimation? Removal of duplicate frames. Made as a part of the inverse telecine process. MPlayer seems to have the whole process in one big whopping filter (ivtc or detc); but heck, what if I need this stuff for other purposes? Notably, some animation tends to have low real fps; like 15, sometimes lower. While experimenting with decimate to capture only the meaningful frames might be a long process, it may be worth the result. > > smartdeinter/smartyuv - "smart" deinterlacing, only deinterlaces motion > > areas and keeps the full resoultion in others > > Similar to pp=md. smartdeinter uses some thresholds (all tune-able) to detect motion areas, and only applies a deinterlacing algorithm to where it has detected motion. (smartyuv is a yuv-optimized version of the same). CPU-intensive, but can do wonders, especially if you spend some time fine-tuning it. I'll have to encode some lectures soon; there the still part of the image (the drawing pad) should be kept at max resolution, while the moving part (the speaker) should be deinterlaced. If I were to use mencoder instead of transcode, I'd lose all the fine-tuning capability. Try pp=md, love it or leave it. Yours, Mikhail Ramendik From arpi at thot.banki.hu Tue Apr 20 17:26:22 2004 From: arpi at thot.banki.hu (Arpi) Date: Tue, 20 Apr 2004 17:26:22 +0200 (CEST) Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: <1082474155.1922.33.camel@localhost.localdomain> Message-ID: <20040420152622.6C90934B2E@mail.mplayerhq.hu> Hi, > > > cshift - chroma lag shifter > > > > What is chroma lag? > > The chrominance component shifted to the left or right compared to > luminance. This seems to happen in some broadcast situations. Granted > this is not *widely* necessary. swscaler can do this, see -ssf > > > decimate - NTSC decimation in pure software > > > > What is NTSC decimation? > > Removal of duplicate frames. Made as a part of the inverse telecine > process. MPlayer seems to have the whole process in one big whopping > filter (ivtc or detc); but heck, what if I need this stuff for other > purposes? there is such filter for duplicate (very similar) frame removal > > > smartdeinter/smartyuv - "smart" deinterlacing, only deinterlaces motion > > > areas and keeps the full resoultion in others > > > > Similar to pp=md. > > smartdeinter uses some thresholds (all tune-able) to detect motion > areas, and only applies a deinterlacing algorithm to where it has > detected motion. (smartyuv is a yuv-optimized version of the same). kerndeint? A'rpi / MPlayer, Astral & ESP-team -- PyMaViS - my answer to worm storm - http://mplayerhq.hu/~arpi/pymavis/ From mr at ramendik.ru Tue Apr 20 17:41:11 2004 From: mr at ramendik.ru (Mikhail Ramendik) Date: Tue, 20 Apr 2004 19:41:11 +0400 Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: <20040420152622.6C90934B2E@mail.mplayerhq.hu> References: <20040420152622.6C90934B2E@mail.mplayerhq.hu> Message-ID: <1082475671.1922.42.camel@localhost.localdomain> Hello, Arpi wrote: > > The chrominance component shifted to the left or right compared to > > luminance. This seems to happen in some broadcast situations. Granted > > this is not *widely* necessary. > > swscaler can do this, see -ssf Oops, thanks. (Looks like the real advantage of transcode is a better filter documentation. With mplayer, who'd have guessed one needs to look to the scaler when one wants to shift color, rather than, well, scale?) > > > > decimate - NTSC decimation in pure software > > > > > > What is NTSC decimation? > > > > Removal of duplicate frames. Made as a part of the inverse telecine > > process. MPlayer seems to have the whole process in one big whopping > > filter (ivtc or detc); but heck, what if I need this stuff for other > > purposes? > > there is such filter for duplicate (very similar) frame removal I have not found it, but it's probably just me. > > > > smartdeinter/smartyuv - "smart" deinterlacing, only deinterlaces motion > > > > areas and keeps the full resoultion in others > > > > > > Similar to pp=md. > > > > smartdeinter uses some thresholds (all tune-able) to detect motion > > areas, and only applies a deinterlacing algorithm to where it has > > detected motion. (smartyuv is a yuv-optimized version of the same). > > kerndeint? Will check this out (will need to compile a new version of mplayer for that). It's a recent addition, however; transcode had one for quite some time. Is this a case of work duplication? Would not a more standard API prevent *that* kind of duplication? The real debate was about the merits of a standard plugin API for various video processing apps. I just used the filter situation as an example. MEncoder and transcode are two Free Software solutions; they lack each other's cool features, and possibly do duplicate work, because no such API is present. Yours, Mikhail Ramendik From arpi at thot.banki.hu Tue Apr 20 18:16:33 2004 From: arpi at thot.banki.hu (Arpi) Date: Tue, 20 Apr 2004 18:16:33 +0200 (CEST) Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: <1082475671.1922.42.camel@localhost.localdomain> Message-ID: <20040420161633.9D311FD@mail.mplayerhq.hu> Hi, > > > The chrominance component shifted to the left or right compared to > > > luminance. This seems to happen in some broadcast situations. Granted > > > this is not *widely* necessary. > > > > swscaler can do this, see -ssf > > Oops, thanks. > > (Looks like the real advantage of transcode is a better filter > documentation. With mplayer, who'd have guessed one needs to look to the > scaler when one wants to shift color, rather than, well, scale?) it was a side-effect of swscaler, and as it can be done, no one wrote a separate filter for this funcion later > > > > > > smartdeinter uses some thresholds (all tune-able) to detect motion > > > areas, and only applies a deinterlacing algorithm to where it has > > > detected motion. (smartyuv is a yuv-optimized version of the same). > > > > kerndeint? > > Will check this out (will need to compile a new version of mplayer for > that). It's a recent addition, however; transcode had one for quite some > time. Is this a case of work duplication? Would not a more standard API > prevent *that* kind of duplication? afair it is ported from transcode > The real debate was about the merits of a standard plugin API for > various video processing apps. I just used the filter situation as an > example. MEncoder and transcode are two Free Software solutions; they > lack each other's cool features, and possibly do duplicate work, because > no such API is present. but why dont you tell this ti the transcode people? our filter api is much more powerful than their one. at least we beleieve so, so we won't go for their api. also they took and ported many of mplayers filters, so they should have support our api, and not inverse. A'rpi / MPlayer, Astral & ESP-team -- PyMaViS - my answer to worm storm - http://mplayerhq.hu/~arpi/pymavis/ From diego at biurrun.de Tue Apr 20 21:48:44 2004 From: diego at biurrun.de (Diego Biurrun) Date: Tue, 20 Apr 2004 21:48:44 +0200 Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: <1082475671.1922.42.camel@localhost.localdomain> References: <20040420152622.6C90934B2E@mail.mplayerhq.hu> <1082475671.1922.42.camel@localhost.localdomain> Message-ID: <16517.32412.807500.396837@biurrun.de> Mikhail Ramendik writes: > Arpi wrote: > > > The chrominance component shifted to the left or right compared to > > > luminance. This seems to happen in some broadcast situations. Granted > > > this is not *widely* necessary. > > > > swscaler can do this, see -ssf > > Oops, thanks. > > (Looks like the real advantage of transcode is a better filter > documentation. With mplayer, who'd have guessed one needs to look to the > scaler when one wants to shift color, rather than, well, scale?) We're always looking for able hands that can improve the documentation. If you want to help out with this you are more than welcome. Diego From mr at ramendik.ru Tue Apr 20 21:55:05 2004 From: mr at ramendik.ru (Mikhail Ramendik) Date: Tue, 20 Apr 2004 23:55:05 +0400 Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: <20040420161633.9D311FD@mail.mplayerhq.hu> References: <20040420161633.9D311FD@mail.mplayerhq.hu> Message-ID: <1082490904.4854.8.camel@localhost.localdomain> Hello, Arpi wrote: > > > kerndeint? > > > > Will check this out (will need to compile a new version of mplayer for > > that). It's a recent addition, however; transcode had one for quite some > > time. Is this a case of work duplication? Would not a more standard API > > prevent *that* kind of duplication? > > afair it is ported from transcode Then why the strange name change, so that a transcode user would never guess it's the same? In Russian, in such cases we say jokingly: "so that the enemies would never guess?". But transcode people are not the enemy, it's all free software ;) > > The real debate was about the merits of a standard plugin API for > > various video processing apps. I just used the filter situation as an > > example. MEncoder and transcode are two Free Software solutions; they > > lack each other's cool features, and possibly do duplicate work, because > > no such API is present. > > but why dont you tell this ti the transcode people? > our filter api is much more powerful than their one. at least we > beleieve so, so we won't go for their api. I just think that first, a clear API should be published and promoted as an open standard - preparing to be "The Linux Video API", not "the MPlayer API". It may be that GStreamer is already set up as that. If this is indeed so, then perhaps MPlayer should support GStreamer? (I may not like it, objects don't seem to belong -- but I think it's important to have a standard even if I don't support it). Once a standard is established - and the word of the MPlayer people means *much* in this regard - I think transcode will be under much pressure to conform. > also they took and ported many of mplayers filters, so they should have > support our api, and not inverse. In fact I agree it should be based on the MPlayer API. But, it should be a clean, simple, FIXED (no new versions without backward compatibility), and well described standard. Like those that drive the Internet. It *may* resolve the licensing debate on the way. Then again it may not. And I am very interested in your opinion on this. In fact, I'm most interested exactly in *your* opinion, as your messages in the archives got me thinking about it. Yours, Mikhail Ramendik From mr at ramendik.ru Tue Apr 20 21:59:29 2004 From: mr at ramendik.ru (Mikhail Ramendik) Date: Tue, 20 Apr 2004 23:59:29 +0400 Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: <16517.32412.807500.396837@biurrun.de> References: <20040420152622.6C90934B2E@mail.mplayerhq.hu> <1082475671.1922.42.camel@localhost.localdomain> <16517.32412.807500.396837@biurrun.de> Message-ID: <1082491168.4854.13.camel@localhost.localdomain> Hello, Diego Biurrun wrote: > > > > The chrominance component shifted to the left or right compared to > > > > luminance. This seems to happen in some broadcast situations. Granted > > > > this is not *widely* necessary. > > > > > > swscaler can do this, see -ssf > > > > Oops, thanks. > > > > (Looks like the real advantage of transcode is a better filter > > documentation. With mplayer, who'd have guessed one needs to look to the > > scaler when one wants to shift color, rather than, well, scale?) > > We're always looking for able hands that can improve the > documentation. If you want to help out with this you are more than > welcome. Might be a good idea. But there is one thing I'd really like to understand, before trying my hand at this. What's the *big* deal of a difference between mencoder and transcode? I mean - from the mencoder developers' viewpoint. To be honest, I have not grasped the difference, other than the commandline syntax... And I promise not to argue. So no flamewar. I just want your words. Yours, Mikhail Ramendik From dalias at aerifal.cx Tue Apr 20 22:25:00 2004 From: dalias at aerifal.cx (D Richard Felker III) Date: Tue, 20 Apr 2004 16:25:00 -0400 Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: <1082475671.1922.42.camel@localhost.localdomain> References: <20040420152622.6C90934B2E@mail.mplayerhq.hu> <1082475671.1922.42.camel@localhost.localdomain> Message-ID: <20040420202500.GA250@brightrain.aerifal.cx> On Tue, Apr 20, 2004 at 07:41:11PM +0400, Mikhail Ramendik wrote: > Hello, > > Arpi wrote: > > > > The chrominance component shifted to the left or right compared to > > > luminance. This seems to happen in some broadcast situations. Granted > > > this is not *widely* necessary. > > > > swscaler can do this, see -ssf > > Oops, thanks. > > (Looks like the real advantage of transcode is a better filter > documentation. With mplayer, who'd have guessed one needs to look to the > scaler when one wants to shift color, rather than, well, scale?) Someone who understands the theory behind video processing. You can make an argument that there should be a good concept manual with chroma shifting as a topic, for people who don't understand. Go ahead and write it if you like. :) But when one filter can naturally do two operations, and do so more efficiently by doing it that way, there's no reason to make two separate filters. > > > > > decimate - NTSC decimation in pure software > > > > > > > > What is NTSC decimation? > > > > > > Removal of duplicate frames. Made as a part of the inverse telecine > > > process. MPlayer seems to have the whole process in one big whopping Inverse telecine does not naturally make duplicate frames. We didn't "hide" the decimation process inside the inverse telecine filters; it's simply not happening. > > > filter (ivtc or detc); but heck, what if I need this stuff for other > > > purposes? > > > > there is such filter for duplicate (very similar) frame removal > > I have not found it, but it's probably just me. It's called *gasp* decimate! > > > > > smartdeinter/smartyuv - "smart" deinterlacing, only deinterlaces motion > > > > > areas and keeps the full resoultion in others > > > > > > > > Similar to pp=md. > > > > > > smartdeinter uses some thresholds (all tune-able) to detect motion > > > areas, and only applies a deinterlacing algorithm to where it has > > > detected motion. (smartyuv is a yuv-optimized version of the same). > > > > kerndeint? > > Will check this out (will need to compile a new version of mplayer for > that). It's a recent addition, however; transcode had one for quite some > time. Is this a case of work duplication? Would not a more standard API > prevent *that* kind of duplication? No. It would just mean there was essentially one app. With lots of stupid mindless frontends for the same basic code. All with the same limitations. And the same bugs. And the same slowness. > The real debate was about the merits of a standard plugin API for > various video processing apps. I just used the filter situation as an > example. MEncoder and transcode are two Free Software solutions; they > lack each other's cool features, and possibly do duplicate work, because > no such API is present. There's a little "duplicate" work, but for a good reason: because one project does not like the approach the other one takes to solving a problem. If they did, they would just reuse/port code from the other, or not even write the other one to begin with. Rich From dalias at aerifal.cx Tue Apr 20 22:34:00 2004 From: dalias at aerifal.cx (D Richard Felker III) Date: Tue, 20 Apr 2004 16:34:00 -0400 Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: <1082490904.4854.8.camel@localhost.localdomain> References: <20040420161633.9D311FD@mail.mplayerhq.hu> <1082490904.4854.8.camel@localhost.localdomain> Message-ID: <20040420203400.GB250@brightrain.aerifal.cx> On Tue, Apr 20, 2004 at 11:55:05PM +0400, Mikhail Ramendik wrote: > > > > kerndeint? > > > > > > Will check this out (will need to compile a new version of mplayer for > > > that). It's a recent addition, however; transcode had one for quite some > > > time. Is this a case of work duplication? Would not a more standard API > > > prevent *that* kind of duplication? > > > > afair it is ported from transcode > > Then why the strange name change, so that a transcode user would never > guess it's the same? The transcode one has a very stupid name. WTF does "smart" mean? It's windows-speak. "smartyuv" is an especially dumb name, since the basic filter should either support multiple colorspaces (oh I forgot, transcode's api is too broken to do this) or use the standard colorspace for video (yuv). Now kerndeint is a very stupid name too. It's a motion-adaptive deinterlacer, which is totally different from a kernel deinterlacer (e.g. pp=fd). > > > The real debate was about the merits of a standard plugin API for > > > various video processing apps. I just used the filter situation as an > > > example. MEncoder and transcode are two Free Software solutions; they > > > lack each other's cool features, and possibly do duplicate work, because > > > no such API is present. > > > > but why dont you tell this ti the transcode people? > > our filter api is much more powerful than their one. at least we > > beleieve so, so we won't go for their api. > > I just think that first, a clear API should be published and promoted as > an open standard - preparing to be "The Linux Video API", not "the > MPlayer API". It's even more stupid to call it something to do with Linux. Linux is a kernel/operating system (depending on your viewpoint :) which has NOTHING to do with video. Why should something that's usable on any semi-sane OS be called the "Linux Video API"?? > It may be that GStreamer is already set up as that. If this is indeed > so, then perhaps MPlayer should support GStreamer? (I may not like it, > objects don't seem to belong -- but I think it's important to have a > standard even if I don't support it). Uhg. This kind of blind standard-worship really annoys me. Even if a standard is really horrible, you'll support it just for the sake of having a standard?? > Once a standard is established - and the word of the MPlayer people > means *much* in this regard - I think transcode will be under much > pressure to conform. I couldn't care less what transcode does. I've always found it slow, awkward, and just plain broken. > In fact I agree it should be based on the MPlayer API. But, it should be > a clean, simple, FIXED (no new versions without backward compatibility), > and well described standard. Like those that drive the Internet. No. The internet needs to keep working and interoperating with old and new systems which are upgraded (or not) independently of one another, and which use independent implementations of the same standards. There's no such need with the MPlayer API. I'm against gratuitous breaking of compatibility, but sometimes it must be done to overcome limitations in the old design. > It *may* resolve the licensing debate on the way. Then again it may not. IMO it has nothing to do with the licensing debate, unless you mean the stuff you originally wrote about allowing proprietary software to use our code. Rich From mr at ramendik.ru Tue Apr 20 22:52:29 2004 From: mr at ramendik.ru (Mikhail Ramendik) Date: Wed, 21 Apr 2004 00:52:29 +0400 Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: <20040420203400.GB250@brightrain.aerifal.cx> References: <20040420161633.9D311FD@mail.mplayerhq.hu> <1082490904.4854.8.camel@localhost.localdomain> <20040420203400.GB250@brightrain.aerifal.cx> Message-ID: <1082494348.4854.25.camel@localhost.localdomain> Hello, D Richard Felker III wrote: > > > afair it is ported from transcode > > > > Then why the strange name change, so that a transcode user would never > > guess it's the same? > > The transcode one has a very stupid name. WTF does "smart" mean? It's > windows-speak. "smartyuv" is an especially dumb name, since the basic > filter should either support multiple colorspaces (oh I forgot, > transcode's api is too broken to do this) or use the standard > colorspace for video (yuv). > > Now kerndeint is a very stupid name too. It's a motion-adaptive > deinterlacer, which is totally different from a kernel deinterlacer > (e.g. pp=fd). OK. So why the name change from one stupid name to another stupid name? And smartdeinter is at least understandable as Windows-speak; kerndeint is not understandable period (for me a "kernel" is an OS kernel anyway, and I suspect it's the same for the majority of Linux and BSD users). > > > but why dont you tell this ti the transcode people? > > > our filter api is much more powerful than their one. at least we > > > beleieve so, so we won't go for their api. > > > > I just think that first, a clear API should be published and promoted as > > an open standard - preparing to be "The Linux Video API", not "the > > MPlayer API". > > It's even more stupid to call it something to do with Linux. Linux is > a kernel/operating system (depending on your viewpoint :) which has > NOTHING to do with video. Why should something that's usable on any > semi-sane OS be called the "Linux Video API"?? Point taken. Why not "Open Video API" then? > > It may be that GStreamer is already set up as that. If this is indeed > > so, then perhaps MPlayer should support GStreamer? (I may not like it, > > objects don't seem to belong -- but I think it's important to have a > > standard even if I don't support it). > > Uhg. This kind of blind standard-worship really annoys me. Even if a > standard is really horrible, you'll support it just for the sake of > having a standard?? Well, I see what Microsoft ended up doing when they rejected standards. I see what Linus Torvalds ended up doing when he went about trying to implement a standard (Posix). > > Once a standard is established - and the word of the MPlayer people > > means *much* in this regard - I think transcode will be under much > > pressure to conform. > > I couldn't care less what transcode does. I've always found it slow, > awkward, and just plain broken. Slow? Exactly how *can* it be slow (or fast), when by far the most CPU time is spent in either codecs or filters, and the much of the code there is the same? > > In fact I agree it should be based on the MPlayer API. But, it should be > > a clean, simple, FIXED (no new versions without backward compatibility), > > and well described standard. Like those that drive the Internet. > > No. The internet needs to keep working and interoperating with old and > new systems which are upgraded (or not) independently of one another, > and which use independent implementations of the same standards. > There's no such need with the MPlayer API. I'm against gratuitous > breaking of compatibility, but sometimes it must be done to overcome > limitations in the old design. And how then can one write anything that is, well not mplayer? Like a visual recoding tool, a video editor, etc. Write them against a current version of the MPlayer API, and then have to modify it every time it changes? > > It *may* resolve the licensing debate on the way. Then again it may not. > > IMO it has nothing to do with the licensing debate, unless you mean > the stuff you originally wrote about allowing proprietary software to > use our code. What's "use"? Do you consider any processing of your code's output by proprietary software "use"? If I somehow use mencoder to stream video over the Net, and somebody receives it and plays it on MS Media Player, is that wrong? I *am* for the GPL. It's just a question of where use ends and interoperability begins. Yours, Mikhail Ramendik From arpi at thot.banki.hu Tue Apr 20 23:13:08 2004 From: arpi at thot.banki.hu (Arpi) Date: Tue, 20 Apr 2004 23:13:08 +0200 (CEST) Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: <1082494348.4854.25.camel@localhost.localdomain> Message-ID: <20040420211308.0AD5B3518C@mail.mplayerhq.hu> Hi, > > > Once a standard is established - and the word of the MPlayer people > > > means *much* in this regard - I think transcode will be under much > > > pressure to conform. > > > > I couldn't care less what transcode does. I've always found it slow, > > awkward, and just plain broken. > > Slow? Exactly how *can* it be slow (or fast), when by far the most CPU > time is spent in either codecs or filters, and the much of the code > there is the same? No. MEncoder is an MPlayer with ao/vo replaced with encoders & muxers. And while the whole MPlayer video pipeline (demuxers, decoders, filters) were designed for fast, realtime processing, transcode was always made to be offline (non realtime) transcoder. MPlayer filter api is much more complex than transcode's one, as it supports multiple colorspaces, partial (for example, only one of 3 planes) changes, slices, direct rendering/filtering etc. These can speed up processing a lot, and keep better image quality (afaik transcode sometimes does yuv2->rgb, filter, rgb->yuv, because of its lame filter api / filter implementations). Also, as mencoder shares the whole stream/demuxer/decoder/filter layer with mplayer, it supports every mplayer formats, from quicktime to real streaming. Transcode afaik supports a very limited set of input formats and codecs. imho advantages of mencoder over transcode: - many supported input formats, codecs - faster, due to DR and slices in filtering api - sometimes better quality, due to less colorspace conversions disadvantages: - less output formats/muxers anyway afaik there is a way to pipe mplayer's output to transcode, so by combining these two you can get the best results. A'rpi / MPlayer, Astral & ESP-team -- PyMaViS - my answer to worm storm - http://mplayerhq.hu/~arpi/pymavis/ From dalias at aerifal.cx Tue Apr 20 23:20:33 2004 From: dalias at aerifal.cx (D Richard Felker III) Date: Tue, 20 Apr 2004 17:20:33 -0400 Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: <1082494348.4854.25.camel@localhost.localdomain> References: <20040420161633.9D311FD@mail.mplayerhq.hu> <1082490904.4854.8.camel@localhost.localdomain> <20040420203400.GB250@brightrain.aerifal.cx> <1082494348.4854.25.camel@localhost.localdomain> Message-ID: <20040420212033.GF250@brightrain.aerifal.cx> On Wed, Apr 21, 2004 at 12:52:29AM +0400, Mikhail Ramendik wrote: > Hello, > > D Richard Felker III wrote: > > > > > afair it is ported from transcode > > > > > > Then why the strange name change, so that a transcode user would never > > > guess it's the same? > > > > The transcode one has a very stupid name. WTF does "smart" mean? It's > > windows-speak. "smartyuv" is an especially dumb name, since the basic > > filter should either support multiple colorspaces (oh I forgot, > > transcode's api is too broken to do this) or use the standard > > colorspace for video (yuv). > > > > Now kerndeint is a very stupid name too. It's a motion-adaptive > > deinterlacer, which is totally different from a kernel deinterlacer > > (e.g. pp=fd). > > OK. So why the name change from one stupid name to another stupid name? Well maybe it uses a kernel deinterlacer on the parts it decides to deinterlace. I don't know. I haven't read the code or even tried it. > And smartdeinter is at least understandable as Windows-speak; kerndeint How is that understandible? What information does it convey? Nothing. > is not understandable period (for me a "kernel" is an OS kernel anyway, > and I suspect it's the same for the majority of Linux and BSD users). We don't cater to people who are too stupid to realize that the same word can mean different things in different contexts. The people who will be confused by this are the same stupid nerds who expect to see a gui element when someone asks for a menu at a restaurant. FYI, a kernel is a function k which is used to generate an integral operator K, as in: /` Kf(x) = | k(x,y) f(y) dy ./ Often k(x,y) will be a function of x-y only, in which case you actually have a convolution. > Point taken. Why not "Open Video API" then? Because that sounds pompous, like we have some authority to define what the API should be for everyone. Instead we'll make an API we like. If others like it, they can use it. If not, they can write their own. > > Uhg. This kind of blind standard-worship really annoys me. Even if a > > standard is really horrible, you'll support it just for the sake of > > having a standard?? > > Well, I see what Microsoft ended up doing when they rejected standards. > I see what Linus Torvalds ended up doing when he went about trying to > implement a standard (Posix). This is not a valid analogy, for many reasons. > > I couldn't care less what transcode does. I've always found it slow, > > awkward, and just plain broken. > > Slow? Exactly how *can* it be slow (or fast), when by far the most CPU > time is spent in either codecs or filters, and the much of the code > there is the same? Most of the cpu time is spent wastefully copying buffers or converting between colorspaces. > And how then can one write anything that is, well not mplayer? Like a > visual recoding tool, a video editor, etc. Write them against a current > version of the MPlayer API, and then have to modify it every time it > changes? Only modify if you want to use the new libs. Otherwise you're free to stick with the old ones. > > > It *may* resolve the licensing debate on the way. Then again it may not. > > > > IMO it has nothing to do with the licensing debate, unless you mean > > the stuff you originally wrote about allowing proprietary software to > > use our code. > > What's "use"? Do you consider any processing of your code's output by > proprietary software "use"? If I somehow use mencoder to stream video > over the Net, and somebody receives it and plays it on MS Media Player, > is that wrong? No. What's wrong is for someone to make a proprietary NLE program that uses GPL codecs, filters, etc. Rich From dalias at aerifal.cx Tue Apr 20 23:21:41 2004 From: dalias at aerifal.cx (D Richard Felker III) Date: Tue, 20 Apr 2004 17:21:41 -0400 Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: <20040420211308.0AD5B3518C@mail.mplayerhq.hu> References: <1082494348.4854.25.camel@localhost.localdomain> <20040420211308.0AD5B3518C@mail.mplayerhq.hu> Message-ID: <20040420212141.GG250@brightrain.aerifal.cx> On Tue, Apr 20, 2004 at 11:13:08PM +0200, Arpi wrote: > imho advantages of mencoder over transcode: > - many supported input formats, codecs > - faster, due to DR and slices in filtering api > - sometimes better quality, due to less colorspace conversions > > disadvantages: > - less output formats/muxers Add one: totally broken a/v sync model! Rich From mr at ramendik.ru Tue Apr 20 23:29:01 2004 From: mr at ramendik.ru (Mikhail Ramendik) Date: Wed, 21 Apr 2004 01:29:01 +0400 Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: <20040420212141.GG250@brightrain.aerifal.cx> References: <1082494348.4854.25.camel@localhost.localdomain> <20040420211308.0AD5B3518C@mail.mplayerhq.hu> <20040420212141.GG250@brightrain.aerifal.cx> Message-ID: <1082496540.4854.29.camel@localhost.localdomain> Hello, D Richard Felker III wrote: > > imho advantages of mencoder over transcode: > > - many supported input formats, codecs > > - faster, due to DR and slices in filtering api > > - sometimes better quality, due to less colorspace conversions > > > > disadvantages: > > - less output formats/muxers > Add one: totally broken a/v sync model! Is that on mencoder or transcode? Yours, Mikhail Ramendik P.S. I'll go test out mencoder vs. transcode speed on the exact same set of codecs and filters. You got me intrigued. From arpi at thot.banki.hu Tue Apr 20 23:35:39 2004 From: arpi at thot.banki.hu (Arpi) Date: Tue, 20 Apr 2004 23:35:39 +0200 (CEST) Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: <1082496540.4854.29.camel@localhost.localdomain> Message-ID: <20040420213539.CA606BF@mail.mplayerhq.hu> Hi, > > > imho advantages of mencoder over transcode: > > > - many supported input formats, codecs > > > - faster, due to DR and slices in filtering api > > > - sometimes better quality, due to less colorspace conversions > > > > > > disadvantages: > > > - less output formats/muxers > > Add one: totally broken a/v sync model! > > Is that on mencoder or transcode? i guess, both :)) A'rpi / MPlayer, Astral & ESP-team -- PyMaViS - my answer to worm storm - http://mplayerhq.hu/~arpi/pymavis/ From dalias at aerifal.cx Wed Apr 21 00:37:06 2004 From: dalias at aerifal.cx (D Richard Felker III) Date: Tue, 20 Apr 2004 18:37:06 -0400 Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: <1082496540.4854.29.camel@localhost.localdomain> References: <1082494348.4854.25.camel@localhost.localdomain> <20040420211308.0AD5B3518C@mail.mplayerhq.hu> <20040420212141.GG250@brightrain.aerifal.cx> <1082496540.4854.29.camel@localhost.localdomain> Message-ID: <20040420223706.GJ250@brightrain.aerifal.cx> On Wed, Apr 21, 2004 at 01:29:01AM +0400, Mikhail Ramendik wrote: > Hello, > > D Richard Felker III wrote: > > > > imho advantages of mencoder over transcode: > > > - many supported input formats, codecs > > > - faster, due to DR and slices in filtering api > > > - sometimes better quality, due to less colorspace conversions > > > > > > disadvantages: > > > - less output formats/muxers > > Add one: totally broken a/v sync model! > > Is that on mencoder or transcode? I haven't used transcode enough to know if it has working a/v sync. MEncoder's a/v sync should not desync on average (although some users report that it does...), but the premises on which it's based are dumb, so you get unnecessary frame dropping and duplication which makes the video choppy. Rich From mosgalin at VM10124.spb.edu Wed Apr 21 10:16:16 2004 From: mosgalin at VM10124.spb.edu (Vladimir Mosgalin) Date: Wed, 21 Apr 2004 12:16:16 +0400 (MSD) Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: <20040420202500.GA250@brightrain.aerifal.cx> References: <20040420152622.6C90934B2E@mail.mplayerhq.hu> <1082475671.1922.42.camel@localhost.localdomain> <20040420202500.GA250@brightrain.aerifal.cx> Message-ID: On Tue, 20 Apr 2004, D Richard Felker III wrote: DRFI>> > > > > decimate - NTSC decimation in pure software DRFI>> > > > DRFI>> > > > What is NTSC decimation? DRFI>> > > DRFI>> > > Removal of duplicate frames. Made as a part of the inverse telecine DRFI>> > > process. MPlayer seems to have the whole process in one big whopping DRFI> DRFI>Inverse telecine does not naturally make duplicate frames. We didn't DRFI>"hide" the decimation process inside the inverse telecine filters; DRFI>it's simply not happening. DRFI> DRFI>> > > filter (ivtc or detc); but heck, what if I need this stuff for other DRFI>> > > purposes? DRFI>> > DRFI>> > there is such filter for duplicate (very similar) frame removal DRFI>> DRFI>> I have not found it, but it's probably just me. DRFI> DRFI>It's called *gasp* decimate! You are a little wrong here. Sure, since you never tried those filters in transcode you might think so, but the following sequence of filters -J ivtc,32detect=force_mode=3,decimate in transcode has no working alternative in mplayer. I tried a lot, but found nothing. decimate filter in mplayer is different, that's all. I ended up using transcode for encoding dvd to avi with ffv1 codec and then recompressing it to mpeg4 using mplayer. Yes, ivtc filters in mplayer are overally better. But _there are_ cases when transcode gives desired result and mplayer doesn't. smartdeint is another very good ported from vdub filter (that's why it is working in rgb, and smartyuv was made afterwards as "special yuv version for transcode") that imho is the best deinterlacer. It's a shame it isn't present in mplayer, but I haven't looked at kerndeint yet. I hope it is just ported smartdeint with stupid name, not something written from scratch... Also, for filters like this transcode can be used in yuv mode, so there is no extra colorspace conversion. -- Vladimir From dalias at aerifal.cx Wed Apr 21 10:45:06 2004 From: dalias at aerifal.cx (D Richard Felker III) Date: Wed, 21 Apr 2004 04:45:06 -0400 Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: References: <20040420152622.6C90934B2E@mail.mplayerhq.hu> <1082475671.1922.42.camel@localhost.localdomain> <20040420202500.GA250@brightrain.aerifal.cx> Message-ID: <20040421084505.GV250@brightrain.aerifal.cx> On Wed, Apr 21, 2004 at 12:16:16PM +0400, Vladimir Mosgalin wrote: > On Tue, 20 Apr 2004, D Richard Felker III wrote: > > DRFI>> > > > > decimate - NTSC decimation in pure software > DRFI>> > > > > DRFI>> > > > What is NTSC decimation? > DRFI>> > > > DRFI>> > > Removal of duplicate frames. Made as a part of the inverse telecine > DRFI>> > > process. MPlayer seems to have the whole process in one big whopping > DRFI> > DRFI>Inverse telecine does not naturally make duplicate frames. We didn't > DRFI>"hide" the decimation process inside the inverse telecine filters; > DRFI>it's simply not happening. > DRFI> > DRFI>> > > filter (ivtc or detc); but heck, what if I need this stuff for other > DRFI>> > > purposes? > DRFI>> > > DRFI>> > there is such filter for duplicate (very similar) frame removal > DRFI>> > DRFI>> I have not found it, but it's probably just me. > DRFI> > DRFI>It's called *gasp* decimate! > > You are a little wrong here. Sure, since you never tried those filters > in transcode you might think so, but the following sequence of filters > -J ivtc,32detect=force_mode=3,decimate > > in transcode has no working alternative in mplayer. I tried a lot, but What is this sequence supposed to do? > found nothing. decimate filter in mplayer is different, that's all. I What does transcode's decimate filter do? Drop 1 in 5 frames, picking the one with the least difference? The new divtc filter in mplayer (silly name, oh well) does that. > Yes, ivtc filters in mplayer are overally better. But _there are_ cases > when transcode gives desired result and mplayer doesn't. If you have example clips where the various filters mess up, I'd appreciate seeing them. > smartdeint is another very good ported from vdub filter (that's why it > is working in rgb, and smartyuv was made afterwards as "special yuv > version for transcode") that imho is the best deinterlacer. It's a shame > it isn't present in mplayer, but I haven't looked at kerndeint yet. I > hope it is just ported smartdeint with stupid name, not something > written from scratch... I think it's the same thing. I tried it on telecined content because I didn't have any true interlaced material handy, and I must say I wasn't impressed. The kernel it uses for deinterlacing the interlaced parts is bad (produces nasty edge-ghosts) and it deinterlaces wherever there's motion, rather than just the areas that have combing. Rich From mosgalin at VM10124.spb.edu Wed Apr 21 11:32:49 2004 From: mosgalin at VM10124.spb.edu (Vladimir Mosgalin) Date: Wed, 21 Apr 2004 13:32:49 +0400 (MSD) Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: <20040421084505.GV250@brightrain.aerifal.cx> References: <20040420152622.6C90934B2E@mail.mplayerhq.hu> <1082475671.1922.42.camel@localhost.localdomain> <20040420202500.GA250@brightrain.aerifal.cx> <20040421084505.GV250@brightrain.aerifal.cx> Message-ID: On Wed, 21 Apr 2004, D Richard Felker III wrote: DRFI>> You are a little wrong here. Sure, since you never tried those filters DRFI>> in transcode you might think so, but the following sequence of filters DRFI>> -J ivtc,32detect=force_mode=3,decimate DRFI>> DRFI>> in transcode has no working alternative in mplayer. I tried a lot, but DRFI>What is this sequence supposed to do? It's supposed to er.. remove telecine, I presume ;) It was a strange and hard case, looked like material that was deinterlaced after telecine, but not quite, and it was on dvd. DRFI>> found nothing. decimate filter in mplayer is different, that's all. I DRFI> DRFI>What does transcode's decimate filter do? Drop 1 in 5 frames, picking DRFI>the one with the least difference? The new divtc filter in mplayer Yes. DRFI>(silly name, oh well) does that. Maybe, but it is _really_ new ;) DRFI>> Yes, ivtc filters in mplayer are overally better. But _there are_ cases DRFI>> when transcode gives desired result and mplayer doesn't. DRFI> DRFI>If you have example clips where the various filters mess up, I'd DRFI>appreciate seeing them. Sorry.. I have only slow modem connection, and uploading even parts of dvd's is a hard thing to do. DRFI>> smartdeint is another very good ported from vdub filter (that's why it DRFI>> is working in rgb, and smartyuv was made afterwards as "special yuv DRFI>> version for transcode") that imho is the best deinterlacer. It's a shame DRFI>> it isn't present in mplayer, but I haven't looked at kerndeint yet. I DRFI>> hope it is just ported smartdeint with stupid name, not something DRFI>> written from scratch... DRFI> DRFI>I think it's the same thing. I tried it on telecined content because I DRFI>didn't have any true interlaced material handy, and I must say I DRFI>wasn't impressed. The kernel it uses for deinterlacing the interlaced DRFI>parts is bad (produces nasty edge-ghosts) and it deinterlaces wherever DRFI>there's motion, rather than just the areas that have combing. If you are interested, there is some info about kerndeint (from its author): http://neuron2.net/journal/journal.html And yes, it looks like it is the same filter as smartdeint, or something close to it; at least, they are written by the same person. They seem to use the similar technique to detect motion, but possibly deinterlacing algorithm is different. In smartdeint, you can choose either lb, li or ci filter (speaking in mplayer terms ;) for actual deinterlacing. I've yet to find [dis]advantages of kerndeint comparing to smartdeint... -- Vladimir From dalias at aerifal.cx Wed Apr 21 15:59:57 2004 From: dalias at aerifal.cx (D Richard Felker III) Date: Wed, 21 Apr 2004 09:59:57 -0400 Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: References: <20040420152622.6C90934B2E@mail.mplayerhq.hu> <1082475671.1922.42.camel@localhost.localdomain> <20040420202500.GA250@brightrain.aerifal.cx> <20040421084505.GV250@brightrain.aerifal.cx> Message-ID: <20040421135957.GZ250@brightrain.aerifal.cx> On Wed, Apr 21, 2004 at 01:32:49PM +0400, Vladimir Mosgalin wrote: > On Wed, 21 Apr 2004, D Richard Felker III wrote: > > DRFI>> You are a little wrong here. Sure, since you never tried those filters > DRFI>> in transcode you might think so, but the following sequence of filters > DRFI>> -J ivtc,32detect=force_mode=3,decimate > DRFI>> > DRFI>> in transcode has no working alternative in mplayer. I tried a lot, but > DRFI>What is this sequence supposed to do? > > It's supposed to er.. remove telecine, I presume ;) It was a strange and > hard case, looked like material that was deinterlaced after telecine, > but not quite, and it was on dvd. Hmm, but you shouldn't need a complicated filter chain like that to remove telecine. The steps make no sense... > DRFI>> Yes, ivtc filters in mplayer are overally better. But _there are_ cases > DRFI>> when transcode gives desired result and mplayer doesn't. > DRFI> > DRFI>If you have example clips where the various filters mess up, I'd > DRFI>appreciate seeing them. > > Sorry.. I have only slow modem connection, and uploading even parts of > dvd's is a hard thing to do. :( > DRFI>I think it's the same thing. I tried it on telecined content because I > DRFI>didn't have any true interlaced material handy, and I must say I > DRFI>wasn't impressed. The kernel it uses for deinterlacing the interlaced > DRFI>parts is bad (produces nasty edge-ghosts) and it deinterlaces wherever > DRFI>there's motion, rather than just the areas that have combing. > > If you are interested, there is some info about kerndeint (from its > author): > http://neuron2.net/journal/journal.html Thanks, very informative. > And yes, it looks like it is the same filter as smartdeint, or something > close to it; at least, they are written by the same person. They seem to > use the similar technique to detect motion, but possibly deinterlacing > algorithm is different. In smartdeint, you can choose either lb, li or > ci filter (speaking in mplayer terms ;) for actual deinterlacing. I've > yet to find [dis]advantages of kerndeint comparing to smartdeint... Ah. li and ci are correct deinterlacers but the output looks "low resolution". lb is incorrect but looks ok if there's only low motion. fd (aka ffmpeg deinterlacer, aka lavcdeint, which kerndeint seems to mimic) is completely wrong (but based on formulae in some mpeg spec...) and produces a ghost that looks like an edge-detect image of the other field over top of the picture. Rich From mosgalin at VM10124.spb.edu Wed Apr 21 18:29:31 2004 From: mosgalin at VM10124.spb.edu (Vladimir Mosgalin) Date: Wed, 21 Apr 2004 20:29:31 +0400 (MSD) Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: <20040421135957.GZ250@brightrain.aerifal.cx> References: <20040420152622.6C90934B2E@mail.mplayerhq.hu> <1082475671.1922.42.camel@localhost.localdomain> <20040420202500.GA250@brightrain.aerifal.cx> <20040421084505.GV250@brightrain.aerifal.cx> <20040421135957.GZ250@brightrain.aerifal.cx> Message-ID: On Wed, 21 Apr 2004, D Richard Felker III wrote: DRFI>Hmm, but you shouldn't need a complicated filter chain like that to DRFI>remove telecine. The steps make no sense... Nope, it's just transcode way of performing ivtc. It is written in documentation --- Additionally, when the telecine pattern shifts (as a result of, maybe, field editing by the DVD authors) a couple of interlaced frames will pass through. These should be deinterlaced on their own. You shouldn't use global deinterlacing after ivtc - it would blur out the otherwise perfect progressive frames that get reconstructed from 'ivtc'. However, transcode's '32detect' filter comes to the rescue: It first checks whether the frame is interlaced, and only then it forces a frame deinterlacing. So, this is what I recommend for your sessions: transcode -M 0 -f 23.976 -J ivtc,32detect=force_mode=3,decimate ... --- DRFI>> And yes, it looks like it is the same filter as smartdeint, or something DRFI>> close to it; at least, they are written by the same person. They seem to DRFI>> use the similar technique to detect motion, but possibly deinterlacing DRFI>> algorithm is different. In smartdeint, you can choose either lb, li or DRFI>> ci filter (speaking in mplayer terms ;) for actual deinterlacing. I've DRFI>> yet to find [dis]advantages of kerndeint comparing to smartdeint... DRFI> DRFI>Ah. li and ci are correct deinterlacers but the output looks "low DRFI>resolution". lb is incorrect but looks ok if there's only low motion. DRFI>fd (aka ffmpeg deinterlacer, aka lavcdeint, which kerndeint seems to DRFI>mimic) is completely wrong (but based on formulae in some mpeg DRFI>spec...) and produces a ghost that looks like an edge-detect image of DRFI>the other field over top of the picture. Well, result from kerndeint looks almost like cubic interpolation to me. I liked smartdeint in blend mode more... -- Vladimir From dalias at aerifal.cx Wed Apr 21 19:42:36 2004 From: dalias at aerifal.cx (D Richard Felker III) Date: Wed, 21 Apr 2004 13:42:36 -0400 Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: References: <20040420152622.6C90934B2E@mail.mplayerhq.hu> <1082475671.1922.42.camel@localhost.localdomain> <20040420202500.GA250@brightrain.aerifal.cx> <20040421084505.GV250@brightrain.aerifal.cx> <20040421135957.GZ250@brightrain.aerifal.cx> Message-ID: <20040421174236.GF250@brightrain.aerifal.cx> On Wed, Apr 21, 2004 at 08:29:31PM +0400, Vladimir Mosgalin wrote: > On Wed, 21 Apr 2004, D Richard Felker III wrote: > > DRFI>Hmm, but you shouldn't need a complicated filter chain like that to > DRFI>remove telecine. The steps make no sense... > > Nope, it's just transcode way of performing ivtc. It is written in > documentation > --- > Additionally, when the telecine pattern shifts (as a result > of, maybe, field editing by the DVD authors) a couple of > interlaced frames will pass through. These should be deinterlaced > on their own. You shouldn't use global deinterlacing after ivtc > - it would blur out the otherwise perfect progressive frames > that get reconstructed from 'ivtc'. This most likely indicates that the ivtc filter is buggy. It should never output any interlaced material for purely telecined content. If the original is a mix of telecined film and video, then all bets are off... > However, transcode's '32detect' > filter comes to the rescue: It first checks whether the frame > is interlaced, and only then it forces a frame deinterlacing. This will result in ugly flicker in your movie where frames incorrectly get deinterlaced rather than properly matched. You'd be better off deinterlacing all the frames with a blend filter so it's uniform at least... > So, this is what I recommend for your sessions: > > transcode -M 0 -f 23.976 -J ivtc,32detect=force_mode=3,decimate ... I still don't understand how decimate is useful... > DRFI>Ah. li and ci are correct deinterlacers but the output looks "low > DRFI>resolution". lb is incorrect but looks ok if there's only low motion. > DRFI>fd (aka ffmpeg deinterlacer, aka lavcdeint, which kerndeint seems to > DRFI>mimic) is completely wrong (but based on formulae in some mpeg > DRFI>spec...) and produces a ghost that looks like an edge-detect image of > DRFI>the other field over top of the picture. > > Well, result from kerndeint looks almost like cubic interpolation to me. It's definitely not. As far as I can tell it's the same lowpass/highpass thing the Decomb author described on that web site you cited. Cubic would "look bad" but not create artifacts. > I liked smartdeint in blend mode more... I agree. The kernel they use is absolutely horrible, but lots of people seem to like it for some dumb reason. I think it's supposed to look ok with very-low-motion live action video, but it's very very bad on animation or high-motion video. Rich From mr at ramendik.ru Wed Apr 21 19:44:30 2004 From: mr at ramendik.ru (Mikhail Ramendik) Date: Wed, 21 Apr 2004 21:44:30 +0400 (MSD) Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: <20040421135957.GZ250@brightrain.aerifal.cx> References: <20040420152622.6C90934B2E@mail.mplayerhq.hu><1082475671.1922.42.camel@localhost.localdomain><20040420202500.GA250@brightrain.aerifal.cx><20040421084505.GV250@brightrain.aerifal.cx> <20040421135957.GZ250@brightrain.aerifal.cx> Message-ID: <2305.213.234.233.150.1082569470.squirrel@webmail.sub.ru> Hello, D Richard Felker III said: >> DRFI>If you have example clips where the various filters mess up, I'd >> DRFI>appreciate seeing them. >> >> Sorry.. I have only slow modem connection, and uploading even parts of >> dvd's is a hard thing to do. > > :( To Vladimir: St.Petersburg has a lot of good Internet clubs. Uploading 10-20 MB would be easy. After all you only need a second or two of motion. If not, please contact me the next time you plan to visit Moscow; I'll try to arrange uploading it. To Richard: where to upload? Or should we put it up on some public server for download? Yours, Mikhail Ramendik From dalias at aerifal.cx Wed Apr 21 21:49:31 2004 From: dalias at aerifal.cx (D Richard Felker III) Date: Wed, 21 Apr 2004 15:49:31 -0400 Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: <2305.213.234.233.150.1082569470.squirrel@webmail.sub.ru> References: <20040421135957.GZ250@brightrain.aerifal.cx> <2305.213.234.233.150.1082569470.squirrel@webmail.sub.ru> Message-ID: <20040421194931.GH250@brightrain.aerifal.cx> On Wed, Apr 21, 2004 at 09:44:30PM +0400, Mikhail Ramendik wrote: > Hello, > > D Richard Felker III said: > > >> DRFI>If you have example clips where the various filters mess up, I'd > >> DRFI>appreciate seeing them. > >> > >> Sorry.. I have only slow modem connection, and uploading even parts of > >> dvd's is a hard thing to do. > > > > :( > > To Vladimir: St.Petersburg has a lot of good Internet clubs. Uploading > 10-20 MB would be easy. After all you only need a second or two of motion. > If not, please contact me the next time you plan to visit Moscow; I'll try > to arrange uploading it. > > To Richard: where to upload? Or should we put it up on some public server > for download? MPHQ of course. Rich From diego at biurrun.de Wed Apr 21 23:45:55 2004 From: diego at biurrun.de (Diego Biurrun) Date: Wed, 21 Apr 2004 23:45:55 +0200 Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: <20040421194931.GH250@brightrain.aerifal.cx> References: <20040421135957.GZ250@brightrain.aerifal.cx> <2305.213.234.233.150.1082569470.squirrel@webmail.sub.ru> <20040421194931.GH250@brightrain.aerifal.cx> Message-ID: <16518.60307.113322.8257@biurrun.de> D Richard Felker III writes: > On Wed, Apr 21, 2004 at 09:44:30PM +0400, Mikhail Ramendik wrote: > > To Richard: where to upload? Or should we put it up on some public server > > for download? > > MPHQ of course. This means mplayerhq.hu just in case you did not now. Further details can be found at http://www.mplayerhq.hu/DOCS/HTML/en/bugreports.html Diego From ranma at gmx.at Thu Apr 22 00:53:10 2004 From: ranma at gmx.at (Tobias Diedrich) Date: Thu, 22 Apr 2004 00:53:10 +0200 Subject: [MPlayer-G2-dev] Re: transcode filters In-Reply-To: References: <20040420152622.6C90934B2E@mail.mplayerhq.hu> <1082475671.1922.42.camel@localhost.localdomain> <20040420202500.GA250@brightrain.aerifal.cx> Message-ID: <20040421225310.GC25759@melchior.yamamaya.is-a-geek.org> Vladimir Mosgalin wrote: > smartdeint is another very good ported from vdub filter (that's why it > is working in rgb, and smartyuv was made afterwards as "special yuv > version for transcode") that imho is the best deinterlacer. It's a shame > it isn't present in mplayer, but I haven't looked at kerndeint yet. I > hope it is just ported smartdeint with stupid name, not something > written from scratch... AFAIK smartdeint was written by Donald Graft too, right? AFAICS in the avisynth forums the newer kernel deinterlacer is supposed to be better than the smart deinterlacer. -- Tobias PGP: http://9ac7e0bc.2ya.com Be vigilant! From ranma at gmx.at Thu Apr 22 00:43:39 2004 From: ranma at gmx.at (Tobias Diedrich) Date: Thu, 22 Apr 2004 00:43:39 +0200 Subject: [MPlayer-G2-dev] Re: transcode filters In-Reply-To: <1082475671.1922.42.camel@localhost.localdomain> References: <20040420152622.6C90934B2E@mail.mplayerhq.hu> <1082475671.1922.42.camel@localhost.localdomain> Message-ID: <20040421224339.GB25759@melchior.yamamaya.is-a-geek.org> Mikhail Ramendik wrote: > > kerndeint? > > Will check this out (will need to compile a new version of mplayer for > that). It's a recent addition, however; transcode had one for quite some > time. Is this a case of work duplication? Would not a more standard API > prevent *that* kind of duplication? kerndeint is a port of a Donald Graft filter from avisynth (Nice windoze tool). -- Tobias PGP: http://9ac7e0bc.2ya.com German uses a strange mixture of big- and little-endianness: 376 is pronounced as "Dreihundertsechsundsiebzig", i.e. "three hundred six and seventy". -- http://en.wikipedia.org/wiki/Endianess From mosgalin at VM10124.spb.edu Thu Apr 22 08:37:54 2004 From: mosgalin at VM10124.spb.edu (Vladimir Mosgalin) Date: Thu, 22 Apr 2004 10:37:54 +0400 (MSD) Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: <20040421174236.GF250@brightrain.aerifal.cx> References: <20040420152622.6C90934B2E@mail.mplayerhq.hu> <1082475671.1922.42.camel@localhost.localdomain> <20040420202500.GA250@brightrain.aerifal.cx> <20040421084505.GV250@brightrain.aerifal.cx> <20040421135957.GZ250@brightrain.aerifal.cx> <20040421174236.GF250@brightrain.aerifal.cx> Message-ID: On Wed, 21 Apr 2004, D Richard Felker III wrote: DRFI>This most likely indicates that the ivtc filter is buggy. It should DRFI>never output any interlaced material for purely telecined content. DRFI>If the original is a mix of telecined film and video, then all bets DRFI>are off... Yes, it was about mix. In "good" case 32detect won't be needed. Just ivts,decimate. DRFI>> However, transcode's '32detect' DRFI>> filter comes to the rescue: It first checks whether the frame DRFI>> is interlaced, and only then it forces a frame deinterlacing. DRFI> DRFI>This will result in ugly flicker in your movie where frames DRFI>incorrectly get deinterlaced rather than properly matched. You'd be DRFI>better off deinterlacing all the frames with a blend filter so it's DRFI>uniform at least... Um.. But what about cases when there are 500-1000 interlaced frames after imperfect ivtc process in the whole movie on complex cases? They are noticable when they are interlaced, but they aren't if they will just flicker a little. On the other hand, quality of this 32detect isn't perfect too. It tends to skips frames with slight interlacing pattern. DRFI>> So, this is what I recommend for your sessions: DRFI>> DRFI>> transcode -M 0 -f 23.976 -J ivtc,32detect=force_mode=3,decimate ... DRFI>I still don't understand how decimate is useful... ivtc creates 5 progressive frames, decimate actually throws one of them. Nothing to do about it, it's just the way. 32detect is inserted because the case was hard. In easy cases, of course, every filter inserted between them will potentially ruin the process. But. As the side effect (the cases where ivtc will fail, but due to movie nature - say, it was deinterlaced) decimate, which will still work on that frames, /may/ do something useful and throw away right frame. That's the rare case, but I needed this once. DRFI>> Well, result from kerndeint looks almost like cubic interpolation to me. DRFI> DRFI>It's definitely not. As far as I can tell it's the same DRFI>lowpass/highpass thing the Decomb author described on that web site DRFI>you cited. Cubic would "look bad" but not create artifacts. I took two cases: animation with complex non-interlaced still frame, and interlaced pal dvd (the sequence was recorded from the camera). Then I ran mplayer -vf pp,pp=ci (I compared with other filters too) in one desktop and -vf pp,kerndeint. Then I was switching between them. The two pictures were wery close. Another oddity: on still frame, kerndeint was working _only_ if the output driver is xv, xvidix or sdl, but not with x11. Adding scale filter at the end of the chain disabled its effect with any driver. This was regardless thresh=0 (which I suppose is required to force deinterlacing still frame). What kind of bug is this? -- Vladimir From dalias at aerifal.cx Fri Apr 23 01:13:39 2004 From: dalias at aerifal.cx (D Richard Felker III) Date: Thu, 22 Apr 2004 19:13:39 -0400 Subject: [MPlayer-G2-dev] transcode filters In-Reply-To: References: <20040420152622.6C90934B2E@mail.mplayerhq.hu> <1082475671.1922.42.camel@localhost.localdomain> <20040420202500.GA250@brightrain.aerifal.cx> <20040421084505.GV250@brightrain.aerifal.cx> <20040421135957.GZ250@brightrain.aerifal.cx> <20040421174236.GF250@brightrain.aerifal.cx> Message-ID: <20040422231339.GB250@brightrain.aerifal.cx> On Thu, Apr 22, 2004 at 10:37:54AM +0400, Vladimir Mosgalin wrote: > On Wed, 21 Apr 2004, D Richard Felker III wrote: > > DRFI>This most likely indicates that the ivtc filter is buggy. It should > DRFI>never output any interlaced material for purely telecined content. > DRFI>If the original is a mix of telecined film and video, then all bets > DRFI>are off... > > Yes, it was about mix. In "good" case 32detect won't be needed. Just > ivts,decimate. Let me explain why this is bad design. At the inverse-telecine stage, you already know which fields are duplicate, based on (here's the key) more information! Once the frames get to the decimate filter, this information is lost. So decimate has to make the decision again, based on less information. And it can in principle make the wrong decision, even though the right decision was already computer at the ivtc stage. That's why I claim transcode's approach is bad. > DRFI>This will result in ugly flicker in your movie where frames > DRFI>incorrectly get deinterlaced rather than properly matched. You'd be > DRFI>better off deinterlacing all the frames with a blend filter so it's > DRFI>uniform at least... > > Um.. But what about cases when there are 500-1000 interlaced frames > after imperfect ivtc process in the whole movie on complex cases? They > are noticable when they are interlaced, but they aren't if they will > just flicker a little. Why are there 500-1000 interlaced frames? If there's really a mix of video and film/animation content in the original, then it makes sense. But if the original is entirely telecined film/animation, there should be at most 5-10 interlaced frames in the output, and preferably 0. If there are that many, it means that the ivtc algorithm really sucks (probably it's too dependent on tracking phase or not spatially localized like it should be). > Another oddity: on still frame, kerndeint was working _only_ if the > output driver is xv, xvidix or sdl, but not with x11. Adding scale > filter at the end of the chain disabled its effect with any driver. > This was regardless thresh=0 (which I suppose is required to force > deinterlacing still frame). What kind of bug is this? Show me the command lines or something. Maybe kerndeint is buggy. Necessary filters should be auto-loaded... Rich From Reimar.Doeffinger at stud.uni-karlsruhe.de Fri Apr 23 23:39:14 2004 From: Reimar.Doeffinger at stud.uni-karlsruhe.de (=?ISO-8859-15?Q?Reimar_D=F6ffinger?=) Date: Fri, 23 Apr 2004 23:39:14 +0200 Subject: [MPlayer-G2-dev] Experimental vo_gl with (limited) support for hardware YUV. Message-ID: <40898D02.6080701@stud.uni-karlsruhe.de> Hi, just thought I let everyone have a look at what I'm currently working on. It can play YUV directly on nVidia GPUs better than GeForce3, but probably won't work at all with older cards :( - it's still a long way till it's finished, but I hope the concept will be extensible enough. On my PC it uses slightly less CPU even than xv (even with sync-to-vblank enabled). As always comments or test results are welcome. Greetings, Reimar D?ffinger -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: vo_gl.c URL: