From alex at fsn.hu  Sat Nov  1 16:38:14 2003
From: alex at fsn.hu (Alex Beregszaszi)
Date: Sat, 1 Nov 2003 16:38:14 +0100
Subject: [MPlayer-G2-dev] MID proposal
Message-ID: <20031101163814.65ff6232.alex@fsn.hu>

Hi,

Here's my idea of a IMGFMT replacement: MID stands for MPlayer Image
Description.

MID {
imgfmt (?)
type (rgb or yuv)
bpp
depth
chroma_x_shift
chroma_y_shift
num_planes (>1 -> planar)
fields (interlacing)
}

MPI {
mid
flags
type
width, height
x,y,w,h
planes
stride
...
}

The advantage of this is that it would be safe to change query_format to
the following: query_format(vo_instance, mid, width, height) and other
apperances of complicated imgfmt usage, it would reduce code.

-- 
Alex Beregszaszi <alex at fsn.hu>
(MPlayer Core Developer -- http://www.mplayerhq.hu/)


From dalias at aerifal.cx  Sat Nov  1 18:17:42 2003
From: dalias at aerifal.cx (D Richard Felker III)
Date: Sat, 1 Nov 2003 12:17:42 -0500
Subject: [MPlayer-G2-dev] MID proposal
In-Reply-To: <20031101163814.65ff6232.alex@fsn.hu>
References: <20031101163814.65ff6232.alex@fsn.hu>
Message-ID: <20031101171742.GP2856@brightrain.aerifal.cx>

On Sat, Nov 01, 2003 at 04:38:14PM +0100, Alex Beregszaszi wrote:
> Hi,
> 
> Here's my idea of a IMGFMT replacement: MID stands for MPlayer Image
> Description.
> 
> MID {
> imgfmt (?)
> type (rgb or yuv)
> bpp
> depth
> chroma_x_shift
> chroma_y_shift
> num_planes (>1 -> planar)
> fields (interlacing)
> }
> 
> MPI {
> mid
> flags
> type
> width, height
> x,y,w,h
> planes
> stride
> ...
> }
> 
> The advantage of this is that it would be safe to change query_format to
> the following: query_format(vo_instance, mid, width, height) and other
> apperances of complicated imgfmt usage, it would reduce code.

I agree something like this is _definitely_ desirable. Having a switch
statement in query_format where you have to list every single planar
format is idiotic...

As for the specifics, I'm not sure it's complete. IMO there should
more than just num_planes, but a way to specify
packed(yuy2)/planar(yv12)/semiplanar(nv12)... Maybe 1/3/2 planes is
a suitable way to identify this...? From my perspective, the main info
filters need is just the type of planar layout; beyond that, most
filter's don't care about the specifics.

Rich


From dalias at aerifal.cx  Sat Nov  1 19:07:24 2003
From: dalias at aerifal.cx (D Richard Felker III)
Date: Sat, 1 Nov 2003 13:07:24 -0500
Subject: [MPlayer-G2-dev] MID proposal
In-Reply-To: <20031101163814.65ff6232.alex@fsn.hu>
References: <20031101163814.65ff6232.alex@fsn.hu>
Message-ID: <20031101180723.GQ2856@brightrain.aerifal.cx>

On Sat, Nov 01, 2003 at 04:38:14PM +0100, Alex Beregszaszi wrote:
> Hi,
> 
> Here's my idea of a IMGFMT replacement: MID stands for MPlayer Image
> Description.

Some more questions...

> MID {
> imgfmt (?)

Rename to fourcc.

> type (rgb or yuv)

Rename to colorspace or something.

> bpp
> depth

What is the point of depth? How is it defined?

> chroma_x_shift
> chroma_y_shift

I always found these names excessively verbose to type when writing
filters. Any better ideas?

> num_planes (>1 -> planar)

See my other mail.

> fields (interlacing)

Hm? Does this just tell if the image is interlaced, or does it contain
other flags about the interlacing format?

> }
> 
> MPI {
> mid
> flags
> type
> width, height
> x,y,w,h

The whole width, height, x, y, w, h system is nonsense. x,y are never
used and not even supported. They're much better handled by just
adjusting the actual pointers. Also lots of filters don't understand
the difference between w and width, which is very silly to begin with.
In the new vp code I'm working on, width/height/x/y are entirely
eliminated, and the stride restrictions are cleaned up a lot. There's
now:

#define VP_STRIDE_MB_ALIGNED 0x1    // stride%(16*bpp) == 0
#define VP_STRIDE_PIXEL_ALIGNED 0x2 // stride%bpp == 0
#define VP_STRIDE_EXACT_WIDTH 0x4   // stride==bpp*w
#define VP_STRIDE_COMMON 0x8        // stride[1,2]==stride[0]>>shift

Codecs which wanted MP_IMGFLAG_ACCEPT_WIDTH before can now just use
VP_STRIDE_PIXEL_ALIGNED and stride/bpp will be the old width field.
(Note for simplicity here I use bpp=bytes/pixel, even though it really
means bits... :)

Rich



From arpi at thot.banki.hu  Sat Nov  1 19:33:13 2003
From: arpi at thot.banki.hu (Arpi)
Date: Sat, 1 Nov 2003 19:33:13 +0100
Subject: [MPlayer-G2-dev] MID proposal
In-Reply-To: <20031101163814.65ff6232.alex@fsn.hu>
Message-ID: <200311011833.hA1IXDiR031218@mail.mplayerhq.hu>

Hi,

> Here's my idea of a IMGFMT replacement: MID stands for MPlayer Image
> Description.
> 
> MID {
> imgfmt (?)
> type (rgb or yuv)
or special: mpeg, mjpeg, cymk etc

> bpp
> depth
> chroma_x_shift
> chroma_y_shift
> num_planes (>1 -> planar)

imho:
0   -> packed
>=1 -> planar (think of grayscale)

> fields (interlacing)

why here? whats the purpose of it?

btw the definitions of MID should be constant, from a
big table also used for config layer's TYPE_IMGFMT and so on.

> MPI {
> mid
> flags
> type
> width, height
> x,y,w,h
> planes
> stride
> ...
> }
> 
> The advantage of this is that it would be safe to change query_format to
> the following: query_format(vo_instance, mid, width, height) and other
> apperances of complicated imgfmt usage, it would reduce code.

agree, i also had this in mind, but i dropped due to big changes
all over the code (i wanted to avoid big changes which delay g2 a lot)


A'rpi / Astral & ESP-team

--
Developer of MPlayer G2, the Movie Framework for all - http://www.MPlayerHQ.hu


From kinali at gmx.net  Sat Nov  1 19:48:46 2003
From: kinali at gmx.net (Attila Kinali)
Date: Sat, 1 Nov 2003 19:48:46 +0100
Subject: [MPlayer-G2-dev] MID proposal
In-Reply-To: <20031101163814.65ff6232.alex@fsn.hu>
References: <20031101163814.65ff6232.alex@fsn.hu>
Message-ID: <20031101194846.332de131.kinali@gmx.net>

On Sat, 1 Nov 2003 16:38:14 +0100
Alex Beregszaszi <alex at fsn.hu> wrote:

> bpp
> depth

Sorry, for my ignorance, but what's the difference between
bpp and depth ?

		Attila Kinali

-- 
egp ist vergleichbar mit einem ikea bausatz fuer flugzeugtraeger
			-- reeler in +kaosu


From dalias at aerifal.cx  Sat Nov  1 19:52:40 2003
From: dalias at aerifal.cx (D Richard Felker III)
Date: Sat, 1 Nov 2003 13:52:40 -0500
Subject: [MPlayer-G2-dev] MID proposal
In-Reply-To: <200311011833.hA1IXDiR031218@mail.mplayerhq.hu>
References: <20031101163814.65ff6232.alex@fsn.hu>
	<200311011833.hA1IXDiR031218@mail.mplayerhq.hu>
Message-ID: <20031101185240.GR2856@brightrain.aerifal.cx>

On Sat, Nov 01, 2003 at 07:33:13PM +0100, Arpi wrote:
> Hi,
> 
> > Here's my idea of a IMGFMT replacement: MID stands for MPlayer Image
> > Description.
> > 
> > MID {
> > imgfmt (?)
> > type (rgb or yuv)
> or special: mpeg, mjpeg, cymk etc

Hmm..what do you mean?
Maybe type (colorspace) should be able to take on 3 values (and more
in the future if needed): RGB, YUV, and "SPECIAL". Filters then could
reject "SPECIAL" type outright unless they recognize the imgfmt.

> imho:
> 0   -> packed
> >=1 -> planar (think of grayscale)

Agree.

> > fields (interlacing)
> 
> why here? whats the purpose of it?

I agree it doesn't belong.

> > The advantage of this is that it would be safe to change query_format to
> > the following: query_format(vo_instance, mid, width, height) and other
> > apperances of complicated imgfmt usage, it would reduce code.
> 
> agree, i also had this in mind, but i dropped due to big changes
> all over the code (i wanted to avoid big changes which delay g2 a lot)

Porting filters will require changes anyway. I volunteer to do most of
it once I'm done with vp layer.

Rich


From alex at fsn.hu  Sat Nov  1 20:06:47 2003
From: alex at fsn.hu (Alex Beregszaszi)
Date: Sat, 1 Nov 2003 20:06:47 +0100
Subject: [MPlayer-G2-dev] MID proposal
In-Reply-To: <200311011833.hA1IXDiR031218@mail.mplayerhq.hu>
References: <20031101163814.65ff6232.alex@fsn.hu>
	<200311011833.hA1IXDiR031218@mail.mplayerhq.hu>
Message-ID: <20031101200647.5b27894f.alex@fsn.hu>

Hi,

> > MID {
> > imgfmt (?)
> > type (rgb or yuv)
> or special: mpeg, mjpeg, cymk etc
Let's see the actual cases:
0: yuv
1: rgb
2: mpeg
3: mjpeg
further types (cmyk :D)

Or what about adding a 'compressed' type that would indicate
mpeg1/mjpeg/mpeg4/asv2 (you know, asus cards can en/decode asv) and
others.

> > chroma_x_shift
> > chroma_y_shift
What about naming them cr_h/v (chroma horizontal / vertical)

> > num_planes (>1 -> planar)
> 
> imho:
> 0   -> packed
> >=1 -> planar (think of grayscale)
Or the flags field from mpi: (i forgot it first :)
MPI_IMGFLAG_PLANAR, YUV, SWAPPED, RGB_PALETTE

Or what about merging flags with type?

> > fields (interlacing)
> why here? whats the purpose of it?
probably it has not much sense (i can see only one, if you read the
thread about jpeg_enc)
 
> agree, i also had this in mind, but i dropped due to big changes
> all over the code (i wanted to avoid big changes which delay g2 a lot)
but without big changes it will become the same bloat as g1 after some
time

A new idea: the new swscaler should based on a similar structure.

And also we could make a mid_to_imgfmt function, so we could really get
rid of internal imgfmt usage, only some filters/modules/whatever which
need to convert to the ancient format could use it.

-- 
Alex Beregszaszi <alex at fsn.hu>
(MPlayer Core Developer -- http://www.mplayerhq.hu/)


From arpi at thot.banki.hu  Sat Nov  1 20:11:45 2003
From: arpi at thot.banki.hu (Arpi)
Date: Sat, 1 Nov 2003 20:11:45 +0100
Subject: [MPlayer-G2-dev] MID proposal
In-Reply-To: <20031101200647.5b27894f.alex@fsn.hu>
Message-ID: <200311011911.hA1JBjIi012192@mail.mplayerhq.hu>

Hi,

> > > MID {
> > > imgfmt (?)
> > > type (rgb or yuv)
> > or special: mpeg, mjpeg, cymk etc
> Let's see the actual cases:
> 0: yuv
> 1: rgb
> 2: mpeg
> 3: mjpeg
> further types (cmyk :D)
> 
> Or what about adding a 'compressed' type that would indicate
> mpeg1/mjpeg/mpeg4/asv2 (you know, asus cards can en/decode asv) and
> others.

i vote for yuv, rgb, special
or maybe yuv, rgb, special, compressed

so uncompressed but non-rgb non-yuv could go to special

> > > chroma_x_shift
> > > chroma_y_shift
> What about naming them cr_h/v (chroma horizontal / vertical)

or csv/csh (ala scale filter's shift params)

> > > num_planes (>1 -> planar)
> > 
> > imho:
> > 0   -> packed
> > >=1 -> planar (think of grayscale)
> Or the flags field from mpi: (i forgot it first :)
> MPI_IMGFLAG_PLANAR, YUV, SWAPPED, RGB_PALETTE

we still need a plane number

> Or what about merging flags with type?

yes, it would be the best!!

> > > fields (interlacing)
> > why here? whats the purpose of it?
> probably it has not much sense (i can see only one, if you read the
> thread about jpeg_enc)
i didnt read that yet

> > agree, i also had this in mind, but i dropped due to big changes
> > all over the code (i wanted to avoid big changes which delay g2 a lot)
> but without big changes it will become the same bloat as g1 after some
> time
> 
> A new idea: the new swscaler should based on a similar structure.

new swscaler? r u dreaming? :)

> And also we could make a mid_to_imgfmt function, so we could really get
> rid of internal imgfmt usage, only some filters/modules/whatever which
> need to convert to the ancient format could use it.

dunno
as Rich said, if it's well done, we dont need imgfmt anymore
btw, what about adding short name and longer description too?
for config_type_imgfmt and various debug/nondebug printfs...


A'rpi / Astral & ESP-team

--
Developer of MPlayer G2, the Movie Framework for all - http://www.MPlayerHQ.hu


From alex at fsn.hu  Sat Nov  1 20:14:21 2003
From: alex at fsn.hu (Alex Beregszaszi)
Date: Sat, 1 Nov 2003 20:14:21 +0100
Subject: [MPlayer-G2-dev] MID proposal
In-Reply-To: <20031101180723.GQ2856@brightrain.aerifal.cx>
References: <20031101163814.65ff6232.alex@fsn.hu>
	<20031101180723.GQ2856@brightrain.aerifal.cx>
Message-ID: <20031101201421.76ddcf4c.alex@fsn.hu>

Hi,

> > MID {
> > imgfmt (?)
> 
> Rename to fourcc.
Why?

We could create a new list of format fourccs which could indicate only
one format described by MID, but the imgfmt is the ancient fourcc.

> > type (rgb or yuv)
> 
> Rename to colorspace or something.
See the mail from Arpi.
 
> > bpp
> > depth
> 
> What is the point of depth? How is it defined?
Depth is the number of bits used for describe a pixel, bpp means the
bytes which are used to store a pixel. RGB24 has depth=24 but can have
both bpp=24 and bpp=32.
 
> > chroma_x_shift
> > chroma_y_shift
> 
> I always found these names excessively verbose to type when writing
> filters. Any better ideas?
See my other mail (cr_h/v)
 
> > fields (interlacing)
> 
> Hm? Does this just tell if the image is interlaced, or does it contain
> other flags about the interlacing format?
If we leave it in MID it could contain more flags. Dunno if it should be
present or not.

[snip]
> The whole width, height, x, y, w, h system is nonsense. x,y are never
> used and not even supported. They're much better handled by just
> adjusting the actual pointers. Also lots of filters don't understand
> the difference between w and width, which is very silly to begin with.
Imho it has sense, but lot of filter/vo writers are dumb and can't make
difference.

Width/height defines the dimensions of the whole image pointed by MPI.
x,y,w,h descibre a rectangle of which part of the image should be drawn.

Yes, there are two ways: only pass a MPI with the actual image, but that
would still need x,y to be defined to place it correctly on the screen.
The other way is to leave our current scheme and fix the filters. Imho
this is a better approach.

Probably you will ask about the sense of this whole partial mpi stuff:
it has not much sense currently, but we could have some filters/decoders
which will support partial decoding / filtering.

-- 
Alex Beregszaszi <alex at fsn.hu>
(MPlayer Core Developer -- http://www.mplayerhq.hu/)


From alex at fsn.hu  Sat Nov  1 20:17:19 2003
From: alex at fsn.hu (Alex Beregszaszi)
Date: Sat, 1 Nov 2003 20:17:19 +0100
Subject: [MPlayer-G2-dev] MID proposal
In-Reply-To: <200311011911.hA1JBjIi012192@mail.mplayerhq.hu>
References: <20031101200647.5b27894f.alex@fsn.hu>
	<200311011911.hA1JBjIi012192@mail.mplayerhq.hu>
Message-ID: <20031101201719.3d4ad857.alex@fsn.hu>

Hi,

> > A new idea: the new swscaler should based on a similar structure.
> 
> new swscaler? r u dreaming? :)
Yes, else I wouldn't code on opensource projects.

-- 
Alex Beregszaszi <alex at fsn.hu>
(MPlayer Core Developer -- http://www.mplayerhq.hu/)


From dalias at aerifal.cx  Sat Nov  1 20:31:44 2003
From: dalias at aerifal.cx (D Richard Felker III)
Date: Sat, 1 Nov 2003 14:31:44 -0500
Subject: [MPlayer-G2-dev] MID proposal
In-Reply-To: <20031101201421.76ddcf4c.alex@fsn.hu>
References: <20031101163814.65ff6232.alex@fsn.hu>
	<20031101180723.GQ2856@brightrain.aerifal.cx>
	<20031101201421.76ddcf4c.alex@fsn.hu>
Message-ID: <20031101193144.GS2856@brightrain.aerifal.cx>

On Sat, Nov 01, 2003 at 08:14:21PM +0100, Alex Beregszaszi wrote:
> Hi,
> 
> > > MID {
> > > imgfmt (?)
> > 
> > Rename to fourcc.
> Why?
> 
> We could create a new list of format fourccs which could indicate only
> one format described by MID, but the imgfmt is the ancient fourcc.
> 
> > > type (rgb or yuv)
> > 
> > Rename to colorspace or something.
> See the mail from Arpi.
>  
> > > bpp
> > > depth
> > 
> > What is the point of depth? How is it defined?
> Depth is the number of bits used for describe a pixel, bpp means the
> bytes which are used to store a pixel. RGB24 has depth=24 but can have
> both bpp=24 and bpp=32.

I know that much. But how is depth defined for YUV? Is there even any
point of knowing the depth..?

> > > fields (interlacing)
> > 
> > Hm? Does this just tell if the image is interlaced, or does it contain
> > other flags about the interlacing format?
> If we leave it in MID it could contain more flags. Dunno if it should be
> present or not.

IMO not. Info about interlacing does definitely belong in the video
pipeline, BUT most of the time it wouldn't even be known yet when
you're using MIDs, and like Arpi said MIDs should come from a table of
constants.

> > The whole width, height, x, y, w, h system is nonsense. x,y are never
> > used and not even supported. They're much better handled by just
> > adjusting the actual pointers. Also lots of filters don't understand
> > the difference between w and width, which is very silly to begin with.
> Imho it has sense, but lot of filter/vo writers are dumb and can't make
> difference.
> 
> Width/height defines the dimensions of the whole image pointed by MPI.
> x,y,w,h descibre a rectangle of which part of the image should be drawn.

But that's not how it is right now. As it stands, width is essentially
the same thing as stride/bpp, which filters should not even care
about.

> Yes, there are two ways: only pass a MPI with the actual image, but that
> would still need x,y to be defined to place it correctly on the screen.
> The other way is to leave our current scheme and fix the filters. Imho
> this is a better approach.
> 
> Probably you will ask about the sense of this whole partial mpi stuff:
> it has not much sense currently, but we could have some filters/decoders
> which will support partial decoding / filtering.

This is NOT a clean way to do it. It's very limited (forces you to
only have one rectangle) and also forces all filters to support this
bloated ugliness of working on partial images explicitly. I have a
much better design for new vp layer:

Let's say you want to blur a small rectangle of your picture. Here's
how the filter chain works:


vd ----> vf_subfilter ----> vo
            |   ^
            |   |
            V   |
           vf_blur

vf_subfilter would pull an image from vd, then pull an image from
vf_blur, which would in turn pull from vf_subfilter, getting only the
small part of the image it was supposed to work on. The blurred image
would be returned to vf_subfilter, and then vf_subfilter would finish
things up and return the final image to vo. Of course direct rendering
and/or exporting would happen all over the place, so it would be very
fast.

Anyway, IMO the x,y stuff is not acceptable. It's too much of a burden
to ask every single filter to know how to process partial images by
itself, and there's no advantage.

Rich





From alex at fsn.hu  Sat Nov  1 20:33:43 2003
From: alex at fsn.hu (Alex Beregszaszi)
Date: Sat, 1 Nov 2003 20:33:43 +0100
Subject: [MPlayer-G2-dev] MID proposal
In-Reply-To: <20031101163814.65ff6232.alex@fsn.hu>
References: <20031101163814.65ff6232.alex@fsn.hu>
Message-ID: <20031101203343.50055e69.alex@fsn.hu>

Hi,

here comes v2 :)

MPlayer Image Description
-----------------------------------

Flags: YUV, RGB, SWAPPED, PALETTE, COMPRESSED, SPECIAL (is this really
needed?!)
BPP: bytes (bits?) needed for storing a pixel
Depth: bits used for a pixel
CSH: horizontal chroma shift (only used if YUV)
CSV: vertical chroma shift (only used if YUV)
Num Planes: 1 for packed, 2 (for y + interleaved uv (NV21/12)), 3, 4
(for IF04 and maybe later yuv+alpha?)
Imgfmt: backward compatibility fourcc?!

Btw, imho we need a separate data block for storing palette in mpi (we
should avoid that planes[1] hack) and also a palette_type or
palette_bits (currently mplayer treats all palettes as 8->24 bits)

Hm, one new issue come to my mind: ARGB vs RGBA, aka the place of alpha
byte. I hate these endianess issues :)

-- 
Alex Beregszaszi <alex at fsn.hu>
(MPlayer Core Developer -- http://www.mplayerhq.hu/)


From dalias at aerifal.cx  Sat Nov  1 20:52:59 2003
From: dalias at aerifal.cx (D Richard Felker III)
Date: Sat, 1 Nov 2003 14:52:59 -0500
Subject: [MPlayer-G2-dev] MID proposal
In-Reply-To: <20031101203343.50055e69.alex@fsn.hu>
References: <20031101163814.65ff6232.alex@fsn.hu>
	<20031101203343.50055e69.alex@fsn.hu>
Message-ID: <20031101195259.GT2856@brightrain.aerifal.cx>

On Sat, Nov 01, 2003 at 08:33:43PM +0100, Alex Beregszaszi wrote:
> Hi,
> 
> here comes v2 :)

Alex says write my thoughts to the list, so..

> MPlayer Image Description
> -----------------------------------
> 
> Flags: YUV, RGB, SWAPPED, PALETTE, COMPRESSED, SPECIAL (is this really
> needed?!)
> BPP: bytes (bits?) needed for storing a pixel
> Depth: bits used for a pixel
> CSH: horizontal chroma shift (only used if YUV)
> CSV: vertical chroma shift (only used if YUV)
> Num Planes: 1 for packed, 2 (for y + interleaved uv (NV21/12)), 3, 4
> (for IF04 and maybe later yuv+alpha?)
> Imgfmt: backward compatibility fourcc?!

Agree!

> Btw, imho we need a separate data block for storing palette in mpi (we
> should avoid that planes[1] hack) and also a palette_type or
> palette_bits (currently mplayer treats all palettes as 8->24 bits)

Agree, YUV palettes should also be possible.

> Hm, one new issue come to my mind: ARGB vs RGBA, aka the place of alpha
> byte. I hate these endianess issues :)

A is idiotic, there is no alpha. Just say RGB or BGR with no A.

>From vp.h:

// MPlayer Imageformat Description

#define MID_FLAG_YUV        0x01
#define MID_FLAG_RGB        0x02
#define MID_FLAG_SPECIAL    0x04
#define MID_FLAG_SWAPPED    0x10
#define MID_FLAG_PALETTE    0x20

typedef struct mid_s {
        int flags;
        int bpp;
        int csh, csv; // chroma shift, horiz. and vert.
        int num_planes; // 0=packed, 1=gray, 2=semiplanar(nv12),
        3=full planar
        unsigned int imgfmt; // legacy
        char *name, *longname;
} mid_t;

IMO we still need a way of telling where the channels are stored in
packed modes, especially RGB (list of bit sizes and bit offsets).

Rich


From alex at fsn.hu  Sat Nov  1 20:51:48 2003
From: alex at fsn.hu (Alex Beregszaszi)
Date: Sat, 1 Nov 2003 20:51:48 +0100
Subject: [MPlayer-G2-dev] MID proposal
In-Reply-To: <20031101193144.GS2856@brightrain.aerifal.cx>
References: <20031101163814.65ff6232.alex@fsn.hu>
	<20031101180723.GQ2856@brightrain.aerifal.cx>
	<20031101201421.76ddcf4c.alex@fsn.hu>
	<20031101193144.GS2856@brightrain.aerifal.cx>
Message-ID: <20031101205148.5d58259f.alex@fsn.hu>

Hi,

> > > What is the point of depth? How is it defined?
> > Depth is the number of bits used for describe a pixel, bpp means the
> > bytes which are used to store a pixel. RGB24 has depth=24 but can
> > have both bpp=24 and bpp=32.
> 
> I know that much. But how is depth defined for YUV? Is there even any
> point of knowing the depth..?
For YUV you must round down depth, but imho it has not much sense to
speak about depth in case of YUV. Depth field is only usable for RGB
colorspace so we wouldn't need those IMGFMT_RGB/BGR_DEPTH macros. 

 
> IMO not. Info about interlacing does definitely belong in the video
> pipeline, BUT most of the time it wouldn't even be known yet when
> you're using MIDs, and like Arpi said MIDs should come from a table of
> constants.
Let's drop this field for now.

[snip]
> Anyway, IMO the x,y stuff is not acceptable. It's too much of a burden
> to ask every single filter to know how to process partial images by
> itself, and there's no advantage.
Your mechanism is interesting, and yes, it's up to you if you drop these
fields or not. Note: I only copied MPI from the current headers. If we
all agree on a MID definition, we can work on a new MPI using MID.

-- 
Alex Beregszaszi <alex at fsn.hu>
(MPlayer Core Developer -- http://www.mplayerhq.hu/)


From ivan at cacad.com  Sat Nov  1 21:03:08 2003
From: ivan at cacad.com (Ivan Kalvachev)
Date: Sat, 1 Nov 2003 22:03:08 +0200 (EET)
Subject: [MPlayer-G2-dev] MID proposal
In-Reply-To: <20031101201719.3d4ad857.alex@fsn.hu>
References: 
	<20031101200647.5b27894f.alex@fsn.hu><200311011911.hA1JBjIi012192@mail.mplayerhq.hu>
	<20031101201719.3d4ad857.alex@fsn.hu>
Message-ID: <1545.194.12.248.133.1067716988.squirrel@mail.cacad.com>

Alex Beregszaszi said:
> Hi,
>
>> > A new idea: the new swscaler should based on a similar structure.
>>
>> new swscaler? r u dreaming? :)
> Yes, else I wouldn't code on opensource projects.
>
> --
> Alex Beregszaszi <alex at fsn.hu>
> (MPlayer Core Developer -- http://www.mplayerhq.hu/)
>
OK, I am constanly pointing the good side of XvImage
here is the structure, it is quite clear and quite overloaded,
strip it ;)
And don't worry, you wont become blind if you see it (once:)

typedef struct {
  int id;                      /* Unique descriptor for the format */
  int type;                    /* XvRGB, XvYUV */
  int byte_order;              /* LSBFirst, MSBFirst */
  char guid[16];               /* Globally Unique IDentifier */
  int bits_per_pixel;
  int format;                  /* XvPacked, XvPlanar */
  int num_planes;

  /* for RGB formats */
  int depth;
  unsigned int red_mask;
  unsigned int green_mask;
  unsigned int blue_mask;

  /* for YUV formats */
  unsigned int y_sample_bits;
  unsigned int u_sample_bits;
  unsigned int v_sample_bits;
  unsigned int horz_y_period;
  unsigned int horz_u_period;
  unsigned int horz_v_period;
  unsigned int vert_y_period;
  unsigned int vert_u_period;
  unsigned int vert_v_period;
  char component_order[32];    /* eg. UYVY */
  int scanline_order;          /* XvTopToBottom, XvBottomToTop */
} XvImageFormatValues;

OK here are my points:
1. merge some strucures so we wont' have different structures for
different formats (yuv,rgb)
2. _period and sample_bits could be in array.
3. samplebits is good for and e.g. RGB, but we will need and shift bits(e.g.
BGR565 B<<10 G <5 R<0 and G=g>>(8-samplebits[g_ofs])

4. component_order is nice idea. but donno how nv12 will be described;)
I had the idea to have recursive modes, but it will be too complicated.
5. rgb mask are good to have, some (sdl) releay on them
6. as MID will be static we could also add fourcc code, short name, long
name etc.... All stuff that will be used only once :)

Best Regards
   Ivan Kalvachev
  iive




From dalias at aerifal.cx  Sat Nov  1 21:18:42 2003
From: dalias at aerifal.cx (D Richard Felker III)
Date: Sat, 1 Nov 2003 15:18:42 -0500
Subject: [MPlayer-G2-dev] MID proposal
In-Reply-To: <1545.194.12.248.133.1067716988.squirrel@mail.cacad.com>
References: <20031101201719.3d4ad857.alex@fsn.hu>
	<1545.194.12.248.133.1067716988.squirrel@mail.cacad.com>
Message-ID: <20031101201842.GU2856@brightrain.aerifal.cx>

On Sat, Nov 01, 2003 at 10:03:08PM +0200, Ivan Kalvachev wrote:
> Alex Beregszaszi said:
> > Hi,
> >
> >> > A new idea: the new swscaler should based on a similar structure.
> >>
> >> new swscaler? r u dreaming? :)
> > Yes, else I wouldn't code on opensource projects.
> >
> > --
> > Alex Beregszaszi <alex at fsn.hu>
> > (MPlayer Core Developer -- http://www.mplayerhq.hu/)
> >
> OK, I am constanly pointing the good side of XvImage
> here is the structure, it is quite clear and quite overloaded,
> strip it ;)
> And don't worry, you wont become blind if you see it (once:)

Most MPlayer developers are already quite aware that stories about
going blind from seeing stuff are just myths...

> typedef struct {
>   int id;                      /* Unique descriptor for the format */
>   int type;                    /* XvRGB, XvYUV */
>   int byte_order;              /* LSBFirst, MSBFirst */
>   char guid[16];               /* Globally Unique IDentifier */
>   int bits_per_pixel;
>   int format;                  /* XvPacked, XvPlanar */
>   int num_planes;
> 
>   /* for RGB formats */
>   int depth;
>   unsigned int red_mask;
>   unsigned int green_mask;
>   unsigned int blue_mask;
> 
>   /* for YUV formats */
>   unsigned int y_sample_bits;
>   unsigned int u_sample_bits;
>   unsigned int v_sample_bits;
>   unsigned int horz_y_period;
>   unsigned int horz_u_period;
>   unsigned int horz_v_period;
>   unsigned int vert_y_period;
>   unsigned int vert_u_period;
>   unsigned int vert_v_period;
>   char component_order[32];    /* eg. UYVY */
>   int scanline_order;          /* XvTopToBottom, XvBottomToTop */
> } XvImageFormatValues;
> 
> OK here are my points:
> 1. merge some strucures so we wont' have different structures for
> different formats (yuv,rgb)

Yes, XvImageFormat is very ugly!

> 2. _period and sample_bits could be in array.
> 3. samplebits is good for and e.g. RGB, but we will need and shift bits(e.g.
> BGR565 B<<10 G <5 R<0 and G=g>>(8-samplebits[g_ofs])

Yes, we already know this.. :)

> 4. component_order is nice idea. but donno how nv12 will be described;)
> I had the idea to have recursive modes, but it will be too complicated.

Yes definitely too complicated!
IMO nv12 is handled just like fully planar modes. Default order is
U,V; set the "swapped" flag if it's opposite.

> 5. rgb mask are good to have, some (sdl) releay on them

Useless. You can generate them from sizes/offsets, and SDL sucks. IMO
SDL shouldn't even be supported in G2 since there's no way to support
it correctly, and since MPlayer's native drivers already support a lot
more than SDL anyway.

> 6. as MID will be static we could also add fourcc code, short name, long
> name etc.... All stuff that will be used only once :)

Yes.

Rich


From ivan at cacad.com  Sat Nov  1 22:22:31 2003
From: ivan at cacad.com (Ivan Kalvachev)
Date: Sat, 1 Nov 2003 23:22:31 +0200 (EET)
Subject: [MPlayer-G2-dev] MID proposal
In-Reply-To: <20031101201842.GU2856@brightrain.aerifal.cx>
References: 
	<20031101201719.3d4ad857.alex@fsn.hu><1545.194.12.248.133.1067716988.squirrel@mail.cacad.com>
	<20031101201842.GU2856@brightrain.aerifal.cx>
Message-ID: <1725.194.12.248.133.1067721751.squirrel@mail.cacad.com>

D Richard Felker III said:
> On Sat, Nov 01, 2003 at 10:03:08PM +0200, Ivan Kalvachev wrote:
>> OK, I am constanly pointing the good side of XvImage
>> here is the structure, it is quite clear and quite overloaded,
>> strip it ;)
>> And don't worry, you wont become blind if you see it (once:)
>
> Most MPlayer developers are already quite aware that stories about
> going blind from seeing stuff are just myths...
>
>> typedef struct {
[snip]
>> } XvImageFormatValues;
>>
>> OK here are my points:
>> 1. merge some strucures so we wont' have different structures for
>> different formats (yuv,rgb)
>
> Yes, XvImageFormat is very ugly!
>
>> 2. _period and sample_bits could be in array.
>> 3. samplebits is good for and e.g. RGB, but we will need and shift
>> bits(e.g.
>> BGR565 B<<10 G <5 R<0 and G=g>>(8-samplebits[g_ofs])
>
> Yes, we already know this.. :)
>
>> 4. component_order is nice idea. but donno how nv12 will be described;)
>> I had the idea to have recursive modes, but it will be too complicated.
>
> Yes definitely too complicated!
> IMO nv12 is handled just like fully planar modes. Default order is
> U,V; set the "swapped" flag if it's opposite.
Ok i got another idea, just in for the protocol, probably
it is too complicated to be used for something else than
learning formats. The basic is that in the plane cannot have
another plane format, so it can be only packed format.

e.g. for some imagine format
we have num_planes=3
plane 0 (bytes) - Y(bits=8, shift=0)
plane 1 (words) - U (bits=3, shift=5), V(bits=3, shift=2),
None(bits=2,shift=0)
plane 1 (bytes) - A (bits=8,shift=0)


>
>> 5. rgb mask are good to have, some (sdl) releay on them
>
> Useless. You can generate them from sizes/offsets, and SDL sucks. IMO
> SDL shouldn't even be supported in G2 since there's no way to support
> it correctly, and since MPlayer's native drivers already support a lot
> more than SDL anyway.
Agree.

Best Regards
   Ivan Kalvachev
  iive



From arpi at thot.banki.hu  Sat Nov  1 23:50:06 2003
From: arpi at thot.banki.hu (Arpi)
Date: Sat, 1 Nov 2003 23:50:06 +0100
Subject: [MPlayer-G2-dev] MID proposal
In-Reply-To: <20031101193144.GS2856@brightrain.aerifal.cx>
Message-ID: <200311012250.hA1Mo6hK027891@mail.mplayerhq.hu>

Hi,

> > > What is the point of depth? How is it defined?
> > Depth is the number of bits used for describe a pixel, bpp means the
> > bytes which are used to store a pixel. RGB24 has depth=24 but can have
> > both bpp=24 and bpp=32.
> 
> I know that much. But how is depth defined for YUV? Is there even any
> point of knowing the depth..?

for planar formats, it should be component depth, imho
ie. 8 for most YUV formats, even NV12

mayeb we should add 'component' field too
it could help for nv12 (comp=3 planes=2), rgb vs rgba (comp=4) etc

> > > > fields (interlacing)
> > > 
> > > Hm? Does this just tell if the image is interlaced, or does it contain
> > > other flags about the interlacing format?
> > If we leave it in MID it could contain more flags. Dunno if it should be
> > present or not.
> 
> IMO not. Info about interlacing does definitely belong in the video
> pipeline, BUT most of the time it wouldn't even be known yet when
> you're using MIDs, and like Arpi said MIDs should come from a table of
> constants.

yes. MID shouldnt contani variables at all.
every single field in MID struct must be constant for a given imgfmt

> > > The whole width, height, x, y, w, h system is nonsense. x,y are never
> > > used and not even supported. They're much better handled by just
> > > adjusting the actual pointers. Also lots of filters don't understand
> > > the difference between w and width, which is very silly to begin with.
> > Imho it has sense, but lot of filter/vo writers are dumb and can't make
> > difference.
> > 
> > Width/height defines the dimensions of the whole image pointed by MPI.
> > x,y,w,h descibre a rectangle of which part of the image should be drawn.
> 
> But that's not how it is right now. As it stands, width is essentially
> the same thing as stride/bpp, which filters should not even care
> about.

yes

> > Yes, there are two ways: only pass a MPI with the actual image, but that
> > would still need x,y to be defined to place it correctly on the screen.
> > The other way is to leave our current scheme and fix the filters. Imho
> > this is a better approach.
> > 
> > Probably you will ask about the sense of this whole partial mpi stuff:
> > it has not much sense currently, but we could have some filters/decoders
> > which will support partial decoding / filtering.
> 
> This is NOT a clean way to do it. It's very limited (forces you to
> only have one rectangle) and also forces all filters to support this

agree. x,y must go, it's unused in g1 too
in my original plans (for g1's vf) it was for crop/expand stuff, i wanted
to implement crop/expand by just changing those fields... was bad idea

> bloated ugliness of working on partial images explicitly. I have a

sure. we need well working slices, with x and w parameters (not y-only as in
g1)

> much better design for new vp layer:
> 
> Let's say you want to blur a small rectangle of your picture. Here's
> how the filter chain works:
> 
> 
> vd ----> vf_subfilter ----> vo
>             |   ^
>             |   |
>             V   |
>            vf_blur

huh


A'rpi / Astral & ESP-team

--
Developer of MPlayer G2, the Movie Framework for all - http://www.MPlayerHQ.hu


From dalias at aerifal.cx  Sun Nov  2 00:37:48 2003
From: dalias at aerifal.cx (D Richard Felker III)
Date: Sat, 1 Nov 2003 18:37:48 -0500
Subject: [MPlayer-G2-dev] MID proposal
In-Reply-To: <200311012250.hA1Mo6hK027891@mail.mplayerhq.hu>
References: <20031101193144.GS2856@brightrain.aerifal.cx>
	<200311012250.hA1Mo6hK027891@mail.mplayerhq.hu>
Message-ID: <20031101233748.GV2856@brightrain.aerifal.cx>

On Sat, Nov 01, 2003 at 11:50:06PM +0100, Arpi wrote:
> > > Probably you will ask about the sense of this whole partial mpi stuff:
> > > it has not much sense currently, but we could have some filters/decoders
> > > which will support partial decoding / filtering.
> > 
> > This is NOT a clean way to do it. It's very limited (forces you to
> > only have one rectangle) and also forces all filters to support this
> 
> agree. x,y must go, it's unused in g1 too
> in my original plans (for g1's vf) it was for crop/expand stuff, i wanted
> to implement crop/expand by just changing those fields... was bad idea
> 
> > bloated ugliness of working on partial images explicitly. I have a
> 
> sure. we need well working slices, with x and w parameters (not y-only as in
> g1)

Yes, perhaps. I don't see how slices will help performance much if you
do them in a strange order tho. Anyway slices aren't really related to
this mess in mp_image.

> > much better design for new vp layer:
> > 
> > Let's say you want to blur a small rectangle of your picture. Here's
> > how the filter chain works:
> > 
> > 
> > vd ----> vf_subfilter ----> vo
> >             |   ^
> >             |   |
> >             V   |
> >            vf_blur
> 
> huh

Just a cool example of nonlinear filter chains. The (hypothetical)
vf_subfilter attaches an entire chain to itself at both ends to allow
you to apply filters to just a part of an image.

Rich


From dalias at aerifal.cx  Sun Nov  2 01:00:05 2003
From: dalias at aerifal.cx (D Richard Felker III)
Date: Sat, 1 Nov 2003 19:00:05 -0500
Subject: [MPlayer-G2-dev] MID proposal
In-Reply-To: <20031101163814.65ff6232.alex@fsn.hu>
References: <20031101163814.65ff6232.alex@fsn.hu>
Message-ID: <20031102000005.GW2856@brightrain.aerifal.cx>

On Sat, Nov 01, 2003 at 04:38:14PM +0100, Alex Beregszaszi wrote:
> Hi,
> 
> Here's my idea of a IMGFMT replacement: MID stands for MPlayer Image
> Description.

Here's what I have so far:

// MPlayer Imageformat Description

#define MID_TYPE_YUV        1
#define MID_TYPE_RGB        2
#define MID_TYPE_PALETTE    3
#define MID_TYPE_SPECIAL    4

#define MID_FLAG_SWAPPED      0x1

typedef struct mid_s {
        int type;
        int flags;
        int bpp;
	int depth;
        int csh, csv; // chroma shift, horiz. and vert.
        int num_planes; // 0=packed, 1=gray, 2=semiplanar(nv12), 3=full planar
        int chan_bits[4]; // sizes and offsets of channels (in bits). order is:
        int chan_offs[4]; // R,G,B[,A] for RGB; Y1,Y2,U,V for packed YUV
        unsigned int imgfmt; // legacy "fourcc"
        char *name, *longname;
} mid_t;

Some comments:

Q: Why separate type and flags again?
A: if (mid.type==MID_TYPE_YUV) is cleaner than
   if ((mid.flags & MID_FLAG_TYPEMASK) == MID_TYPE_YUV).

Q: Why have flags if there's only one flag?
A: Good question.

Q: Should num_planes really be the determining factor for what type of
   planar arrangement we have? Or should it just tell how many
   pointers in the planes[] array are valid, and should be use flags
   or different basic types for different planar arrangements?
A: I'm not sure. I would consider something like the following but I
   don't know if I like it:

#define MID_YUV_PLANAR     1
#define MID_YUV_PACKED     2
#define MID_YUV_SEMIPLANAR 3
#define MID_RGB            4
#define MID_PALETTE        5
#define MID_SPECIAL        6

Q: Why is palette a type rather than a flag?
A: Type tells the filter what type of data is stored in the buffers
   which planes[] points to. Indexed (palette) images do not have data
   that can be treated as meaningful colors (RGB or YUV) here, so IMO
   it's a different type in itself. Also it doesn't make sense to
   consider indexed images as YUV or RGB in nature, since the
   colorspace is a property of the palette, not the data itself (in
   fact you can convert the colorspace of the palette at essentially
   no cost!).

Q: What is the "special" type?
A: Anything other than a standard-format YUV, RGB, or indexed image.
   Some examples are compressed images (for hardware decoder boards)
   and dct/mv data for xvmc. Normally filters will reject special type
   images.

Q: Doesn't storing arbitrary channel layout (chan_bits/chan_offs)
   force filters to handle any strange format and thus be slow?
A: No. In fact only standard formats should be supported. This data is
   just here for filters which are written in C, and don't want to
   special case 555/565 or RGB/BGR, but instead use the same generic
   code for all cases. The same information in chan_bits/chan_offs is
   already available via depth, bpp, and the status of the swapped
   flag.

Rich


From michaelni at gmx.at  Sun Nov  2 02:13:45 2003
From: michaelni at gmx.at (Michael Niedermayer)
Date: Sun, 2 Nov 2003 02:13:45 +0100
Subject: [MPlayer-G2-dev] MID proposal
In-Reply-To: <20031102000005.GW2856@brightrain.aerifal.cx>
References: <20031101163814.65ff6232.alex@fsn.hu>
	<20031102000005.GW2856@brightrain.aerifal.cx>
Message-ID: <200311020213.45639.michaelni@gmx.at>

Hi

On Sunday 02 November 2003 01:00, D Richard Felker III wrote:
> On Sat, Nov 01, 2003 at 04:38:14PM +0100, Alex Beregszaszi wrote:
> > Hi,
> >
> > Here's my idea of a IMGFMT replacement: MID stands for MPlayer Image
> > Description.
>
> Here's what I have so far:
>
> // MPlayer Imageformat Description
>
> #define MID_TYPE_YUV        1
> #define MID_TYPE_RGB        2
> #define MID_TYPE_PALETTE    3
> #define MID_TYPE_SPECIAL    4
>
> #define MID_FLAG_SWAPPED      0x1
>
> typedef struct mid_s {
>         int type;
>         int flags;
>         int bpp;
> 	int depth;
>         int csh, csv; // chroma shift, horiz. and vert.
>         int num_planes; // 0=packed, 1=gray, 2=semiplanar(nv12), 3=full
> planar int chan_bits[4]; // sizes and offsets of channels (in bits). order
> is: int chan_offs[4]; // R,G,B[,A] for RGB; Y1,Y2,U,V for packed YUV
> unsigned int imgfmt; // legacy "fourcc"
>         char *name, *longname;
> } mid_t;
>
> Some comments:
>
> Q: Why separate type and flags again?
> A: if (mid.type==MID_TYPE_YUV) is cleaner than
>    if ((mid.flags & MID_FLAG_TYPEMASK) == MID_TYPE_YUV).
hmm if(mid.flags & MID_TYPE_YUV) would be an option too

[...]
-- 
Michael
level[i]= get_vlc(); i+=get_vlc();		(violates patent EP0266049)
median(mv[y-1][x], mv[y][x-1], mv[y+1][x+1]);	(violates patent #5,905,535)
buf[i]= qp - buf[i-1];				(violates patent #?)
for more examples, see http://mplayerhq.hu/~michael/patent.html
stop it, see http://petition.eurolinux.org & http://petition.ffii.org/eubsa/en


From dalias at aerifal.cx  Sun Nov  2 08:09:21 2003
From: dalias at aerifal.cx (D Richard Felker III)
Date: Sun, 2 Nov 2003 02:09:21 -0500
Subject: [MPlayer-G2-dev] basic g2 vp (video pipeline) code
Message-ID: <20031102070921.GY2856@brightrain.aerifal.cx>

Hey devs! It's far from finished, but here's a glimpse at the new
video pipeline layer for mplayer-g2. The main functions of interest
are vp_pull_image, vp_get_image, and vp_release_image. vp_config and
vp_insert_filter are somewhat relevant to the dreaded auto-insertion
hack too. :)

Please read and make comments. I have no idea if the code actually
compiles but it should be roughly correct. Some key points:

Image dimensions: You don't pick the image size when requesting an
image like you did in G1. Instead it's implicit from how config() was
called. Along with this change, there's a much clearer distinction
between actual picture dimensions and allocated (stride). Thus filters
will be sure to have the correct values for width/height so they don't
end up processing extra junk/blackspace outside the image (possibly
causing them to behave wrong!).

Aspect: Use sample aspect ratio rather than display aspect ratio! This
means we don't have to constantly recompute d_width/d_height (making
bad estimates and introducing error) every time we crop/expand/etc.
Most filters can just pass sample aspect through unchanged, or making
simple changes such as doubling the sample width or height (the only
notable exception is scale). Monitor aspect can get figured in at the
_end_ of the chain (where it belongs) instead of the beginning.

Stride restrictions: Greatly simplified, and the flag names are
actually consistent with what they mean now. From vp.h:
// Note: Stride restrictions are for compatibility with legacy codecs ONLY!
//       ALL filters and vo MUST accept any stride as input!
//       Aligned stride is always implicitly preferred when not forbidden!!

PTS: We now use a discrete time base fraction instead of floats.
Filters are free to modify the time base if they wish. This should
prevent error accumulation during lengthy encodes and help mencoder
automatically select a good time base for the output file. Decoders
should select the correct natural timebase (msec for asf, 60000/1001
for ntsc mpeg, 50/1 for pal mpeg, fps for fixed-fps formats, etc.).

DR and slices: "Slices" are replaced with an indirect buffer type,
indirect meaning that you can't write directly to it, only through
the draw_indirect() function. Both DR and indirect rendering (slices)
are accomplished by requesting special buffer types from vp_get_image.
Rather than falling back to automatic (allocated) buffer, vp_get_image
will _fail_ if DR/indirect buffer is not available! This is to give
the source filter/codec more flexibility in deciding which buffer type
to use. (For example, it may be better to fallback to export instead
of automatic buffer if DR fails.) Unlike in G1, indirect rendering
does not have to be done in "slices", but any region can be drawn via
draw_indirect().

The auto-insertion hack: ... :)) It should be pretty self-explanatory.
Basically, when a new filter is auto-inserted at vp_config() time, the
link between the two filters is split in two. The part connected to
the dest filter remains in place, and a new "sister" link is created
to remain with the source filter. Then scale (or whatever you like) is
inserted in between. When execution gets back to vp_pull_image, it
sees that the link it's processing has a sister, so it stuffs the
image back to the sister and clears the sister field, then calls
pull_image on the link again, this time getting an image from the
newly inserted conversion filter. It's hard to explain so RTFS!! :)

Comments are welcome, but please make an attempt to understand the
code before just flaming!

Rich

-------------- next part --------------



#include "vp.h"



vp_node_t *vp_open_filter(...)
{
	// FIXME: how do we load modules?
}

int vp_insert_filter(vp_link_t **link, vp_node_t *vf)
{
	vf->out = *link;
	vf->in = *link = calloc(1, sizeof(vp_link_t));
	vf->in->src = vf->out->src;
	vf->out->src = vf;
	vf->in->dest = vf;
}


int vp_config(vp_link_t **link, int w, int h, int samp_w, int samp_h, mid_t *mid)
{
	vp_node_t *dest = *link->dest;
	int flags = dest->query_format(dest, mid);
	if (!flags) {
		vp_link_t *orig = *link;
		vp_insert_filter(link, vf_open_filter("scale"));
		orig->sister = *link;
	}
	link->w = w;
	link->h = h;
	link->mid = mid;
	return *link->dest->config(*link, w, h, samp_w, samp_h, mid);
}



mp_image_t *vp_get_image(vp_link_t *link, int type, int flags)
{
	int i, j;
	mp_image_t *img = NULL;
	mp_image_t **pool = link->image_pool;
	int stride[MAX_PLANES];
	mid_t *mid = link->mid;
	int num_planes = mid->num_planes;
	int bpp = mid->bpp;
	int width = link->w;

	if (type == MPI_AUTO) {
		memset(stride, 0, sizeof stride);
		switch(mid->type) {
		case MID_TYPE_YUV:
			switch (num_planes) {
			case 0:
				stride[0] = width*bpp/8;
				break;
			case 2:
				stride[0] = width;
				stride[1] = (2*width)>>mid->csh;
				break;
			case 4:
				stride[3] = width>>mid->csh;
			case 3:
				stride[2] = width>>mid->csh;
				stride[1] = width>>mid->csh;
			case 1:
				stride[0] = width;
				break;
			default:
				return NULL;
			}
			break;
		case MID_TYPE_RGB:
		case MID_TYPE_PALETTE:
			stride[0] = width*bpp/8;
			break;
		default:
			if (type != MPI_EXPORT && type != MPI_DIRECT)
				return NULL;
		}

		// Always MB-align stride unless EXACT_WIDTH is demanded.
		if (!(flags & MPI_STRIDE_EXACT_WIDTH)) {
			if (mid->type == MID_TYPE_YUV) {
				int align = (8<<mid->csh)-1;
				switch (mid->num_planes) {
				case 0:
				case 1:
					stride[0] = (stride[0]+15)&~15;
					break;
				case 2:
					stride[0] = (stride[0]+align)&~align;
					stride[1] = (stride[1]+15)&~15;
					break;
				case 4:
					stride[3] = (stride[3]+7)&~7;
				case 3:
					stride[0] = (stride[0]+align)&~align;
					stride[1] = (stride[1]+7)&~7;
					stride[2] = (stride[2]+7)&~7;
					break;
				}
			} else {
				stride[0] = (stride[0]+15)&~15;
			}
		}

		// Search for already-allocated image
		for (i=0; i<link->image_pool_size; i++) {
			if (pool[i]->lock) continue;
			if (!pool[i]->planes[0]) continue;
			if (pool[i]->w != link->w) continue;
			if (pool[i]->h != link->h) continue;
			if (pool[i]->mid != link->mid) continue;
			for (j=0; j<MAX_PLANES; j++)
				if (stride[j] != pool[i]->stride[j]) break;
			if (j != MAX_PLANES) continue;
			img = pool[i];
			break;
		}
	}
	if (!img) {
		for (i=0; i<link->image_pool_size; i++) {
			if (!pool[i]->lock) {
				img = pool[i];
				if (img->type == MPI_AUTO) {
					free(img->planes[0]);
				}
				memset(img, 0, sizeof(mp_image_t));
				break;
			}
		}
	}
	if (!img) {
		i = ++link->image_pool_size;
		pool = link->image_pool = realloc(pool, i*sizeof(mp_image_t *));
		img = pool[--i] = calloc(1, sizeof(mp_image_t));
	}
	img->type = type;
	img->link = link;
	img->w = link->w;
	img->h = link->h;
	img->mid = link->mid;
	img->cw = (img->w + (1<<mid->csh)) >> mid->csh;
	img->ch = (img->h + (1<<mid->csv)) >> mid->csv;
	
	switch (type) {
	case MPI_AUTO:
		if (!img->planes[0]) {
			img->planes[0] = memalign(64, img->h*stride[0]
				+ img->ch*(stride[1]+stride[2]+stride[3]));
			if (num_planes > 1) img->planes[1] = img->planes[1] + stride[0]*img->h;
			if (num_planes > 2) img->planes[2] = img->planes[2] + stride[1]*img->ch;
			// IF09 crap...
			if (num_planes > 3) img->planes[3] = img->planes[3] + stride[2]*img->ch;
			for (i=0; i<MAX_PLANES; i++)
				img->stride[i] = stride[i];
			memset(planes[0], 0, stride[0]*link->h);
			if (planes[1]) memset(planes[1], 128, stride[1]*link->ch);
			if (planes[2]) memset(planes[2], 128, stride[2]*link->ch);
			// Swapped means YVU, unswapped is YUV...is this ok?
			if ((mid->flags & MID_FLAG_SWAPPED) && num_planes > 2) {
				unsigned char *temp = planes[1];
				planes[1] = planes[2];
				planes[2] = temp;
			}
		}
		break;
	case MPI_EXPORT:
		owner = link->src;
		break;
	case MPI_INDIRECT:
		flags = MPI_MEM_INDIRECT;
	case MPI_DIRECT:
		if (!link->dest->get_buffer) return NULL;
		if (!link->dest->get_buffer(link, flags, img))
			return NULL;
		img->owner = link->dest;
		break;
	}
	img->lock++;
	return img;
}

void vp_lock_image(mp_image_t *img)
{
	img->lock++;
}

void vp_release_image(mp_image_t *img)
{
	if (--img->lock) return;
	if (img->owner && img->owner->release_buffer)
		img->owner->release_buffer(img->owner, img);
}

mp_image_t *vp_pull_image(vp_link_t *link, int drop)
{
	mp_image_t *img = link->img_redir;
	if (img) {
		link->img_redir = NULL;
		return img;
	}
again:
	img = link->src->pull_image(link, drop);
	if (link->sister) {
		link->sister->img_redir = img;
		link->sister = NULL;
		goto again;
	}
	return img;
}

-------------- next part --------------



#include "cfg.h"
typedef module_info_t vf_info_t;
typedef struct vp_link_s vp_link_t;
typedef struct vp_node_s vp_node_t;
typedef struct vp_priv_s vp_priv_t;
typedef struct mp_image_s mp_image_t;




// MPlayer Imageformat Description

#define MID_TYPE_YUV        1
#define MID_TYPE_RGB        2
#define MID_TYPE_PALETTE    3
#define MID_TYPE_SPECIAL    4

#define MID_FLAG_SWAPPED      0x1

typedef struct mid_s {
	int type;
	int flags;
	int bpp;
	int depth;
	int csh, csv; // chroma shift, horiz. and vert.
	int num_planes; // 0=packed, 1=gray, 2=semiplanar(nv12), 3=full planar
	int chan_bits[4]; // sizes and offsets of channels (in bits). order is:
	int chan_offs[4]; // R,G,B[,A] for RGB; Y1,Y2,U,V for packed YUV
	unsigned int imgfmt; // legacy "fourcc"
	char *name, *longname;
} mid_t;



// Buffer management types
#define MPI_AUTO     1 // auto-managed by link layer
#define MPI_EXPORT   2 // exported from source to dest
#define MPI_DIRECT   3 // exported from dest to source (DR)
#define MPI_INDIRECT 4 // indirect rendering via slices, no pointers

// Buffer memory restrictions
#define MPI_MEM_READABLE 0x1 // buffer is in readable memory
#define MPI_MEM_PRESERVE 0x2 // dest will not clobber buffer contents
#define MPI_MEM_REUSABLE 0x4 // source can reuse & re-return buffer >1 times
#define MPI_MEM_INDIRECT 0x8 // indirect buffer

// Stride restrictions (define B==bpp/8 for packed, B==1 for planar)
// Note: Stride restrictions are for compatibility with legacy codecs ONLY!
//       ALL filters and vo MUST accept any stride as input!
//       Aligned stride is always implicitly preferred when not forbidden!!
#define MPI_STRIDE_MB_ALIGNED    0x0100 // stride%(16*B) == 0
#define MPI_STRIDE_PIXEL_ALIGNED 0x0200 // stride%B == 0
#define MPI_STRIDE_EXACT_WIDTH   0x0400 // stride == w*B
#define MPI_STRIDE_POSITIVE      0x0800 // stride > 0
#define MPI_STRIDE_COMMON_STRIDE 0x1000 // stride[1,2] == stride[0]>>csh
#define MPI_STRIDE_COMMON_PLANE  0x2000 // planes[1] == planes[0]+stride[0]*h..
#define MPI_STRIDE_MASK          0x3F00

// PTS flags
#define MPI_PTS_PTS_VALID      0x01
#define MPI_PTS_DURATION_VALID 0x02
#define MPI_PTS_REL_PTS_VALID  0x04
#define MPI_PTS_PTS_ACCURATE   0x08
#define MPI_PTS_DISCONTINUITY  0x10

struct mp_image_s {
	// Buffer management
	int buftype;
	int flags;
	vp_node_t *owner;
	vp_link_t *link;
	int lock;
	// Format & dimensions
	mid_t *mid;
	int w, h;
	int cw, ch;
	// Pointers & strides
	unsigned char *planes[MAX_PLANES];
	int stride[MAX_PLANES];
	unsigned char *qscale;
	int qstride;
	int qscale_type;
	unsigned char *palette;
	mid_t *palette_mid;
	// Frame information
	int pict_type;
	int color_type;
	int field_flags;
	// Timing
	int time_base_num;
	int time_base_denom;
	long long pts;
	int duration;
	int rel_pts;
	unsigned int pts_flags;
	// Private data for use by owner
	void *priv;
};

struct vp_link_s {
	vp_node_t *src, *dest;
	vp_link_t *sister;
	mp_image_t *img_redir;
	int configured;
	int w, h;
	mid_t *mid;
	mp_image_t **image_pool;
	int image_pool_size;
};

struct vp_node_s {
	vp_info_t *info;
	config_data_t *cfg;
	// Links
	vp_link_t *in, *out;
	// Basic VP API
	mp_image_t *(*pull_image)(vp_link_t *link);
	int (*query_format)(vp_link_t *link, mid_t *mid);
	int (*config)(vp_link_t *link, int w, int h, int samp_w, int samp_h, mid_t *mid);
	int (*control)(vp_link_t *link, int request, void *data);
	int (*close)(vp_node_t *self);
	// Direct rendering
	int (*get_buffer)(vp_link_t *link, int flags, mp_image_t *img);
	void (*release_buffer)(vp_node_t *self, mp_image_t *img);
	// Slices
	void (*draw_indirect)(vp_link_t *link, mp_image_t *img,
		unsigned char **src, int *stride, int x, int y, int w, int h);
	// Private data for this node of the pipeline
	vp_priv_t *priv;
};





From kinali at gmx.net  Sun Nov  2 11:29:51 2003
From: kinali at gmx.net (Attila Kinali)
Date: Sun, 2 Nov 2003 11:29:51 +0100
Subject: [MPlayer-G2-dev] MID proposal
In-Reply-To: <20031101195259.GT2856@brightrain.aerifal.cx>
References: <20031101163814.65ff6232.alex@fsn.hu>
	<20031101203343.50055e69.alex@fsn.hu>
	<20031101195259.GT2856@brightrain.aerifal.cx>
Message-ID: <20031102112951.16e50cc9.kinali@gmx.net>

On Sat, 1 Nov 2003 14:52:59 -0500
D Richard Felker III <dalias at aerifal.cx> wrote:

> > Hm, one new issue come to my mind: ARGB vs RGBA, aka the place of alpha
> > byte. I hate these endianess issues :)
> 
> A is idiotic, there is no alpha. Just say RGB or BGR with no A.

Not really, think about subs or osd. It'll be imho nice
if we just could handle them as a stream like normal
video too.
But on the other hand it may complicate things for
the normal filters which dont need it.

		Attila Kinali

-- 
egp ist vergleichbar mit einem ikea bausatz fuer flugzeugtraeger
			-- reeler in +kaosu


From arpi at thot.banki.hu  Sun Nov  2 13:17:10 2003
From: arpi at thot.banki.hu (Arpi)
Date: Sun, 2 Nov 2003 13:17:10 +0100
Subject: [MPlayer-G2-dev] basic g2 vp (video pipeline) code
In-Reply-To: <20031102070921.GY2856@brightrain.aerifal.cx>
Message-ID: <200311021217.hA2CHAUZ002421@mail.mplayerhq.hu>

Hi,

> Hey devs! It's far from finished, but here's a glimpse at the new
> video pipeline layer for mplayer-g2. The main functions of interest
> are vp_pull_image, vp_get_image, and vp_release_image. vp_config and
> vp_insert_filter are somewhat relevant to the dreaded auto-insertion
> hack too. :)

ok, i'm answering yoru question on irc:

<dalias> in g2, any object to requiring that all filters/vo be able to
    accept images with any stride?

YES.
it limits g2 vf layer to use fully controlled filter code only.
ie you cannot call 3rd party code which doesnt accept any kind of stride,
be it to-mpeg transcoder, codecs (remember, video encoders are connected
to vf layer anyway, but it can be workarounded by ugly hacks), external
filters and so on.

it should be handled, at least by auto-inserting expand filter.


A'rpi / Astral & ESP-team

--
Developer of MPlayer G2, the Movie Framework for all - http://www.MPlayerHQ.hu


From dalias at aerifal.cx  Sun Nov  2 19:11:34 2003
From: dalias at aerifal.cx (D Richard Felker III)
Date: Sun, 2 Nov 2003 13:11:34 -0500
Subject: [MPlayer-G2-dev] basic g2 vp (video pipeline) code
In-Reply-To: <200311021217.hA2CHAUZ002421@mail.mplayerhq.hu>
References: <20031102070921.GY2856@brightrain.aerifal.cx>
	<200311021217.hA2CHAUZ002421@mail.mplayerhq.hu>
Message-ID: <20031102181134.GZ2856@brightrain.aerifal.cx>

On Sun, Nov 02, 2003 at 01:17:10PM +0100, Arpi wrote:
> Hi,
> 
> > Hey devs! It's far from finished, but here's a glimpse at the new
> > video pipeline layer for mplayer-g2. The main functions of interest
> > are vp_pull_image, vp_get_image, and vp_release_image. vp_config and
> > vp_insert_filter are somewhat relevant to the dreaded auto-insertion
> > hack too. :)
> 
> ok, i'm answering yoru question on irc:
> 
> <dalias> in g2, any object to requiring that all filters/vo be able to
>     accept images with any stride?
> 
> YES.
> it limits g2 vf layer to use fully controlled filter code only.
> ie you cannot call 3rd party code which doesnt accept any kind of stride,
> be it to-mpeg transcoder, codecs (remember, video encoders are connected
> to vf layer anyway, but it can be workarounded by ugly hacks), external
> filters and so on.
> 
> it should be handled, at least by auto-inserting expand filter.

OK. IMO there are a couple solutions. One, like you said, is loading
expand. I'm not sure exactly how this works. Does it expand the image
up to stride by adding a small border, or copy the image to a new
buffer with stride = width*bpp? I assume the latter since the former
isn't always possible (if stride[0] != stride[1]<<csh, etc.).

Part of the problem with auto-inserting expand is that it has to be
done at config time, when you don't yet know what stride will be.
Maybe stride should be negotiated at config-time? That would help
codecs/filters that don't like stride changing from one frame to
another, but it also might interfere with DR1 of B frames for IPB
codecs. It would also add a lot of complexity to the config process.

So should we do like G1 and auto-load expand if the destination vf/ve
has restrictions on which strides it will accept? Will expand behave
well and just do nothing in the case where the strides already meet
the dest's requirements?

Rich



From dalias at aerifal.cx  Sun Nov  2 19:13:37 2003
From: dalias at aerifal.cx (D Richard Felker III)
Date: Sun, 2 Nov 2003 13:13:37 -0500
Subject: [MPlayer-G2-dev] MID proposal
In-Reply-To: <20031102112951.16e50cc9.kinali@gmx.net>
References: <20031101163814.65ff6232.alex@fsn.hu>
	<20031101203343.50055e69.alex@fsn.hu>
	<20031101195259.GT2856@brightrain.aerifal.cx>
	<20031102112951.16e50cc9.kinali@gmx.net>
Message-ID: <20031102181337.GA2856@brightrain.aerifal.cx>

On Sun, Nov 02, 2003 at 11:29:51AM +0100, Attila Kinali wrote:
> On Sat, 1 Nov 2003 14:52:59 -0500
> D Richard Felker III <dalias at aerifal.cx> wrote:
> 
> > > Hm, one new issue come to my mind: ARGB vs RGBA, aka the place of alpha
> > > byte. I hate these endianess issues :)
> > 
> > A is idiotic, there is no alpha. Just say RGB or BGR with no A.
> 
> Not really, think about subs or osd. It'll be imho nice
> if we just could handle them as a stream like normal
> video too.

Handling them this way is slow. That's why Arpi designed a special
format for OSD data that facilitates optimized alphablending. Even if
you ever did want to treat them as images, RGBA would be silly...YUVA
maybe, but...?

> But on the other hand it may complicate things for
> the normal filters which dont need it.

Yes...

Rich


From arpi at thot.banki.hu  Sun Nov  2 19:58:29 2003
From: arpi at thot.banki.hu (Arpi)
Date: Sun, 2 Nov 2003 19:58:29 +0100
Subject: [MPlayer-G2-dev] basic g2 vp (video pipeline) code
In-Reply-To: <20031102181134.GZ2856@brightrain.aerifal.cx>
Message-ID: <200311021858.hA2IwT9B025362@mail.mplayerhq.hu>

Hi,

> > <dalias> in g2, any object to requiring that all filters/vo be able to
> >     accept images with any stride?
> > 
> > YES.
> > it limits g2 vf layer to use fully controlled filter code only.
> > ie you cannot call 3rd party code which doesnt accept any kind of stride,
> > be it to-mpeg transcoder, codecs (remember, video encoders are connected
> > to vf layer anyway, but it can be workarounded by ugly hacks), external
> > filters and so on.
> > 
> > it should be handled, at least by auto-inserting expand filter.
> 
> OK. IMO there are a couple solutions. One, like you said, is loading
> expand. I'm not sure exactly how this works. Does it expand the image
> up to stride by adding a small border, or copy the image to a new
> buffer with stride = width*bpp? I assume the latter since the former

the later, of course

> isn't always possible (if stride[0] != stride[1]<<csh, etc.).
> 
> Part of the problem with auto-inserting expand is that it has to be
> done at config time, when you don't yet know what stride will be.

yes

> Maybe stride should be negotiated at config-time? That would help

nope, its impossiblee imho in many cases, and adds yet another
limitation...

dont forget that config() happens befor ethe first frame is passed,
so in case of 3rd party codecs, or anything special, you wont know
the wanted stride before processing first frame

> So should we do like G1 and auto-load expand if the destination vf/ve
> has restrictions on which strides it will accept? Will expand behave
> well and just do nothing in the case where the strides already meet
> the dest's requirements?

it depends on implementation of new vf_expand...


A'rpi / Astral & ESP-team

--
Developer of MPlayer G2, the Movie Framework for all - http://www.MPlayerHQ.hu


From dalias at aerifal.cx  Sun Nov  2 20:34:06 2003
From: dalias at aerifal.cx (D Richard Felker III)
Date: Sun, 2 Nov 2003 14:34:06 -0500
Subject: [MPlayer-G2-dev] basic g2 vp (video pipeline) code
In-Reply-To: <20031102070921.GY2856@brightrain.aerifal.cx>
References: <20031102070921.GY2856@brightrain.aerifal.cx>
Message-ID: <20031102193406.GF2856@brightrain.aerifal.cx>

On Sun, Nov 02, 2003 at 02:09:21AM -0500, D Richard Felker III wrote:
> mp_image_t *vp_pull_image(vp_link_t *link, int drop)
> {
> 	mp_image_t *img = link->img_redir;
> 	if (img) {
> 		link->img_redir = NULL;
> 		return img;
> 	}
> again:
> 	img = link->src->pull_image(link, drop);
> 	if (link->sister) {
> 		link->sister->img_redir = img;
> 		link->sister = NULL;
> 		goto again;
> 	}
> 	return img;
> }

One other feature worth mentioning, the drop flag to vp_pull_image: If
you intend to ignore the contents of a frame (for -framedrop, a/v sync
in fixed-fps output, or whatever reason), pass the drop flag when
calling vp_pull_image. The filter is _still_ required to return a
valid mpi for timing purposes, but the actual contents of the image
can be meaningless. Filters are recommended but not required to
propagate the drop flag back through the chain...and of course if it
gets all the way back to the codec and the next frame is a B frame,
the codec can skip decoding.

Optional ideas: require filters to propagate drop flag if its value is
greater than 1? Include a way to signal hard framedrop?

Rich