On Wed, 30 Jan 2008 17:12:24 +0300, Pavel Modilaynen wrote:

I think there is another and more powerfull approach could be used to add this feature to mplayer without xmms and libvisual. It could be implemented as additional feature which is enabled in command line (maybe with -viz [oscill | spectrum | ... ]) only if only one stream is present - audio.

This feature could be useful for various diagnostic / test / demonstration purposes.

Any advices / objections?

Thank you for your consideration.

i think it could be implemented as a video filter for example, look at these filters: mplayer -vf test file or mplayer -vf rgbtest file also ffplay has a simple audio visualizer, maybe it can be ported? -compn

...

Check out libvisual, and search the MPlayer m/l archives. There is already plugin support for MPlayer in libvisual, apparently.

As I understood, there is xmms is needed for using mplayer with libvisual and its xmms-plugins. If it is true, IMHO it is too inconvenient for lots of application and bulky.

I think there is another and more powerfull approach could be used to add this feature to mplayer without xmms and libvisual. It could be implemented as additional feature which is enabled in command line (maybe with -viz [oscill | spectrum | ... ]) only if only one stream is present - audio.

This feature could be useful for various diagnostic / test / demonstration purposes.

Any advices / objections?

Thank you for your consideration.

IIRC, the issue was the messing with video and audio streams. The way it was implemented before was that the audio was output via an audio fifo to libvisual as well as the regular output. Then libvisual generated the required images using optimized routines and exported those images to a fifo. The image fifo was read by -vf bmovl and used to display the image on MPlayer vo. IMHO I really think that reinventing the wheel in terms of re-writing libvisual image creation algs is not such a good idea, and the internal messing with audio/video streams would not be accepted into MPlayer (at least that was the stance of developers some years ago, according to m/l archives I looked at, IIRC). The export of audio and import of the video overlay images was at least tolerated. I am not sure how the audio export/bypass worked exactly, but surely you can google the m/l thread from years back, as I did. See the thread from around 2003-11-14 by Gustavo Sverzut Barbieri. I don't think the export audio stuff was limited to xmms, a libvisual plugin to interface exists for libvisual, which i _think_ can interface with the exported audio fifo directly. HTH //Andrew

Wednesday, January 30, 2008, 8:05:35 PM, Andrew Calkin wrote:

...

I think there is another and more powerfull approach could be used to add this feature to mplayer without xmms and libvisual. It could be implemented as additional feature which is enabled in command line (maybe with -viz [oscill | spectrum | ... ]) only if only one stream is present - audio.

This feature could be useful for various diagnostic / test / demonstration purposes.

IIRC, the issue was the messing with video and audio streams. The way it was implemented before was that the audio was output via an audio fifo to libvisual as well as the regular output. Then libvisual generated the required images using optimized routines and exported those images to a fifo. The image fifo was read by -vf bmovl and used to display the image on MPlayer vo.

I see ... It is really messy and inconvenient. There is a lot of obvious drawbacks in this approach (I suspect everebody sees that).

IMHO I really think that reinventing the wheel in terms of re-writing libvisual image creation algs is not such a good idea, and the internal messing with audio/video streams would not be accepted into MPlayer (at least that was the stance of developers some years ago, according to m/l archives I looked at, IIRC).

Of course, I don't want to reinvent the wheel or mess existing code of mplayer. God forbid :) I only propose a new approach to this issue. This approach is very local and with little invasion to mplayer code (only to demux_audio.c at least). Thank you for your consideration. -- Pavel Modilaynen

Friday, February 1, 2008, 7:56:51 PM, Alban Bedel wrote:

On Fri, 1 Feb 2008 18:23:45 +0300 Pavel Modilaynen <kopli@mail.ru> wrote:

...

I only propose a new approach to this issue. This approach is very local and with little invasion to mplayer code (only to demux_audio.c at least).

I once did an implementation, which I still have laying around. Bascialy it use a special demuxer that take another demuxer as input. The demuxer decode the audio and put it as video and audio stream. The visual code is then implemented like a normal video codec.

But why can't we do much easier adding to demux_audio.c initialization of video stream. It's just a few strings. And then we generate the video packets like demux_gif.c do. This is the simpliest method as you can see. To make this audio dependent we just need to read audio output buffer, transform correspondingly and fill video buffer in this demuxer. Of course there is no need to reinvent the wheel. Transformation might be done by freq for example. So, we have only one demuxer, and we force all other mplayer stuff do its work like it does in case of video+audio. What do you thing about it ? Thank you for your consideration. -- Pavel Modilaynen

Monday, February 4, 2008, 7:04:21 PM, Alban Bedel wrote:

On Mon, 4 Feb 2008 15:05:35 +0300 Pavel Modilaynen <kopli@mail.ru> wrote:

...

This is the simpliest method as you can see. To make this audio dependent we just need to read audio output buffer, transform correspondingly and fill video buffer in this demuxer.

I think you are missing the point that the data comming from the demuxer is still encoded.

It seems to be you didn't catch my idea. I meant that in demux_audio.c: demux_audio_fill_buffer () call for video stream (assuming we've created a faked video stream) it is possible to read audio output buffer (just want to emphasize this - audio output buffer with data after audio decoder). I.e. we have raw audio samples and we can do with them whatever we want. We just have to deal with endiannes, sample size, sign which is not painfull and this variety can be easily converted to one common format (for example signed 16 bit). Then it is trivial to draw an oscillogram of audio signal which I've already done and it works fine. Nevertheless I consider my current modifications a little bit hacky and messy and I can propose this feature for mplayer in the following preliminary form: New files: - vz.c/vz.h - main audio visualization functionality (like ad.c, vf.c,.. ), table of visualizers, entries; - vz_xxx.c - visualizer's implementation itself / access to external libs (sticking to the conception of libmpcodecs) And functionality might be the following: - in demux_audio.c: demux_audio_open() - check if visualization needed, and call vz_init () which calls appropriate vz_xxx.c: init () - in demux_audio.c: demux_audio_fill_buffer () - check if visualization was enabled - pass control to vz_process() which calls vz_xxx.c: process () That is intuitively understandable imho. I do not want to mess the MPlayer code, and my proposals I think is a minimal harm for this great benefit that this feature gives. But what do fathers of MPlayer developers think about it? Thank you for your consideration. -- Pavel Modilaynen

Tuesday, February 5, 2008, 7:18:12 PM, Reimar Doffinger wrote:

On Tue, Feb 05, 2008 at 06:47:33PM +0300, Pavel Modilaynen wrote:

...

New files: - vz.c/vz.h - main audio visualization functionality (like ad.c, vf.c,.. ), table of visualizers, entries; - vz_xxx.c - visualizer's implementation itself / access to external libs (sticking to the conception of libmpcodecs)

I don't see why you can't just use a normal video filter. Just add a new IMGFMT_AUDIO or so.

And how in this case will I work with audio path inside video filter? It doesn't know and doesn't have to know about any audio. This would be real mess if not. And IMGFMT_AUDIO sounds crazy IMHO :) In my proposed approach video filter support, of course, remains. And what is really important - it's exactly the same. As I read in MPlayer documentation - the purpose of vf is resize/reformat/recolor and not "generation of video or images". So, I think, the best place for entry - is the audio demuxer (or mplayer.c itself ??). There is nothing awful in this - we just ask an audio demuxer - would you please convert the audio going through to the video. And it answers - yes, of course, which conversion do you want? dimensions?fps?spectrum? And you still have possibility to use any vf like in case of true video.

...

And functionality might be the following: - in demux_audio.c: demux_audio_open() - check if visualization needed, and call vz_init () which calls appropriate vz_xxx.c: init () - in demux_audio.c: demux_audio_fill_buffer () - check if visualization was enabled - pass control to vz_process() which calls vz_xxx.c: process () That is intuitively understandable imho.

Well, I can overcome my personal opinions if necessary, but my honest and personal opinion is that it is a (if done right only small) mess for a useless feature ;-) Keep in mind that it is "MPlayer - The Movie Player" :-)

The future will judge the feature. :) Why somebody is obliged to find some other libs to do this simple task, wasting time and etc. Yes it's certainly the Movie Player, but why it can't be a little bit more powerfull :) BTW, can you advice any fft lib that converts data to image that can be used to help MPlayer to do the audio visualization. Thank you for your consideration. -- Pavel Modilaynen

On Tue, 5 Feb 2008 17:18:12 +0100 Reimar Döffinger <Reimar.Doeffinger@stud.uni-karlsruhe.de> wrote:

Well, I can overcome my personal opinions if necessary, but my honest and personal opinion is that it is a (if done right only small) mess for a useless feature ;-) Keep in mind that it is "MPlayer - The Movie Player" :-)

Still it can also play "audio only" files, so here is my take on it :) As said earlier it is some old code, I just updated it to work with the changes to the decode_audio() interface. There is no doc nor configure support, but that's not the point atm as it is unfinished. To test it you need goom (goom.sf.net) and to add -lgoom2 your LDFLAGS. Comments and suggestions are welcome, the original idea was to then replace vd_goom with a vd_visuals wich would take care of converting the raw pcm data to what the selected visual need as input. Albeu

On Sun, 17 Feb 2008 18:51:34 +0100 Benjamin Zores <ben@geexbox.org> wrote:

Alban Bedel a écrit :

...

On Tue, 5 Feb 2008 17:18:12 +0100 Reimar Döffinger <Reimar.Doeffinger@stud.uni-karlsruhe.de> wrote:

...

Well, I can overcome my personal opinions if necessary, but my honest and personal opinion is that it is a (if done right only small) mess for a useless feature ;-) Keep in mind that it is "MPlayer - The Movie Player" :-)

Still it can also play "audio only" files, so here is my take on it :) As said earlier it is some old code, I just updated it to work with the changes to the decode_audio() interface. There is no doc nor configure support, but that's not the point atm as it is unfinished. To test it you need goom (goom.sf.net) and to add -lgoom2 your LDFLAGS.

Comments and suggestions are welcome, the original idea was to then replace vd_goom with a vd_visuals wich would take care of converting the raw pcm data to what the selected visual need as input.

What's the status of this finally ? Is it planned to get in someday ?

Good question. I think the solution I implemented could be accepted as it's not very obstrusive. But as no one commented it is hard to tell. I can bring the patch to a commitable version. But as it is a somewhat controversial feature, I would prefer having confirmation that it might be accepted before investing more time in it.

I've tested it successfully and it's for sure quite an interesting future. However, it might be a good thing to import libgoom to MPlayer then, especially there's not much releases of it and a few distros have it (besides a bug exist in upstream release, latest one being from 2005 ..., that tries to do 3DNow code unconditionnally).

We could do that, but honestly I haven't put much thought into such things yet. First, let's see if the code needing goom is accepted. Albeu

On Sun, 17 Feb 2008 20:36:42 +0100 Reimar Döffinger <Reimar.Doeffinger@stud.uni-karlsruhe.de> wrote:

Hello, On Sun, Feb 17, 2008 at 08:30:45PM +0100, Alban Bedel wrote:

...

On Sun, 17 Feb 2008 18:51:34 +0100 Benjamin Zores <ben@geexbox.org> wrote:

...

What's the status of this finally ? Is it planned to get in someday ?

Good question. I think the solution I implemented could be accepted as it's not very obstrusive. But as no one commented it is hard to tell.

I assumed that my opinion is clear: A demuxer calling a decoder is a horrible breaking of layering

I see it more as a demuxer filter ;) I don't find it pretty either, but it is currently the simplest way imho.

and implementing it as vd makes it really hard to pass any options at all without making everything global options.

Well, that's a lack from the vd api. It could be fixed.

Now a different question is what a nice way to do this would be.

If you drop the "demuxer filter" mplayer.c will need quiet some modifications to still setup a video chain when sh_video == NULL. Then the audio decoding would need to be modified to also feed the video chain. A/V sync will probably need some change too (I want visuals with perfect A/V sync). All in all, I'm not yet convinced that such approch would be worse it. It sound like quiet a burden for a minor feature.

For that part of the question would also be what kind of information the visualization needs and in which cases it is most important to work. Since the IMGFMT_AUDIO idea was not liked, a different possibility would be IMGFMT_GRAY16, where the audio could be placed each channel in one line.

I think this is a moot point. Even if the visuals are vf, they have to be at the start of the chain. So you can use any API you like to ask them to generate a frame, and such "image format" are never needed. But if we do end up with mp_image containing audio data at some point, I would obviously vote for IMGFMT_AUDIO over a perversion of an existing format.

Supporting anything besides s16le and some fixed samplerate would not (easily, non-hackishly) be possible like this though.

Independently of how we interface it with mplayer, we will need an API similar to the vd for the visuals. And I think it would be a good idea that it include a format converter/resampler for commons PCM formats. We can't expect visuals to support many formats and forcing the soundcard to use what the visual need is obviously not an option.

This would need some more changes to mplayer.c though.

Way too much imho, but I may still investigate such a solution.

On the plus side, it would make things like -vf rgbtest less of a pain to use at the same time though.

I doubt it, but we'll see. Albeu