[FFmpeg-devel] MOD support for FFmpeg (My GSoC 2010 task starts tomorrow)

[Peter Ross]

On Thu, May 27, 2010 at 05:43:26PM +0200, Vitor Sessak wrote:
> On 05/27/2010 03:35 PM, Ronald S. Bultje wrote:
> >Hi,
> >
> >On Thu, May 27, 2010 at 7:56 AM, Peter Ross<pross at xvid.org>  wrote:
> >>On Thu, May 27, 2010 at 04:31:59AM +0200, Michael Niedermayer wrote:
> >>>On Sun, May 23, 2010 at 11:27:37PM +0200, Sebastian Vater wrote:
> >>>I dont think the 1. and. 2. are exclusive, it surely should be possible
> >>>to have a seperate lib and at the same time provide "mod" decoding 
> >>>support
> >>>through the existing APIs.
> >>>And we need to support it through existing apis because anything else 
> >>>would
> >>>be quite inconvenient for applications.
> >>>I am a bit undecided though about seperate lib or no seperate lib
> >>
> >>I toyed with ProTracker playback in FFmpeg last year, but misplaced the 
> >>code
> >>and gave up. My approach was a hybrid between the ones Sebastian listed, 
> >>namely:
> >>
> >>* treat a module as an AVMEDIA_TYPE_AUDIO stream containing 
> >>CODEC_TYPE_PATTERN.
> >>
> >>* add data structure(s) to AVCodecContext that describe the contents of a
> >>sound pattern/module file. The structure must support concepts likes 
> >>Patterns,
> >>Notes, and Special Commands, etc. (TuComposer will have this already.)
> >>
> >>* write demuxers for various module formats. The demuxer would read the 
> >>file
> >>and converts it into the AVCodecContext pattern data structure.
> >>
> >>* write a libavcodec decoder called 'Pattern decoder', that takes 
> >>CODEC_TYPE_PATTERN
> >>as input. The decoder will process the AVCodecContext structure and 
> >>outputs audio
> >>samples (SAMPLE_FMT_xxx). Esentially this codec would invoke the 
> >>TuComposer
> >>functionality.
> >>
> >>* write muxers. thus allowing conversion between alike-module formats.
> >
> >I had something similar in mind, we (Vitor, me and Sebastian)
> >discussed this on IRC earlier. However, we'd like to more-or-less keep
> >the mod format even in "decoded" samples.
> >
> >- write demuxers that can "parse" mod files. These would be small,
> >like raw demuxers. The output is AVMEDIA_TYPE_AUDIO with
> >CODEC_TYPE_S3M/XYZ etc.
> >- write decoders that can "decode" mod file b{it,yte}streams into
> >collections of notes, etc. The output would be SAMPLE_FMT_MOD or
> >SAMPLE_FMT_COMPOSER. Mans wants it to be called SAMPLE_FMT_KITTEN.
> 
> I'm still undecided of what is best, AVMEDIA_TYPE_AUDIO + 
> SAMPLE_FMT_COMPOSER or AVMEDIA_TYPE_COMPOSER. But the argument for 
> AVMEDIA_TYPE_AUDIO of having audioconvert.c do the PCM conversion in a 
> way that is transparent for caller apps is appealing.
> 
> >- the "tracker" would then be a converter in the style of
> >audioconvert.c, which converts from SAMPLE_FMT_KITTY to
> >SAMPLE_FMT_S16/U8/FLOAT
> 
> >- a FIXME here's is how a user would choose the samplerate, maybe a
> >AVCodecContext.request_sample_rate in the style of channel_mask?
> 
> Maybe "-sampler_flags" in the same way we have "-sws_flags"?
> 
> >- Vitor had another bunch of comments here but I forgot what it was
> >exactly. Vitor?
> 
> Yes, I'd say that a very important piece of code would be the data 
> structures holding the decoded collection of notes, instruments, etc. 
> This will be what the decoders will output (and what we mean by 
> SAMPLE_FMT_COMPOSER). Since this should be able to describe all 
> supported MOD formats, it should be pretty flexible. I think it should 
> be part of libavcodec. Also, the caller app might want to known which 
> instruments are being played and when and how, so this structs should be 
> part of the public API (which implies tough review). IMHO, this should 
> be the first thing to be reviewed and committed, since it blocks all the 
> rest.

Agree 100%. The structures need to also need to be flexible enough to
support VST instruments that are used in modern tracker file formats
(such as Cubase, Orion and Renoise).

In an ideal world, FFmpeg would provide portable open-source implementation
of popular VST instruments.

-- Peter
(A907 E02F A6E5 0CD2 34CD 20D2 6760 79C5 AC40 DD6B)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20100528/55c6745f/attachment.pgp>