[FFmpeg-devel] MOD support for FFmpeg (My GSoC 2010 task starts tomorrow)

[Ronald S. Bultje]

Hi,

On Thu, May 27, 2010 at 7:56 AM, Peter Ross <pross at xvid.org> wrote:
> On Thu, May 27, 2010 at 04:31:59AM +0200, Michael Niedermayer wrote:
>> On Sun, May 23, 2010 at 11:27:37PM +0200, Sebastian Vater wrote:
>> I dont think the 1. and. 2. are exclusive, it surely should be possible
>> to have a seperate lib and at the same time provide "mod" decoding support
>> through the existing APIs.
>> And we need to support it through existing apis because anything else would
>> be quite inconvenient for applications.
>> I am a bit undecided though about seperate lib or no seperate lib
>
> I toyed with ProTracker playback in FFmpeg last year, but misplaced the code
> and gave up. My approach was a hybrid between the ones Sebastian listed, namely:
>
> * treat a module as an AVMEDIA_TYPE_AUDIO stream containing CODEC_TYPE_PATTERN.
>
> * add data structure(s) to AVCodecContext that describe the contents of a
> sound pattern/module file. The structure must support concepts likes Patterns,
> Notes, and Special Commands, etc. (TuComposer will have this already.)
>
> * write demuxers for various module formats. The demuxer would read the file
> and converts it into the AVCodecContext pattern data structure.
>
> * write a libavcodec decoder called 'Pattern decoder', that takes CODEC_TYPE_PATTERN
> as input. The decoder will process the AVCodecContext structure and outputs audio
> samples (SAMPLE_FMT_xxx). Esentially this codec would invoke the TuComposer
> functionality.
>
> * write muxers. thus allowing conversion between alike-module formats.

I had something similar in mind, we (Vitor, me and Sebastian)
discussed this on IRC earlier. However, we'd like to more-or-less keep
the mod format even in "decoded" samples.

- write demuxers that can "parse" mod files. These would be small,
like raw demuxers. The output is AVMEDIA_TYPE_AUDIO with
CODEC_TYPE_S3M/XYZ etc.
- write decoders that can "decode" mod file b{it,yte}streams into
collections of notes, etc. The output would be SAMPLE_FMT_MOD or
SAMPLE_FMT_COMPOSER. Mans wants it to be called SAMPLE_FMT_KITTEN.
- the "tracker" would then be a converter in the style of
audioconvert.c, which converts from SAMPLE_FMT_KITTY to
SAMPLE_FMT_S16/U8/FLOAT
- a FIXME here's is how a user would choose the samplerate, maybe a
AVCodecContext.request_sample_rate in the style of channel_mask?
- Vitor had another bunch of comments here but I forgot what it was
exactly. Vitor?

The advantage of this approach is that we can render between mod
formats without complete re-encoding, we only have to reformat the
b{it,yte}stream from the note collections.

We've suggested that Sebastian starts with #1, then moves to #2. For
#3 he could write a tucomposer wrapper to start with, but eventually
we'd want something in FFmpeg. He can start from tucomp code
where-ever he wants, but the code will eventually live in FFmpeg SVN
and thus go through full review here. He'll probably need a ff-soc
repo account to put his work-in-progress because keeping track of 100
patches will make us dizzy.

What do others think of this approach?

Ronald