[FFmpeg-devel] [PATCH] Whisper audio filter

[Vittorio Palmisano]

Thanks Michael, I will try to answer your comments.

> > +ffmpeg -i input.mp4 -vn -af "aformat=sample_rates=16000:channel_layouts=mono,whisper=
>
> Is there a reason why we convert to 16khz mono here ?

It is the only format supported by the whisper.cpp library.

> > +model=../whisper.cpp/models/ggml-base.en.bin\
>
> It would be nice if the models would be in a standard location, so teh user
> just has to specify the model name and not the path

I think that this functionality should be implemented inside the
whisper.cpp library, so they can manage the exact model location and
the download process. I will propose a change.

> I tried this:
>
> ./ffmpeg -i matrixbench_mpeg2.mpg -vn -af "aformat=sample_rates=16000:channel_layouts=mono,whisper=model=/home/michael/whisper.cpp/models/ggml-base.en.bin:language=en:queue=3000:destination=output.srt:format=srt" -f null -
>
> but the output.srt is empty (0 bytes)

Can you enable verbose logging?

> libavfilter/af_whisper.c:75:49: error: parameter name omitted
>    75 | static void cb_log_disable(enum ggml_log_level, const char *, void *) {}
>       |                                                 ^~~~~~~~~~~~
> libavfilter/af_whisper.c:75:63:

I don't see this error using the gcc13 compiler. Do you use a
different compiler or some other flags?

> > +    wctx->audio_buffer_fill_size = 0;
> > +
> > +    wctx->next_pts = AV_NOPTS_VALUE;
> > +
> > +    wctx->avio_context = NULL;
>
> arent things already initialized to 0 ?

Yes, maybe we can keep the AV_NOPTS_VALUE assignment (it is not zero).

-- 
/Vittorio Palmisano/