[FFmpeg-devel] [PATCH 5/5] Add AudioToolbox audio input device.

[Romain Beauxis]

Le mer. 19 janv. 2022 à 10:59, Marvin Scholz <epirat07 at gmail.com> a écrit :
>
>
>
> On 19 Jan 2022, at 15:42, Romain Beauxis wrote:
>
> Hi, thanks for the patch. I've not done a full code review yet, just a
> few
> initial remarks below:
>
> > This patch adds support for a new, audio-specific input device using
> > the documented and battle-tested AUHAL input. This provides a pendant
> > to the AudioToolbox audio-only output.
> >
> > A couple of advantages for this:
> > * It avoids a lot of the complexity of supporting audio and video in a
> > single input
> > * The AUHAL API seems tested, documented and robust
> > * This implementation hopefully gives good control over audio latency
> > and also minimizes data copy
> > From: Romain Beauxis <toots at rastageeks.org>
> > To: ffmpeg-devel at ffmpeg.org
> > Subject: [PATCH] Add AudioToolbox audio input device.
> > Date: 18. January 2022 at 23:29
> > Signed-off-by: Romain Beauxis <toots at rastageeks.org>
> > ---
> >  configure                      |   5 +
> >  doc/indevs.texi                |  44 ++++
> >  libavdevice/Makefile           |   1 +
> >  libavdevice/alldevices.c       |   1 +
> >  libavdevice/audiotoolbox_dec.m | 466
> > +++++++++++++++++++++++++++++++++
> >  5 files changed, 517 insertions(+)
> >  create mode 100644 libavdevice/audiotoolbox_dec.m
> >
> > diff --git a/configure b/configure
> > index 1413122d87..80e39aae44 100755
> > --- a/configure
> > +++ b/configure
> > @@ -204,6 +204,7 @@ External library support:
> >    --disable-avfoundation   disable Apple AVFoundation framework
> > [autodetect]
> >    --enable-avisynth        enable reading of AviSynth script files
> > [no]
> >    --disable-bzlib          disable bzlib [autodetect]
> > +  --disable-coremedia      disable Apple CoreMedia framework
> > [autodetect]
> >    --disable-coreimage      disable Apple CoreImage framework
> > [autodetect]
> >    --enable-chromaprint     enable audio fingerprinting with
> > chromaprint [no]
> >    --enable-frei0r          enable frei0r video filtering [no]
> > @@ -1750,6 +1751,7 @@ EXTERNAL_AUTODETECT_LIBRARY_LIST="
> >      appkit
> >      avfoundation
> >      bzlib
> > +    coremedia
> >      coreimage
> >      iconv
> >      libxcb
> > @@ -3493,6 +3495,8 @@ alsa_outdev_deps="alsa"
> >  avfoundation_indev_deps="avfoundation corevideo coremedia pthreads"
> >  avfoundation_indev_suggest="coregraphics applicationservices"
> >  avfoundation_indev_extralibs="-framework Foundation"
> > +audiotoolbox_indev_deps="coremedia audiotoolbox"
> > +audiotoolbox_indev_extralibs="-framework CoreMedia -framework
> > AudioToolbox"
> >  audiotoolbox_outdev_deps="audiotoolbox pthreads"
> >  audiotoolbox_outdev_extralibs="-framework AudioToolbox -framework
> > CoreAudio"
> >  bktr_indev_deps_any="dev_bktr_ioctl_bt848_h machine_ioctl_bt848_h
> > dev_video_bktr_ioctl_bt848_h dev_ic_bt8xx_h"
> > @@ -6340,6 +6344,7 @@ check_lib camera2ndk "stdbool.h stdint.h
> > camera/NdkCameraManager.h" ACameraManag
> >  enabled appkit       && check_apple_framework AppKit
> >  enabled audiotoolbox && check_apple_framework AudioToolbox
> >  enabled avfoundation && check_apple_framework AVFoundation
> > +enabled coremedia    && check_apple_framework CoreMedia
> >  enabled coreimage    && check_apple_framework CoreImage
> >  enabled metal        && check_apple_framework Metal
> >  enabled videotoolbox && check_apple_framework VideoToolbox
> > diff --git a/doc/indevs.texi b/doc/indevs.texi
> > index 858c0fa4e4..30a91d304f 100644
> > --- a/doc/indevs.texi
> > +++ b/doc/indevs.texi
> > @@ -103,6 +103,50 @@ Set the maximum number of frames to buffer.
> > Default is 5.
> >
> >  @end table
> >
> > + at section AudioToolbox
> > +
> > +AudioToolbox input device.
> > +
> > +Allows native input from CoreAudio devices on OSX.
> Nit: Nowadays it's macOS instead of OSX
>
> > +
> > +All available devices can be enumerated by using
> > @option{-list_devices true}, listing
> > +all device names, and corresponding unique ID.
>
> Instead of adding another device that uses a custom list-devices option,
> could you
> instead implement the .get_device_list callback? (See alsa or pulse
> modules for an
> example) Then listing would work properly with the `ffmpeg -sources`
> command and
> devices could be iterated over using the avdevice APIs as well.
>
> (Ideally we would have that for AVFoundation too but its really hard now
> to do that
> in a backwards compatible manner)

Is there a way to access the result of the get_devices_callback from
the ffmpeg CLI? I have searched but haven't found any so far..

> > +
> > + at subsection Options
> > +
> > +AudioToolbox supports the following options:
> > +
> > + at table @option
> > +
> > + at item channels
> > +Set the number of channels. Default is device's default.
> > +
> > + at item frames_queue_length
> > +Maximum of buffers in the input queue
> > +
> > + at item buffer_frame_size
> > +Buffer frame size, gouverning internal latency
> > +
> > + at item big_endian
> > +Return big endian samples
> > +
> > + at item sample_format
> > +Sample format
> > +
> > + at end table
> > +
> > + at subsection Examples
> > +
> > + at itemize
> > +
> > + at item
> > +Print the list of supported devices
> > + at example
> > +$ ffmpeg -f audiotoolbox -list_devices true -i ""
> > + at end example
> > +
> > + at end itemize
> > +
> >  @section avfoundation
> […]