[FFmpeg-devel] [PATCH 5/5] Add AudioToolbox audio input device.

[Marvin Scholz]

On 19 Jan 2022, at 15:42, Romain Beauxis wrote:

Hi, thanks for the patch. I've not done a full code review yet, just a 
few
initial remarks below:

> This patch adds support for a new, audio-specific input device using 
> the documented and battle-tested AUHAL input. This provides a pendant 
> to the AudioToolbox audio-only output.
>
> A couple of advantages for this:
> * It avoids a lot of the complexity of supporting audio and video in a 
> single input
> * The AUHAL API seems tested, documented and robust
> * This implementation hopefully gives good control over audio latency 
> and also minimizes data copy
> From: Romain Beauxis <toots at rastageeks.org>
> To: ffmpeg-devel at ffmpeg.org
> Subject: [PATCH] Add AudioToolbox audio input device.
> Date: 18. January 2022 at 23:29
> Signed-off-by: Romain Beauxis <toots at rastageeks.org>
> ---
>  configure                      |   5 +
>  doc/indevs.texi                |  44 ++++
>  libavdevice/Makefile           |   1 +
>  libavdevice/alldevices.c       |   1 +
>  libavdevice/audiotoolbox_dec.m | 466 
> +++++++++++++++++++++++++++++++++
>  5 files changed, 517 insertions(+)
>  create mode 100644 libavdevice/audiotoolbox_dec.m
>
> diff --git a/configure b/configure
> index 1413122d87..80e39aae44 100755
> --- a/configure
> +++ b/configure
> @@ -204,6 +204,7 @@ External library support:
>    --disable-avfoundation   disable Apple AVFoundation framework 
> [autodetect]
>    --enable-avisynth        enable reading of AviSynth script files 
> [no]
>    --disable-bzlib          disable bzlib [autodetect]
> +  --disable-coremedia      disable Apple CoreMedia framework 
> [autodetect]
>    --disable-coreimage      disable Apple CoreImage framework 
> [autodetect]
>    --enable-chromaprint     enable audio fingerprinting with 
> chromaprint [no]
>    --enable-frei0r          enable frei0r video filtering [no]
> @@ -1750,6 +1751,7 @@ EXTERNAL_AUTODETECT_LIBRARY_LIST="
>      appkit
>      avfoundation
>      bzlib
> +    coremedia
>      coreimage
>      iconv
>      libxcb
> @@ -3493,6 +3495,8 @@ alsa_outdev_deps="alsa"
>  avfoundation_indev_deps="avfoundation corevideo coremedia pthreads"
>  avfoundation_indev_suggest="coregraphics applicationservices"
>  avfoundation_indev_extralibs="-framework Foundation"
> +audiotoolbox_indev_deps="coremedia audiotoolbox"
> +audiotoolbox_indev_extralibs="-framework CoreMedia -framework 
> AudioToolbox"
>  audiotoolbox_outdev_deps="audiotoolbox pthreads"
>  audiotoolbox_outdev_extralibs="-framework AudioToolbox -framework 
> CoreAudio"
>  bktr_indev_deps_any="dev_bktr_ioctl_bt848_h machine_ioctl_bt848_h 
> dev_video_bktr_ioctl_bt848_h dev_ic_bt8xx_h"
> @@ -6340,6 +6344,7 @@ check_lib camera2ndk "stdbool.h stdint.h 
> camera/NdkCameraManager.h" ACameraManag
>  enabled appkit       && check_apple_framework AppKit
>  enabled audiotoolbox && check_apple_framework AudioToolbox
>  enabled avfoundation && check_apple_framework AVFoundation
> +enabled coremedia    && check_apple_framework CoreMedia
>  enabled coreimage    && check_apple_framework CoreImage
>  enabled metal        && check_apple_framework Metal
>  enabled videotoolbox && check_apple_framework VideoToolbox
> diff --git a/doc/indevs.texi b/doc/indevs.texi
> index 858c0fa4e4..30a91d304f 100644
> --- a/doc/indevs.texi
> +++ b/doc/indevs.texi
> @@ -103,6 +103,50 @@ Set the maximum number of frames to buffer. 
> Default is 5.
>
>  @end table
>
> + at section AudioToolbox
> +
> +AudioToolbox input device.
> +
> +Allows native input from CoreAudio devices on OSX.
Nit: Nowadays it's macOS instead of OSX

> +
> +All available devices can be enumerated by using 
> @option{-list_devices true}, listing
> +all device names, and corresponding unique ID.

Instead of adding another device that uses a custom list-devices option, 
could you
instead implement the .get_device_list callback? (See alsa or pulse 
modules for an
example) Then listing would work properly with the `ffmpeg -sources` 
command and
devices could be iterated over using the avdevice APIs as well.

(Ideally we would have that for AVFoundation too but its really hard now 
to do that
in a backwards compatible manner)

> +
> + at subsection Options
> +
> +AudioToolbox supports the following options:
> +
> + at table @option
> +
> + at item channels
> +Set the number of channels. Default is device's default.
> +
> + at item frames_queue_length
> +Maximum of buffers in the input queue
> +
> + at item buffer_frame_size
> +Buffer frame size, gouverning internal latency
> +
> + at item big_endian
> +Return big endian samples
> +
> + at item sample_format
> +Sample format
> +
> + at end table
> +
> + at subsection Examples
> +
> + at itemize
> +
> + at item
> +Print the list of supported devices
> + at example
> +$ ffmpeg -f audiotoolbox -list_devices true -i ""
> + at end example
> +
> + at end itemize
> +
>  @section avfoundation
[…]