[FFmpeg-devel] [PATCH 5/5] Add AudioToolbox audio input device.

[Marvin Scholz]

On 24 Jan 2022, at 16:42, Romain Beauxis wrote:

> Hi Marvin,
>
> Le mer. 19 janv. 2022 à 10:59, Marvin Scholz <epirat07 at gmail.com> a 
> écrit :
>> On 19 Jan 2022, at 15:42, Romain Beauxis wrote:
>>
>> Hi, thanks for the patch. I've not done a full code review yet, just 
>> a
>> few
>> initial remarks below:
>>
>>> This patch adds support for a new, audio-specific input device using
>>> the documented and battle-tested AUHAL input. This provides a 
>>> pendant
>>> to the AudioToolbox audio-only output.
>>>
>>> A couple of advantages for this:
>>> * It avoids a lot of the complexity of supporting audio and video in 
>>> a
>>> single input
>>> * The AUHAL API seems tested, documented and robust
>>> * This implementation hopefully gives good control over audio 
>>> latency
>>> and also minimizes data copy
>
> Thanks!
>
> Do you need more time for this review? We discovered another issue
> with the dynamic array of video devices and I'd like to send a revised
> series soon.

Well if you want to address the things I mentioned last time
it will anyway change a bunch of things probably, so it
might be easier if I just wait for the new patch.

Additionally it might be useful to send it independent of the
whole patchset as IIUC it does not really depend on any of the
previous patches.

> Also, I think I'll reorder the patches so that the most
> trivial one (unique device name, probing API call updates) can be
> applied right away.
>
> Let me know!
> Romain
>
>>> From: Romain Beauxis <toots at rastageeks.org>
>>> To: ffmpeg-devel at ffmpeg.org
>>> Subject: [PATCH] Add AudioToolbox audio input device.
>>> Date: 18. January 2022 at 23:29
>>> Signed-off-by: Romain Beauxis <toots at rastageeks.org>
>>> ---
>>>  configure                      |   5 +
>>>  doc/indevs.texi                |  44 ++++
>>>  libavdevice/Makefile           |   1 +
>>>  libavdevice/alldevices.c       |   1 +
>>>  libavdevice/audiotoolbox_dec.m | 466
>>> +++++++++++++++++++++++++++++++++
>>>  5 files changed, 517 insertions(+)
>>>  create mode 100644 libavdevice/audiotoolbox_dec.m
>>>
>>> diff --git a/configure b/configure
>>> index 1413122d87..80e39aae44 100755
>>> --- a/configure
>>> +++ b/configure
>>> @@ -204,6 +204,7 @@ External library support:
>>>    --disable-avfoundation   disable Apple AVFoundation framework
>>> [autodetect]
>>>    --enable-avisynth        enable reading of AviSynth script files
>>> [no]
>>>    --disable-bzlib          disable bzlib [autodetect]
>>> +  --disable-coremedia      disable Apple CoreMedia framework
>>> [autodetect]
>>>    --disable-coreimage      disable Apple CoreImage framework
>>> [autodetect]
>>>    --enable-chromaprint     enable audio fingerprinting with
>>> chromaprint [no]
>>>    --enable-frei0r          enable frei0r video filtering [no]
>>> @@ -1750,6 +1751,7 @@ EXTERNAL_AUTODETECT_LIBRARY_LIST="
>>>      appkit
>>>      avfoundation
>>>      bzlib
>>> +    coremedia
>>>      coreimage
>>>      iconv
>>>      libxcb
>>> @@ -3493,6 +3495,8 @@ alsa_outdev_deps="alsa"
>>>  avfoundation_indev_deps="avfoundation corevideo coremedia pthreads"
>>>  avfoundation_indev_suggest="coregraphics applicationservices"
>>>  avfoundation_indev_extralibs="-framework Foundation"
>>> +audiotoolbox_indev_deps="coremedia audiotoolbox"
>>> +audiotoolbox_indev_extralibs="-framework CoreMedia -framework
>>> AudioToolbox"
>>>  audiotoolbox_outdev_deps="audiotoolbox pthreads"
>>>  audiotoolbox_outdev_extralibs="-framework AudioToolbox -framework
>>> CoreAudio"
>>>  bktr_indev_deps_any="dev_bktr_ioctl_bt848_h machine_ioctl_bt848_h
>>> dev_video_bktr_ioctl_bt848_h dev_ic_bt8xx_h"
>>> @@ -6340,6 +6344,7 @@ check_lib camera2ndk "stdbool.h stdint.h
>>> camera/NdkCameraManager.h" ACameraManag
>>>  enabled appkit       && check_apple_framework AppKit
>>>  enabled audiotoolbox && check_apple_framework AudioToolbox
>>>  enabled avfoundation && check_apple_framework AVFoundation
>>> +enabled coremedia    && check_apple_framework CoreMedia
>>>  enabled coreimage    && check_apple_framework CoreImage
>>>  enabled metal        && check_apple_framework Metal
>>>  enabled videotoolbox && check_apple_framework VideoToolbox
>>> diff --git a/doc/indevs.texi b/doc/indevs.texi
>>> index 858c0fa4e4..30a91d304f 100644
>>> --- a/doc/indevs.texi
>>> +++ b/doc/indevs.texi
>>> @@ -103,6 +103,50 @@ Set the maximum number of frames to buffer.
>>> Default is 5.
>>>
>>>  @end table
>>>
>>> + at section AudioToolbox
>>> +
>>> +AudioToolbox input device.
>>> +
>>> +Allows native input from CoreAudio devices on OSX.
>> Nit: Nowadays it's macOS instead of OSX
>>
>>> +
>>> +All available devices can be enumerated by using
>>> @option{-list_devices true}, listing
>>> +all device names, and corresponding unique ID.
>>
>> Instead of adding another device that uses a custom list-devices 
>> option,
>> could you
>> instead implement the .get_device_list callback? (See alsa or pulse
>> modules for an
>> example) Then listing would work properly with the `ffmpeg -sources`
>> command and
>> devices could be iterated over using the avdevice APIs as well.
>>
>> (Ideally we would have that for AVFoundation too but its really hard 
>> now
>> to do that
>> in a backwards compatible manner)
>>
>>> +
>>> + at subsection Options
>>> +
>>> +AudioToolbox supports the following options:
>>> +
>>> + at table @option
>>> +
>>> + at item channels
>>> +Set the number of channels. Default is device's default.
>>> +
>>> + at item frames_queue_length
>>> +Maximum of buffers in the input queue
>>> +
>>> + at item buffer_frame_size
>>> +Buffer frame size, gouverning internal latency
>>> +
>>> + at item big_endian
>>> +Return big endian samples
>>> +
>>> + at item sample_format
>>> +Sample format
>>> +
>>> + at end table
>>> +
>>> + at subsection Examples
>>> +
>>> + at itemize
>>> +
>>> + at item
>>> +Print the list of supported devices
>>> + at example
>>> +$ ffmpeg -f audiotoolbox -list_devices true -i ""
>>> + at end example
>>> +
>>> + at end itemize
>>> +
>>>  @section avfoundation
>> […]