[FFmpeg-devel] [PATCH] lavf/dashenc: Add support for per-stream container type selection.
Andrey Semashev
andrey.semashev at gmail.com
Wed Nov 14 10:11:41 EET 2018
On 11/12/18 3:55 PM, Andrey Semashev wrote:
> On 11/12/18 3:12 PM, Jeyapal, Karthick wrote:
>>
>> On 11/12/18 5:20 PM, Andrey Semashev wrote:
>>> On 11/12/18 8:20 AM, Jeyapal, Karthick wrote:
>>>>
>>>> On 11/8/18 10:27 PM, Andrey Semashev wrote:
>>>>> This commit restores the ability to create DASH streams with codecs
>>>>> that require different containers that was lost after commit
>>>>> 2efdbf7367989cf9d296c25fa3d2aff8d6e25fdd. It extends the
>>>>> dash_segment_type
>>>>> option syntax to allow to specify segment container types for
>>>>> individual
>>>>> streams, in addition to the default container type that is applied to
>>>>> all streams. The extended syntax is backward compatible with the
>>>>> previous
>>>>> syntax.
>>>> Thanks for sending the patch. I understand your requirement completely.
>>>> But I feel that this option for mapping streams with container
>>>> format is little confusing. Also, the relevant code is relatively
>>>> big, and thus difficult to maintain in future.
>>>> I have a middle ground suggestion. If your goal is to achieve the
>>>> earlier behavior broken commits, then I propose the following.
>>>> Option "dash_segment_type" could take one more option "auto"
>>>> (instead of mp4 or webm).
>>>> When "auto" is chosen, the muxer could choose webm format for VP8,
>>>> VP9, vorbis, opus streams and mp4 format for all other streams.
>>>> In this method the previous behavior of dashenc muxer could be
>>>> restored with little addition to the overall code. Also it's usage
>>>> will be simpler and easier to understand.
>>>
>>> This solution might be ok for just restoring the previous capability,
>>> but I think the ability for selecting the container format by the
>>> user is still more useful. For example, Opus can be muxed in both mp4
>>> (although, with experimental flag) and webm, and it may make sense to
>>> some users to select mp4. (In my case, though, I wanted webm, hence
>>> the patch.)
>> In that case they could select "dash_segment_type" as "mp4", in which
>> case all streams including opus will be muxed in mp4 format. I don't
>> see a use-case where somebody wants opus in mp4 and would want other
>> streams in webm format.
>
> Suppose you want to create a DASH stream consisting of VP8, H264, Vorbis
> and Opus substreams to cover the best compatibility with clients (which
> are mostly web browsers, but not necessarilly all of them). AFAIK, you
> cannot put all these codecs neither in mp4 nor webm. An "auto" option
> would put Opus in webm container, leaving clients not supporting webm
> not able to play audio. With explicit container selection one could put
> Opus content in mp4 or both webm and mp4 (in which case the substream
> would be duplicated).
>
> An example of a client that is picky about container format is Safari on
> OS X High Sierra. I don't have one to test, but reportedly it supports
> Opus only in caf format, and I've read someone hacked it to play Opus in
> mp4 if disguised as AAC. I know lavf/dashenc doesn't support caf (yet)
> but it may support it in the future, so the container format selection
> would become even more relevant then.
So, what do we decide about this patch?
>>> Besides the parser, it doesn't add much code, and if I can improve
>>> the patch, please let me know how. If you absolutely don't want this
>>> functionality, that's ok, I'll keep this patch for my local builds
>>> and submit an "auto" option patch instead.
>> I am not absolutely against this patch. But I am just trying to find
>> if there is a use-case for such an advanced option. If there is a
>> practical requirement for such a use-case, then I agree that we should
>> review and push this patch for sure.
>> But if the requirement is just theoretical at this point, then I would
>> prefer the "auto" option patch. Maybe we could revisit this advanced
>> options patch when a real requirement comes up.
>>>
>>>>> ---
>>>>> doc/muxers.texi | 8 ++-
>>>>> libavformat/dashenc.c | 161
>>>>> +++++++++++++++++++++++++++++++++++-------
>>>>> 2 files changed, 140 insertions(+), 29 deletions(-)
>>>>>
>>>>> diff --git a/doc/muxers.texi b/doc/muxers.texi
>>>>> index 62f4091e31..4418b38c76 100644
>>>>> --- a/doc/muxers.texi
>>>>> +++ b/doc/muxers.texi
>>>>> @@ -289,8 +289,12 @@ Set container format (mp4/webm) options using
>>>>> a @code{:} separated list of
>>>>> key=value parameters. Values containing @code{:} special
>>>>> characters must be
>>>>> escaped.
>>>>> - at item dash_segment_type @var{dash_segment_type}
>>>>> -Possible values:
>>>>> + at item -dash_segment_type @var{dash_segment_type}
>>>>> +Sets the container type for dash segment files. Syntax is "<type>
>>>>> <type>:a,b,c <type>:d,e" where <type> is
>>>>> +the container type and a, b, c, d and e are the indices of the
>>>>> mapped streams. When no indices are specified,
>>>>> +the container type is set for all streams.
>>>>> +
>>>>> +Possible container type values:
>>>>> @item mp4
>>>>> If this flag is set, the dash segment files will be in in
>>>>> ISOBMFF format. This is the default format.
>>>>> diff --git a/libavformat/dashenc.c b/libavformat/dashenc.c
>>>>> index f8b3d106d5..626dc76413 100644
>>>>> --- a/libavformat/dashenc.c
>>>>> +++ b/libavformat/dashenc.c
>>>>> @@ -84,6 +84,8 @@ typedef struct OutputStream {
>>>>> int64_t first_pts, start_pts, max_pts;
>>>>> int64_t last_dts, last_pts;
>>>>> int bit_rate;
>>>>> + SegmentType segment_type;
>>>>> + const char *format_name;
>>>>> char codec_str[100];
>>>>> int written_len;
>>>>> @@ -131,8 +133,7 @@ typedef struct DASHContext {
>>>>> int64_t timeout;
>>>>> int index_correction;
>>>>> char *format_options_str;
>>>>> - SegmentType segment_type;
>>>>> - const char *format_name;
>>>>> + const char *segment_types_str;
>>>>> } DASHContext;
>>>>> static struct codec_string {
>>>>> @@ -188,14 +189,6 @@ static void dashenc_io_close(AVFormatContext
>>>>> *s, AVIOContext **pb, char *filenam
>>>>> }
>>>>> }
>>>>> -static const char *get_format_str(SegmentType segment_type) {
>>>>> - int i;
>>>>> - for (i = 0; i < SEGMENT_TYPE_NB; i++)
>>>>> - if (formats[i].segment_type == segment_type)
>>>>> - return formats[i].str;
>>>>> - return NULL;
>>>>> -}
>>>>> -
>>>>> static int check_file_extension(const char *filename, const char
>>>>> *extension) {
>>>>> char *dot;
>>>>> if (!filename || !extension)
>>>>> @@ -375,6 +368,8 @@ static void dash_free(AVFormatContext *s)
>>>>> c->nb_as = 0;
>>>>> }
>>>>> + av_freep(&c->segment_types_str);
>>>>> +
>>>>> if (!c->streams)
>>>>> return;
>>>>> for (i = 0; i < s->nb_streams; i++) {
>>>>> @@ -621,13 +616,13 @@ static int
>>>>> write_adaptation_set(AVFormatContext *s, AVIOContext *out, int as_ind
>>>>> if (as->media_type == AVMEDIA_TYPE_VIDEO) {
>>>>> AVStream *st = s->streams[i];
>>>>> avio_printf(out, "\t\t\t<Representation id=\"%d\"
>>>>> mimeType=\"video/%s\" codecs=\"%s\"%s width=\"%d\" height=\"%d\"",
>>>>> - i, c->format_name, os->codec_str, bandwidth_str,
>>>>> s->streams[i]->codecpar->width, s->streams[i]->codecpar->height);
>>>>> + i, os->format_name, os->codec_str, bandwidth_str,
>>>>> s->streams[i]->codecpar->width, s->streams[i]->codecpar->height);
>>>>> if (st->avg_frame_rate.num)
>>>>> avio_printf(out, " frameRate=\"%d/%d\"",
>>>>> st->avg_frame_rate.num, st->avg_frame_rate.den);
>>>>> avio_printf(out, ">\n");
>>>>> } else {
>>>>> avio_printf(out, "\t\t\t<Representation id=\"%d\"
>>>>> mimeType=\"audio/%s\" codecs=\"%s\"%s audioSamplingRate=\"%d\">\n",
>>>>> - i, c->format_name, os->codec_str, bandwidth_str,
>>>>> s->streams[i]->codecpar->sample_rate);
>>>>> + i, os->format_name, os->codec_str, bandwidth_str,
>>>>> s->streams[i]->codecpar->sample_rate);
>>>>> avio_printf(out, "\t\t\t\t<AudioChannelConfiguration
>>>>> schemeIdUri=\"urn:mpeg:dash:23003:3:audio_channel_configuration:2011\"
>>>>> value=\"%d\" />\n",
>>>>> s->streams[i]->codecpar->channels);
>>>>> }
>>>>> @@ -773,6 +768,120 @@ end:
>>>>> return 0;
>>>>> }
>>>>> +static inline void set_all_segment_types(AVFormatContext *s,
>>>>> int format_idx)
>>>>> +{
>>>>> + DASHContext *c = s->priv_data;
>>>>> + for (int i = 0; i < s->nb_streams; ++i) {
>>>>> + OutputStream *os = &c->streams[i];
>>>>> + os->segment_type = formats[format_idx].segment_type;
>>>>> + os->format_name = formats[format_idx].str;
>>>>> + }
>>>>> +}
>>>>> +
>>>>> +static int parse_segment_types(AVFormatContext *s)
>>>>> +{
>>>>> + DASHContext *c = s->priv_data;
>>>>> + const char *p = c->segment_types_str;
>>>>> + enum { type_expected, type_parsed, parsing_streams } state;
>>>>> + int i, format_idx;
>>>>> +
>>>>> + // Set the default container type in case if some streams are
>>>>> not mentioned in the string
>>>>> + set_all_segment_types(s, 0);
>>>>> +
>>>>> + if (!p)
>>>>> + return 0;
>>>>> +
>>>>> + // Parse per-stream container types: mp4 webm:0,1,2 and so on
>>>>> + state = type_expected;
>>>>> + format_idx = 0;
>>>>> + while (*p) {
>>>>> + if (*p == ' ') {
>>>>> + if (state == type_parsed)
>>>>> + set_all_segment_types(s, format_idx);
>>>>> +
>>>>> + state = type_expected;
>>>>> + ++p;
>>>>> + continue;
>>>>> + }
>>>>> +
>>>>> + if (state == type_expected) {
>>>>> + for (i = 0; i < SEGMENT_TYPE_NB; i++) {
>>>>> + if (av_strstart(p, formats[i].str, &p)) {
>>>>> + state = type_parsed;
>>>>> + format_idx = i;
>>>>> + break;
>>>>> + }
>>>>> + }
>>>>> +
>>>>> + if (state != type_parsed) {
>>>>> + av_log(s, AV_LOG_ERROR, "DASH segment type not
>>>>> recognized at position %d of \"%s\"\n", (int)(p -
>>>>> c->segment_types_str + 1), c->segment_types_str);
>>>>> + return AVERROR_MUXER_NOT_FOUND;
>>>>> + }
>>>>> +
>>>>> + continue;
>>>>> + }
>>>>> +
>>>>> + if (state == type_parsed) {
>>>>> + if (*p != ':') {
>>>>> + av_log(s, AV_LOG_ERROR, "Unexpected character at
>>>>> position %d of \"%s\", a colon is expected\n", (int)(p -
>>>>> c->segment_types_str + 1), c->segment_types_str);
>>>>> + return AVERROR(EINVAL);
>>>>> + }
>>>>> +
>>>>> + state = parsing_streams;
>>>>> + ++p;
>>>>> + continue;
>>>>> + }
>>>>> +
>>>>> + if (state == parsing_streams) {
>>>>> + while (*p && *p != ' ') {
>>>>> + if (*p == ',') {
>>>>> + ++p;
>>>>> + } else if (*p == 'a' || *p == 'v') {
>>>>> + // Select all streams of the given type
>>>>> + enum AVMediaType type = (*p == 'v') ?
>>>>> AVMEDIA_TYPE_VIDEO : AVMEDIA_TYPE_AUDIO;
>>>>> +
>>>>> + for (i = 0; i < s->nb_streams; ++i) {
>>>>> + if (s->streams[i]->codecpar->codec_type ==
>>>>> type) {
>>>>> + OutputStream *os = &c->streams[i];
>>>>> + os->segment_type =
>>>>> formats[format_idx].segment_type;
>>>>> + os->format_name =
>>>>> formats[format_idx].str;
>>>>> + }
>>>>> + }
>>>>> +
>>>>> + ++p;
>>>>> + } else {
>>>>> + // Select a stream by index
>>>>> + OutputStream *os;
>>>>> + const char* end_p = p;
>>>>> + i = strtol(p, (char**)&end_p, 10);
>>>>> + if (p == end_p) {
>>>>> + av_log(s, AV_LOG_ERROR, "Failed to parse
>>>>> stream index at position %d of \"%s\"\n", (int)(p -
>>>>> c->segment_types_str + 1), c->segment_types_str);
>>>>> + return AVERROR(EINVAL);
>>>>> + }
>>>>> + if (i < 0 || i >= s->nb_streams) {
>>>>> + av_log(s, AV_LOG_ERROR, "Selected stream
>>>>> %d not found!\n", i);
>>>>> + return AVERROR(EINVAL);
>>>>> + }
>>>>> +
>>>>> + os = &c->streams[i];
>>>>> + os->segment_type =
>>>>> formats[format_idx].segment_type;
>>>>> + os->format_name = formats[format_idx].str;
>>>>> +
>>>>> + p = end_p;
>>>>> + }
>>>>> + }
>>>>> +
>>>>> + state = type_expected;
>>>>> + }
>>>>> + }
>>>>> +
>>>>> + // Complete segment type setup if no streams are specified
>>>>> after the container type
>>>>> + if (state == type_parsed)
>>>>> + set_all_segment_types(s, format_idx);
>>>>> +
>>>>> + return 0;
>>>>> +}
>>>>> +
>>>>> static int write_manifest(AVFormatContext *s, int final)
>>>>> {
>>>>> DASHContext *c = s->priv_data;
>>>>> @@ -992,6 +1101,9 @@ static int dash_init(AVFormatContext *s)
>>>>> if ((ret = parse_adaptation_sets(s)) < 0)
>>>>> return ret;
>>>>> + if ((ret = parse_segment_types(s)) < 0)
>>>>> + return ret;
>>>>> +
>>>>> for (i = 0; i < s->nb_streams; i++) {
>>>>> OutputStream *os = &c->streams[i];
>>>>> AdaptationSet *as = &c->as[os->as_idx - 1];
>>>>> @@ -1017,13 +1129,10 @@ static int dash_init(AVFormatContext *s)
>>>>> if (!ctx)
>>>>> return AVERROR(ENOMEM);
>>>>> - c->format_name = get_format_str(c->segment_type);
>>>>> - if (!c->format_name)
>>>>> - return AVERROR_MUXER_NOT_FOUND;
>>>>> - if (c->segment_type == SEGMENT_TYPE_WEBM) {
>>>>> - if ((!c->single_file &&
>>>>> check_file_extension(c->init_seg_name, c->format_name) != 0) ||
>>>>> - (!c->single_file &&
>>>>> check_file_extension(c->media_seg_name, c->format_name) != 0) ||
>>>>> - (c->single_file &&
>>>>> check_file_extension(c->single_file_name, c->format_name) != 0)) {
>>>>> + if (os->segment_type == SEGMENT_TYPE_WEBM) {
>>>>> + if ((!c->single_file &&
>>>>> check_file_extension(c->init_seg_name, os->format_name) != 0) ||
>>>>> + (!c->single_file &&
>>>>> check_file_extension(c->media_seg_name, os->format_name) != 0) ||
>>>>> + (c->single_file &&
>>>>> check_file_extension(c->single_file_name, os->format_name) != 0)) {
>>>>> av_log(s, AV_LOG_WARNING,
>>>>> "One or many segment file names doesn't
>>>>> end with .webm. "
>>>>> "Override -init_seg_name and/or
>>>>> -media_seg_name and/or "
>>>>> @@ -1031,7 +1140,7 @@ static int dash_init(AVFormatContext *s)
>>>>> }
>>>>> }
>>>>> - ctx->oformat = av_guess_format(c->format_name, NULL,
>>>>> NULL);
>>>>> + ctx->oformat = av_guess_format(os->format_name, NULL, NULL);
>>>>> if (!ctx->oformat)
>>>>> return AVERROR_MUXER_NOT_FOUND;
>>>>> os->ctx = ctx;
>>>>> @@ -1075,7 +1184,7 @@ static int dash_init(AVFormatContext *s)
>>>>> return ret;
>>>>> }
>>>>> - if (c->segment_type == SEGMENT_TYPE_MP4) {
>>>>> + if (os->segment_type == SEGMENT_TYPE_MP4) {
>>>>> if (c->streaming)
>>>>> av_dict_set(&opts, "movflags",
>>>>> "frag_every_frame+dash+delay_moov+global_sidx", 0);
>>>>> else
>>>>> @@ -1140,7 +1249,7 @@ static int dash_write_header(AVFormatContext *s)
>>>>> // Flush init segment
>>>>> // Only for WebM segment, since for mp4 delay_moov is
>>>>> set and
>>>>> // the init segment is thus flushed after the first
>>>>> packets.
>>>>> - if (c->segment_type == SEGMENT_TYPE_WEBM &&
>>>>> + if (os->segment_type == SEGMENT_TYPE_WEBM &&
>>>>> (ret = flush_init_segment(s, os)) < 0)
>>>>> return ret;
>>>>> }
>>>>> @@ -1311,7 +1420,7 @@ static int dash_flush(AVFormatContext *s, int
>>>>> final, int stream)
>>>>> }
>>>>> if (!c->single_file) {
>>>>> - if (c->segment_type == SEGMENT_TYPE_MP4 &&
>>>>> !os->written_len)
>>>>> + if (os->segment_type == SEGMENT_TYPE_MP4 &&
>>>>> !os->written_len)
>>>>> write_styp(os->ctx->pb);
>>>>> } else {
>>>>> snprintf(os->full_path, sizeof(os->full_path),
>>>>> "%s%s", c->dirname, os->initfile);
>>>>> @@ -1501,7 +1610,7 @@ static int dash_write_packet(AVFormatContext
>>>>> *s, AVPacket *pkt)
>>>>> }
>>>>> //write out the data immediately in streaming mode
>>>>> - if (c->streaming && c->segment_type == SEGMENT_TYPE_MP4) {
>>>>> + if (c->streaming && os->segment_type == SEGMENT_TYPE_MP4) {
>>>>> int len = 0;
>>>>> uint8_t *buf = NULL;
>>>>> if (!os->written_len)
>>>>> @@ -1597,9 +1706,7 @@ static const AVOption options[] = {
>>>>> { "timeout", "set timeout for socket I/O operations",
>>>>> OFFSET(timeout), AV_OPT_TYPE_DURATION, { .i64 = -1 }, -1, INT_MAX,
>>>>> .flags = E },
>>>>> { "index_correction", "Enable/Disable segment index
>>>>> correction logic", OFFSET(index_correction), AV_OPT_TYPE_BOOL, {
>>>>> .i64 = 0 }, 0, 1, E },
>>>>> { "format_options","set list of options for the container
>>>>> format (mp4/webm) used for dash", OFFSET(format_options_str),
>>>>> AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, E},
>>>>> - { "dash_segment_type", "set dash segment files type",
>>>>> OFFSET(segment_type), AV_OPT_TYPE_INT, {.i64 = SEGMENT_TYPE_MP4 },
>>>>> 0, SEGMENT_TYPE_NB - 1, E, "segment_type"},
>>>>> - { "mp4", "make segment file in ISOBMFF format", 0,
>>>>> AV_OPT_TYPE_CONST, {.i64 = SEGMENT_TYPE_MP4 }, 0, UINT_MAX, E,
>>>>> "segment_type"},
>>>>> - { "webm", "make segment file in WebM format", 0,
>>>>> AV_OPT_TYPE_CONST, {.i64 = SEGMENT_TYPE_WEBM }, 0, UINT_MAX, E,
>>>>> "segment_type"},
>>>>> + { "dash_segment_type", "DASH segment files type. Syntax:
>>>>> \"<type> <type>:<ids>\" where <type> is one of mp4 or webm and
>>>>> <ids> are comma-separated stream ids", OFFSET(segment_types_str),
>>>>> AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, E },
>>>>> { NULL },
>>>>> };
>>>>
>>>
>>> _______________________________________________
>>> ffmpeg-devel mailing list
>>> ffmpeg-devel at ffmpeg.org
>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>
>
More information about the ffmpeg-devel
mailing list