[FFmpeg-devel] [PATCH] Add mkdir support for HLS localtime-generated segment files

Johan Ström johan at stromnet.se
Tue Feb 16 21:03:52 CET 2016


On 16/02/16 00:59, Michael Niedermayer wrote:
> On Mon, Feb 15, 2016 at 10:35:57PM +0100, Johan Ström wrote:
>> Thanks for your attention!
>>
>> Updated patch attached. It has been rebased against current master
>> (avio_open2->ffio_open_whitelist changes) with no issues.
>>
>> On 14/02/16 01:29, Michael Niedermayer wrote:
>>> On Wed, Feb 03, 2016 at 10:44:28PM +0100, Johan Ström wrote:
>>>> Hi,
>>>>
>>>> this patch adds -use_localtime_mkdir option to the HLS encoder.
>>>> Use with -use_localtime, and set -hls_segment_filename to a path
>>>> which contains a subdirectory i.e.
>>>> /some/path/%Y%m%d/%Y%m%dT%H%M%S-%s.ts
>>>> This will mkdir the %Y%m%d-part of the path if it does not already exist.
>>>>
>>>> In addition, each filename in the playlist output will be prefixed
>>>> with this subdirectory (if playlist and segment shares the same base
>>>> path).
>> ...
>>> missing update to the docs
>> Actually, there was no docs for the -use_localtime property at all.
>> I've added for both use_localtime and use_localtime_mkdir now.
> what happens if the path is not a local file but for example
> http://...
>
> if that otherwise works but fails with the new option then it should
> be documented and maybe tested for. It may seem logic for us but a
> user might be confused why the option doesnt work for that
That is not possible in any mode, and neither would I expect it to.

The following is with -use_localtime 1 -hls_segment_filename 
http://localhost/test/%d.ts:

[tcp @ 0x80401e880] Connection to tcp://localhost:80 failed (Connection 
refused), trying next address
Output #0, hls, to '/storage/dvr/ipcam//test/playlist.m3u8':
   Metadata:
     title           : RTSP Session
     encoder         : Lavf57.25.100
     Stream #0:0: Video: h264, yuv420p, 352x288, q=2-31, 25 fps, 25 tbr, 
25 tbn, 25 tbc
Stream mapping:
   Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
..
[tcp @ 0x804065080] Connection to tcp://back-1:80 failed (Connection 
refused), trying next address
[tcp @ 0x804065080] Connection to tcp://back-1:80 failed (Connection 
refused), trying next address
frame=  198 fps= 29 q=-1.0 Lsize=N/A time=00:00:07.92 bitrate=N/A 
speed=1.17x
video:350kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB 
muxing overhead: unknown
Exiting normally, received signal 2.

Same thing without -use_localtime 1.
So, it seems it tries to use some kind of tcp transport (rather than HTTP).

I did notice a minor typo in my docs patch, so a new patch is attached.

>>> also can you add a fate test for this feature ?
>> Sorry, I have no idea how those works or how to write one. As far as
>> I can see there are none for HLS today which I could extend on..
> yes, it would be great to have hls fate tests
> without such tests someone might break hls without him noticing ...
> but thats orthogonal to the patch

Indeed. But I'm probably not the right one to write them.

-------------- next part --------------
From 443289a36c71bfecbc748ad706d2fa35d3659e90 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Johan=20Str=C3=B6m?= <johan at stromnet.se>
Date: Wed, 3 Feb 2016 22:20:07 +0100
Subject: [PATCH] hlsenc: add use_localtime_mkdir option to automatically
 create time-based directory

Use with -use_localtime, and set -hls_segment_filename to a path which
contains a subdirectory i.e. /some/path/%Y%m%d/%Y%m%dT%H%M%S-%s.ts
This will mkdir the %Y%m%d-part of the path if it does not already
exist.
In addition, each filename in the playlist output will be prefixed with
this subdirectory (if playlist and segment shares the same base path).
---
 doc/muxers.texi      | 20 ++++++++++++++++++++
 libavformat/hlsenc.c | 50 +++++++++++++++++++++++++++++++++++++++++++-------
 2 files changed, 63 insertions(+), 7 deletions(-)

diff --git a/doc/muxers.texi b/doc/muxers.texi
index 2e6bb4c..f2ad7fe 100644
--- a/doc/muxers.texi
+++ b/doc/muxers.texi
@@ -318,6 +318,26 @@ ffmpeg in.nut -hls_segment_filename 'file%03d.ts' out.m3u8
 This example will produce the playlist, @file{out.m3u8}, and segment files:
 @file{file000.ts}, @file{file001.ts}, @file{file002.ts}, etc.
 
+ at item use_localtime
+Use strftime on @var{filename} to expand the segment filename with localtime.
+The segment number (%d) is not available in this mode.
+ at example
+ffmpeg in.nut -use_localtime 1 -hls_segment_filename 'file-%Y%m%d-%s.ts' out.m3u8
+ at end example
+This example will produce the playlist, @file{out.m3u8}, and segment files:
+ at file{file-20160215-1455569023.ts}, @file{file-20160215-1455569024.ts}, etc.
+
+ at item use_localtime_mkdir
+Used together with -use_localtime, it will create up to one subdirectory which
+is expanded in @var{filename}.
+ at example
+ffmpeg in.nut -use_localtime 1 -use_localtime_mkdir 1 -hls_segment_filename '%Y%m%d/file-%Y%m%d-%s.ts' out.m3u8
+ at end example
+This example will create a directory 201560215 (if it does not exist), and then
+produce the playlist, @file{out.m3u8}, and segment files:
+ at file{201560215/file-20160215-1455569023.ts}, @file{201560215/file-20160215-1455569024.ts}, etc.
+
+
 @item hls_key_info_file @var{key_info_file}
 Use the information in @var{key_info_file} for segment encryption. The first
 line of @var{key_info_file} specifies the key URI written to the playlist. The
diff --git a/libavformat/hlsenc.c b/libavformat/hlsenc.c
index 7ab7cbb..8c61b6d 100644
--- a/libavformat/hlsenc.c
+++ b/libavformat/hlsenc.c
@@ -82,6 +82,7 @@ typedef struct HLSContext {
     char *segment_filename;
 
     int use_localtime;      ///< flag to expand filename with localtime
+    int use_localtime_mkdir;///< flag to mkdir dirname in timebased filename
     int allowcache;
     int64_t recording_time;
     int has_video;
@@ -304,16 +305,35 @@ static int hls_mux_init(AVFormatContext *s)
 }
 
 /* Create a new segment and append it to the segment list */
-static int hls_append_segment(HLSContext *hls, double duration, int64_t pos,
-                              int64_t size)
+static int hls_append_segment(struct AVFormatContext *s, HLSContext *hls, double duration,
+                              int64_t pos, int64_t size)
 {
     HLSSegment *en = av_malloc(sizeof(*en));
+    char *tmp, *p;
+    const char *pl_dir, *filename;
     int ret;
 
     if (!en)
         return AVERROR(ENOMEM);
 
-    av_strlcpy(en->filename, av_basename(hls->avf->filename), sizeof(en->filename));
+    filename = av_basename(hls->avf->filename);
+
+    if (hls->use_localtime_mkdir) {
+        /* Possibly prefix with mkdir'ed subdir, if playlist share same
+         * base path. */
+        tmp = av_strdup(s->filename);
+        if (!tmp) {
+            av_free(en);
+            return AVERROR(ENOMEM);
+        }
+
+        pl_dir = av_dirname(tmp);
+        p = hls->avf->filename;
+        if (strstr(p, pl_dir) == p)
+            filename = hls->avf->filename + strlen(pl_dir) + 1;
+        av_free(tmp);
+    }
+    av_strlcpy(en->filename, filename, sizeof(en->filename));
 
     if(hls->has_subtitle)
         av_strlcpy(en->sub_filename, av_basename(hls->vtt_avf->filename), sizeof(en->sub_filename));
@@ -508,7 +528,22 @@ static int hls_start(AVFormatContext *s)
                 av_log(oc, AV_LOG_ERROR, "Could not get segment filename with use_localtime\n");
                 return AVERROR(EINVAL);
             }
-       } else if (av_get_frame_filename(oc->filename, sizeof(oc->filename),
+
+            if (c->use_localtime_mkdir) {
+                const char *dir;
+                char *fn_copy = av_strdup(oc->filename);
+                if (!fn_copy) {
+                    return AVERROR(ENOMEM);
+                }
+                dir = av_dirname(fn_copy);
+                if (mkdir(dir, 0777) == -1 && errno != EEXIST) {
+                    av_log(oc, AV_LOG_ERROR, "Could not create directory %s with use_localtime_mkdir\n", dir);
+                    av_free(fn_copy);
+                    return AVERROR(errno);
+                }
+                av_free(fn_copy);
+            }
+        } else if (av_get_frame_filename(oc->filename, sizeof(oc->filename),
                                   c->basename, c->wrap ? c->sequence % c->wrap : c->sequence) < 0) {
             av_log(oc, AV_LOG_ERROR, "Invalid segment filename template '%s' you can try use -use_localtime 1 with it\n", c->basename);
             return AVERROR(EINVAL);
@@ -778,7 +813,7 @@ static int hls_write_packet(AVFormatContext *s, AVPacket *pkt)
 
         new_start_pos = avio_tell(hls->avf->pb);
         hls->size = new_start_pos - hls->start_pos;
-        ret = hls_append_segment(hls, hls->duration, hls->start_pos, hls->size);
+        ret = hls_append_segment(s, hls, hls->duration, hls->start_pos, hls->size);
         hls->start_pos = new_start_pos;
         if (ret < 0)
             return ret;
@@ -825,7 +860,7 @@ static int hls_write_trailer(struct AVFormatContext *s)
     if (oc->pb) {
         hls->size = avio_tell(hls->avf->pb) - hls->start_pos;
         ff_format_io_close(s, &oc->pb);
-        hls_append_segment(hls, hls->duration, hls->start_pos, hls->size);
+        hls_append_segment(s, hls, hls->duration, hls->start_pos, hls->size);
     }
 
     if (vtt_oc) {
@@ -871,7 +906,8 @@ static const AVOption options[] = {
     {"round_durations", "round durations in m3u8 to whole numbers", 0, AV_OPT_TYPE_CONST, {.i64 = HLS_ROUND_DURATIONS }, 0, UINT_MAX,   E, "flags"},
     {"discont_start", "start the playlist with a discontinuity tag", 0, AV_OPT_TYPE_CONST, {.i64 = HLS_DISCONT_START }, 0, UINT_MAX,   E, "flags"},
     {"omit_endlist", "Do not append an endlist when ending stream", 0, AV_OPT_TYPE_CONST, {.i64 = HLS_OMIT_ENDLIST }, 0, UINT_MAX,   E, "flags"},
-    { "use_localtime", "set filename expansion with strftime at segment creation", OFFSET(use_localtime), AV_OPT_TYPE_BOOL, {.i64 = 0 }, 0, 1, E },
+    {"use_localtime", "set filename expansion with strftime at segment creation", OFFSET(use_localtime), AV_OPT_TYPE_BOOL, {.i64 = 0 }, 0, 1, E },
+    {"use_localtime_mkdir", "create last directory component in strftime-generated filename", OFFSET(use_localtime_mkdir), AV_OPT_TYPE_BOOL, {.i64 = 0 }, 0, 1, E },
     {"method", "set the HTTP method", OFFSET(method), AV_OPT_TYPE_STRING, {.str = NULL},  0, 0,    E},
 
     { NULL },
-- 
2.7.0



More information about the ffmpeg-devel mailing list