[FFmpeg-user] problems consuming live single file hls

Daniel Oberhoff danieloberhoff at googlemail.com
Fri Jan 11 19:29:20 EET 2019


Am 10.01.19 um 18:04 schrieb Carl Eugen Hoyos:
> 2019-01-10 14:59 GMT+01:00, Daniel Oberhoff <danieloberhoff at googlemail.com>:
>
>> I previously reported on problems with consuming a live single
>> file hls stream. It was quite embedded in the application so i
>> failed to make a good reproducible case. Now i reduced it.
> Please test current FFmpeg git head.

same behavior (i attach logs again)

> Are you missing the "-re" option?
> Iiuc, you trying to read as-fast-as-possible an input
> that is just created: How is this supposed to work,
> what behaviour do you expect?

I dont think -re is applicable. Actually it may be the consumer is
slower than the producer, which is ok. Or it may be slower for some time
and later catch up. So, yes, i want to consume as fast as possible.
Since hls is a streaming format i would actually expect this to work. It
does for multi file. The rationale is simple: the producer produces a
m3u8 plalist, appending to it new segments as they are produced. i.e.
after they have been fully written. Then the consumer sees them (by
polling the m3u8) and consumes them. This is how i would expect it to
work over http also, i.e. the consumer polls the playlist and consumes
any new segments. We simply shortcut here and do it locally via the file
system, which should be ok. Am I missing something? As I said, for
multi-file hls we dont have these problems, and i am not sure why single
file should be different. The only difference really is that instead of
whole files the data is in a byte range of a file.

Best


Daniel


-------------- next part --------------
ffmpeg version N-92950-gc65c4ce773 Copyright (c) 2000-2019 the FFmpeg developers
  built with Apple LLVM version 10.0.0 (clang-1000.11.45.5)
  configuration: 
  libavutil      56. 25.100 / 56. 25.100
  libavcodec     58. 43.100 / 58. 43.100
  libavformat    58. 25.100 / 58. 25.100
  libavdevice    58.  6.101 / 58.  6.101
  libavfilter     7. 47.100 /  7. 47.100
  libswscale      5.  4.100 /  5.  4.100
  libswresample   3.  4.100 /  3.  4.100
[hls,applehttp @ 0x7fb600000e00] Opening 'test_single_file_mp4.m4s' for reading
    Last message repeated 1 times
Input #0, hls,applehttp, from 'test_single_file_mp4.m3u8':
  Duration: N/A, start: 28.000000, bitrate: 51 kb/s
  Program 0 
    Metadata:
      variant_bitrate : 0
    Stream #0:0: Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 51 kb/s, 30 fps, 30 tbr, 15360 tbn, 30720 tbc
    Metadata:
      variant_bitrate : 0
      compatible_brands: iso6mp41
      major_brand     : iso5
      minor_version   : 512
      encoder         : Lavf58.25.100
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
Output #0, null, to '/dev/null':
  Metadata:
    encoder         : Lavf58.25.100
    Stream #0:0: Video: wrapped_avframe, yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 30 fps, 30 tbn, 30 tbc
    Metadata:
      variant_bitrate : 0
      compatible_brands: iso6mp41
      major_brand     : iso5
      minor_version   : 512
      encoder         : Lavc58.43.100 wrapped_avframe
[hls,applehttp @ 0x7fb600000e00] Opening 'test_single_file_mp4.m4s' for reading
    Last message repeated 3 times
[h264 @ 0x7fb60100d600] Invalid NAL unit 0, skipping.
frame=  156 fps= 79 q=-0.0 size=N/A time=00:00:05.20 bitrate=N/A speed=2.64x    
[h264 @ 0x7fb60100dc00] Invalid NAL unit size (-503519357 > 3150).
[h264 @ 0x7fb60100dc00] Error splitting the input into NAL units.
[h264 @ 0x7fb601810800] Invalid NAL unit size (-1920240576 > 3132).
[h264 @ 0x7fb601810800] Error splitting the input into NAL units.
[h264 @ 0x7fb601002800] Invalid NAL unit size (-805746105 > 3132).
[h264 @ 0x7fb601002800] Error splitting the input into NAL units.
[h264 @ 0x7fb60100d000] Invalid NAL unit size (271284657 > 3114).
[h264 @ 0x7fb60100d000] Error splitting the input into NAL units.
[h264 @ 0x7fb60100d600] error while decoding MB 34 8, bytestream -6
[h264 @ 0x7fb60100d600] data partitioning is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[h264 @ 0x7fb60100d600] If you want to help, upload a sample of this file to ftp://upload.ffmpeg.org/incoming/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel at ffmpeg.org)
[h264 @ 0x7fb60100d600] concealing 2975 DC, 2975 AC, 2975 MV errors in I frame
Error while decoding stream #0:0: Invalid data found when processing input
[h264 @ 0x7fb60100d600] Invalid NAL unit size (453635919 > 3121).
[h264 @ 0x7fb60100d600] Error splitting the input into NAL units.
Error while decoding stream #0:0: Invalid data found when processing input
[h264 @ 0x7fb60100dc00] Invalid NAL unit size (-1955315659 > 3109).
[h264 @ 0x7fb60100dc00] Error splitting the input into NAL units.
Error while decoding stream #0:0: Invalid data found when processing input
[h264 @ 0x7fb601810800] Invalid NAL unit size (-653815108 > 3125).
[h264 @ 0x7fb601810800] Error splitting the input into NAL units.
[h264 @ 0x7fb601002800] Invalid NAL unit size (692335841 > 3116).
[h264 @ 0x7fb601002800] Error splitting the input into NAL units.
Error while decoding stream #0:0: Invalid data found when processing input
    Last message repeated 1 times
[h264 @ 0x7fb60100d000] Invalid NAL unit size (1717580837 > 3124).
[h264 @ 0x7fb60100d000] Error splitting the input into NAL units.
[h264 @ 0x7fb60100d600] Invalid NAL unit size (268134308 > 3108).
Error while decoding stream #0:0: Invalid data found when processing input
[h264 @ 0x7fb60100d600] Error splitting the input into NAL units.
Error while decoding stream #0:0: Invalid data found when processing input
[h264 @ 0x7fb60100dc00] Invalid NAL unit size (628277483 > 3107).
[h264 @ 0x7fb60100dc00] Error splitting the input into NAL units.
Error while decoding stream #0:0: Invalid data found when processing input
[h264 @ 0x7fb601810800] Invalid NAL unit size (1047275371 > 3104).
[h264 @ 0x7fb601810800] Error splitting the input into NAL units.
Error while decoding stream #0:0: Invalid data found when processing input
[h264 @ 0x7fb601002800] Invalid NAL unit size (616960908 > 3101).
[h264 @ 0x7fb601002800] Error splitting the input into NAL units.
[h264 @ 0x7fb60100d000] Invalid NAL unit size (-478814450 > 3116).
[h264 @ 0x7fb60100d000] Error splitting the input into NAL units.
Error while decoding stream #0:0: Invalid data found when processing input
    Last message repeated 1 times
[h264 @ 0x7fb60100d600] Invalid NAL unit size (580686732 > 3101).
[h264 @ 0x7fb60100d600] Error splitting the input into NAL units.
Error while decoding stream #0:0: Invalid data found when processing input
[h264 @ 0x7fb60100dc00] Invalid NAL unit size (318166377 > 3107).
[h264 @ 0x7fb60100dc00] Error splitting the input into NAL units.
Error while decoding stream #0:0: Invalid data found when processing input
[h264 @ 0x7fb601810800] Invalid NAL unit size (1949148582 > 3102).
[h264 @ 0x7fb601810800] Error splitting the input into NAL units.
Error while decoding stream #0:0: Invalid data found when processing input
[h264 @ 0x7fb601002800] Invalid NAL unit size (-94566289 > 3082).
[h264 @ 0x7fb601002800] Error splitting the input into NAL units.
Error while decoding stream #0:0: Invalid data found when processing input
[h264 @ 0x7fb60100d000] Invalid NAL unit size (-170880014 > 3110).
[h264 @ 0x7fb60100d000] Error splitting the input into NAL units.
Error while decoding stream #0:0: Invalid data found when processing input
[h264 @ 0x7fb60100d600] Invalid NAL unit size (-534996253 > 3117).
[h264 @ 0x7fb60100d600] Error splitting the input into NAL units.
Error while decoding stream #0:0: Invalid data found when processing input
[h264 @ 0x7fb60100dc00] Invalid NAL unit size (1305816859 > 3110).
[h264 @ 0x7fb60100dc00] Error splitting the input into NAL units.
[hls,applehttp @ 0x7fb600000e00] Opening 'test_single_file_mp4.m4s' for reading
    Last message repeated 36 times
Error while decoding stream #0:0: Invalid data found when processing input
    Last message repeated 1 times
frame=  159 fps= 13 q=-0.0 size=N/A time=00:00:05.30 bitrate=N/A speed=0.428x    
Error while decoding stream #0:0: Invalid data found when processing input
    Last message repeated 1 times
frame=  159 fps= 13 q=-0.0 Lsize=N/A time=00:00:05.30 bitrate=N/A speed=0.428x    
video:83kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
-------------- next part --------------
ffmpeg version N-92950-gc65c4ce773 Copyright (c) 2000-2019 the FFmpeg developers
  built with Apple LLVM version 10.0.0 (clang-1000.11.45.5)
  configuration: 
  libavutil      56. 25.100 / 56. 25.100
  libavcodec     58. 43.100 / 58. 43.100
  libavformat    58. 25.100 / 58. 25.100
  libavdevice    58.  6.101 / 58.  6.101
  libavfilter     7. 47.100 /  7. 47.100
  libswscale      5.  4.100 /  5.  4.100
  libswresample   3.  4.100 /  3.  4.100
Input #0, lavfi, from 'testsrc=duration=100:size=1280x720:rate=30':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: rawvideo (RGB[24] / 0x18424752), rgb24, 1280x720 [SAR 1:1 DAR 16:9], 30 tbr, 30 tbn, 30 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (h264_videotoolbox))
Press [q] to stop, [?] for help
[h264_videotoolbox @ 0x7fdc7b811800] Color range not set for yuv420p. Using MPEG range.
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m4s' for writing
Output #0, hls, to 'test_single_file_mp4.m3u8':
  Metadata:
    encoder         : Lavf58.25.100
    Stream #0:0: Video: h264 (h264_videotoolbox), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 30 fps, 15360 tbn, 30 tbc
    Metadata:
      encoder         : Lavc58.43.100 h264_videotoolbox
frame=   56 fps=0.0 q=-0.0 size=N/A time=00:00:01.76 bitrate=N/A speed= 3.5x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
    Last message repeated 1 times
frame=  152 fps=151 q=-0.0 size=N/A time=00:00:04.93 bitrate=N/A speed= 4.9x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
frame=  234 fps=155 q=-0.0 size=N/A time=00:00:07.66 bitrate=N/A speed=5.08x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
    Last message repeated 1 times
frame=  331 fps=165 q=-0.0 size=N/A time=00:00:10.90 bitrate=N/A speed=5.42x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
    Last message repeated 1 times
frame=  428 fps=170 q=-0.0 size=N/A time=00:00:14.13 bitrate=N/A speed=5.62x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
frame=  487 fps=161 q=-0.0 size=N/A time=00:00:16.06 bitrate=N/A speed=5.32x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
frame=  567 fps=161 q=-0.0 size=N/A time=00:00:18.73 bitrate=N/A speed=5.32x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
frame=  665 fps=165 q=-0.0 size=N/A time=00:00:22.00 bitrate=N/A speed=5.46x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
    Last message repeated 1 times
frame=  765 fps=169 q=-0.0 size=N/A time=00:00:25.33 bitrate=N/A speed= 5.6x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
    Last message repeated 1 times
frame=  863 fps=172 q=-0.0 size=N/A time=00:00:28.60 bitrate=N/A speed=5.68x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
frame=  951 fps=172 q=-0.0 size=N/A time=00:00:31.53 bitrate=N/A speed= 5.7x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
    Last message repeated 1 times
frame= 1039 fps=172 q=-0.0 size=N/A time=00:00:34.46 bitrate=N/A speed= 5.7x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
frame= 1109 fps=169 q=-0.0 size=N/A time=00:00:36.80 bitrate=N/A speed=5.62x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
    Last message repeated 1 times
frame= 1207 fps=171 q=-0.0 size=N/A time=00:00:40.06 bitrate=N/A speed=5.69x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
frame= 1307 fps=173 q=-0.0 size=N/A time=00:00:43.40 bitrate=N/A speed=5.75x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
    Last message repeated 1 times
frame= 1403 fps=174 q=-0.0 size=N/A time=00:00:46.60 bitrate=N/A speed=5.78x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
frame= 1500 fps=175 q=-0.0 size=N/A time=00:00:49.83 bitrate=N/A speed=5.82x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
    Last message repeated 1 times
frame= 1600 fps=177 q=-0.0 size=N/A time=00:00:53.16 bitrate=N/A speed=5.87x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
    Last message repeated 1 times
frame= 1699 fps=178 q=-0.0 size=N/A time=00:00:56.46 bitrate=N/A speed= 5.9x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
frame= 1799 fps=179 q=-0.0 size=N/A time=00:00:59.80 bitrate=N/A speed=5.94x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
    Last message repeated 1 times
frame= 1869 fps=177 q=-0.0 size=N/A time=00:01:02.10 bitrate=N/A speed=5.87x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
frame= 1948 fps=176 q=-0.0 size=N/A time=00:01:04.73 bitrate=N/A speed=5.84x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
    Last message repeated 1 times
frame= 2047 fps=177 q=-0.0 size=N/A time=00:01:08.03 bitrate=N/A speed=5.88x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
frame= 2146 fps=178 q=-0.0 size=N/A time=00:01:11.33 bitrate=N/A speed=5.91x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
    Last message repeated 1 times
frame= 2234 fps=178 q=-0.0 size=N/A time=00:01:14.26 bitrate=N/A speed= 5.9x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
frame= 2331 fps=178 q=-0.0 size=N/A time=00:01:17.50 bitrate=N/A speed=5.92x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
    Last message repeated 1 times
frame= 2420 fps=178 q=-0.0 size=N/A time=00:01:20.46 bitrate=N/A speed=5.92x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
frame= 2521 fps=179 q=-0.0 size=N/A time=00:01:23.83 bitrate=N/A speed=5.95x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
    Last message repeated 1 times
frame= 2623 fps=180 q=-0.0 size=N/A time=00:01:27.23 bitrate=N/A speed=5.98x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
    Last message repeated 1 times
frame= 2722 fps=180 q=-0.0 size=N/A time=00:01:30.53 bitrate=N/A speed=   6x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
frame= 2821 fps=181 q=-0.0 size=N/A time=00:01:33.83 bitrate=N/A speed=6.02x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
    Last message repeated 1 times
frame= 2921 fps=181 q=-0.0 size=N/A time=00:01:37.16 bitrate=N/A speed=6.04x    
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
[Parsed_testsrc_0 @ 0x7fdc7b50e5c0] EOF timestamp not reliable
[hls @ 0x7fdc7c800000] Opening 'test_single_file_mp4.m3u8.tmp' for writing
frame= 3000 fps=182 q=-0.0 Lsize=N/A time=00:01:40.00 bitrate=N/A speed=6.05x    
video:9258kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pEpkey.asc
Type: application/pgp-keys
Size: 5658 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-user/attachments/20190111/e402b40b/attachment.key>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pEpkey.asc
Type: application/pgp-keys
Size: 5657 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-user/attachments/20190111/e402b40b/attachment-0001.key>


More information about the ffmpeg-user mailing list