[FFmpeg-user] Live streaming from Flash Player 11 h264, dropping (almost) all frames.

Thu Dec 15 05:03:33 CET 2011

Hey, I'm putting together a webcam conferencing application, and I'm counting on using Flash Player 11 for encoding h264.

I'm running into difficulties getting ffmpeg to transcode the streams. After the first couple frames all of the rest are dropped.

I'm mostly interested in live transcoding for multi-bitrate delivery and picture in picture composition, but dumped streams reproduce the issue:
https://s3.amazonaws.com/ottw_dev/streamdump/dump.f4v (https://s3.amazonaws.com/ottw_dev/streamdump/dump.f4v?AWSAccessKeyId=AKIAI66KR7KZCRFPA3FA&Expires=1324005133&Signature=JUyVAUJ12EkfPsEznbruUl62mvE%3D)
($ rtmpdump --live -o dump.f4v -r rtmp://localhost/live/streampublisher/demo)

$ ffmpeg -loglevel debug -i dump.f4v out.mp4
ffmpeg version N-31338-gdcca7d6, Copyright (c) 2000-2011 the FFmpeg developers
  built on Dec 14 2011 15:42:53 with clang 2.1 (tags/Apple/clang-163.7.1)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD --enable-shared --enable-gpl --enable-version3 --enable-nonfree --enable-hardcoded-tables --cc=/usr/bin/clang --enable-libx264 --enable-libfaac --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libxvid --enable-librtmp --cc=clang
  libavutil    51. 32. 0 / 51. 32. 0
  libavcodec   53. 45. 0 / 53. 45. 0
  libavformat  53. 25. 0 / 53. 25. 0
  libavdevice  53.  4. 0 / 53.  4. 0
  libavfilter   2. 53. 0 /  2. 53. 0
  libswscale    2.  1. 0 /  2.  1. 0
  libpostproc  51.  2. 0 / 51.  2. 0
[flv @ 0x7f912203d800] Format flv probed with size=2048 and score=100
[h264 @ 0x7f912204e400] err{or,}_recognition separate: 1; 1
[h264 @ 0x7f912204e400] err{or,}_recognition combined: 1; 10001
[nellymoser @ 0x7f912203fc00] err{or,}_recognition separate: 1; 1
[nellymoser @ 0x7f912203fc00] err{or,}_recognition combined: 1; 10001
[nellymoser @ 0x7f912203fc00] Unsupported bit depth: 0
[flv @ 0x7f912203d800] All info found
rfps: 0.083333 0.003837
    Last message repeated 1 times
rfps: 0.166667 0.015349
    Last message repeated 1 times

Seems stream 0 codec frame rate differs from container frame rate: 48.00 (48/1) -> 0.08 (1/12)
Input #0, flv, from 'dump.f4v':
  Duration: 00:00:36.75, start: 0.000000, bitrate: 330 kb/s
    Stream #0:0, 42, 1/1000: Video: h264 (Baseline), yuv420p, 320x240 [SAR 1:1 DAR 4:3], 1/48, 0.08 tbr, 1k tbn, 48 tbc
    Stream #0:1, 56, 1/1000: Audio: nellymoser, 22050 Hz, mono, s16
File 'out.mp4' already exists. Overwrite ? [y/N] y
w:320 h:240 pixfmt:yuv420p tb:1/1000000 sar:1/1 sws_param:
[libx264 @ 0x7f912204c000] err{or,}_recognition separate: 1; 1
[libx264 @ 0x7f912204c000] err{or,}_recognition combined: 1; 10001
[libx264 @ 0x7f912204c000] using mv_range_thread = 40
[libx264 @ 0x7f912204c000] using SAR=1/1
[libx264 @ 0x7f912204c000] using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.1 Cache64
[libx264 @ 0x7f912204c000] profile High, level 1.2
[libx264 @ 0x7f912204c000] 264 - core 118 r8974+3 e361f4a - H.264/MPEG-4 AVC codec - Copyleft 2003-2011 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=1 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[libfaac @ 0x7f912204ce00] err{or,}_recognition separate: 1; 1
[libfaac @ 0x7f912204ce00] err{or,}_recognition combined: 1; 10001
[h264 @ 0x7f912204e400] err{or,}_recognition separate: 1; 10001
[h264 @ 0x7f912204e400] err{or,}_recognition combined: 1; 10001
[nellymoser @ 0x7f912203fc00] err{or,}_recognition separate: 1; 10001
[nellymoser @ 0x7f912203fc00] err{or,}_recognition combined: 1; 10001
[nellymoser @ 0x7f912203fc00] Unsupported bit depth: 0
Output #0, mp4, to 'out.mp4':
  Metadata:
    encoder         : Lavf53.25.0
    Stream #0:0, 0, 1/1: Video: h264 (![0][0][0] / 0x0021), yuv420p, 320x240 [SAR 1:1 DAR 4:3], 12/1, q=-1--1, 1 tbn, 0.08 tbc
    Stream #0:1, 0, 1/22050: Audio: aac (@[0][0][0] / 0x0040), 22050 Hz, mono, s16, 128 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (h264 -> libx264)
  Stream #0:1 -> #0:1 (nellymoser -> libfaac)
Press [q] to stop, [?] for help
[libx264 @ 0x7f912204c000] using mv_range_thread = 40
*** drop!
    Last message repeated 21 times
[h264 @ 0x7f912204e400] AVC: nal size 21102612
[h264 @ 0x7f912204e400] no frame!
Error while decoding stream #0:0
*** drop!
…

...
    Last message repeated 23 times
[h264 @ 0x7f912204e400] AVC: nal size 21102612
[h264 @ 0x7f912204e400] no frame!
Error while decoding stream #0:0
*** drop!
    Last message repeated 22 times
[h264 @ 0x7f912204e400] AVC: nal size 21102612
[h264 @ 0x7f912204e400] no frame!
Error while decoding stream #0:0
*** drop!
    Last message repeated 21 times
[libx264 @ 0x7f912204c000] frame=   0 QP=13.69 NAL=3 Slice:I Poc:0   I:300  P:0    SKIP:0    size=15365 bytes
[libx264 @ 0x7f912204c000] frame=   1 QP=14.37 NAL=2 Slice:P Poc:2   I:15   P:48   SKIP:237  size=454 bytes
[libx264 @ 0x7f912204c000] frame=   2 QP=15.27 NAL=2 Slice:P Poc:6   I:139  P:161  SKIP:0    size=11100 bytes
[libx264 @ 0x7f912204c000] frame=   3 QP=16.19 NAL=0 Slice:B Poc:4   I:134  P:137  SKIP:0    size=9986 bytes
[libx264 @ 0x7f912204c000] frame=   4 QP=16.07 NAL=2 Slice:P Poc:8   I:158  P:142  SKIP:0    size=8920 bytes
frame=    5 fps=  5 q=-1.0 Lsize=     221kB time=00:00:36.00 bitrate=  50.3kbits/s dup=0 drop=545     
video:45kB audio:171kB global headers:0kB muxing overhead 2.064660%
[libx264 @ 0x7f912204c000] frame I:1     Avg QP:13.69  size: 15365
[libx264 @ 0x7f912204c000] frame P:3     Avg QP:15.24  size:  6825
[libx264 @ 0x7f912204c000] frame B:1     Avg QP:16.19  size:  9986
[libx264 @ 0x7f912204c000] consecutive B-frames: 60.0% 40.0%  0.0%  0.0%
[libx264 @ 0x7f912204c000] mb I  I16..4: 28.7%  7.0% 64.3%
[libx264 @ 0x7f912204c000] mb P  I16..4:  5.0% 19.8%  9.9%  P16..4: 14.7% 15.3%  9.0%  0.0%  0.0%    skip:26.3%
[libx264 @ 0x7f912204c000] mb B  I16..4:  5.7% 21.0% 18.0%  B16..8: 17.3% 14.0%  7.4%  direct:16.7%  skip: 0.0%  L0: 9.7% L1:62.4% BI:28.0%
[libx264 @ 0x7f912204c000] 8x8 transform intra:35.1% inter:26.4%
[libx264 @ 0x7f912204c000] coded y,uvDC,uvAC intra: 82.1% 99.2% 95.0% inter: 53.3% 64.5% 57.0%
[libx264 @ 0x7f912204c000] i16 v,h,dc,p: 32% 24% 30% 14%
[libx264 @ 0x7f912204c000] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 25% 20% 29%  3%  3%  4%  5%  4%  6%
[libx264 @ 0x7f912204c000] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 29% 20% 15%  7%  6%  6%  5%  7%  5%
[libx264 @ 0x7f912204c000] i8c dc,h,v,p: 57% 14% 17% 12%
[libx264 @ 0x7f912204c000] Weighted P-Frames: Y:66.7% UV:66.7%
[libx264 @ 0x7f912204c000] ref P L0: 58.8% 17.4%  6.2% 14.9%  2.8%
[libx264 @ 0x7f912204c000] ref B L0: 73.6% 26.4%
[libx264 @ 0x7f912204c000] kb/s:6.11

If I try to pull the live stream and republish it to a different rtmp address like so:
ffmpeg -i 'rtmp://localhost/live/streampublisher/demo2 live=1' -acodec copy -vcodec copy -f flv rtmp://localhost/live/streampublisher/out

It errors out with:
[flv @ 0x2bf0da0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 160 >= 160

If I try to change the -vcodec it just drops all of the frames like for the dump.

Is there anything I can do to get this working? Has anyone been successful getting h264 from FP 11 to play nice with ffmpeg?