[FFmpeg-user] Video and audio out of sync after some time

[Nithiwat Maneesint]

Hi, I am quite new to ffmpeg, so I have no idea how to go on.

I am writing a Java application that receives video and audio data from
server.
The video is encoded in VP8 and the audio is Opus. I receive the data in
bytes
then I decode the video to YUV420p and the audio to pcm_s16le in Java,
then I send the decoded video and audio data (in bytes) to ffmpeg via
pipe:0 (video)
and a udp socket (audio).

The problem is that, the video has variable FPS, so after a few seconds,
the video
will go faster than the sound. As I am new to ffmpeg, I have no idea how to
properly
deal with this. I really am in need of suggestions. Followng is the command
I use and the output:

*Command:*
E:/ffmpeg/bin/ffmpeg.exe",
 "-y",
"-f", "rawvideo",
"-vcodec", "rawvideo",
"-s", "400x300",
"-r", "16",
"-pix_fmt", "rgb24",
"-i", "pipe:0", // pipe 0 for video
"-f", "s16le", // means 16bit input
"-acodec", "pcm_s16le", // means raw 16bit input
"-ar", "48000", // the input will have 48000 Hz
"-ac", "1", // the input will have 1 channel (mono)
"-i", "udp://127.0.0.1:" + port0 + "?listen", // means that the input will
arrive from the udp socket, audio 0
"-vcodec", "libvpx",
"-vf", "vflip",
"-vsync", "cfr",
"-acodec", "libvorbis",
"-start_at_zero",
"-f", "webm",
"http:\\localhost:8080\publish\first?password=secret"


*Output:*
 ffmpeg version N-76262-g2ccc1b3 Copyright (c) 2000-2015 the FFmpeg
developers
   built with gcc 5.2.0 (GCC)
   configuration: --enable-gpl --enable-version3 --disable-w32threads
--enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r
--enable-gnutls --enable-iconv --enable-libass --enable-libbluray
--enable-libbs2b --enable-libcaca --enable-libdcadec --enable-libfreetype
--enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug
--enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb
--enable-libopenjpeg --enable-libopus --enable-librtmp
--enable-libschroedinger --enable-libsoxr --enable-libspeex
--enable-libtheora --enable-libtwolame --enable-libvidstab
--enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis
--enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264
--enable-libx265 --enable-libxavs --enable-libxvid --enable-lzma
--enable-decklink --enable-zlib
   libavutil      55.  4.100 / 55.  4.100
   libavcodec     57.  9.100 / 57.  9.100
   libavformat    57. 11.100 / 57. 11.100
   libavdevice    57.  0.100 / 57.  0.100
   libavfilter     6. 13.100 /  6. 13.100
   libswscale      4.  0.100 /  4.  0.100
   libswresample   2.  0.100 /  2.  0.100
   libpostproc    54.  0.100 / 54.  0.100
 Input #0, rawvideo, from 'pipe:0':
   Duration: N/A, start: 0.000000, bitrate: 46080 kb/s
     Stream #0:0: Video: rawvideo (RGB[24] / 0x18424752), rgb24, 400x300,
46080 kb/s, 16 tbr, 16 tbn, 16 tbc
 Guessed Channel Layout for  Input Stream #1.0 : mono
 Input #1, s16le, from 'udp:\\127.0.0.1:50042?listen':
   Duration: N/A, bitrate: 768 kb/s
     Stream #1:0: Audio: pcm_s16le, 48000 Hz, 1 channels, s16, 768 kb/s
 [libvpx @ 0000004db840bf60] v1.4.0
 Output #0, webm, to 'http:\\localhost:8080\publish\first?password=secret':
   Metadata:
     encoder         : Lavf57.11.100
     Stream #0:0: Video: vp8 (libvpx), yuv420p, 400x300, q=-1--1, 200 kb/s,
16 fps, 1k tbn, 16 tbc
     Metadata:
       encoder         : Lavc57.9.100 libvpx
     Stream #0:1: Audio: vorbis (libvorbis), 48000 Hz, mono, fltp
     Metadata:
       encoder         : Lavc57.9.100 libvorbis
 Stream mapping:
   Stream #0:0 -> #0:0 (rawvideo (native) -> vp8 (libvpx))
   Stream #1:0 -> #0:1 (pcm_s16le (native) -> vorbis (libvorbis))
 [rawvideo @ 0000004db835ce60] Thread message queue blocking; consider
raising the thread_queue_size option (current value: 8)
 [s16le @ 0000004db83bfda0] Thread message queue blocking; consider raising
the thread_queue_size option (current value: 8)
 frame=   25 fps=0.0 q=0.0 size=      36kB time=00:00:01.56 bitrate=
187.8kbits/s
 frame=   34 fps= 33 q=0.0 size=      62kB time=00:00:02.12 bitrate=
238.0kbits/s
 frame=   42 fps= 28 q=0.0 size=      62kB time=00:00:02.62 bitrate=
192.7kbits/s
 frame=   49 fps= 24 q=0.0 size=      90kB time=00:00:03.06 bitrate=
240.5kbits/s
 frame=   57 fps= 22 q=0.0 size=      90kB time=00:00:03.56 bitrate=
206.8kbits/s
 frame=   66 fps= 22 q=0.0 size=     123kB time=00:00:04.12 bitrate=
245.1kbits/s
 frame=   74 fps= 21 q=0.0 size=     123kB time=00:00:04.62 bitrate=
218.6kbits/s
 frame=   82 fps= 20 q=0.0 size=     157kB time=00:00:05.12 bitrate=
250.9kbits/s
 frame=   90 fps= 20 q=0.0 size=     157kB time=00:00:05.62 bitrate=
228.6kbits/s
 frame=   98 fps= 19 q=0.0 size=     189kB time=00:00:06.12 bitrate=
252.3kbits/s
 frame=  106 fps= 19 q=0.0 size=     221kB time=00:00:06.62 bitrate=
273.4kbits/s
 frame=  115 fps= 19 q=0.0 size=     221kB time=00:00:07.18 bitrate=
252.0kbits/s
 frame=  122 fps= 19 q=0.0 size=     253kB time=00:00:07.62 bitrate=
271.9kbits/s
 frame=  131 fps= 18 q=0.0 size=     268kB time=00:00:08.18 bitrate=
268.0kbits/s
 frame=  139 fps= 18 q=0.0 size=     268kB time=00:00:08.68 bitrate=
252.5kbits/s
 frame=  144 fps= 18 q=0.0 size=     300kB time=00:00:09.00 bitrate=
273.4kbits/s
 frame=  150 fps= 17 q=0.0 size=     300kB time=00:00:09.37 bitrate=
262.5kbits/s
 frame=  159 fps= 17 q=0.0 size=     300kB time=00:00:09.93 bitrate=
247.6kbits/s
 frame=  165 fps= 17 q=0.0 size=     331kB time=00:00:10.31 bitrate=
262.8kbits/s
 frame=  180 fps= 18 q=0.0 size=     363kB time=00:00:11.25 bitrate=
264.5kbits/s
 frame=  188 fps= 18 q=0.0 size=     363kB time=00:00:11.75 bitrate=
253.3kbits/s
 frame=  192 fps= 17 q=0.0 size=     395kB time=00:00:12.00 bitrate=
269.9kbits/s