[Libav-user] Syncing a seperate audio and video stream

[Ludwig Ertl]

Hi,

I want to use ffmpeg to decode Audio and video streams by a
Trendnet TV-IP512P IP Camera in order to feed it to ffserver for streaming.
I found a document on the Internet which is describing the video and
audio format container used by the IP camera:
http://www.paillassou.com/DCS-2121/CGI_2121.pdf
See Section 8.2 (Advanced ip-Camera Stream(ACS) Header.

Unfortunately, Audio and Video streams are served seperately under different
URLs (See Section 4.1.5 and 4.1.6): /video/ACVS.cgi and /audio/ACAS.cgi

Now I wrote 2 decoding plugins for libavformat (which I'd like to
contribute to libavformat once they are working as expected), which I have
attached in this message.
They basically work fine, but as I'm completely new to libav, I don't have
an idea how to sync those 2 streams together.
Currently, what I'm doing for testing is:

ffmpeg -i http://admin:xxx@192.168.210.69/video/ACVS.cgi -i
http://admin:xxx@192.168.210.69/audio/ACAS.cgi test.mpg

So I have 2 seperate streams which are of course out of sync.
There is a timestamp field in the frame-Header of each audio/videoframe,
which
is just a unix-timestamp with msec precision, both from the same clock
source.
So this information could be used to sync the streams, but I have no clue
how
this could possibly work, as Audio- and Videodecoder plugins don't know any-
thing from each other and even if they would (via an external variable or
some
ugly hack like that), I don't have a clue how to sync them. I suspect that
it
may have something to do with PTS and DTS timestamps, but I don't know how
they are used for audio and video sync in seperate streams.

Can you give me some hints on how to solve this problem? I can provide
sample
data from the camera, if you want.
As you may see from the code, I'm also quite unsure how to correctly
calculate
PTS (and possibly DTS) values, but the current code at least somehow works,
if I don't use any seeking or offsets (i.e. ffmpeg -ss parameter).

I hope that there is a possibility for synchronisation.
What I have already tried was using the clock as PTS for both audio and
video:

av_set_pts_info(st, 64, 1, 1000000);  /* 64 bits pts in us */
pkt->pts = ac->hdr.ulTimeSec * 1000000LL + ac->hdr.ulTimeUSec;

But this just resulted in a totally garbled video stream.

Regards,
DI (FH) Ludwig Ertl
Software Engineering
Computer Software Production GmbH
Fockygasse 4
1120 Wien
Web: http://www.csp.at
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: acsdec.c
URL: <http://ffmpeg.org/pipermail/libav-user/attachments/20120320/0024d022/attachment.c>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: acsvdec.c
URL: <http://ffmpeg.org/pipermail/libav-user/attachments/20120320/0024d022/attachment-0001.c>