[Libav-user] security camera app: bookmark+seek mp4 by wall-clock? low latency? visual timestamps?

[Alex Cohn]

On Mon, Aug 29, 2011 at 04:40, Camera Man <i.like.privacy.too at gmail.com> wrote:

> (a) is there a general way to bookmark frames while writing a file in such a
> way that I can seek to them directly when playing, without searching? pts is
> NOT the answer, as explained below.
>
> This is only relevant for key frames, but yes, you can. When writing
> the stream, you can embed a copy of SPS/PPS preceeding every key
> frame. Keep the file offset for this copy of SPS in your database, and
> when you need it, you can simply start reading the file from this
> offset.
>
> As I mentioned in my original post, this works fine for .h264 file, but for
> some reason not for .avi or .mp4 - perhaps I am missing some magic
> incantation that someone on the list is aware of?

I am not sure about .avi files, but for .mp4 the file header should
have an index to all key frames. This makes the search for a key frame
relatively fast, but not immediate, because the index does not include
(alas!) the byte offset of the frame.

OTOH, if you are anyway responsible for writing the file, what makes
you use the mov format (.mp4)? Maybe you can write to copies of the
same video stream, one as .mp4 and one as .h264?

> (c) is there a way to force "low delay" handling of a stream despite its
> SPS/PPS description? for some cameras, video decoding is lagging 4 frames
> after the packet arrival, which - at 5fps, is almost 1 second delay. (one
> suggestion given below, looking for more options)
>
> Why can't you fix the SPS to require low delay?
>
> I can (and probably will), but that requires bit manipulation that I was
> hoping to avoid; I hoped there was a way to do it without patching either
> libavcodec or the bitstream.

That's a good point. Actually you can try to provide your substitute function

   int ff_h264_get_profile(SPS *sps) { return FF_PROFILE_H264_CONSTRAINED; }

You can either convince the linker to pick up your version of this
function (depends on your OS and toolchain), or if you are not worried
about the LGPL restrictions at this moment you can simply remove the
implementation form libavcodec/h264.c.

Maybe, the correct value for your stream will be
FF_PROFILE_H264_BASELINE, not *_CONSTRAINED. Both constants are
defined in libavcodec/avcodec.h.

> (d) is there a way, other than subtitles, to add a visual timestamp to the
> file while writing it, without decoding+overlaying+reencoding?
>
> What's wrong with using subtitles?
>
> Nothing wrong, and is probably what I am going to do; It would require
> adding a different subtitle per frame. I was hoping there's a standard or
> accepted way to do this that I was unaware of - I'm sure I'm not the first
> person to run into this.

This request of VISUAL timestamp leaves no choice. You can certainly
display the timestamp in your video player without subtitles, based on
the packet info alone. Many existing video players actually provide
for such capability. But I don't think there is a way to set up a
movie (in any format) to request all players to display the timestamp.
Actually, there is not way to force all video players to display your
subtitles.. Which means that the only plausible way to enforce visual
timestamps is with decoding/overlay/encoding.

BR,
Alex