[FFmpeg-user] editing video stream with KLV metadata
Moritz Barsnick
barsnick at gmx.net
Mon May 7 15:33:13 EEST 2018
Hi Claire,
On Mon, May 07, 2018 at 10:41:44 +0000, Claire Mantel wrote:
> Yes, KLV is a standard for encoding data (https://en.wikipedia.org/wiki/KLV)
> I believe it's quite commonly used in commercial UAVs (drones) to tag the video with info like timestamp, GPS coordinates and pitch, roll and yaw of the camera.
I see.
> I tried the first command and it works, so thanks a lot!
Very nice, thanks for the feedback.
> So I guess it means ffmpeg is able to tell which KLV data is
> associated with which frame as the editing also edited the KLV stream
> properly.
That means that the data stream has timestamps as well (which seems
necessary, if you want to correlate changing GPS positions with a
recording).
> I'm also trying to output the KLV data in some text file, I tried
> ffmpeg -i video_out_2018_05_01_14_11_53.ts -f ffmetadata testMetadata.txt
> ffmpeg -i video_out_2018_05_01_14_11_53.ts -map 0:d -f ffmetadata testMetadata.txt
> ffmpeg -i video_out_2018_05_01_14_11_53.ts -c:d copy -map 0:d -f ffmetadata testMetadata.txt
> but I only get " ;FFMETADATA1 encoder=Lavf58.13.100 [STREAM]" in my testMetadata.txt file
Again, as far as ffmpeg is concerned, that KLV stream is a binary data
stream with unknown content. While that content may be "metadata", what
ffmpeg usually considers as metadata is additional data which is
"attached" to a stream, not contained. That's why your ffmpeg metadata
commands are to no avail.
> Would you have any input?
You need to access the binary data of the stream. One method is
outlined here:
https://stackoverflow.com/a/29461404/3974309
So you can dump that data to a file. As far as I know, it will lose its
timestamps though. (I *may* be wrong here.) You would need to read the
KLV specification to understand that data, ffmpeg doesn't.
There was once a suggestion for an "fftextdata" muxer (and demuxer)
which might suit your needs. It would dump each received packet with a
timestamp and the base64-encoded hex data payload. Again, you need to
interpret the data on your own.
The patch was never accepted (and perhaps was never meant to be):
https://ffmpeg.org/pipermail/ffmpeg-devel/2016-May/194445.html
I am attaching a rebased version, in case it's something which could
help you. I assume it doesn't help, as you actually want to have the
KLV data interpreted?
Med vennlig hilsen,
Moritz
-------------- next part --------------
diff -urN ffmpeg-snapshot-2018-02-07/doc/demuxers.texi ffmpeg-snapshot-2018-02-07-textdata/doc/demuxers.texi
--- ffmpeg-snapshot-2018-02-07/doc/demuxers.texi 2018-01-14 18:20:06.000000000 +0100
+++ ffmpeg-snapshot-2018-02-07-textdata/doc/demuxers.texi 2018-02-08 17:25:37.000000000 +0100
@@ -471,6 +471,46 @@
@end example
@end itemize
+ at anchor{fftextdata}
+ at section fftextdata, fftd
+
+FFmpeg text data demuxer.
+
+This special demuxer allows to read serialized data base64-encoded and
+remux it. It is especially useful for injecting opaque data streams.
+
+The fftextdata bytestream consists of a sequence of packets. Each
+packet starts with a timestamps expressed in a format recognized by
+FFmpeg (see
+ at ref{time duration syntax,,the Time duration section in the ffmpeg-utils(1) manual,ffmpeg-utils})
+followed by a sequence of spaces and the base64 encoded data for the
+packet, terminated by ";". The data representation may contain
+interleaved space characters (a space, a tab, or a newline) which are
+ignored.
+
+At the moment a single stream can be represented by an fftextdata
+bytestream.
+
+If an input filename is "fftextdata" or "fftd" then the file format is
+recognized as fftextdata.
+
+ at subsection Options
+ at table @option
+ at item codec_name
+Set the codec name for the packets data.
+ at end table
+
+ at subsection Examples
+
+ at itemize
+ at item
+Inject timed_id3 packed data stored into the data.fftd file into the
+output file.
+ at example
+ffmpeg -i input.mp4 -codec_name timed_id3 -f fftextdata -i data.fftd -y -map 0 -map 1 -c copy output.ts
+ at end example
+ at end itemize
+
@section libgme
The Game Music Emu library is a collection of video game music file emulators.
diff -urN ffmpeg-snapshot-2018-02-07/doc/muxers.texi ffmpeg-snapshot-2018-02-07-textdata/doc/muxers.texi
--- ffmpeg-snapshot-2018-02-07/doc/muxers.texi 2018-01-24 18:20:05.000000000 +0100
+++ ffmpeg-snapshot-2018-02-07-textdata/doc/muxers.texi 2018-02-08 17:25:37.000000000 +0100
@@ -263,6 +263,37 @@
When no assignment is defined, this defaults to an AdaptationSet for each stream.
@end table
+ at section fftextdata, fftd
+
+FFmpeg text data muxer.
+
+The fftextdata bytestream consists of a sequence of packets. Each
+packet starts with a timestamps expressed in a format recognized by
+FFmpeg (see
+ at ref{time duration syntax,,the Time duration section in the ffmpeg-utils(1) manual,ffmpeg-utils})
+followed by a sequence of spaces and the base64 encoded data for the
+packet, terminated by ";". The data representation may contain
+interleaved space characters (a space, a tab, or a newline) which are
+ignored.
+
+At the moment only a single stream can be represented by an fftextdata
+bytestream.
+
+This muxer can be used to reinject the stream (e.g. a data stream) in
+a different output, or to provide serialized data of the encoded data.
+
+The output can then be read using the fftextdata demuxer.
+
+ at subsection Examples
+
+ at itemize
+ at item
+Store a data stream to an output file:
+ at example
+ffmpeg -i INPUT -codec copy -map 0 -an -vn data.fftd
+ at end example
+ at end itemize
+
@anchor{framecrc}
@section framecrc
diff -urN ffmpeg-snapshot-2018-02-07/libavformat/allformats.c ffmpeg-snapshot-2018-02-07-textdata/libavformat/allformats.c
--- ffmpeg-snapshot-2018-02-07/libavformat/allformats.c 2018-02-07 18:20:05.000000000 +0100
+++ ffmpeg-snapshot-2018-02-07-textdata/libavformat/allformats.c 2018-02-08 17:26:22.000000000 +0100
@@ -127,6 +127,8 @@
extern AVOutputFormat ff_f4v_muxer;
extern AVInputFormat ff_ffmetadata_demuxer;
extern AVOutputFormat ff_ffmetadata_muxer;
+extern AVInputFormat ff_fftextdata_demuxer;
+extern AVOutputFormat ff_fftextdata_muxer;
extern AVOutputFormat ff_fifo_muxer;
extern AVOutputFormat ff_fifo_test_muxer;
extern AVInputFormat ff_filmstrip_demuxer;
diff -urN ffmpeg-snapshot-2018-02-07/libavformat/fftextdatadec.c ffmpeg-snapshot-2018-02-07-textdata/libavformat/fftextdatadec.c
--- ffmpeg-snapshot-2018-02-07/libavformat/fftextdatadec.c 1970-01-01 01:00:00.000000000 +0100
+++ ffmpeg-snapshot-2018-02-07-textdata/libavformat/fftextdatadec.c 2018-02-08 17:25:37.000000000 +0100
@@ -0,0 +1,212 @@
+/*
+ * Copyright (c) 2016 Stefano Sabatini
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * timestamped data virtual demuxer
+ */
+
+#include "libavutil/base64.h"
+#include "libavutil/bprint.h"
+#include "libavutil/opt.h"
+#include "libavutil/parseutils.h"
+#include "avformat.h"
+#include "internal.h"
+
+typedef struct {
+ const AVClass *class; /**< Class for private options. */
+ int nb_packets;
+ AVBPrint bp;
+ const char *codec_name;
+} FFTextdataContext;
+
+av_cold static int fftextdata_read_close(AVFormatContext *avctx)
+{
+ FFTextdataContext *td = avctx->priv_data;
+
+ av_bprint_finalize(&td->bp, NULL);
+ return 0;
+}
+
+av_cold static int fftextdata_read_header(AVFormatContext *s)
+{
+ FFTextdataContext *td = s->priv_data;
+ AVStream *st;
+ const AVCodecDescriptor *cd;
+
+ st = avformat_new_stream(s, NULL);
+ if (!st)
+ return AVERROR(ENOMEM);
+
+ cd = avcodec_descriptor_get_by_name(td->codec_name);
+ if (!cd) {
+ av_log(s, AV_LOG_ERROR, "Impossible to find a codec with name '%s'\n",
+ td->codec_name);
+ return AVERROR(EINVAL);
+ }
+
+ st->codecpar->codec_type = cd->type;
+ st->codecpar->codec_id = cd->id;
+ avpriv_set_pts_info(st, 64, 1, 1000000);
+
+ av_bprint_init(&(td->bp), 0, 1);
+ td->nb_packets = 0;
+
+ return 0;
+}
+
+static inline int is_space(char c)
+{
+ return c == ' ' || c == '\t' || c == '\r' || c == '\n';
+}
+
+static int read_word(AVIOContext *avio, AVBPrint *bp)
+{
+ int c;
+
+ av_bprint_clear(bp);
+
+ /* skip spaces */
+ do {
+ c = avio_r8(avio);
+ if (!c)
+ goto end;
+ } while (is_space(c));
+
+ /* read word */
+ av_bprint_chars(bp, c, 1);
+ do {
+ c = avio_r8(avio);
+ if (!c)
+ goto end;
+ if (is_space(c)) {
+ avio_skip(avio, -1);
+ goto end;
+ }
+ av_bprint_chars(bp, c, 1);
+ } while (1);
+
+end:
+ return bp->len;
+}
+
+static int read_data(AVIOContext *avio, AVBPrint *bp)
+{
+ int c;
+
+ av_bprint_clear(bp);
+
+ /* skip spaces */
+ do {
+ c = avio_r8(avio);
+ if (!c)
+ goto end;
+ } while (is_space(c));
+
+ /* read data chunk */
+ av_bprint_chars(bp, c, 1);
+ do {
+ c = avio_r8(avio);
+ if (!c || c == ';')
+ goto end;
+ if (is_space(c)) {
+ continue;
+ }
+ av_bprint_chars(bp, c, 1);
+ } while (1);
+
+end:
+ return bp->len;
+}
+
+static int fftextdata_read_packet(AVFormatContext *s, AVPacket *pkt)
+{
+ FFTextdataContext *td = s->priv_data;
+ AVIOContext *avio = s->pb;
+ int ret;
+ AVBPrint *bp = &(td->bp);
+
+ pkt->pos = avio_tell(avio);
+
+ /* read PTS */
+ ret = read_word(avio, bp);
+ if (ret == 0)
+ return AVERROR_EOF;
+
+ ret = av_parse_time(&pkt->pts, bp->str, 1);
+ if (ret < 0) {
+ av_log(s, AV_LOG_ERROR, "Invalid time specification '%s' for data packet #%d\n",
+ bp->str, td->nb_packets);
+ return ret;
+ }
+
+ ret = read_data(avio, bp);
+ if (ret == 0) {
+ av_log(s, AV_LOG_WARNING, "Incomplete packet #%d with no data at the end of the data stream\n",
+ td->nb_packets);
+ return AVERROR_EOF;
+ }
+
+ pkt->size = AV_BASE64_DECODE_SIZE(ret);
+ pkt->data = av_malloc(pkt->size);
+ if (ret < 0)
+ return ret;
+
+ ret = av_base64_decode(pkt->data, bp->str, pkt->size);
+ if (ret < 0) {
+ av_freep(&pkt->data);
+ return ret;
+ }
+
+ pkt->size = ret;
+ pkt->flags |= AV_PKT_FLAG_KEY;
+ td->nb_packets++;
+
+ return ret;
+}
+
+#define OFFSET(x) offsetof(FFTextdataContext, x)
+
+#define D AV_OPT_FLAG_DECODING_PARAM
+
+#define OFFSET(x) offsetof(FFTextdataContext, x)
+
+static const AVOption options[] = {
+ { "codec_name", "set output codec name", OFFSET(codec_name), AV_OPT_TYPE_STRING, {.str = "bin_data"}, CHAR_MIN, CHAR_MAX, D },
+ { NULL },
+};
+
+static const AVClass fftextdata_class = {
+ .class_name = "fftexdata demuxer",
+ .item_name = av_default_item_name,
+ .option = options,
+ .version = LIBAVUTIL_VERSION_INT,
+};
+
+AVInputFormat ff_fftextdata_demuxer = {
+ .name = "fftextdata",
+ .long_name = NULL_IF_CONFIG_SMALL("Timestamped data virtual demuxer"),
+ .extensions = "fftextdata,fftd",
+ .priv_data_size = sizeof(FFTextdataContext),
+ .read_header = fftextdata_read_header,
+ .read_packet = fftextdata_read_packet,
+ .read_close = fftextdata_read_close,
+ .priv_class = &fftextdata_class,
+};
diff -urN ffmpeg-snapshot-2018-02-07/libavformat/fftextdataenc.c ffmpeg-snapshot-2018-02-07-textdata/libavformat/fftextdataenc.c
--- ffmpeg-snapshot-2018-02-07/libavformat/fftextdataenc.c 1970-01-01 01:00:00.000000000 +0100
+++ ffmpeg-snapshot-2018-02-07-textdata/libavformat/fftextdataenc.c 2018-02-08 17:25:37.000000000 +0100
@@ -0,0 +1,103 @@
+/*
+ * Copyright (c) 2016 Stefano Sabatini
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * timestamped data virtual muxer
+ */
+
+#include "avformat.h"
+#include "libavutil/base64.h"
+
+typedef struct {
+ uint8_t *buf;
+ size_t buf_size;
+} FFTextdataContext;
+
+static int fftextdata_write_header(AVFormatContext *s)
+{
+ FFTextdataContext *td = s->priv_data;
+
+ td->buf = NULL;
+ td->buf_size = 0;
+
+ return 0;
+}
+
+static int fftextdata_write_trailer(AVFormatContext *s)
+{
+ FFTextdataContext *td = s->priv_data;
+
+ av_freep(&td->buf);
+ td->buf_size = 0;
+
+ return 0;
+}
+
+static int fftextdata_write_packet(AVFormatContext *s, AVPacket *pkt)
+{
+ FFTextdataContext *td = s->priv_data;
+ char ts[32];
+ size_t encoded_data_size;
+ AVStream *st = s->streams[pkt->stream_index];
+ int64_t pts = pkt->pts;
+ double secs;
+ int hours, mins;
+
+ if (st->start_time != AV_NOPTS_VALUE)
+ pts += st->start_time;
+
+ secs = (double)pkt->pts * av_q2d(st->time_base);
+ mins = (int)secs / 60;
+ secs = secs - mins * 60;
+ hours = mins / 60;
+ mins %= 60;
+ snprintf(ts, sizeof(ts), "%d:%02d:%09.6f", hours, mins, secs);
+ avio_put_str(s->pb, ts);
+ avio_skip(s->pb, -1);
+ avio_w8(s->pb, '\n');
+
+ encoded_data_size = AV_BASE64_SIZE(pkt->size);
+ if (encoded_data_size > td->buf_size) {
+ td->buf = av_realloc_f(td->buf, encoded_data_size, 1);
+ if (!td->buf)
+ return AVERROR(ENOMEM);
+ td->buf_size = encoded_data_size;
+ }
+
+ av_base64_encode(td->buf, td->buf_size, pkt->data, pkt->size);
+ avio_put_str(s->pb, td->buf);
+ avio_skip(s->pb, -1);
+
+ avio_put_str(s->pb, "\n;\n");
+ avio_skip(s->pb, -1);
+
+ return 0;
+}
+
+AVOutputFormat ff_fftextdata_muxer = {
+ .name = "fftextdata",
+ .long_name = NULL_IF_CONFIG_SMALL("Timestamped data virtual muxer"),
+ .extensions = "fftextdata,fftd",
+ .priv_data_size = sizeof(FFTextdataContext),
+ .write_header = fftextdata_write_header,
+ .write_packet = fftextdata_write_packet,
+ .write_trailer = fftextdata_write_trailer,
+};
diff -urN ffmpeg-snapshot-2018-02-07/libavformat/Makefile ffmpeg-snapshot-2018-02-07-textdata/libavformat/Makefile
--- ffmpeg-snapshot-2018-02-07/libavformat/Makefile 2018-02-07 18:20:05.000000000 +0100
+++ ffmpeg-snapshot-2018-02-07-textdata/libavformat/Makefile 2018-02-08 17:25:37.000000000 +0100
@@ -163,6 +163,8 @@
OBJS-$(CONFIG_EPAF_DEMUXER) += epafdec.o pcm.o
OBJS-$(CONFIG_FFMETADATA_DEMUXER) += ffmetadec.o
OBJS-$(CONFIG_FFMETADATA_MUXER) += ffmetaenc.o
+OBJS-$(CONFIG_FFTEXTDATA_DEMUXER) += fftextdatadec.o
+OBJS-$(CONFIG_FFTEXTDATA_MUXER) += fftextdataenc.o
OBJS-$(CONFIG_FIFO_MUXER) += fifo.o
OBJS-$(CONFIG_FIFO_TEST_MUXER) += fifo_test.o
OBJS-$(CONFIG_FILMSTRIP_DEMUXER) += filmstripdec.o
More information about the ffmpeg-user
mailing list