[FFmpeg-devel] [PATCH] movie video source
Michael Niedermayer
michaelni
Fri Jan 28 20:49:55 CET 2011
On Fri, Jan 28, 2011 at 12:27:45AM +0100, Stefano Sabatini wrote:
> On date Sunday 2011-01-16 19:35:33 +0100, Stefano Sabatini encoded:
> > On date Thursday 2011-01-06 22:55:18 +0100, Michael Niedermayer encoded:
> > > On Thu, Jan 06, 2011 at 09:43:00PM +0100, Stefano Sabatini wrote:
> > > > On date Tuesday 2011-01-04 23:48:24 +0100, Stefano Sabatini encoded:
> > > > > On date Tuesday 2011-01-04 18:48:48 +0100, Nicolas George encoded:
> > > > > > Le quintidi 15 niv?se, an CCXIX, Stefano Sabatini a ?crit?:
> > > > > > > What do you suggest to do, to make the cmdutils.c PtsCorrectionContext
> > > > > > > API public and use it (av_pts_correction_init(),
> > > > > > > av_pts_correction_guess()) or to duplicate the code?
> > > > > >
> > > > > > Something like that was evoked three weeks ago in the thread about the API
> > > > > > example for lavfi. Since then, I thought I might try to implement it.
> > > > > >
> > > > > > My idea was to add the PtsCorrectionContext structure to the end of
> > > > > > AVCodecContext and add some sort of avcodec_set_stream_time_base function:
> > > > > > this way, avcodec_decode_video2 would be able to use the pts and dts fields
> > > > > > in AVPacket directly without depending on libavformat.
> > > > >
> > > > > My idea was to implement a dumb transposition of the correct_pts API
> > > > > in cmdutils.c:
> > > > >
> > > > > AVPTSCorrectionContext
> > > > > void av_pts_correction_init(AVPTSCorrectionContext *ctx);
> > > > > int64_t av_pts_correction_guess(AVPTSCorrectionContext *ctx, int64_t pts, int64_t dts);
> > > > >
> > > > > this would be defined in libavcodec/avcodec.h (as I don't see any
> > > > > dependency on libavformat).
> > > >
> > > > Michael? (I'm not going to even start coding if the design will be
> > > > rejected).
> > >
> > > What is missing primarely is that pts from the demuxer must be routed through
> > > the decoders reordered_opaque. (that is current API)
> > > That is needed or things plain and simple are wrong.
> > >
> > > after that we are in API change land and i dont think this is a prerequesit to
> > > this patch, just a nice to have.
> > >
> > > The idea of putting a opaque PTSCorrectionContext in AVCodecContext and using
> > > the AVPacket.pts/dts for it and exporting the resulting single
> > > (stream) timestamp in AVFrame sounds very nice.
> > > Why a avcodec_set_stream_time_base() would be needed is not clear to me though
> > >
> > > About making the current cmdutils code public, i think Nicolas idea is much
> > > nicer and simpler from a user applications point of view, of course it is more
> > > work to implement though ..
> >
> > Updated work in progress, I managed the PTS correction stuff the ugly
> > way, simply copying all the logic from ffplay.c/cmdutils.c.
> >
> > Also I'm not sure that the way movie->picref is created with
> > avfilter_get_video_buffer_ref_from_arrays() is safe.
>
> Updated with some fixes, depends on the rawdec.c patch I posted today.
>
> Also I don't like at all the decoder_reorder_pts option as it sounds
> overly complicated, and I would rather prefer an API solution (e.g. to
> push the PtsCorrectionContext thing down into the library).
>
> Nicolas, Michael, everyone... suggestions?
> --
> FFmpeg = Fierce and Foolish Most Portable Evangelical Guru
> doc/filters.texi | 54 +++++++
> libavfilter/Makefile | 2
> libavfilter/allfilters.c | 1
> libavfilter/vsrc_movie.c | 350 +++++++++++++++++++++++++++++++++++++++++++++++
> 4 files changed, 407 insertions(+)
> 9de393d4e8b879af118d878e73860c052715f6f2 0007-Add-movie-video-source.patch
> From a018b4bbd785d6fa7ccc8efa9a68198e4ab20ac8 Mon Sep 17 00:00:00 2001
> From: Stefano Sabatini <stefano.sabatini-lala at poste.it>
> Date: Sun, 31 Oct 2010 17:37:10 +0100
> Subject: [PATCH] Add movie video source.
>
> ---
> doc/filters.texi | 54 +++++++
> libavfilter/Makefile | 2 +
> libavfilter/allfilters.c | 1 +
> libavfilter/vsrc_movie.c | 350 ++++++++++++++++++++++++++++++++++++++++++++++
> 4 files changed, 407 insertions(+), 0 deletions(-)
> create mode 100644 libavfilter/vsrc_movie.c
>
> diff --git a/doc/filters.texi b/doc/filters.texi
> index 3842886..c46a82b 100644
> --- a/doc/filters.texi
> +++ b/doc/filters.texi
> @@ -1088,6 +1088,60 @@ to the pad with identifier "in".
> "color=red@@0.2:qcif:10 [color]; [in][color] overlay [out]"
> @end example
>
> + at section movie
> +
> +Read a video stream from a movie container.
> +
> +It accepts the syntax: @var{movie_name}[@var{options}] where
> + at var{movie_name} is the name of the reource to read (not necessarily a
> +file but also a device or a stream accessed through some protocol),
> +
> +and @var{options} is a sequence of @var{key}=@var{value} pairs,
> +separated by ":".
> +
> +The description of the accepted options follows.
> +
> + at table @option
> +
> + at item format_name, f
> +Specifies the format assumed for the movie to read, and can be either
> +the name of a container or an input device. If not specified the
> +format is guessed from @var{movie_name} or by probing.
> +
> + at item seek_point, sp
> +Specifies the seek point in seconds, the frames will be output
> +starting from this seek point, the parameter is evaluated with
> + at code{av_strtod} so the numerical value may be suffixed by an IS
> +postfix.
> +
> + at item stream_index, si
> +Specifies the index of the video stream to read. If the value is -1,
> +the best suited video stream will be automatically selected. Default
> +value is "-1".
> +
> + at end table
> +
> +This filter allows to overlay a second movie on top of main input of
> +a filtergraph as shown in this graph:
> + at example
> +input -----------> deltapts0 --> overlay --> output
> + ^
> + |
> +movie --> scale--> deltapts1 -------+
> + at end example
> +
> +Some examples follow:
> + at example
> +# skip 3.2 seconds from the start of the avi file in.avi, and overlay it
> +# on top of the input labelled as "in".
> +"movie=in.avi:seek_point=3.2, scale=180:-1, setpts=PTS-STARTPTS [movie]; [in] setpts=PTS-START_PTS, [movie] overlay=16:16 [out]"
> +
> +# read from a video4linux2 device, and overlay it on top of the input
> +# labelled as "in"
> +"movie=/dev/video0:f=video4linux2, scale=180:-1, setpts=PTS-STARTPTS [movie]; [in] setpts=PTS-START_PTS, [movie] overlay=16:16 [out]"
> +
> + at end example
> +
> @section nullsrc
>
> Null video source, never return images. It is mainly useful as a
> diff --git a/libavfilter/Makefile b/libavfilter/Makefile
> index 9304c19..1ac9b0c 100644
> --- a/libavfilter/Makefile
> +++ b/libavfilter/Makefile
> @@ -2,6 +2,7 @@ include $(SUBDIR)../config.mak
>
> NAME = avfilter
> FFLIBS = avcore avutil
> +FFLIBS-$(CONFIG_MOVIE_FILTER) += avformat avcodec
> FFLIBS-$(CONFIG_SCALE_FILTER) += swscale
>
> HEADERS = avfilter.h avfiltergraph.h
> @@ -51,6 +52,7 @@ OBJS-$(CONFIG_YADIF_FILTER) += vf_yadif.o
> OBJS-$(CONFIG_BUFFER_FILTER) += vsrc_buffer.o
> OBJS-$(CONFIG_COLOR_FILTER) += vf_pad.o
> OBJS-$(CONFIG_FREI0R_SRC_FILTER) += vf_frei0r.o
> +OBJS-$(CONFIG_MOVIE_FILTER) += vsrc_movie.o
> OBJS-$(CONFIG_NULLSRC_FILTER) += vsrc_nullsrc.o
>
> OBJS-$(CONFIG_NULLSINK_FILTER) += vsink_nullsink.o
> diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
> index b3e6fc5..29a9d68 100644
> --- a/libavfilter/allfilters.c
> +++ b/libavfilter/allfilters.c
> @@ -72,6 +72,7 @@ void avfilter_register_all(void)
> REGISTER_FILTER (BUFFER, buffer, vsrc);
> REGISTER_FILTER (COLOR, color, vsrc);
> REGISTER_FILTER (FREI0R, frei0r_src, vsrc);
> + REGISTER_FILTER (MOVIE, movie, vsrc);
> REGISTER_FILTER (NULLSRC, nullsrc, vsrc);
>
> REGISTER_FILTER (NULLSINK, nullsink, vsink);
> diff --git a/libavfilter/vsrc_movie.c b/libavfilter/vsrc_movie.c
> new file mode 100644
> index 0000000..c55e65d
> --- /dev/null
> +++ b/libavfilter/vsrc_movie.c
> @@ -0,0 +1,350 @@
> +/*
> + * Copyright (c) 2010 Stefano Sabatini
> + * Copyright (c) 2008 Victor Paesa
> + *
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> + */
> +
> +/**
> + * @file
> + * movie video source
> + *
> + * @todo use direct rendering (no allocation of a new frame)
> + * @todo support more than one output stream
> + */
> +
> +#include <float.h>
> +#include "libavutil/avstring.h"
> +#include "libavutil/opt.h"
> +#include "libavcore/imgutils.h"
> +#include "libavformat/avformat.h"
> +#include "avfilter.h"
> +
> +/* #define DEBUG */
> +
> +typedef struct {
> + int64_t num_faulty_pts; /// number of incorrect PTS values so far
> + int64_t num_faulty_dts; /// number of incorrect DTS values so far
> + int64_t last_pts; /// PTS of the last frame
> + int64_t last_dts; /// DTS of the last frame
> +} PtsCorrectionContext;
> +
> +static void init_pts_correction(PtsCorrectionContext *ctx)
> +{
> + ctx->num_faulty_pts = ctx->num_faulty_dts = 0;
> + ctx->last_pts = ctx->last_dts = INT64_MIN;
> +}
> +
> +static int64_t guess_correct_pts(PtsCorrectionContext *ctx,
> + int64_t reordered_pts, int64_t dts)
> +{
> + int64_t pts = AV_NOPTS_VALUE;
> +
> + if (dts != AV_NOPTS_VALUE) {
> + ctx->num_faulty_dts += dts <= ctx->last_dts;
> + ctx->last_dts = dts;
> + }
> + if (reordered_pts != AV_NOPTS_VALUE) {
> + ctx->num_faulty_pts += reordered_pts <= ctx->last_pts;
> + ctx->last_pts = reordered_pts;
> + }
> + if ((ctx->num_faulty_pts<=ctx->num_faulty_dts || dts == AV_NOPTS_VALUE)
> + && reordered_pts != AV_NOPTS_VALUE)
> + pts = reordered_pts;
> + else
> + pts = dts;
> +
> + return pts;
> +}
cant we wait for nicolas implementing this cleanly instead of duplicating
it here?
You can just throw this out and use pts if available and if not dts
until the API is cleanly implemented
[...]
> +
> +static int movie_init(AVFilterContext *ctx)
> +{
> + MovieContext *movie = ctx->priv;
> + AVInputFormat *iformat = NULL;
> + AVCodec *codec;
> + int ret;
> + int64_t timestamp;
> +
> + av_register_all();
> +
> + // Try to find the movie format (container)
> + iformat = movie->format_name ? av_find_input_format(movie->format_name) : NULL;
> +
> + movie->format_ctx = NULL;
> + if ((ret = av_open_input_file(&movie->format_ctx, movie->file_name, iformat, 0, NULL)) < 0) {
> + av_log(ctx, AV_LOG_ERROR,
> + "Failed to av_open_input_file '%s'\n", movie->file_name);
> + return ret;
> + }
> + if ((ret = av_find_stream_info(movie->format_ctx)) < 0) {
> + av_log(ctx, AV_LOG_ERROR, "Failed to find stream info\n");
> + return ret;
> + }
this isnt fatal in rare cases
(for example in case of url_interrupt_cb())
> +
> + // if seeking requested, we execute it
> + if (movie->seek_point > 0) {
> + timestamp = movie->seek_point;
> + // add the stream start time, should it exist
> + if (movie->format_ctx->start_time != AV_NOPTS_VALUE)
> + timestamp += movie->format_ctx->start_time;
> + if (av_seek_frame(movie->format_ctx, -1, timestamp, AVSEEK_FLAG_BACKWARD) < 0) {
> + av_log(ctx, AV_LOG_ERROR, "%s: could not seek to position %"PRId64"\n",
> + movie->file_name, timestamp);
> + }
> + }
> +
> + /* select the video stream */
> + if ((ret = av_find_best_stream(movie->format_ctx, AVMEDIA_TYPE_VIDEO,
> + movie->stream_index, -1, NULL, 0)) < 0) {
> + av_log(ctx, AV_LOG_ERROR, "No video stream with index '%d' found\n",
> + movie->stream_index);
> + return ret;
> + }
> + movie->stream_index = ret;
> + movie->codec_ctx = movie->format_ctx->streams[movie->stream_index]->codec;
> +
> + /*
> + * So now we've got a pointer to the so-called codec context for our video
> + * stream, but we still have to find the actual codec and open it.
> + */
> + codec = avcodec_find_decoder(movie->codec_ctx->codec_id);
> + if (!codec) {
> + av_log(ctx, AV_LOG_ERROR, "Failed to find any codec\n");
> + return AVERROR(EINVAL);
> + }
> +
> + if ((ret = avcodec_open(movie->codec_ctx, codec)) < 0) {
> + av_log(ctx, AV_LOG_ERROR, "Failed to open codec\n");
> + return ret;
> + }
> +
> + if (!(movie->frame = avcodec_alloc_frame()) ) {
> + av_log(ctx, AV_LOG_ERROR, "Failed to alloc frame\n");
> + return AVERROR(ENOMEM);
> + }
> +
> + movie->w = movie->codec_ctx->width;
> + movie->h = movie->codec_ctx->height;
> +
> + av_log(ctx, AV_LOG_INFO, "seek_point:%lld format_name:%s file_name:%s stream_index:%d\n",
> + movie->seek_point, movie->format_name, movie->file_name,
> + movie->stream_index);
> +
> + return 0;
> +}
> +
> +static av_cold int init(AVFilterContext *ctx, const char *args, void *opaque)
> +{
> + MovieContext *movie = ctx->priv;
> + int ret;
> + movie->class = &movie_class;
> + av_opt_set_defaults2(movie, 0, 0);
> +
> + if (args) {
> + movie->file_name = av_get_token(&args, ":");
> + if (*args++ == ':' && (ret = (av_set_options_string(movie, args, "=", ":"))) < 0) {
^
useless ()
> + av_log(ctx, AV_LOG_ERROR, "Error parsing options string: '%s'\n", args);
> + return ret;
> + }
> + }
> + if (!movie->file_name) {
> + av_log(ctx, AV_LOG_ERROR, "No filename provided!\n");
> + return AVERROR(EINVAL);
> + }
> +
> + if (movie->seek_point_d > (INT64_MAX-0.5) / 1000000) {
> + av_log(ctx, AV_LOG_ERROR, "Value for seek point is too big\n");
> + return AVERROR(EINVAL);
> + }
> + movie->seek_point = (int64_t)(movie->seek_point_d * 1000000 + 0.5);
> +
> + init_pts_correction(&movie->pts_correction_ctx);
> +
> + return movie_init(ctx);
> +}
> +
[...]
> +static int movie_get_frame(AVFilterLink *outlink)
> +{
> + MovieContext *movie = outlink->src->priv;
> + AVPacket pkt;
> + int ret, frame_decoded;
> +
> + if (movie->is_done == 1)
> + return 0;
> +
> + while ((ret = av_read_frame(movie->format_ctx, &pkt)) >= 0) {
> + // Is this a packet from the video stream?
> + if (pkt.stream_index == movie->stream_index) {
> + avcodec_decode_video2(movie->codec_ctx, movie->frame, &frame_decoded, &pkt);
> +
> + if (frame_decoded) {
> + /* FIXME: avoid the memcpy */
> + movie->picref = avfilter_get_video_buffer(outlink, AV_PERM_WRITE | AV_PERM_PRESERVE |
> + AV_PERM_REUSE2, outlink->w, outlink->h);
> + av_image_copy(movie->picref->data, movie->picref->linesize,
> + movie->frame->data, movie->frame->linesize,
> + movie->picref->format, outlink->w, outlink->h);
> +
> + if (movie->decoder_reorder_pts == 1) {
> + movie->picref->pts = movie->frame->pkt_pts;
> + } else if (movie->decoder_reorder_pts == 0) {
> + movie->picref->pts = pkt.dts;
movie->frame->pkt_dts
> + } else /* if (movie->decoder_reorder_pts == -1) */ {
> + movie->picref->pts = guess_correct_pts(&movie->pts_correction_ctx, movie->frame->pkt_pts, pkt.dts);
same change here needed (for ffmpeg-mt it will make a difference)
> + }
> +
> + movie->picref->pos = movie->frame->reordered_opaque;
is this working? it looks incomplete
> + movie->picref->video->pixel_aspect = movie->codec_ctx->sample_aspect_ratio;
the format aspect should be favored if available
rest looks ok from a quick look
[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
Many things microsoft did are stupid, but not doing something just because
microsoft did it is even more stupid. If everything ms did were stupid they
would be bankrupt already.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20110128/e51e3bf2/attachment.pgp>
More information about the ffmpeg-devel
mailing list