[FFmpeg-devel] [PATCH] [PATCH] lavfi: add inverse telecine filter
Michael Niedermayer
michaelni at gmx.at
Mon Mar 16 23:57:08 CET 2015
On Tue, Mar 17, 2015 at 12:44:07AM +0530, Himangi Saraogi wrote:
> On 14 March 2015 at 09:42, Michael Niedermayer <michaelni at gmx.at> wrote:
>
> > On Wed, Mar 11, 2015 at 03:20:47AM +0530, Himangi Saraogi wrote:
> > > This is an exact inverse of the telecine filter unlike previously
> > existing
> > > pullup and fieldmatch ones.
> > >
> > > The algorithm was briefly discussed with Carl. The algorithm is not
> > completely
> > > tested, though I do have a some sample suggestions and will be testing on
> > > them soon. Documentation is yet to be added.
> > > ---
> > > Changelog | 1 +
> > > libavfilter/Makefile | 1 +
> > > libavfilter/allfilters.c | 1 +
> > > libavfilter/vf_detelecine.c | 323
> > ++++++++++++++++++++++++++++++++++++++++++++
> > > 4 files changed, 326 insertions(+)
> > > create mode 100644 libavfilter/vf_detelecine.c
> >
> >
> >
> > >
> > > diff --git a/Changelog b/Changelog
> > > index e88359d..341faca 100644
> > > --- a/Changelog
> > > +++ b/Changelog
> > > @@ -3,6 +3,7 @@ releases are sorted from youngest to oldest.
> > >
> > > version <next>:
> > > - FFT video filter
> > > +- Detelecine filter
> > >
> > >
> > > version 2.6:
> > > diff --git a/libavfilter/Makefile b/libavfilter/Makefile
> > > index b184f07..399072c 100644
> > > --- a/libavfilter/Makefile
> > > +++ b/libavfilter/Makefile
> > > @@ -112,6 +112,7 @@ OBJS-$(CONFIG_DECIMATE_FILTER) +=
> > vf_decimate.o
> > > OBJS-$(CONFIG_DEJUDDER_FILTER) += vf_dejudder.o
> > > OBJS-$(CONFIG_DELOGO_FILTER) += vf_delogo.o
> > > OBJS-$(CONFIG_DESHAKE_FILTER) += vf_deshake.o
> > > +OBJS-$(CONFIG_DETELECINE_FILTER) += vf_detelecine.o
> > > OBJS-$(CONFIG_DRAWBOX_FILTER) += vf_drawbox.o
> > > OBJS-$(CONFIG_DRAWGRID_FILTER) += vf_drawbox.o
> > > OBJS-$(CONFIG_DRAWTEXT_FILTER) += vf_drawtext.o
> > > diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
> > > index 043ac56..2e4e2f6 100644
> > > --- a/libavfilter/allfilters.c
> > > +++ b/libavfilter/allfilters.c
> > > @@ -128,6 +128,7 @@ void avfilter_register_all(void)
> > > REGISTER_FILTER(DEJUDDER, dejudder, vf);
> > > REGISTER_FILTER(DELOGO, delogo, vf);
> > > REGISTER_FILTER(DESHAKE, deshake, vf);
> > > + REGISTER_FILTER(DETELECINE, detelecine, vf);
> > > REGISTER_FILTER(DRAWBOX, drawbox, vf);
> > > REGISTER_FILTER(DRAWGRID, drawgrid, vf);
> > > REGISTER_FILTER(DRAWTEXT, drawtext, vf);
> > > diff --git a/libavfilter/vf_detelecine.c b/libavfilter/vf_detelecine.c
> > > new file mode 100644
> > > index 0000000..ce9ba74
> > > --- /dev/null
> > > +++ b/libavfilter/vf_detelecine.c
> > > @@ -0,0 +1,323 @@
> > > +/*
> > > + * Copyright (c) 2015 Himangi Saraogi <himangi774 at gmail.com>
> > > + *
> > > + * This file is part of FFmpeg.
> > > + *
> > > + * FFmpeg is free software; you can redistribute it and/or
> > > + * modify it under the terms of the GNU Lesser General Public
> > > + * License as published by the Free Software Foundation; either
> > > + * version 2.1 of the License, or (at your option) any later version.
> > > + *
> > > + * FFmpeg is distributed in the hope that it will be useful,
> > > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> > > + * Lesser General Public License for more details.
> > > + *
> > > + * You should have received a copy of the GNU Lesser General Public
> > > + * License along with FFmpeg; if not, write to the Free Software
> > > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
> > 02110-1301 USA
> > > + */
> > > +
> > > +/**
> > > + * @file detelecine filter.
> > > + */
> > > +
> > > +#include "libavutil/avstring.h"
> > > +#include "libavutil/imgutils.h"
> > > +#include "libavutil/opt.h"
> > > +#include "libavutil/pixdesc.h"
> > > +#include "avfilter.h"
> > > +#include "formats.h"
> > > +#include "internal.h"
> > > +#include "video.h"
> > > +
> > > +typedef struct {
> > > + const AVClass *class;
> > > + int first_field;
> > > + char *pattern;
> > > + unsigned int pattern_pos;
> > > + unsigned int nskip_fields;
> > > +
> > > + AVRational pts;
> > > + double ts_unit;
> > > + int occupied;
> > > +
> > > + int nb_planes;
> > > + int planeheight[4];
> > > + int stride[4];
> > > +
> > > + AVFrame *frame;
> > > + AVFrame *temp;
> > > +} DetelecineContext;
> > > +
> > > +#define OFFSET(x) offsetof(DetelecineContext, x)
> > > +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM
> > > +
> > > +static const AVOption detelecine_options[] = {
> > > + {"first_field", "select first field", OFFSET(first_field),
> > AV_OPT_TYPE_INT, {.i64=0}, 0, 1, FLAGS, "field"},
> > > + {"top", "select top field first", 0,
> > AV_OPT_TYPE_CONST, {.i64=0}, 0, 0, FLAGS, "field"},
> > > + {"t", "select top field first", 0,
> > AV_OPT_TYPE_CONST, {.i64=0}, 0, 0, FLAGS, "field"},
> > > + {"bottom", "select bottom field first", 0,
> > AV_OPT_TYPE_CONST, {.i64=1}, 0, 0, FLAGS, "field"},
> > > + {"b", "select bottom field first", 0,
> > AV_OPT_TYPE_CONST, {.i64=1}, 0, 0, FLAGS, "field"},
> > > + {"pattern", "pattern that describe for how many fields a frame is
> > to be displayed", OFFSET(pattern), AV_OPT_TYPE_STRING, {.str="23"}, 0, 0,
> > FLAGS},
> > > + {NULL}
> > > +};
> > > +
> > > +AVFILTER_DEFINE_CLASS(detelecine);
> > > +
> > > +static av_cold int init(AVFilterContext *ctx)
> > > +{
> > > + DetelecineContext *s = ctx->priv;
> > > + const char *p;
> > > + int max = 0;
> > > +
> > > + if (!strlen(s->pattern)) {
> > > + av_log(ctx, AV_LOG_ERROR, "No pattern provided.\n");
> > > + return AVERROR_INVALIDDATA;
> > > + }
> > > +
> > > + for (p = s->pattern; *p; p++) {
> > > + if (!av_isdigit(*p)) {
> > > + av_log(ctx, AV_LOG_ERROR, "Provided pattern includes
> > non-numeric characters.\n");
> > > + return AVERROR_INVALIDDATA;
> > > + }
> > > +
> > > + max = FFMAX(*p - '0', max);
> > > + s->pts.num += *p - '0';
> > > + s->pts.den += 2;
> > > + }
> > > +
> > > + s->nskip_fields = 0;
> > > +
> > > + av_log(ctx, AV_LOG_INFO, "Detelecine pattern %s removes up to %d
> > frames per frame, pts advance factor: %d/%d\n",
> > > + s->pattern, (max + 1) / 2, s->pts.num, s->pts.den);
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +static int query_formats(AVFilterContext *ctx)
> > > +{
> > > + AVFilterFormats *pix_fmts = NULL;
> > > + int fmt;
> > > +
> > > + for (fmt = 0; av_pix_fmt_desc_get(fmt); fmt++) {
> > > + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(fmt);
> > > + if (!(desc->flags & AV_PIX_FMT_FLAG_HWACCEL ||
> > > + desc->flags & AV_PIX_FMT_FLAG_PAL ||
> > > + desc->flags & AV_PIX_FMT_FLAG_BITSTREAM))
> > > + ff_add_format(&pix_fmts, fmt);
> > > + }
> > > +
> > > + ff_set_common_formats(ctx, pix_fmts);
> > > + return 0;
> > > +}
> > > +
> > > +static int config_input(AVFilterLink *inlink)
> > > +{
> > > + DetelecineContext *s = inlink->dst->priv;
> > > + const AVPixFmtDescriptor *desc =
> > av_pix_fmt_desc_get(inlink->format);
> > > + int i, ret;
> > > +
> > > + s->temp = ff_get_video_buffer(inlink, inlink->w, inlink->h);
> > > + if (!s->temp)
> > > + return AVERROR(ENOMEM);
> > > +
> > > + s->frame = ff_get_video_buffer(inlink, inlink->w, inlink->h);
> > > + if (!s->frame)
> > > + return AVERROR(ENOMEM);
> > > +
> > > + if ((ret = av_image_fill_linesizes(s->stride, inlink->format,
> > inlink->w)) < 0)
> > > + return ret;
> > > +
> > > + s->planeheight[1] = s->planeheight[2] = FF_CEIL_RSHIFT(inlink->h,
> > desc->log2_chroma_h);
> > > + s->planeheight[0] = s->planeheight[3] = inlink->h;
> > > +
> > > + s->nb_planes = av_pix_fmt_count_planes(inlink->format);
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +static int config_output(AVFilterLink *outlink)
> > > +{
> > > + AVFilterContext *ctx = outlink->src;
> > > + DetelecineContext *s = ctx->priv;
> > > + const AVFilterLink *inlink = ctx->inputs[0];
> > > + AVRational fps = inlink->frame_rate;
> > > +
> > > + if (!fps.num || !fps.den) {
> > > + av_log(ctx, AV_LOG_ERROR, "The input needs a constant frame
> > rate; "
> > > + "current rate of %d/%d is invalid\n", fps.num, fps.den);
> > > + return AVERROR(EINVAL);
> > > + }
> > > + fps = av_mul_q(fps, av_inv_q(s->pts));
> > > + av_log(ctx, AV_LOG_VERBOSE, "FPS: %d/%d -> %d/%d\n",
> > > + inlink->frame_rate.num, inlink->frame_rate.den, fps.num,
> > fps.den);
> > > +
> > > + outlink->flags |= FF_LINK_FLAG_REQUEST_LOOP;
> > > + outlink->frame_rate = fps;
> > > + outlink->time_base = av_mul_q(inlink->time_base, s->pts);
> > > + av_log(ctx, AV_LOG_VERBOSE, "TB: %d/%d -> %d/%d\n",
> > > + inlink->time_base.num, inlink->time_base.den,
> > outlink->time_base.num, outlink->time_base.den);
> > > +
> > > + s->ts_unit = av_q2d(av_inv_q(av_mul_q(fps, outlink->time_base)));
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +static int filter_frame(AVFilterLink *inlink, AVFrame *inpicref)
> > > +{
> > > + AVFilterContext *ctx = inlink->dst;
> > > + AVFilterLink *outlink = ctx->outputs[0];
> > > + DetelecineContext *s = ctx->priv;
> > > + int i, len = 0, ret = 0, out = 0;
> > > +
> > > + if (s->nskip_fields >= 2) {
> > > + s->nskip_fields -= 2;
> > > + return 0;
> > > + } else if (s->nskip_fields >= 1) {
> > > + if (s->occupied) {
> > > + s->occupied = 0;
> > > + s->nskip_fields--;
> > > + }
> > > + else {
> > > + for (i = 0; i < s->nb_planes; i++) {
> > > + av_image_copy_plane(s->temp->data[i],
> > s->temp->linesize[i],
> > > + inpicref->data[i],
> > inpicref->linesize[i],
> > > + s->stride[i],
> > > + s->planeheight[i]);
> > > + }
> > > + s->occupied = 1;
> > > + s->nskip_fields--;
> > > + return 0;
> > > + }
> > > + }
> > > +
> > > + if (s->nskip_fields == 0) {
> > > + while(!len && s->pattern[s->pattern_pos]) {
> > > + len = s->pattern[s->pattern_pos] - '0';
> > > + s->pattern_pos++;
> > > + }
> > > +
> > > + if (!s->pattern[s->pattern_pos])
> > > + s->pattern_pos = 0;
> > > +
> > > + if(!len) { // do not output any field as the entire pattern is
> > zero
> > > + av_frame_free(&inpicref);
> > > + return 0;
> > > + }
> > > +
> > > + if (s->occupied) {
> > > + for (i = 0; i < s->nb_planes; i++) {
> > > + // fill in the EARLIER field from the new pic
> > > + av_image_copy_plane(s->frame->data[i] +
> > s->frame->linesize[i] * s->first_field,
> > > + s->frame->linesize[i] * 2,
> > > + inpicref->data[i] +
> > inpicref->linesize[i] * s->first_field,
> > > + inpicref->linesize[i] * 2,
> > > + s->stride[i],
> > > + (s->planeheight[i] - s->first_field
> > + 1) / 2);
> > > + // fill in the LATER field from the buffered pic
> > > + av_image_copy_plane(s->frame->data[i] +
> > s->frame->linesize[i] * !s->first_field,
> > > + s->frame->linesize[i] * 2,
> > > + s->temp->data[i] +
> > s->temp->linesize[i] * !s->first_field,
> > > + s->temp->linesize[i] * 2,
> > > + s->stride[i],
> > > + (s->planeheight[i] -
> > !s->first_field + 1) / 2);
> > > + }
> > > + len -= 2;
> > > + for (i = 0; i < s->nb_planes; i++) {
> > > + av_image_copy_plane(s->temp->data[i],
> > s->temp->linesize[i],
> > > + inpicref->data[i],
> > inpicref->linesize[i],
> > > + s->stride[i],
> > > + s->planeheight[i]);
> > > + }
> > > + s->occupied = 1;
> > > + out = 1;
> > > + } else {
> > > + if (len >= 2) {
> > > + // output THIS image as-is
> > > + for (i = 0; i < s->nb_planes; i++)
> > > + av_image_copy_plane(s->frame->data[i],
> > s->frame->linesize[i],
> > > + inpicref->data[i],
> > inpicref->linesize[i],
> > > + s->stride[i],
> > > + s->planeheight[i]);
> > > + len -= 2;
> > > + out = 1;
> > > + } else if (len == 1) {
> > > + // fill in the EARLIER field from the new pic
> > > + av_image_copy_plane(s->frame->data[i] +
> > s->frame->linesize[i] * s->first_field,
> > > + s->frame->linesize[i] * 2,
> > > + inpicref->data[i] +
> > inpicref->linesize[i] * s->first_field,
> > > + inpicref->linesize[i] * 2,
> > > + s->stride[i],
> > > + (s->planeheight[i] - s->first_field
> > + 1) / 2);
> > > + // TODO: not sure about the other field
> > > +
> > > + len--;
> > > + out = 1;
> > > + }
> > > + }
> > > +
> > > + if (len == 1 && s->occupied)
> > > + {
> > > + len--;
> > > + s->occupied = 0;
> > > + }
> > > + }
> > > + s->nskip_fields = len;
> > > +
> > > + if (out) {
> > > + AVFrame *frame = av_frame_clone(s->frame);
> > > +
> > > + if (!frame) {
> > > + av_frame_free(&inpicref);
> > > + return AVERROR(ENOMEM);
> > > + }
> > > +
> > > + frame->pts = outlink->frame_count * s->ts_unit;
> >
> > the pts hadling looks wrong
> > also simply testing as in
> > ./ffplay matrixbench_mpeg2.mpg -vf detelecine
> > shows AV desync
> >
> >
> Currently, ts_unit is set to inverse(fps * outlink->time_base) and used to
> get the pts of the output frames as above. This is similar to the telecine
> filter. I suppose outlink->time_base takes care of the input timestamps not
> being guaranteed to start at 0.
>
> Any suggestion on how this should be handled?
The input timestamps are in the input frames (AVFrame.pts)
if the input starts at time 2h 3min 11.234sec then so should the
output otherwise the video stream would become out of sync with
the other streams if there are any
The timebases represent the units of the input and output pts
av_rescale_q() may be usefull to rescale timestamps if any rescaling
is needed
[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
Good people do not need laws to tell them to act responsibly, while bad
people will find a way around the laws. -- Plato
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20150316/6c2473a0/attachment.asc>
More information about the ffmpeg-devel
mailing list