[FFmpeg-soc] libavfilter audio work - qualification task
Stefano Sabatini
stefano.sabatini-lala at poste.it
Tue May 11 00:14:37 CEST 2010
On date Sunday 2010-05-09 20:42:30 -0700, S.N. Hemanth Meenakshisundaram encoded:
> On 05/03/2010 01:32 PM, Stefano Sabatini wrote:
> >On date Monday 2010-05-03 01:11:07 -0700, S.N. Hemanth Meenakshisundaram encoded:
> >>On 04/23/2010 05:03 PM, Stefano Sabatini wrote:
> >>>On date Thursday 2010-04-22 17:19:16 -0700, S.N. Hemanth Meenakshisundaram encoded:
> >>>[...]
> >>+
> >>+ /* FIXME: av_parse_color currently sets alpha to 0 if no alpha is specified.
> >>+ * So we force alpha = 0xFF (opaque), here in such a case.
> >>+ */
> >>+ if(rgba[3] != 0)
> >>+ color[3] = rgba[3];
> >>+ else
> >>+ color[3] = 0xFF;
> >>+ return 0;
> >>+}
> >What if the user wants to specify alpha = 0?
> >
> >I'd rather avoid the if (... != 0) etc check, the user can set the
> >alpha with the 0xRRGGBBAA syntax if he wants, or we could extend the
> >av_parse_color() syntax for specifing color+alpha (for example:
> >color=red/0xff, a nicer syntax suggestion is welcome).
>
> Will fix parseutils fix as a separate patch. For now, this will work.
>
> >
> >So what about:
> >if (alpha) {
> >...
> >} else {
> >...
> >}
> >
> >this should save efficiency if alpha composition is not required. Feel
> >free to skip this and leave it to an eventual further patch.
>
> Done. Alpha now available for the background box as well.
>
> >
> >Condition (text == dtext->text) is always true, also we cannot
> >distinguish between the two cases (imho is better like this), and I
> >see no reason to treat '_' in a special way.
> >
>
> Fixed. '-' is now like any other character.
>
> Also fixed style nits and corrected handling of freetype error messages.
>
> >
> >>Index: Makefile
> >
> >>+libavfilter/vf_drawtext.o libavfilter/vf_drawtext.d: CFLAGS += $(`freetype-config --cflags`)
> >>+FF_EXTRALIBS += $(`freetype-config --libs`)
> >>+
> >No please, both cflags and ldflags should be set in configure.
>
> Done. Moved to configure. No changes required now to original Makefile.
>
> >>--- libavfilter.texi.nodt 2010-05-03 00:39:50.516428954 -0700
> >>+++ libavfilter.texi 2010-05-03 00:28:42.221181682 -0700
> >>@@ -148,6 +148,91 @@
> >> Draw a box with @var{x}:@var{y}:@var{width}:@var{height}:@var{color}
> >> dimensions in a chosen color.
> >>
> >>+ at section drawtext
> >>+
> >>+Draws text string or text from specified file on top of video.
> >Impersonal form (Draws -> Draw) seems preferred in libavfilter.texi,
> >same for the parameter descriptions
>
> Fixed.
>
> >Nit, add empty newline between @end table and @example, improve
> >readability.
>
> Done.
>
> The patch to soc/libavfilter repo is attached. Has changes to
> allfilters.c, libavfilter Makefile and vf_drawtext.c only.
>
> Please let me know if further changes are required.
>
> Regards,
> Hemanth
>
> Index: allfilters.c
> ===================================================================
> --- allfilters.c (revision 5784)
> +++ allfilters.c (working copy)
> @@ -37,6 +37,7 @@
> REGISTER_FILTER (ASPECT, aspect, vf);
> REGISTER_FILTER (CROP, crop, vf);
> REGISTER_FILTER (DRAWBOX, drawbox, vf);
> + REGISTER_FILTER (DRAWTEXT, drawtext, vf);
> REGISTER_FILTER (FIFO, fifo, vf);
> REGISTER_FILTER (FORMAT, format, vf);
> REGISTER_FILTER (FPS, fps, vf);
> Index: Makefile
> ===================================================================
> --- Makefile (revision 5784)
> +++ Makefile (working copy)
> @@ -18,6 +18,7 @@
> OBJS-$(CONFIG_ASPECT_FILTER) += vf_aspect.o
> OBJS-$(CONFIG_CROP_FILTER) += vf_crop.o
> OBJS-$(CONFIG_DRAWBOX_FILTER) += vf_drawbox.o
> +OBJS-$(CONFIG_DRAWTEXT_FILTER) += vf_drawtext.o
> OBJS-$(CONFIG_FIFO_FILTER) += vf_fifo.o
> OBJS-$(CONFIG_FORMAT_FILTER) += vf_format.o
> OBJS-$(CONFIG_FPS_FILTER) += vf_fps.o
> Index: vf_drawtext.c
> ===================================================================
> --- vf_drawtext.c (revision 0)
> +++ vf_drawtext.c (revision 0)
> @@ -0,0 +1,509 @@
> +/*
> + * copyright (c) 2010 S.N. Hemanth Meenakshisundaram
> + * Original vhook author: Gustavo Sverzut Barbieri <gsbarbieri at yahoo.com.br>
> + * Libavfilter version : S.N. Hemanth Meenakshisundaram <smeenaks at ucsd.edu>
> + *
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> + */
> +
> +/**
> + * @file
> + * Drawtext Filter
> + */
> +
> +#include "avfilter.h"
> +#include "parseutils.h"
> +#include "libavcodec/colorspace.h"
> +#include "libavutil/pixdesc.h"
> +
> +#undef time
> +#include <sys/time.h>
> +#include <time.h>
> +
> +#include <ft2build.h>
> +#include <freetype/config/ftheader.h>
> +#include FT_FREETYPE_H
> +#include FT_GLYPH_H
> +
> +typedef struct {
> + const AVClass *class;
> + unsigned char *fontfile; ///< font to be used
> + unsigned char *text; ///< text to be drawn
> + char *textfile; ///< file with text to be drawn
> + unsigned int x; ///< x position to start drawing text
> + unsigned int y; ///< y position to start drawing text
> + unsigned int fontsize; ///< font size to use
> + char *fgcolor_string; ///< foreground color as string
> + char *bgcolor_string; ///< background color as string
> + unsigned char fgcolor[4]; ///< foreground color in YUV
> + unsigned char bgcolor[4]; ///< background/Box color in YUV
> + short int draw_box; ///< draw box around text - true or false
> + short int outline; ///< draw outline in bg color around text
> + int text_height; ///< height of a font symbol
> + int baseline; ///< baseline to draw fonts from
> + int use_kerning; ///< font kerning is used - true/false
> + FT_Library library; ///< freetype font library handle
> + FT_Face face; ///< freetype font face handle
> + FT_Glyph glyphs[256]; ///< array holding glyphs of font
> + FT_Bitmap bitmaps[256]; ///< array holding bitmaps of font
> + int advance[256];
> + int bitmap_left[256];
> + int bitmap_top[256];
> + unsigned int glyphs_index[256];
> + int hsub, vsub; ///< chroma subsampling values
> +} DrawTextContext;
> +
> +#define OFFSET(x) offsetof(DrawTextContext, x)
> +
> +static const AVOption drawtext_options[]= {
> +{"fontfile", "set font file", OFFSET(fontfile), FF_OPT_TYPE_STRING, 0, CHAR_MIN, CHAR_MAX },
> +{"text", "set text", OFFSET(text), FF_OPT_TYPE_STRING, 0, CHAR_MIN, CHAR_MAX },
> +{"textfile", "set text file", OFFSET(textfile), FF_OPT_TYPE_STRING, 0, CHAR_MIN, CHAR_MAX },
> +{"fgcolor", "set foreground color", OFFSET(fgcolor_string), FF_OPT_TYPE_STRING, 0, CHAR_MIN, CHAR_MAX },
> +{"bgcolor", "set background color", OFFSET(bgcolor_string), FF_OPT_TYPE_STRING, 0, CHAR_MIN, CHAR_MAX },
> +{"box", "set box", OFFSET(draw_box), FF_OPT_TYPE_INT, 0, 0, 1 },
> +{"outline", "set outline", OFFSET(outline), FF_OPT_TYPE_INT, 0, 0, 1 },
> +{"fontsize", "set font size", OFFSET(fontsize), FF_OPT_TYPE_INT, 16, 1, 72 },
> +{"x", "set x", OFFSET(x), FF_OPT_TYPE_INT, 0, 0, INT_MAX },
> +{"y", "set y", OFFSET(y), FF_OPT_TYPE_INT, 0, 0, INT_MAX },
> +{NULL},
help text can be vertically aligned as well.
> +};
> +
> +static const char *drawtext_get_name(void *ctx)
> +{
> + return "drawtext";
> +}
> +
> +static const AVClass drawtext_class = {
> + "DrawTextContext",
> + drawtext_get_name,
> + drawtext_options
> +};
> +
> +static int query_formats(AVFilterContext *ctx)
> +{
> + /* FIXME: Add support for other formats */
> + enum PixelFormat pix_fmts[] = {
> + PIX_FMT_YUV420P, PIX_FMT_YUV444P, PIX_FMT_YUV422P,
> + PIX_FMT_YUV411P, PIX_FMT_YUV410P,
> + PIX_FMT_YUV440P, PIX_FMT_NONE
> + };
> +
> + avfilter_set_common_formats(ctx, avfilter_make_format_list(pix_fmts));
> + return 0;
> +}
> +
> +#undef __FTERRORS_H__
> +#define FT_ERROR_START_LIST {
> +#define FT_ERRORDEF(e, v, s) { (e), (s) },
> +#define FT_ERROR_END_LIST { 0, NULL } };
> +
> +struct ft_error
> +{
> + int err;
> + char *err_msg;
> +} ft_errors[] =
> +#include FT_ERRORS_H
> +
> +#define FT_ERRMSG(e) ft_errors[e].err_msg
> +
> +#define MAX_TEXT_SIZE 1024
> +
> +static inline int extract_color(AVFilterContext *ctx, char *color_str, unsigned char *color)
> +{
> + uint8_t rgba[4];
> + uint8_t err;
> + if ((err = av_parse_color(rgba, color_str, ctx))) {
> + return err;
> + }
> + color[0] = RGB_TO_Y(rgba[0], rgba[1], rgba[2]);
> + color[1] = RGB_TO_U(rgba[0], rgba[1], rgba[2], 0);
> + color[2] = RGB_TO_V(rgba[0], rgba[1], rgba[2], 0);
> +
> + /* FIXME: av_parse_color currently sets alpha to 0 if no alpha is specified.
> + * So we force alpha = 0xFF (opaque), here in such a case.
> + */
> + if (rgba[3] != 0)
> + color[3] = rgba[3];
> + else
> + color[3] = 0xFF;
I suppose this was to be skipped.
> + return 0;
> +}
> +
> +static av_cold int init(AVFilterContext *ctx, const char *args, void *opaque)
> +{
> + unsigned short int c;
> + int err;
> + int y_max, y_min;
> + FT_BBox bbox;
> + DrawTextContext *dtext = ctx->priv;
> +
> + dtext->class = &drawtext_class;
> + av_opt_set_defaults2(dtext, 0, 0);
> + dtext->fgcolor_string = av_strdup("black");
> + dtext->bgcolor_string = av_strdup("white");
> +
> + if ((err = (av_set_options_string(dtext, args, "=", ":"))) < 0) {
> + av_log(ctx, AV_LOG_ERROR, "Error parsing options string: '%s'\n", args);
> + return err;
> + }
> +
> + if (!dtext->fontfile) {
> + av_log(ctx, AV_LOG_ERROR, "No font file provided! (=fontfile:<filename>)\n");
> + return AVERROR(EINVAL);
> + }
> +
> + if (dtext->textfile) {
> + FILE *fp;
> + if (dtext->text) {
> + av_log(ctx, AV_LOG_ERROR, "Both text and file provided. Please provide only one.\n");
> + return AVERROR(EINVAL);
> + }
> + if (!(fp = fopen(dtext->textfile, "r"))) {
> + av_log(ctx, AV_LOG_ERROR, "The textfile %s could not be opened.\n", dtext->textfile);
> + return AVERROR(EINVAL);
> + } else {
> + uint16_t read_bytes;
> + char *tbuff = av_malloc(MAX_TEXT_SIZE);
> + if (!tbuff) {
> + av_log(ctx, AV_LOG_ERROR, "Could not allocate read buffer.\n");
> + return AVERROR(ENOMEM);
> + }
> + read_bytes = fread(tbuff, sizeof(char), MAX_TEXT_SIZE-1, fp);
> + if (read_bytes > 0) {
> + tbuff[read_bytes] = 0;
> + av_free(dtext->text);
> + dtext->text = tbuff;
> + } else {
> + av_log(ctx, AV_LOG_ERROR, "The textfile %s could not be read or is empty.\n", dtext->textfile);
> + av_free(tbuff);
> + return AVERROR(EINVAL);
> + }
> + fclose(fp);
> + }
> + }
> +
> + if (!dtext->text) {
> + av_log(ctx, AV_LOG_ERROR, "Either text or a valid file must be provided (=text:<text> or =textfile:<filename>)\n");
> + return AVERROR(EINVAL);
> + }
> +
> + if ((err = extract_color(ctx, dtext->fgcolor_string, dtext->fgcolor))) {
> + av_log(ctx, AV_LOG_ERROR, "Invalid foreground color: '%s'.\n", dtext->fgcolor_string);
> + return err;
> + }
> +
> + if ((err = extract_color(ctx, dtext->bgcolor_string, dtext->bgcolor))) {
> + av_log(ctx, AV_LOG_ERROR, "Invalid background color: '%s'.\n", dtext->fgcolor_string);
> + return err;
> + }
> +
> + if ((err = FT_Init_FreeType(&(dtext->library)))) {
> + av_log(ctx, AV_LOG_ERROR, "Could not load FreeType: %s\n", FT_ERRMSG(err));
> + return AVERROR(EINVAL);
> + }
> +
> + if ((err = FT_New_Face( dtext->library, dtext->fontfile, 0, &(dtext->face)))) {
^
> + av_log(ctx, AV_LOG_ERROR, "Could not load fontface %s: %s\n", dtext->fontfile, FT_ERRMSG(err));
> + return AVERROR(EINVAL);
> + }
> + if ((err = FT_Set_Pixel_Sizes( dtext->face, 0, dtext->fontsize))) {
^
> + av_log(ctx, AV_LOG_ERROR, "Could not set font size to %d pixels: %s\n", dtext->fontsize, FT_ERRMSG(err));
> + return AVERROR(EINVAL);
> + }
> +
> + dtext->use_kerning = FT_HAS_KERNING(dtext->face);
> +
> + /* load and cache glyphs */
> + y_max = -32000;
> + y_min = 32000;
> + /* FIXME: Supports only ASCII text now. Add Unicode support */
> + for (c=0; c <= 255; c++) {
> +
> + if (!strrchr(dtext->text,(unsigned char)c))
> + continue;
> +
> + /* Load char */
> + err = FT_Load_Char( dtext->face, (unsigned char) c, FT_LOAD_RENDER | FT_LOAD_MONOCHROME );
> + if (err)
> + continue; /* ignore errors */
> +
> + dtext->bitmaps [c] = dtext->face->glyph->bitmap;
> + dtext->bitmap_left[c] = dtext->face->glyph->bitmap_left;
> + dtext->bitmap_top [c] = dtext->face->glyph->bitmap_top;
> + dtext->advance [c] = dtext->face->glyph->advance.x >> 6;
> +
> + err = FT_Get_Glyph( dtext->face->glyph, &(dtext->glyphs[c]) );
> + if (err)
> + continue; /* ignore errors */
> +
> + dtext->glyphs_index[c] = FT_Get_Char_Index( dtext->face, (unsigned char) c );
> +
> + /* Measure text height to calculate text_height (or the maximum text height) */
> + FT_Glyph_Get_CBox( dtext->glyphs[c], ft_glyph_bbox_pixels, &bbox );
> + if (bbox.yMax > y_max)
> + y_max = bbox.yMax;
> + if (bbox.yMin < y_min)
> + y_min = bbox.yMin;
> + }
> +
> + dtext->text_height = y_max - y_min;
> + dtext->baseline = y_max;
> +
> + return 0;
> +}
> +
> +static av_cold void uninit(AVFilterContext *ctx)
> +{
> + DrawTextContext *dtext = ctx->priv;
> + av_free(dtext->fontfile);
> + av_free(dtext->text);
> + av_free(dtext->textfile);
> + av_free(dtext->fgcolor_string);
> + av_free(dtext->bgcolor_string);
> + FT_Done_Face(dtext->face);
> + FT_Done_FreeType(dtext->library);
> +}
> +
> +static int config_input(AVFilterLink *link)
> +{
> + DrawTextContext *dtext = link->dst->priv;
> + const AVPixFmtDescriptor *pix_desc = &av_pix_fmt_descriptors[link->format];
> + dtext->hsub = pix_desc->log2_chroma_w;
> + dtext->vsub = pix_desc->log2_chroma_h;
> + return 0;
> +}
> +
> +#define SET_PIXEL(pic_ref, yuv_color, x, y, hsub, vsub) { \
> + luma_pos = (x ) + (y )*pic_ref->linesize[0]; \
> + chroma_pos1 = (x >> hsub) + (y >> vsub)*pic_ref->linesize[1]; \
> + chroma_pos2 = (x >> hsub) + (y >> vsub)*pic_ref->linesize[2]; \
> + pic_ref->data[0][luma_pos] = (yuv_color[3]*yuv_color[0] + (255 - yuv_color[3])*pic_ref->data[0][luma_pos]) >> 8; \
nit+++: align >> 8
> + pic_ref->data[1][chroma_pos1] = (yuv_color[3]*yuv_color[1] + (255 - yuv_color[3])*pic_ref->data[1][chroma_pos1]) >> 8; \
> + pic_ref->data[2][chroma_pos2] = (yuv_color[3]*yuv_color[2] + (255 - yuv_color[3])*pic_ref->data[2][chroma_pos2]) >> 8; \
> +}
> +
> +#define GET_PIXEL(pic_ref, yuv_color, x, y, hsub, vsub) { \
> + yuv_color[0] = pic_ref->data[0][ (x ) + (y )*pic_ref->linesize[0] ]; \
> + yuv_color[1] = pic_ref->data[1][ (x >> hsub) + (y >> vsub)*pic_ref->linesize[1] ]; \
> + yuv_color[2] = pic_ref->data[2][ (x >> hsub) + (y >> vsub)*pic_ref->linesize[2] ]; \
> +}
> +
> +static inline void draw_glyph(AVFilterPicRef *pic_ref, FT_Bitmap *bitmap, unsigned int x,
> + unsigned int y, unsigned int width, unsigned int height,
> + unsigned char yuv_fgcolor[4], unsigned char yuv_color[4],
> + short int outline, int hsub, int vsub)
> +{
> + int r, c;
> + unsigned int luma_pos, chroma_pos1, chroma_pos2;
> + uint8_t spixel, dpixel[4], in_glyph=0;
> +
> + if (bitmap->pixel_mode == ft_pixel_mode_mono) {
> + in_glyph = 0;
> + for (r=0; (r < bitmap->rows) && (r+y < height); r++) {
> + for (c=0; (c < bitmap->width) && (c+x < width); c++) {
> + /* pixel in the pic_ref (destination) */
> + GET_PIXEL(pic_ref, dpixel, (c+x), (y+r), hsub, vsub);
> +
> + /* pixel in the glyph bitmap (source) */
> + spixel = bitmap->buffer[r*bitmap->pitch +c/8] & (0x80>>(c%8));
> +
> + if (spixel)
> + memcpy(dpixel, yuv_fgcolor, 4);
> +
> + if (outline) {
> + /* border detection: */
> + if (!in_glyph && spixel) {
> + /* left border detected */
> + in_glyph = 1;
> + /* draw left pixel border */
> + if (c-1 >= 0)
> + SET_PIXEL(pic_ref, yuv_color, (c+x-1), (y+r), hsub, vsub);
> + } else if ( in_glyph && !spixel ) {
> + /* right border detected */
> + in_glyph = 0;
> + /* 'draw' right pixel border */
> + memcpy(dpixel, yuv_color, 4);
> + }
> +
> + if (in_glyph) {
> + /* see if we have a top/bottom border */
> + /* top */
> + if ((r-1 >= 0) && (! bitmap->buffer[(r-1)*bitmap->pitch +c/8] & (0x80>>(c%8))))
> + /* we have a top border */
> + SET_PIXEL(pic_ref, yuv_color, (c+x), (y+r-1), hsub, vsub);
> +
> + /* bottom border detection */
> + if ((r+1 < height) && (! bitmap->buffer[(r+1)*bitmap->pitch +c/8] & (0x80>>(c%8))))
> + /* draw bottom border */
> + SET_PIXEL(pic_ref, yuv_color, (c+x), (y+r+1), hsub, vsub);
> + }
> + }
> + SET_PIXEL(pic_ref, dpixel, (c+x), (y+r), hsub, vsub);
> + }
> + }
> + }
> +}
> +
> +/* FIXME: There is no support for setting background box alpha here at the moment.
> + * Alpha support will mean we cannot do memset one stride at a time and instead
> + * must set each pixel individually.
> + */
This FIXME should be fixed now, right?
> +
> +static inline void drawbox(AVFilterPicRef *pic_ref, unsigned int x, unsigned int y,
> + unsigned int width, unsigned int height,
> + unsigned char yuv_color[4], int hsub, int vsub)
> +{
> + int i, plane;
> + uint8_t *p;
> +
> + if (yuv_color[3] != 0xFF) {
> + unsigned int j, luma_pos, chroma_pos1, chroma_pos2;
> +
> + for (j = 0; (j < height); j++)
superfluous parentheses around "j < height".
> + for (i = 0; (i < width); i++)
> + SET_PIXEL(pic_ref, yuv_color, (i+x), (y+j), hsub, vsub);
> +
> + } else {
> + for (plane = 0; plane < 3 && pic_ref->data[plane]; plane++) {
> + int hsub1 = plane == 1 || plane == 2 ? hsub : 0;
> + int vsub1 = plane == 1 || plane == 2 ? vsub : 0;
> +
> + p = pic_ref->data[plane] + (y >> vsub1) * pic_ref->linesize[plane] + (x >> hsub1);
> + for (i = 0; i < (height >> vsub1); i++) {
> + memset(p, yuv_color[plane], (width >> hsub1));
> + p += pic_ref->linesize[plane];
> + }
> + }
> + }
> +}
Apart those nits patch looks fine to me (but missing configure and
documentation parts), I assume it has been tested and works.
Please provide the complete patch.
Regards.
More information about the FFmpeg-soc
mailing list