[FFmpeg-user] alphamerge filter use problem
Tim Willison
timbot at oddlystudios.com
Fri Apr 12 16:57:13 CEST 2013
According to AfterEffects, both my moving gradient and the output from the
alphaextract filter have pure zero black pixels.
I also tested by having ffmpeg itself generate the inputs to be sure.
Here's a test I tried:
------generate a fully alpha image (pure alpha)
ffmpeg -f lavfi -i color=c=0x00000000:s=1280x166:r=30:sar=1 -y -pix_fmt
rgba -vcodec png -s 1280x166 -r 30 -vframes 120 alphasquare.mov
------alpha extract the square (pure white - confirmed)
ffmpeg -i alphasquare.mov -vf 'alphaextract' -y -pix_fmt rgb -vcodec png -s
1280x166 -r alphaextracted.mov
------apply white mask to pure white box
ffmpeg -f lavfi -i color=c=0xFFFFFFFF:s=1280x166:r=30:sar=1 -i
alphaextracted.mov -filter_complex '[0:0][1:0]alphamerge' -y -pix_fmt rgba
-vcodec png -s 1280x166 -r 30 -vframes 120 alphacomposite.mov
The resulting video should have pure white pixels, but it doesn't. They
come out more like 235 than 255.
If I skip the in-between step and do it all in one filter chain, however, I
DO get a pure white:
------------------same thing in one line
ffmpeg -f lavfi -i color=c=0x00000000:s=1280x166:r=30:sar=1 -f lavfi -i
color=c=0xFFFFFF:s=1280x166:r=30:sar=1 -filter_complex '[0:0] alphaextract
[alphaextracted];[1:0][alphaextracted] alphamerge' -y -pix_fmt rgba -vcodec
png -s 1280x166 -r 30 -vframes 120 alphacomposite.mov
So....after many combinations like this my best solution was to clean up
the alpha channel rather than try to get the proper combination of
encoding. And it worked pretty well. It seems reliable across various
options and I've tried it even when I'm not using the alphaextract filter,
but rather generating my own luma masks (with format=gray filter).
-t.
On Fri, Apr 12, 2013 at 10:38 AM, Paul B Mahol <onemda at gmail.com> wrote:
> On 4/12/13, Tim Willison <timbot at oddlystudios.com> wrote:
> > That's interesting Paul, you might be on to something there. I found it
> > very coincidental that the alpha from pure black areas was always 16.
> >
> > I'm actually doing two passes, and using rgba for everything.
> > On my first pass, I have a moving gradient which is rgb24, and I apply a
> > mask to it (with alphamerge) that was created using the alphaextract
> filter
> > on an rgba png image (png color-type 6).
> >
> > I am using the output of that (which is now a nice animated gradient in
> the
> > shape of text) as the luma source and doing another alphamerge against my
> > original text image. Now my nice moving gradient reveals the original
> > image.
> >
> > The only place I can think that YUV might be introduced would be in the
> > alphaextract filter. But even that I encode into rgba png before I use
> it.
> > I wonder if there is a better format to use than rgba png encoded.
>
> Does your rgb source have pure black 0 pixels?
> (you can see it with histogram filter)
>
> When converting it to gray8 with alpha extract, is it preserved?
>
> 0 values should remain 0.
>
> >
> > -t.
> >
> > On Fri, Apr 12, 2013 at 9:12 AM, Paul B Mahol <onemda at gmail.com> wrote:
> >
> >> On 4/12/13, Tim Willison <timbot at oddlystudios.com> wrote:
> >> > Hello folks,
> >> > I thought I could share a tip I've used in my experiments with the
> >> > alphaextract and alphamerge filters.
> >> >
> >> > I have found that the result of the alphamerge filter often leaves
> >> > areas
> >> > with some non-zero alpha. Comparing the mask file, areas that are pure
> >> > black are not always reproduced as pure alpha after the merge.
> >>
> >> Looks like you used YUV source for mask. And result is expected because
> >> YUV (limited range, not YUVJ) values can not be 0-16 or 235-255.
> >>
> >> >
> >> > However, I found a way to deal with this by using the lut filter. It
> >> > was
> >> as
> >> > simple as adding this to my alphamerge filter chain:
> >> >
> >> > [text2][out]alphamerge,format=rgba,lutrgb=a=if(gte(val\,18)\,val)
> >> >
> >> > The lutrgb is using an expression that lets me add a threshold for the
> >> > alpha channel. In this case, I'm saying that if the alpha drops below
> >> > 18,
> >> > it should likely be zero, otherwise keep it the same.
> >> > This cleaned up any alpha artefacts very nicely.
> >> >
> >> > Hope that is helpful for somebody.
> >> >
> >> > -t.
> >> >
> >> >
> >> > On Thu, Apr 11, 2013 at 11:36 AM, Tim Willison
> >> > <timbot at oddlystudios.com>wrote:
> >> >
> >> >> Hi Stefano!
> >> >>
> >> >> Thanks very much. Your comment was actually very helpful.
> >> >> I took a second look at how I was viewing the output of my alphamerge
> >> and
> >> >> sure enough - the alpha information was correct. I am now tackling my
> >> >> last
> >> >> problem.
> >> >>
> >> >> My video output from the alphamerge is excellent, but consistently
> >> >> shows
> >> >> a
> >> >> value of 16 in the alpha channel in areas that should be completely
> >> >> transparent. It has the effect of putting a very slight gray layer on
> >> >> everything.
> >> >> I am betting that this is some compression on my output (which is
> >> >> rgba,
> >> >> png codec into a mov).
> >> >>
> >> >> You mentioned using rawvideo. I haven't tried that before, but it
> >> >> sounds
> >> >> like what I might need.
> >> >>
> >> >> Thanks again for your help.
> >> >>
> >> >> -tim.
> >> >>
> >> >>
> >> >> On Wed, Apr 10, 2013 at 11:51 AM, Stefano Sabatini
> >> >> <stefasab at gmail.com>wrote:
> >> >>
> >> >>> On date Sunday 2013-04-07 15:44:07 -0400, Tim Willison wrote:
> >> >>> > Hi, this is my first question ever to a mailing list (or
> >> >>> > anywhere!).
> >> >>> > I have been using ffmpeg for about a year and a half and have
> >> >>> > figured
> >> >>> out
> >> >>> > many things on my own but I'm stumped regarding the alphaextract
> >> >>> > and
> >> >>> > alphamerge filters.
> >> >>> > I am able to get the proper result every time with images, but
> have
> >> >>> > yet
> >> >>> to
> >> >>> > succeed with videos. I have tried many formats and searched
> >> >>> > extensively
> >> >>> but
> >> >>> > have not found a single documented case of someone successfully
> >> >>> > using
> >> >>> this
> >> >>>
> >> >>> > filter combination and sharing an example. The documentation on
> the
> >> >>> > ffmpeg-filters page is very sparse, and does not give any details
> >> >>> > of
> >> >>> what
> >> >>> > formats/codecs are supported.
> >> >>>
> >> >>> Yes that's because ffmpeg-filters is supposed to only document
> >> >>> filters.
> >> >>>
> >> >>> >
> >> >>> > I think this filter would be very useful if I can figure it out.
> >> >>> >
> >> >>> > Step 1 - alphaextract
> >> >>> > This seems to always work, and produces a grayscale video as
> >> expected.
> >> >>> In
> >> >>> > my tests I have used rgba video, as well as quicktime with
> >> >>> > animation
> >> >>> codec
> >> >>> > (bgra).
> >> >>> >
> >> >>> > Step 2 - applying the extracted alpha
> >> >>> > This works every time for images. However while the encode
> >> >>> > completes
> >> >>> with
> >> >>> > video it never provides the masking expected by mapping the
> >> >>> > extracted
> >> >>> alpha
> >> >>> > information to the alpha channel of the target video. In my tests
> >> with
> >> >>> > still images, I found that the output format must be a format that
> >> >>> supports
> >> >>> > an alpha channel, and I have tried several with video (which I
> >> >>> determined
> >> >>> > by looking at the source code of the filter).
> >> >>>
> >> >>> The question is, are you sure your output codec supports alpha (most
> >> >>> don't, indeed I couldn't even tell which ones support alpha). You
> can
> >> >>> try with rawvideo and make sure it is not a codec bug.
> >> >>>
> >> >>> > My command (taken from the docs) looks like this:
> >> >>> >
> >> >>> > ffmpeg -i timbotundelay.mp4 -vf "movie=timbotoverlayextracted.mov
> >> >>> [alpha];
> >> >>> > [in][alpha] alphamerge [out]" -an -y -pix_fmt rgba -vcodec png
> >> >>> composite.mov
> >> >>> > ffmpeg version git-2013-04-06-1177416 Copyright (c) 2000-2013 the
> >> >>> > FFmpeg
> >> >>> > developers
> >> >>> > built on Apr 6 2013 17:31:47 with gcc 4.6.1 (Ubuntu/Linaro
> >> >>> > 4.6.1-9ubuntu3)
> >> >>> > configuration: --enable-gpl --enable-libass --enable-libfaac
> >> >>> > --enable-libfdk-aac --enable-libmp3lame --enable-libopencore-amrnb
> >> >>> > --enable-libopencore-amrwb --enable-libspeex --enable-librtmp
> >> >>> > --enable-libtheora --enable-libvorbis --enable-libvpx
> >> --enable-libx264
> >> >>> > --enable-nonfree --enable-version3 --enable-libfreetype
> >> >>> > --enable-filter=drawtext
> >> >>> > libavutil 52. 25.100 / 52. 25.100
> >> >>> > libavcodec 55. 2.100 / 55. 2.100
> >> >>> > libavformat 55. 1.100 / 55. 1.100
> >> >>> > libavdevice 55. 0.100 / 55. 0.100
> >> >>> > libavfilter 3. 49.100 / 3. 49.100
> >> >>> > libswscale 2. 2.100 / 2. 2.100
> >> >>> > libswresample 0. 17.102 / 0. 17.102
> >> >>> > libpostproc 52. 2.100 / 52. 2.100
> >> >>> > Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'timbotundelay.mp4':
> >> >>> > Metadata:
> >> >>> > major_brand : mp42
> >> >>> > minor_version : 0
> >> >>> > compatible_brands: mp42mp41
> >> >>> > creation_time : 2013-04-07 18:35:55
> >> >>> > Duration: 00:00:04.97, start: 0.000000, bitrate: 241 kb/s
> >> >>> > Stream #0:0(eng): Video: h264 (Main) (avc1 / 0x31637661),
> >> yuv420p,
> >> >>> > 1280x720 [SAR 1:1 DAR 16:9], 35 kb/s, 30 fps, 30 tbr, 30k tbn, 60
> >> >>> > tbc
> >> >>> > Metadata:
> >> >>> > creation_time : 2013-04-07 18:35:55
> >> >>> > handler_name : ?Mainconcept Video Media Handler
> >> >>> > Stream #0:1(eng): Audio: aac (mp4a / 0x6134706D), 48000 Hz,
> >> >>> > stereo,
> >> >>> > fltp, 189 kb/s
> >> >>> > Metadata:
> >> >>> > creation_time : 2013-04-07 18:35:55
> >> >>> > handler_name : #Mainconcept MP4 Sound Media Handler
> >> >>> > Output #0, mov, to 'composite.mov':
> >> >>> > Metadata:
> >> >>> > major_brand : mp42
> >> >>> > minor_version : 0
> >> >>> > compatible_brands: mp42mp41
> >> >>> > encoder : Lavf55.1.100
> >> >>> > Stream #0:0(eng): Video: png (png / 0x20676E70), rgba,
> >> >>> > 1280x720
> >> >>> [SAR
> >> >>> > 1:1 DAR 16:9], q=2-31, 200 kb/s, 15360 tbn, 30 tbc
> >> >>> > Metadata:
> >> >>> > creation_time : 2013-04-07 18:35:55
> >> >>> > handler_name : ?Mainconcept Video Media Handler
> >> >>> > Stream mapping:
> >> >>> > Stream #0:0 -> #0:0 (h264 -> png)
> >> >>> > Press [q] to stop, [?] for help
> >> >>> > frame= 151 fps= 64 q=0.0 Lsize= 1952kB time=00:00:05.03
> >> >>> > bitrate=3177.5kbits/s dup=2 drop=0
> >> >>> > video:1951kB audio:0kB subtitle:0 global headers:0kB muxing
> >> >>> > overhead
> >> >>> > 0.067323%
> >> >>> >
> >> >>> > After this, I see only my red background (timbotundelay.mp4) and
> no
> >> >>> alpha
> >> >>> > applied.
> >> >>>
> >> >>> I tried a variant of your command and seems to work fine here. Also
> >> note
> >> >>> that you need a background to show an image with overlay. Something
> >> >>> like:
> >> >>> ffplay -f lavfi color -vf "movie=composite.mov, [in] overlay"
> >> >>>
> >> >>> should do.
> >> >>>
> >> >>> > Thanks so much to anyone who can help me out here. I'm running out
> >> >>> > of
> >> >>> > things to try. I'd particularly like to see how people are using
> >> >>> > this
> >> >>> > filter successfully.
> >> >>> _______________________________________________
> >> >>> ffmpeg-user mailing list
> >> >>> ffmpeg-user at ffmpeg.org
> >> >>> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
> >> >>>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Tim Willison
> >> >> Lead Product Designer
> >> >> www.oddlystudios.com
> >> >> 495B Queen St. West
> >> >> Toronto, ON.
> >> >> 647 345 6275
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Tim Willison
> >> > Lead Product Designer
> >> > www.oddlystudios.com
> >> > 495B Queen St. West
> >> > Toronto, ON.
> >> > 647 345 6275
> >> > _______________________________________________
> >> > ffmpeg-user mailing list
> >> > ffmpeg-user at ffmpeg.org
> >> > http://ffmpeg.org/mailman/listinfo/ffmpeg-user
> >> >
> >> _______________________________________________
> >> ffmpeg-user mailing list
> >> ffmpeg-user at ffmpeg.org
> >> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
> >>
> >
> >
> >
> > --
> > Tim Willison
> > Lead Product Designer
> > www.oddlystudios.com
> > 495B Queen St. West
> > Toronto, ON.
> > 647 345 6275
> > _______________________________________________
> > ffmpeg-user mailing list
> > ffmpeg-user at ffmpeg.org
> > http://ffmpeg.org/mailman/listinfo/ffmpeg-user
> >
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
--
Tim Willison
Lead Product Designer
www.oddlystudios.com
495B Queen St. West
Toronto, ON.
647 345 6275
More information about the ffmpeg-user
mailing list