[FFmpeg-user] how to detect changes between frames - for a very simple AVI video where most of the frames are exactly identical (just a sporadically updating computer screen capture)

[belcampo]

On 10/27/2011 02:19 AM, Stefan Scott Alexander wrote:
> Hi -
>
> I have a very simple video which is a screen capture of speech-synthesis
> software running -
> It just displays some text, which is motionless. The only "action" is where
> the current word being spoken gets highlighted.
>
> So the first sequence of frames are all exactly identical (the first word in
> the text is highlighted).
> Then suddenly the second word in the text is highlighted during the second
> sequence of frames, etc.
>
> I want to generate an array of timestamps for all the frames in this video
> which differ from the preceding frame.
>
> I've already inspected this video frame-by-frame in VirtualDub.
> There are no fancy "fades" or other special effect going on with the
> highlighting of the words:
> eg frames 1-552 are exactly identical (the highlight is on the first word)
> Then frames 553-662 are all exactly identical (the highlight is on the
> second word), etc.
>
> I want to use ffmpeg to automatically detect the moments when each word gets
> highlighted,
> and output an array of timestamps. So if the text being spoken has 100 words
> being highlighted,
> then I should output an array of 100 timestamps.
>
> (The highlighting is already synched with the audio by the speech-synthesis
> software itself.
> So I can actually ignore the audio.
> The timestamps are guaranteed to automatically line up with the audio,
> since the highlights are already in synch with the audio.)
>
> This is much easier than typical "motion detection" or "scene change
> detection" in video applications.
> In most other videos, there's lots of stuff moving around
> so you have to set the scene change detection "threshold" or "sensitivity".
> eg it could be 30%, which means that whenever 30% or more of the pixels
> changed between frames,
> you're gonna regard that as a scene change.
>
> But for this sort of video, which is really just a subsequence of identical
> frames, then another subsquence of identical frames,
> it should be easy to detect those few moments where the video actually
> changes from one frame to the next.
> If a frame differs from the preceding frame by even just one pixel - well,
> then that's a "scene change" in this case,
> and I want to append the current timestamp to the timestamp array which will
> then get outputted at the end.
>
>
> I saw a similar question here:
>
> http://ffmpeg-users.933282.n4.nabble.com/Scene-detection-td941845.html
>
> It looks like ffmpeg doesn't officially do motion detection or scene change
> detection,
> but I'm encouraged by seeing that ffmpeg has a command-line interface.
>
> ffmpeg does lots of stuff, so I basically am hoping someone can point me to
> the commands which allow frame-by-frame comparisons,
> plus outputting timestamps.
Maybe following this thread and following the suggestions and a 
mentioned script you'll find something that will give you the right 
direction.
http://ffmpeg-users.933282.n4.nabble.com/Creating-a-list-of-I-frames-td2299316.html
>
> Even if ffmpeg can just parse my AVI video into a raw array of frame data -
> I have some other programs which could then process that array of frame
> data
> and detect the changes and compute the timestamps.
>
> Thanks for any pointers!
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-user