[FFmpeg-user] how to detect changes between frames - for a very simple AVI video where most of the frames are exactly identical (just a sporadically updating computer screen capture)

[Stefan Scott Alexander]

Hi -

I have a very simple video which is a screen capture of speech-synthesis
software running -
It just displays some text, which is motionless. The only "action" is where
the current word being spoken gets highlighted.

So the first sequence of frames are all exactly identical (the first word in
the text is highlighted).
Then suddenly the second word in the text is highlighted during the second
sequence of frames, etc.

I want to generate an array of timestamps for all the frames in this video
which differ from the preceding frame.

I've already inspected this video frame-by-frame in VirtualDub.
There are no fancy "fades" or other special effect going on with the
highlighting of the words:
eg frames 1-552 are exactly identical (the highlight is on the first word)
Then frames 553-662 are all exactly identical (the highlight is on the
second word), etc.

I want to use ffmpeg to automatically detect the moments when each word gets
highlighted,
and output an array of timestamps. So if the text being spoken has 100 words
being highlighted,
then I should output an array of 100 timestamps.

(The highlighting is already synched with the audio by the speech-synthesis
software itself.
So I can actually ignore the audio.
The timestamps are guaranteed to automatically line up with the audio,
since the highlights are already in synch with the audio.)

This is much easier than typical "motion detection" or "scene change
detection" in video applications.
In most other videos, there's lots of stuff moving around
so you have to set the scene change detection "threshold" or "sensitivity".
eg it could be 30%, which means that whenever 30% or more of the pixels
changed between frames,
you're gonna regard that as a scene change.

But for this sort of video, which is really just a subsequence of identical
frames, then another subsquence of identical frames,
it should be easy to detect those few moments where the video actually
changes from one frame to the next.
If a frame differs from the preceding frame by even just one pixel - well,
then that's a "scene change" in this case,
and I want to append the current timestamp to the timestamp array which will
then get outputted at the end.


I saw a similar question here:

http://ffmpeg-users.933282.n4.nabble.com/Scene-detection-td941845.html

It looks like ffmpeg doesn't officially do motion detection or scene change
detection,
but I'm encouraged by seeing that ffmpeg has a command-line interface.

ffmpeg does lots of stuff, so I basically am hoping someone can point me to
the commands which allow frame-by-frame comparisons,
plus outputting timestamps.

Even if ffmpeg can just parse my AVI video into a raw array of frame data -
I have some other programs which could then process that array of frame
data
and detect the changes and compute the timestamps.

Thanks for any pointers!