[FFmpeg-user] Timing issues with ffmpeg scene detection & extraction

[Michael]

Hi all,

I would like to do scene detection & frame extraction with ffmpeg, but despite investing multiple days I still have issues with the timing of frames. My goal is to 1) get new scene timestamps 2) get new scene first frame as image and 3) create one still image per second. Can anyone help on this?

First I used ffprobe which outputs the first frame of a newly detected scene/shot.
ffprobe -show_frames -of compact=p=0 -f lavfi "movie=input.avi,select=gt(scene\\,0.3)"
media_type=video|key_frame=1|pkt_pts=19|pkt_pts_time=0.760000|pkt_dts=19|pkt_dts_time=0.760000|pkt_duration=1|pkt_duration_time=0.040000|pkt_pos=77019|width=512|height=288|pix_fmt=rgb24|sample_aspect_ratio=1:1|pict_type=I|coded_picture_number=0|display_picture_number=0|interlaced_frame=0|top_field_first=0|repeat_pict=0|reference=0

Then I tried to extract the frames with ffmpeg (-ss, before and after -i), but the timing seemed to be off, the image flow was not correct.

Next I used ffmpeg with the scene detection integrated, outputing text and images
ffmpeg -v debug -i input.avi -vsync vfr -vf "select='gt(scene,0.3)',showinfo" shot%06d.jpg
[Parsed_showinfo_1 @ 0x17001e0] n:0 pts:19 pts_time:0.76 pos:77019 fmt:rgb24 sar:1/1 s:512x288 i:P iskey:0 type:B checksum:2E612F60 plane_checksum:[2E612F60]

Sidenote: without "-v debug" the number of printed scenes varies every time, seems like a bug.
ffmpeg -i input.avi -vsync vfr -vf "select='gt(scene,0.3)',showinfo" shot%06d.jpg 2>&1 | grep Parsed_showinfo_1 | wc -l

Now I have timestamp and first frame, but I can't extract a correct frame per second. Standard frame extraction is simple, but the timing is off compared to the scene detection frames. I tried

ffmpeg -i ./input.avi -f image2 -vf fps=fps=1 still_%06d.jpg
ffmpeg -i ./input.avi -f image2 -vsync vfr -vf fps=fps=1 still_%06d.jpg

Despite having image "Sec10" "Scene10.2" "Sec11", the "Scene10.2" is after Sec11 in the video, so either the scene extraction or the still extraction is not correct in timing.

Can I do all three tasks [1) scene timestamp, 2) scene image, 3) sec image] in a single command? Or how do I get the same correct timeline for all extracted images?

Thanks
Michael