[FFmpeg-user] optimal exact segmentation without re-encoding

Eduardo Alarcón ealarcong at gmail.com
Fri Jul 7 02:20:06 EEST 2023 

El jue, 6 jul 2023 a las 17:43, David Bernat (<david.bernat at gmail.com>)
escribió:

> On Thu, Jul 6, 2023 at 4:02 PM Eduardo Alarcón <ealarcong at gmail.com>
> wrote:
>
> > El jue, 6 jul 2023 a las 12:16, David Bernat (<david.bernat at gmail.com>)
> > escribió:
> >
> > > At the risk of being rude, Eduardo Alarcón, (and me top-commenting),
> > please
> > > do not thread-hijack.
> > >
> > How is this thread-hijack? do you remember this is a mailing list,
> right? i
> > have done something similar to what you want to do, so i'm telling you
> what
> > i did to help.
> >
> >
> > Carl Zwanzig and I are having a very careful conversation and I wish to
> > > continue with him.
> > > The use of the SEGMENT command requires re-encoding, in my experience,
> > even
> > > though each segment can be set to go from keyframe to keyframe.
> > > If this is incorrect, I look to Carl Zwanzig for more information at
> this
> > > time, commiserate with my detailed reply. Thank you both.
> > >
> >
> > As per example on the documentation, it doesn't require re-encoding :
> >
> > Segment the input file, and create an M3U8 live playlist (can be used as
> > live HLS source):
> > ffmpeg -re -i in.mkv -codec copy -map 0 -f segment -segment_list
> > playlist.m3u8 -segment_list_flags +live -segment_time 10 out%03d.mkv
> >
> > You can use -codec copy and do not need to reencode, ffmpeg will try find
> > the closet keyframe to the segment time you defined without re-encoding,
> as
> > you asked.
> >
> >
> You are correct, and I am wrong. This segmentation method does seem to do
> the similar segmentation
> as my use of copy at key_frames, and is much faster.
>
> It does, however, still possess extra frames when concatenation. If this
> problem is not immediately
> apparent I will post code and the warning messages associated to it later
> in the evening.
>
> Thank you for your response. It was valuable, and not thread-hackingl.
>
> Happy to help.

Try this "template" to generate .ts segment files:
ffmpeg -i <source> -y -codec copy -bsf:v h264_mp4toannexb -f segment
-segment_time <seconds> -segment_time_delta 0.05 %03d.ts

MPEG-TS container is concatenable directly using "cat" from linux, after
that you could use ffmpeg to change the container, .ts to .mov or .mp4
without reencoding (-codec copy).
May be this doesn't generate those extra keyframes, the option
-segment_time_delta 0.05 has something to do with the boundaries where
ffmpeg will make a segment, i remember it was better this way, but i don't
remember exactly why.

Good luck

>
> > >
> > > > On Wed, Jul 5, 2023 at 9:39 PM Carl Zwanzig <cpz at tuunq.com> wrote:
> > > > >
> > > > > > On 7/5/2023 3:29 PM, David Bernat wrote:
> > > > > > > Premise: segment a mov file into about one second segments
> > without
> > > > > > > re-encoding; yet preserving concatenation; such that the
> > > segmentation
> > > > > is
> > > > > > > embarrassingly parallel, for high-speed segmenting.
> > > > > >
> > > > > > What is the purpose of these segmented files (how will they be
> > used)?
> > > > Are
> > > > > > they going to be processed and then assembled together (which may
> > > > involve
> > > > > > decode/recode operations). Depending on the unknown intermediate
> > > work,
> > > > it
> > > > > > may be faster overall to decode all into an uncompressed state
> and
> > > work
> > > > > on
> > > > > > that before concatenating into a reencode pass (which could also
> be
> > > > done
> > > > > > in
> > > > > > segments which are then themselves concatenated).
> > > > > >
> > > > >
> > > > > The parallelization of processing is an essential motivation.
> (Thank
> > > you
> > > > > and looking forward to forwarding this discussion and solution with
> > > you.)
> > > > >
> > > > > Multiple uses but may we focus on two:
> > > > >
> > > > > 1. computer vision and thumbnails:
> > > > >
> > > > > Object detection runs quickly on images; and, in most cases, it is
> > > > > sufficient to summarize a long video by applying object detection
> > > every 1
> > > > > or 2 seconds.
> > > > > Extracting a frame using -ss is very fast but not immediate: my
> speed
> > > > tests
> > > > > indicate that a CPU can achieve about 60 image extractions in about
> > 15
> > > > > seconds.
> > > > > ffmpeg -ss can be embarrassingly parallelized and does speed up
> from
> > > > about
> > > > > 18 seconds to 15 seconds.
> > > > > The precise timestamp of the image is not essential (within a few
> > > frames
> > > > is
> > > > > more than sufficient) and so ffmpeg -ss is sufficient.
> > > > > Accelerating this even further is a wonderful benefit.
> > > > > Furthermore, if the images are already segmented, each segment can
> be
> > > > > handled separately on a different cloud unit or CPU.
> > > > > In this case, motivating a storage option that is segmented is the
> > key.
> > > > > Notice that this method does not strictly require that
> recomposition
> > is
> > > > > achievable.
> > > > >
> > > > > 2. storage and generic processing:
> > > > >
> > > > > This adds the additional requirement that recomposition is
> > achievable.
> > > > > Numerous use cases apply the parallelization scheme described
> above.
> > > > > It would be hugely beneficial for the storage efficiencies gained
> > from
> > > > the
> > > > > above also constitute the full storage of the video.
> > > > > In an extreme case, you can imagine a video player, in which the
> one
> > > > second
> > > > > video clips are also serving a make-shift streaming solution.
> > > > > Whether this is the best example of that use case the concept is
> > > > sufficient
> > > > > and additional use cases are welcomed.
> > > > >
> > > > >
> > > > > > Also, please post some of the commands you've used.
> > > > > >
> > > > >
> > > > > Here is an overview of the processes I am using at the moment:
> > > > >
> > > > > *This series of commands identifies the KEYFRAMES.*
> > > > >
> > > > > # cmd = "ffprobe -loglevel error -select_streams v:0 -show_entries
> > > > > packet=pts_time,flags -of csv=print_section=0 IMG_9209.MOV | awk
> > -F','
> > > > > '/K/ {print $1}'"
> > > > > # result = subprocess.run(cmd, shell=True, cwd=os.getcwd(),
> > > > > capture_output=True)
> > > > >
> > > > > # key_frames = [float(t) for t in result.stdout.decode().split()]
> ==>
> > > > > [0.000000, 1.068333, 2.135000, 3.211667, 4.295000]
> > > > >
> > > > > This command creates SEGMENTs from KEYFRAME to KEYFRAME but is not
> > > > > precisely exact, and hence fails the concatenation requirement.
> > > > >
> > > > > # cmd = "ffmpeg -y -ss {key_frames[i]} -to {key_frames[i+1]} -i
> > > > > IMG_9209.MOV -c copy segment_{i}.move"
> > > > >
> > > > > # The output of SEEK will have ffprobe results at timestamps like
> > this
> > > > > (with K being keyframe and D being negative):
> > > > > # [0.000000,K_ -0.066667,_D -0.100000,_D -0.033333,_D 0.135000,__
> > > > > 0.066667,__ 0.033333,__ 0.100000,__ 0.285000,__ ...
> > > > > # ... 0.826667,__ 0.910000,__ 1.076667,K_ 1.035000,__ 0.993333,__
> > > > > 1.243333,__ 1.160000,__]
> > > > >
> > > > > # Notice that 2.135000 - 1.068333 = 1.076667 as this is
> > segment_2.mov;
> > > > > and timestamps are reset to start at zero.
> > > > >
> > > > > This command TRIMs each SEGMENT and does successfully fulfill the
> > > > > concatenation requirement; but is very, very slow (0.25x real time)
> > > > >
> > > > > # cmd = "ffmpeg -i segment_2.mov -vf trim=0:1.076667
> > > segment_2_trim.mov"
> > > > > #
> > > > > # The output of TRIM takes 4-5 seconds despite being a 1.07 second
> > > > > clip: re-encoding is always slow. But, it is correct.
> > > > > # [0.000000,K_ 0.033333,__ 0.016667,__ 0.008333,__ 0.025000,__
> > > > > 0.066667,__ 0.050000,__ 0.041667,__ ...
> > > > > # ... 1.016667,__ 1.058333,__ 1.041667,__ 1.033333,__ 1.050000,__]
> > > > >
> > > > > This python code launches CONCAT:
> > > > >
> > > > > # s = time.time()
> > > > > # tmp = tempfile.NamedTemporaryFile(delete=False)
> > > > > # relpath = os.path.relpath(os.getcwd(), tmp.name)
> > > > > # with open(tmp.name, "w") as f:
> > > > > #     [f.write(f"file '{relpath}/segment_{i}.mov'\n") for i in
> > > range(28)]
> > > > > # cmd = f"ffmpeg -y -f concat -safe 0 -i {tmp.name} -c copy
> > > > > concatenated.mov"
> > > > > # subprocess.run(cmd.split(), shell=False, cwd=os.getcwd())
> > > > > # print(time.time()-s)
> > > > >
> > > > > [this next one does not work: it is intended to create keyframes
> > every
> > > > > one second. though this entire avenue may be unnecessary.]
> > > > >
> > > > > # cmd = f"ffmpeg -y -i IMG_9209.MOV -force_key_frames
> > > > > expr:gte(t,n_forced*1) -c copy keyframes.mov"
> > > > >
> > > > >
> > > > > Thank you.
> > > > > DB
> > > > >
> > > > >
> > > >
> > > > >
> > > > >
> > > > >
> > > > > > z!
> > > > > > _______________________________________________
> > > > > > ffmpeg-user mailing list
> > > > > > ffmpeg-user at ffmpeg.org
> > > > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> > > > > >
> > > > > > To unsubscribe, visit link above, or email
> > > > > > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
> > > > > >
> > > > > _______________________________________________
> > > > > ffmpeg-user mailing list
> > > > > ffmpeg-user at ffmpeg.org
> > > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> > > > >
> > > > > To unsubscribe, visit link above, or email
> > > > > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
> > > > >
> > > > _______________________________________________
> > > > ffmpeg-user mailing list
> > > > ffmpeg-user at ffmpeg.org
> > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> > > >
> > > > To unsubscribe, visit link above, or email
> > > > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
> > > >
> > > _______________________________________________
> > > ffmpeg-user mailing list
> > > ffmpeg-user at ffmpeg.org
> > > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> > >
> > > To unsubscribe, visit link above, or email
> > > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
> > >
> > _______________________________________________
> > ffmpeg-user mailing list
> > ffmpeg-user at ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
> >
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
>

More information about the ffmpeg-user mailing list