[FFmpeg-devel] [PATCH v5 24/25] fftools/ffmpeg: Introduce subtitle filtering and new frame-based subtitle encoding

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> Andreas Rheinhardt
> Sent: Saturday, June 25, 2022 3:01 PM
> To: ffmpeg-devel at ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v5 24/25] fftools/ffmpeg:
> Introduce subtitle filtering and new frame-based subtitle encoding
> 
> Soft Works:
> >
> >
> > ________________________________________
> > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> on behalf of
> Andreas Rheinhardt <andreas.rheinhardt at outlook.com>
> > Sent: Saturday, June 25, 2022 1:42 PM
> > To: ffmpeg-devel at ffmpeg.org
> > Subject: Re: [FFmpeg-devel] [PATCH v5 24/25] fftools/ffmpeg:
> Introduce subtitle filtering and new frame-based subtitle encoding
> >
> > Michael Niedermayer:
> >> On Sat, Jun 25, 2022 at 09:57:56AM +0000, softworkz wrote:
> >>> From: softworkz <softworkz at hotmail.com>
> >>>
> >>> This commit actually enables subtitle filtering in ffmpeg by
> >>> sending and receiving subtitle frames to and from a filtergraph.
> >>>
> >>> The heartbeat functionality from the previous sub2video
> implementation
> >>> is removed and now provided by the 'subfeed' filter.
> >>> The other part of sub2video functionality is retained by
> >>> auto-insertion of the new graphicsub2video filter.
> >>>
> >>> Justification for changed test refs:
> >>>
> >>> - sub2video
> >>>   The previous results were incorrect. The command line for this
> test
> >>>   specifies -r 5 (5 frames per second), which is now fulfilled by
> the
> >>>   additional frames in the reference output.
> >>>   Example: The first subtitle time is 499000, the second is
> 15355000,
> >>>   which means 0.5s and 15.35s with a difference of 14.85s.
> >>>   15s * 5fps = 75 frames and that's now the exact number of video
> >>>   frames between these two subtitle events.
> >>>
> >>> - sub2video_basic
> >>>   The previous results had some incorrect output because multiple
> >>>   frames had the same dts
> >>>   The non-empty content frames are visually identical, the
> different
> >>>   CRC is due to the different blending algorithm that is being
> used.
> >>>
> >>> - sub2video_time_limited
> >>>   Subtitle frames are emitted to the filter graphs at a 5 fps
> rate
> >>>   by default. The time limit for this test is 15s * 5fps = 75
> frames
> >>>   which matches the count in the new ref.
> >>>
> >>> - sub-dvb
> >>>   Running ffprobe -show_frames on the source file shows that
> there
> >>>   are 7 subtitle frames with 0 rects in the source at the start
> >>>   and 2 at the end. This translates to the 14 and 4 additional
> >>>   entries in the new test results.
> >>>
> >>> - filter-overlay-dvdsub-2397
> >>>   Overlay results have slightly different CRCs due to different
> >>>   blending implementation
> >>>
> >>> - sub-scc
> >>>   The first entry is no longer in the output because it is before
> >>>   the actual start time and the strim filter removes such entries
> >>>   now (like for video and audio)
> >>>
> >>> Signed-off-by: softworkz <softworkz at hotmail.com>
> >>> ---
> >>>  fftools/ffmpeg.c                          |  613 +++++-----
> >>>  fftools/ffmpeg.h                          |   17 +-
> >>>  fftools/ffmpeg_filter.c                   |  270 +++--
> >>>  fftools/ffmpeg_hw.c                       |    2 +-
> >>>  fftools/ffmpeg_opt.c                      |   28 +-
> >>>  tests/ref/fate/filter-overlay-dvdsub-2397 |  182 +--
> >>>  tests/ref/fate/sub-dvb                    |  162 +--
> >>>  tests/ref/fate/sub-scc                    |    1 -
> >>>  tests/ref/fate/sub2video                  | 1091
> +++++++++++++++++-
> >>>  tests/ref/fate/sub2video_basic            | 1238
> +++++++++++++++++++--
> >>>  tests/ref/fate/sub2video_time_limited     |   78 +-
> >>>  11 files changed, 3010 insertions(+), 672 deletions(-)
> >>
> >> seems to break fate
> >>
> >> Input #0, ass, from 'fate-suite//sub/1ededcbd7b.ass':
> >>   Duration: N/A, bitrate: N/A
> >>   Stream #0:0: Subtitle: ass
> >> Codec AVOption threads (set the number of threads) specified for
> input file #0 (fate-suite//sub/1ededcbd7b.ass) has not been used for
> any stream. The most likely reason is either wrong type (e.g. a video
> option with no video streams) or that it is a private option of some
> decoder which was not actually used for any stream.
> >> Codec AVOption thread_type (select multithreading type) specified
> for input file #0 (fate-suite//sub/1ededcbd7b.ass) has not been used
> for any stream. The most likely reason is either wrong type (e.g. a
> video option with no video streams) or that it is a private option of
> some decoder which was not actually used for any stream.
> >> Stream mapping:
> >>   Stream #0:0 -> #0:0 (ass (ssa) -> ass (ssa))
> >> cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is
> harmless if it occurs once at the start per stream)
> >> subtitle input filter: decoding size 1280x720
> >> [out_0_0 @ 0x557cab64bbc0] Subtitle format change from 1 to 3
> >> Assertion c > 0 failed at libavutil/mathematics.c:61
> >
> > That's an av_assert2-assert. softworkz probably didn't test these.
> >
> >
> > Yup. To be honest, I wasn't even aware that I'm supposed to. It
> seems patchwork doesn't do either?
> >
> > Do I need a different configure or make fate?
> >
> 
> The supported way to set it is to pass --assert-level=2 to configure.
> (You can also modify config.h by adding "#define ASSERT_LEVEL 2"; if
> you
> have not used assert-level during configure, you should have no
> ASSERT_LEVEL in config.h, but better check this.)

Both ways working fine, thanks a lot for the explanation.

@Michael - thanks for testing locally. I was able to identify the issue
which came from adding the new strim filter.

Update is on the way.

Thanks,
softworkz