[Libav-user] Pulling valid subtitle from lavfi

[Deron]

On 12/19/23 1:36 AM, Peter Krefting wrote:
> Hi!
>
>> The video and audio comes out just fine, but the subtitle packets all 
>> have a length of 30 bytes, and the data appears to be garbage (well, 
>> could be raw data I've not explored that).
>
> Are you getting bitmap subtitles? If AVSubtitle::format == 0, you have 
> bitmap subtitles.
>
> The format is not immediately obvious, have a look at these posts for 
> some help on how to use them:
>
> https://stackoverflow.com/a/54143538
> https://stackoverflow.com/a/60560764
>
> The former has a code snippet that lets you convert an AVSubtitle into 
> an AVFrame that can be input into an encoder to make an image or video 
> stream for overlay.
>
Thanks for the assist!

The issue was a bit more basic. I was not using avcodec_decode_subtitle2 
and avcodec_encode_subtitle. Easy to overlook if you don't know you need 
it. My documentation is only the ffmpeg tool code itself, which has all 
kind of conditional code to handle exceptions with no explanation of 
what it is trying to fix.

In light of this new information and deciphering what ffmpeg is doing, I 
can decode the subtitle packets coming out of the lavfi filter and back 
out. I'm still not 100%  on the AVSubtitle structure and what it means.

1) AVSubtitle::start_display_time and AVSubtitle::end_display_time are 
always in milliseconds?

2) AVSubtitle::start_display_time and AVSubtitle::end_display_time are 
offsets from AVSubtitle::pts?

Would be great if AVSubtitle had time_base and opaque, but even better 
if it was an AVFrame as referenced in comments in ffmpeg.


The biggest questions are around this subtitle heartbeat in ffmpeg. What 
issue is it trying to solve? I'm guessing I'll eventually run into the 
issue myself and then it will all be made clear, but I'd like to learn 
something the easy way once in awhile :-) Maybe to cause HLS 
segmentation to output an empty subtitle file to keep it numerically in 
sync with the video file name count?


Why does ffmpeg (conditionally) decompose the AVSubtitle into smaller 
AVSubtitles before feeding it to avcodec_encode_subtitle?


The next issue is I often get "[Closed caption Decoder @ 0x7fffa40a4f40] 
Data Ignored since exceeding screen width" errors. I see in 
libavcodec/ccaption_dec.c where the error is generated, but it is not 
some configurable value for width but instead a fixed value of 32 
characters. What would cause this? No demux errors reported, and the 
receiver reports no uncorrected errors in the data stream. Even ffmpeg 
will report this, so not specific to something I am doing.


Deron