[FFmpeg-user] metadata on subtitle track's not readable by ffprobe
Robin van der Linden
linden at venom.one
Sat Sep 17 15:05:22 EEST 2022
Hello,
I'm building an industry grade streaming platform where I use ffmpeg and
ffprobe to convert and parse information from media objects. I'm using
ffprobe to read all available information from a mp4 file using the
following command:
ffprobe -print_format json -export_all true -show_streams -show_format
-loglevel quiet -hide_banner /path/to/file.mp4
But if I take a closer look at the output of it, I'm not able to check
if a subtitle is enforced, marked as SDH or auxiliary content. The only
thing I really see is when the subtitle is marked as a default track
which does not help at all, e.g.:
{
"index": 1,
"codec_name": "mov_text",
"codec_long_name": "MOV text",
"codec_type": "subtitle",
"codec_tag_string": "tx3g",
"codec_tag": "0x67337874",
"width": 1920,
"height": 162,
"id": "0x2",
"r_frame_rate": "0/0",
"avg_frame_rate": "0/0",
"time_base": "1/1000",
"start_pts": 0,
"start_time": "0.000000",
"duration_ts": 28237,
"duration": "28.237000",
"bit_rate": "44",
"nb_frames": "8",
"extradata_size": 48,
"disposition": {
"default": 1,
"dub": 0,
"original": 0,
"comment": 0,
"lyrics": 0,
"karaoke": 0,
"forced": 0,
"hearing_impaired": 0,
"visual_impaired": 0,
"clean_effects": 0,
"attached_pic": 0,
"timed_thumbnails": 0,
"captions": 0,
"descriptions": 0,
"metadata": 0,
"dependent": 0,
"still_image": 0
},
"tags": {
"creation_time": "2022-09-17T11:23:47.000000Z",
"language": "eng"
}
},
You might wonder what I mean by a subtitle is marked as SDH, enforced or
auxiliary content. These are extended information of a specific track
element of a subtitle embedded into a mp4 container, in most cases these
embedded subtitles come in tx3g format. The most advanced tool I know of
to write metadata and tracks into a mp4 container is called "Subler" and
it's sadly only available on MAC OS since many years.
Subler is extremely conform with apple's definition of mp4 containers
and how to "metarise" them, by that it very nicely integrates with
QuickTime, iOS etc. Subler also allows me to set a subtitle track of a
mp4 as SDH, enforced, auxiliary content etc. But I'm not able to read
this information later using ffprobe, which is absolutely mandatory for
me as I repack media from these highly metarised mp4 containers into
HLS/Dash streams where I really want to know if a subtitle is marked as
enforced or SDH. Otherwise, I cannot display this information to the
end-user using e.g. VideoJS, as all the Video players out there parse
their subtitle track names from a m3u8/mpd manifest you pass them. So
the only information I can rely on while building a HLS/DASH stream from
a mp4 input is the ffprobe's output, which currently does not cover
these fields I have with subler and would really solve this issue. The
information is there, but I can't read it using ffprobe to put it in
more simple words. The only thing I can do at the moment is to compare
the nb_frames value and if I have 2 subtitles in the same language.
Maybe I have 2x English subtitles, one is shorter than the other, than I
maybe know what of the two is the enforced subtitle track and which is
the full subtitle track ... This workaround kinda sucks, and I still
have no clue what track is marked as SDH.
I created an example mp4 file with 4 subtitle tracks in it, each of the
subtitle tracks covers a specific type of metadata setup which I want to
be able to read using ffprobe. I also included screenshots where I show
how each subtitle track is metarised under the use of Subler.
https://drive.google.com/file/d/1kShvOA6_Emx3COrMudn3V2__zErj8_-c/view?usp=sharing
If I parse the example file using the command I showd above, I get backe
the following which does not reflect the metadata I set befor using
subler, execpt of one single field "default": 1, this is the only one
that works for me:
{
"streams": [
{
"index": 0,
"codec_name": "h264",
"codec_long_name": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part
10",
"profile": "High",
"codec_type": "video",
"codec_tag_string": "avc1",
"codec_tag": "0x31637661",
"width": 1920,
"height": 1080,
"coded_width": 1920,
"coded_height": 1080,
"closed_captions": 0,
"film_grain": 0,
"has_b_frames": 2,
"pix_fmt": "yuv420p",
"level": 40,
"chroma_location": "left",
"field_order": "progressive",
"refs": 1,
"is_avc": "true",
"nal_length_size": "4",
"id": "0x1",
"r_frame_rate": "24000/1001",
"avg_frame_rate": "24000/1001",
"time_base": "1/24000",
"start_pts": 0,
"start_time": "0.000000",
"duration_ts": 677677,
"duration": "28.236542",
"bit_rate": "10833691",
"bits_per_raw_sample": "8",
"nb_frames": "677",
"extradata_size": 41,
"disposition": {
"default": 1,
"dub": 0,
"original": 0,
"comment": 0,
"lyrics": 0,
"karaoke": 0,
"forced": 0,
"hearing_impaired": 0,
"visual_impaired": 0,
"clean_effects": 0,
"attached_pic": 0,
"timed_thumbnails": 0,
"captions": 0,
"descriptions": 0,
"metadata": 0,
"dependent": 0,
"still_image": 0
},
"tags": {
"language": "eng",
"handler_name": "VideoHandler",
"vendor_id": "[0][0][0][0]"
}
},
{
"index": 1,
"codec_name": "mov_text",
"codec_long_name": "MOV text",
"codec_type": "subtitle",
"codec_tag_string": "tx3g",
"codec_tag": "0x67337874",
"width": 1920,
"height": 162,
"id": "0x2",
"r_frame_rate": "0/0",
"avg_frame_rate": "0/0",
"time_base": "1/1000",
"start_pts": 0,
"start_time": "0.000000",
"duration_ts": 28237,
"duration": "28.237000",
"bit_rate": "44",
"nb_frames": "8",
"extradata_size": 48,
"disposition": {
"default": 1,
"dub": 0,
"original": 0,
"comment": 0,
"lyrics": 0,
"karaoke": 0,
"forced": 0,
"hearing_impaired": 0,
"visual_impaired": 0,
"clean_effects": 0,
"attached_pic": 0,
"timed_thumbnails": 0,
"captions": 0,
"descriptions": 0,
"metadata": 0,
"dependent": 0,
"still_image": 0
},
"tags": {
"creation_time": "2022-09-17T11:23:47.000000Z",
"language": "eng"
}
},
{
"index": 2,
"codec_name": "mov_text",
"codec_long_name": "MOV text",
"codec_type": "subtitle",
"codec_tag_string": "tx3g",
"codec_tag": "0x67337874",
"width": 1920,
"height": 162,
"id": "0x3",
"r_frame_rate": "0/0",
"avg_frame_rate": "0/0",
"time_base": "1/1000",
"start_pts": 0,
"start_time": "0.000000",
"duration_ts": 28237,
"duration": "28.237000",
"bit_rate": "74",
"nb_frames": "14",
"extradata_size": 48,
"disposition": {
"default": 0,
"dub": 0,
"original": 0,
"comment": 0,
"lyrics": 0,
"karaoke": 0,
"forced": 0,
"hearing_impaired": 0,
"visual_impaired": 0,
"clean_effects": 0,
"attached_pic": 0,
"timed_thumbnails": 0,
"captions": 0,
"descriptions": 0,
"metadata": 0,
"dependent": 0,
"still_image": 0
},
"tags": {
"creation_time": "2022-09-17T11:23:47.000000Z",
"language": "eng"
}
},
{
"index": 3,
"codec_name": "mov_text",
"codec_long_name": "MOV text",
"codec_type": "subtitle",
"codec_tag_string": "tx3g",
"codec_tag": "0x67337874",
"width": 1920,
"height": 162,
"id": "0x4",
"r_frame_rate": "0/0",
"avg_frame_rate": "0/0",
"time_base": "1/1000",
"start_pts": 0,
"start_time": "0.000000",
"duration_ts": 28237,
"duration": "28.237000",
"bit_rate": "82",
"nb_frames": "14",
"extradata_size": 48,
"disposition": {
"default": 0,
"dub": 0,
"original": 0,
"comment": 0,
"lyrics": 0,
"karaoke": 0,
"forced": 0,
"hearing_impaired": 0,
"visual_impaired": 0,
"clean_effects": 0,
"attached_pic": 0,
"timed_thumbnails": 0,
"captions": 0,
"descriptions": 0,
"metadata": 0,
"dependent": 0,
"still_image": 0
},
"tags": {
"creation_time": "2022-09-17T11:23:47.000000Z",
"language": "eng"
}
},
{
"index": 4,
"codec_name": "mov_text",
"codec_long_name": "MOV text",
"codec_type": "subtitle",
"codec_tag_string": "tx3g",
"codec_tag": "0x67337874",
"width": 1920,
"height": 162,
"id": "0x5",
"r_frame_rate": "0/0",
"avg_frame_rate": "0/0",
"time_base": "1/1000",
"start_pts": 0,
"start_time": "0.000000",
"duration_ts": 28237,
"duration": "28.237000",
"bit_rate": "94",
"nb_frames": "14",
"extradata_size": 48,
"disposition": {
"default": 0,
"dub": 0,
"original": 0,
"comment": 0,
"lyrics": 0,
"karaoke": 0,
"forced": 0,
"hearing_impaired": 0,
"visual_impaired": 0,
"clean_effects": 0,
"attached_pic": 0,
"timed_thumbnails": 0,
"captions": 0,
"descriptions": 0,
"metadata": 0,
"dependent": 0,
"still_image": 0
},
"tags": {
"creation_time": "2022-09-17T11:23:47.000000Z",
"language": "deu"
}
}
],
"format": {
"filename": "subtitle_sample_1920x1080.mp4",
"nb_streams": 5,
"nb_programs": 0,
"format_name": "mov,mp4,m4a,3gp,3g2,mj2",
"format_long_name": "QuickTime / MOV",
"start_time": "0.000000",
"duration": "28.237000",
"size": "38253389",
"bit_rate": "10837805",
"probe_score": 100,
"tags": {
"major_brand": "mp42",
"minor_version": "512",
"compatible_brands": "isomiso2avc1mp41",
"tagc": "public.main-program-content"
}
}
}
It would really be awesome if somebody could add compatibility here, as
in the end I just need to be able to read if the subtitle is enforced or
SDH, everything else is just "Main Program" to me. Adding this feature
would also make it much simpler for other developers to build proper
streaming outputs.
--
Kind regards,
Robin Linden
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0x257F9C8D.asc
Type: application/pgp-keys
Size: 1753 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-user/attachments/20220917/ba817230/attachment.key>
More information about the ffmpeg-user
mailing list