[FFmpeg-devel] Experiences in using ffmpeg to transcode broadcast video

[Mika Raento]

A follow-up (I'll write this up better somewhere too once I get a chance).

I've rewritten my transcoding pipeline, avoiding -copyts and it looks
pretty promising. I now:

1. cut the input mpegts into pieces that have a) the same aspect ratio and
b) monotonic timestamps (for both audio and video); I cut it using dd
(rather than ffmpeg), which can't go wrong :-)

2. sync audio and video, overlay subtitles and transcode the pieces
(skipping pieces that are too small to contain decodable audio and video)
into mp4, with a GOP length suitable for fragmenting

3. concatenate the transcoded mp4 pieces with the concat demuxer

4. fragment the concatenated mp4 into ismv

Now I'm using ffmpeg's reasonably robust handling of mpegts (given
monotonic timestamps) without needing -copyts, as the concat demuxer will
fix up the timestamps for the pieces so that the pieces can all start at 0
and I'm syncing up audio and video before concatenating.

There are still places that can be a bit flaky (what is 'too small'?,
handling those frames that straddle the discontinuity where dts is on one
side and pts on the other, maybe badly broken audio input can't be synced).

    Mika


On 29 October 2014 13:49, Mika Raento <mikie at iki.fi> wrote:

> Tickets for sub2video and async not working with non-monotonic input,
> using -copyts:
>
> https://trac.ffmpeg.org/ticket/4062
> https://trac.ffmpeg.org/ticket/4064
>
> Hm. Lots of my issues seem to relate to using -copyts. I had trouble
> keeping all video and all audio if there were periods where one or the
> other were missing (esp. in the beginning). Maybe I could make this simpler
> if I'd relax that requirement (I don't know how much happier my users are
> if they get audio but no video, rather than nothing).
>
>     Mika
>
> On 29 October 2014 10:38, Mika Raento <mikie at iki.fi> wrote:
>
>> segment muxer PIDs were fixed in
>> https://github.com/FFmpeg/FFmpeg/commit/502fc3b3d4b36015562d19d74f27d0a4ff835c4e
>> by me
>>
>> On 29 October 2014 10:17, Mika Raento <mikie at iki.fi> wrote:
>>
>>> Hi
>>>
>>> Thanks for the feedback.
>>>
>>> I'll try to capture at least some of these in tickets, and reply on this
>>> thread as I go along.
>>>
>>> Async not working with damaged audio input is
>>> https://trac.ffmpeg.org/ticket/2693 (existing ticket)
>>>
>>> Sub2video not working with negative timestamps is
>>> https://trac.ffmpeg.org/ticket/4062 (new ticket)
>>>
>>>     Mika
>>>
>>> On 27 October 2014 01:00, Michael Niedermayer <michaelni at gmx.at> wrote:
>>>
>>>> On Wed, Sep 24, 2014 at 10:43:40AM +0300, Mika Raento wrote:
>>>> > Dear all
>>>> >
>>>> > This mail is meant mainly as a note to other potential users, but
>>>> > possibly as input to development - time allowing I might be able to
>>>> pick
>>>> > up some of the pieces myself. It's also a thank-you for all the hard
>>>> > work in ffmpeg.
>>>> >
>>>> > I've successfully implemented a transcoding pipeline for producing
>>>> > multi-bitrate fragmented mp4 files from broadcast DVB input. More
>>>> > concretely, I'm taking in broadcast TS captures with either mpeg2
>>>> video
>>>> > and mp2 audio (SD, varying aspect ratio) or h264 video and aac audio,
>>>> > both potentially with dvbsub. From that I'm producing ISMV output with
>>>> > multiple bitrate h264 video at fixed 16:9 aspect ratio and multiple
>>>> > bitrate aac audio, with burned subtitles. The ISMV outputs are
>>>> > post-processed with tools/ismindex and with the hls muxer.
>>>> >
>>>> > There are number of limitations in ffmpeg that I've had to work
>>>> around:
>>>> >
>>>> > - I haven't gotten sub2video or async working without reasonably
>>>> >   monotonous DTS. Broadcast TS streams can easily contain backward
>>>> jumps
>>>> >   in time (e.g., a cheapo source that plays mp4 files and starts each
>>>> file
>>>> >   at 0). The fix is to cut the TS into pieces at timestamp jump
>>>> locations
>>>> >   and using '-itsoffset' to rewrite the timestamps and then
>>>> concatenate.
>>>> >   I'm using the segment and concat muxers for that.
>>>> > - Sub2video doesn't work with negative timestamps, so I use
>>>> '-itsoffset'
>>>> >   to get positive timestamps
>>>> > - For HD streams, I need to scale up the sub2video results from SD to
>>>> >   HD. Sub2video doesn't handle the HD subtitle geometries. I'm not
>>>> >   enough of an expert to know whether that's the issue, or whether
>>>> that's
>>>> >   just the way it's supposed to work with SD subs (typical) with HD
>>>> video.
>>>> > - For columnboxing, I use the scale, pad and setdar video filters.
>>>> These
>>>> >   work fine, but their parameters are only evaluated at start, so I
>>>> need
>>>> >   to cut the video into pieces with a single aspect ratio first and
>>>> >   concatenate later.
>>>> > - Audio sync (using aresample) gets confused if the input contains
>>>> >   errors, so I need to first re-encode audio (with '-copyts') and only
>>>> >   after that synchronize.
>>>> > - The TS PIDs are not kept over the segment muxer, so I given them on
>>>> >   the command line with '-streamid'.
>>>>
>>>> a bit late reply but
>>>> bug reports or feature requests for all these are welcome unless there
>>>> are already tickets for them assuming these issues still exist
>>>>
>>>> thx
>>>>
>>>> [...]
>>>>
>>>> --
>>>> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>>>>
>>>> Why not whip the teacher when the pupil misbehaves? -- Diogenes of
>>>> Sinope
>>>>
>>>> _______________________________________________
>>>> ffmpeg-devel mailing list
>>>> ffmpeg-devel at ffmpeg.org
>>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>>>
>>>>
>>>
>>
>