[FFmpeg-user] Chunked request when using "-progress"

[Tom Evans]

On Thu, Dec 13, 2012 at 5:16 PM, Вадим Лазовский
<vadim.lazovskiy at gmail.com> wrote:
> I've just installed nginx 1.3.9 with chunked input support.
> And still it behaves not as I am expecting.
>
> As described in manpage:
>
>        -progress url (global)
>            Send program-friendly progress information to url.
>
>            Progress information is written approximately every second and
> at the end of the encoding process. It is made of "key=value" lines. key
> consists of only alphanumeric characters. The last key of
>            a sequence of progress information is always "progress".
>
> I captured HTTP request that ffmpeg performs for progress:
>
> POST /progress.php HTTP/1.1
> Transfer-Encoding: chunked
> User-Agent: Lavf54.29.104
> Accept: */*
> Connection: close
> Host: converter.citylink-rk.ru
>
> 8f
> frame=91
> fps=0.0
> stream_0_0_q=23.0
> total_size=205437
> out_time_ms=4388571
> out_time=00:00:04.388571
> dup_frames=1
> drop_frames=0
> progress=continue
>
> 92
> frame=210
> fps=209.5
> stream_0_0_q=11.0
> total_size=569264
> out_time_ms=9357642
> out_time=00:00:09.357642
> dup_frames=1
> drop_frames=0
> progress=continue
>
> 92
> frame=368
> fps=244.8
> stream_0_0_q=4.0
> total_size=814739
> out_time_ms=15952109
> out_time=00:00:15.952109
> dup_frames=1
> drop_frames=0
> progress=continue
>
> 94
> frame=440
> fps=219.0
> stream_0_0_q=10.0
> total_size=1921815
> out_time_ms=18924263
> out_time=00:00:18.924263
> dup_frames=1
> drop_frames=0
> progress=continue
>
> 94
> frame=499
> fps=198.4
> stream_0_0_q=15.0
> total_size=2448688
> out_time_ms=21408798
> out_time=00:00:21.408798
> dup_frames=1
> drop_frames=0
> progress=continue
>
> 94
> frame=559
> fps=185.4
> stream_0_0_q=16.0
> total_size=2809055
> out_time_ms=23870113
> out_time=00:00:23.870113
> dup_frames=1
> drop_frames=0
> progress=continue
>
> 92
> frame=594
> fps=164.6
> stream_0_0_q=32766.0
> total_size=3312591
> out_time_ms=25379410
> out_time=00:00:25.379410
> dup_frames=1
> drop_frames=0
> progress=end
>
> 0
>
> FFmpeg sends POST-request headers at start and after that it sends a chunk
> with current progress each second.
> *It is not possible to use this chunks in realtime because web server waits
> for full request body (terminated by last zero-length chunk),*
> *so web-server passes this request after the process of convertation
> (ffmpeg process) completed.*
> This information is non-realtime and useless.

That depends upon what you have handling the request. There are many
web servers that can handle chunked progressive uploads, handling each
chunk as it arrives.

Remember that people do not just randomly add features because they
are missing. Someone required and desired this feature, with chunked
progressive uploads, and they wrote something to do it.

>
> I expect ffmpeg making separate POST-request with current progress
> information. Ex.:
>
> POST /progress.php HTTP/1.1
> User-Agent: Lavf54.29.104
> Accept: */*
> Host: conv.example.com
> Content-length: NN
>
> frame=91
> fps=0.0
> stream_0_0_q=23.0
> total_size=205437
> out_time_ms=4388571
> out_time=00:00:04.388571
> dup_frames=1
> drop_frames=0
> progress=continue
>
> POST /progress.php HTTP/1.1
> User-Agent: Lavf54.29.104
> Accept: */*
> Host: conv.example.com
> Content-length: NN
>
> frame=210
> fps=209.5
> stream_0_0_q=11.0
> total_size=569264
> out_time_ms=9357642
> out_time=00:00:09.357642
> dup_frames=1
> drop_frames=0
> progress=continue
>
> and so on.
>
> All these requests can be made in the sope of one keepalive connection.
> If web-server does not support keepalive or closes the connection for some
> reason, ffmpeg should re-establish the connection for a new request.
>
> Requests could be identified by input name / output name / process id /
> etc. and some unique key. ex:
>
> input=/tmp/in.avi
> key=ad24d5qw
>
> This pairs must match to identify continious progress.
>

This is what Nicolas was suggesting could be added as an additional
(ie different) protocol for the progress option.

Cheers

Tom