[FFmpeg-devel] TCP timeout for HTTP/HTTPS connections.
Steven Liu
lq at chinaffmpeg.org
Mon Nov 18 12:16:00 EET 2019
> 在 2019年11月18日，17:58，Tom Gaudasiński <tomg at records.headdesk.com.au> 写道：
>
hls can hold to option after the patch:
rw_timeout
-rw_timeout <int64> ED........ Timeout for IO operations (in microseconds) (from 0 to I64_MAX) (default 0)
This can be used when you downloading segment or m3u8 file, the http response too long time.
-m3u8_hold_counters <int> .D........ Maximum number of times requests when the m3u8 file not be refresh (from 0 to INT_MAX) (default 1000)
this can be used when you can success download m3u8 but the m3u8 always refresh without new segments.
> So I have applied Steven's m3u8 patch, and have installed an
> AVIOInterruptCB, but I'm still getting blocked here:
>
>
>
> #0 0x00007ffff791d729 in __GI___poll (fds=fds at entry=0x7fffcf663118,
> nfds=nfds at entry=1, timeout=4999,
> timeout at entry=<error reading variable: That operation is not
> available on integers of more than 8 bytes.>)
> at ../sysdeps/unix/sysv/linux/poll.c:29
> #1 0x00007ffff47951ab in send_dg (ansp2_malloced=<optimised out>,
> resplen2=<optimised out>,
> anssizp2=<optimised out>, ansp2=<optimised out>, anscp=<optimised
> out>, gotsomewhere=<synthetic pointer>,
> v_circuit=<synthetic pointer>, ns=<optimised out>,
> terrno=0x7fffcf6630e8, anssizp=0x7fffcf663210,
> ansp=0x7fffcf6630d8, buflen2=<optimised out>, buf2=<optimised out>,
> buflen=<optimised out>,
> buf=<optimised out>, statp=<optimised out>) at res_send.c:1112
> #2 __res_context_send (ctx=ctx at entry=0x7fffc0050e00,
> buf=buf at entry=0x7fffcf663240 "̦\001",
> buflen=buflen at entry=60, buf2=buf2 at entry=0x7fffcf66327c
> "\344\272\001", buflen2=buflen2 at entry=60,
> ans=<optimised out>, ans at entry=0x7fffcf663e50 "̦\201\202",
> anssiz=<optimised out>, ansp=<optimised out>,
> ansp2=<optimised out>, nansp2=<optimised out>, resplen2=<optimised
> out>, ansp2_malloced=<optimised out>)
> at res_send.c:519
> #3 0x00007ffff47927a3 in __GI___res_context_query
> (ctx=ctx at entry=0x7fffc0050e00,
> name=name at entry=0x7fffcf665070 "abcradiolivehls-lh.akamaihd.net",
> class=class at entry=1,
> type=type at entry=439963904, answer=answer at entry=0x7fffcf663e50
> "̦\201\202", anslen=anslen at entry=2048,
> answerp=0x7fffcf6646a0, answerp2=0x7fffcf6646a8,
> nanswerp2=0x7fffcf664690, resplen2=0x7fffcf664694,
> answerp2_malloced=0x7fffcf664698) at res_query.c:216
> #4 0x00007ffff47933cc in __res_context_querydomain
> (answerp2_malloced=0x7fffcf664698, resplen2=0x7fffcf664694,
> nanswerp2=0x7fffcf664690, answerp2=0x7fffcf6646a8,
> answerp=0x7fffcf6646a0, anslen=2048,
> answer=0x7fffcf663e50 "̦\201\202", type=439963904, class=1, domain=0x0,
> name=0x7fffcf665070 "abcradiolivehls-lh.akamaihd.net",
> ctx=0x7fffc0050e00) at res_query.c:601
> #5 __GI___res_context_search (ctx=ctx at entry=0x7fffc0050e00,
> name=name at entry=0x7fffcf665070 "abcradiolivehls-lh.akamaihd.net",
> class=class at entry=1,
> type=type at entry=439963904, answer=answer at entry=0x7fffcf663e50
> "̦\201\202", anslen=anslen at entry=2048,
> answerp=<optimised out>, answerp2=<optimised out>,
> nanswerp2=<optimised out>, resplen2=<optimised out>,
> answerp2_malloced=<optimised out>) at res_query.c:370
> #6 0x00007ffff007b07a in _nss_dns_gethostbyname4_r (
> name=name at entry=0x7fffcf665070 "abcradiolivehls-lh.akamaihd.net",
> pat=pat at entry=0x7fffcf6647f8,
> buffer=0x7fffcf664a80 "\377\002", buflen=1024,
> errnop=errnop at entry=0x7fffcf66e658,
> herrnop=herrnop at entry=0x7fffcf66e6bc, ttlp=0x0) at
> nss_dns/dns-host.c:371
> #7 0x00007ffff79100b2 in gaih_inet (name=<optimised out>,
> name at entry=0x7fffcf665070 "abcradiolivehls-lh.akamaihd.net",
> service=service at entry=0x7fffcf664a20,
> req=req at entry=0x7fffcf664f30, pai=pai at entry=0x7fffcf664a08,
> naddrs=naddrs at entry=0x7fffcf664a04,
> tmpbuf=tmpbuf at entry=0x7fffcf664a70) at
> ../sysdeps/posix/getaddrinfo.c:770
> #8 0x00007ffff79110a7 in __GI_getaddrinfo (name=<optimised out>,
> service=<optimised out>, hints=0x7fffcf664f30,
> pai=0x7fffcf664f08) at ../sysdeps/posix/getaddrinfo.c:2265
> #9 0x00007ffff074b97f in tcp_open (h=0x7fffc05ba140,
> uri=0x7fffc05ba1a0 "tcp://abcradiolivehls-lh.akamaihd.net:80",
> flags=3) at libavformat/tcp.c:151
> #10 0x00007ffff05a3a8e in ffurl_connect (uc=0x7fffc05ba140,
> options=0x7fffcf669458) at libavformat/avio.c:210
> #11 0x00007ffff05a4214 in ffurl_open_whitelist (puc=0x7fffc0be9088,
> filename=0x7fffcf666db0 "tcp://abcradiolivehls-lh.akamaihd.net:80",
> flags=3, int_cb=0x7fffc05baeb0,
> options=0x7fffcf669458, whitelist=0x7fffc05b32c0
> "http,https,tls,rtp,tcp,udp,crypto,httpproxy",
> blacklist=0x0, parent=0x7fffc05bae80) at libavformat/avio.c:344
> #12 0x00007ffff0607397 in http_open_cnx_internal (h=0x7fffc05bae80,
> options=0x7fffcf669458)
> at libavformat/http.c:236
> #13 0x00007ffff060749b in http_open_cnx (h=0x7fffc05bae80,
> options=0x7fffcf669458) at libavformat/http.c:263
> #14 0x00007ffff060842f in http_open (h=0x7fffc05bae80,
> uri=0x7fffc05baee0
> "http://abcradiolivehls-lh.akamaihd.net/i/abcjazz_1@327288/index_32_a-p.m3u8?sd=10&rebase=on",
> flags=1, options=0x7fffcf669458) at libavformat/http.c:580
> #15 0x00007ffff05a3a64 in ffurl_connect (uc=0x7fffc05bae80,
> options=0x7fffcf669458) at libavformat/avio.c:206
> #16 0x00007ffff05a4214 in ffurl_open_whitelist (puc=0x7fffcf669360,
> filename=0x7fffc0005700
> "http://abcradiolivehls-lh.akamaihd.net/i/abcjazz_1@327288/index_32_a-p.m3u8?sd=10&rebase=on",
> flags=1, int_cb=0x7fffc0001148, options=0x7fffcf669458,
> whitelist=0x7fffc000d210
> "http,https,tls,rtp,tcp,udp,crypto,httpproxy", blacklist=0x0, parent=0x0)
> at libavformat/avio.c:344
> #17 0x00007ffff05a87a3 in ffio_open_whitelist (s=0x7fffcf6693e0,
> filename=0x7fffc0005700
> "http://abcradiolivehls-lh.akamaihd.net/i/abcjazz_1@327288/index_32_a-p.m3u8?sd=10&rebase=on",
> flags=1, int_cb=0x7fffc0001148, options=0x7fffcf669458,
> whitelist=0x7fffc000d210
> "http,https,tls,rtp,tcp,udp,crypto,httpproxy", blacklist=0x0)
> at libavformat/aviobuf.c:1200
> #18 0x00007ffff06db328 in io_open_default (s=0x7fffc0000c80,
> pb=0x7fffcf6693e0,
> url=0x7fffc0005700
> "http://abcradiolivehls-lh.akamaihd.net/i/abcjazz_1@327288/index_32_a-p.m3u8?sd=10&rebase=on",
> flags=1, options=0x7fffcf669458) at libavformat/options.c:124
> #19 0x00007ffff05f602e in parse_playlist (c=0x7fffc0005200,
> url=0x7fffc0005700
> "http://abcradiolivehls-lh.akamaihd.net/i/abcjazz_1@327288/index_32_a-p.m3u8?sd=10&rebase=on",
> pls=0x7fffc0005700, in=0x0) at libavformat/hls.c:740
> #20 0x00007ffff05f847d in read_data (opaque=0x7fffc0005700,
> buf=0x7fffc006e928 "", buf_size=32768)
> at libavformat/hls.c:1417
> #21 0x00007ffff05a66d8 in read_packet_wrapper (s=0x7fffc0006700,
> buf=0x7fffc006e928 "", size=32768)
> at libavformat/aviobuf.c:538
> #22 0x00007ffff05a697f in fill_buffer (s=0x7fffc0006700) at
> libavformat/aviobuf.c:587
> #23 0x00007ffff05a5f45 in avio_feof (s=0x7fffc0006700) at
> libavformat/aviobuf.c:367
> #24 0x00007ffff05fa878 in hls_read_packet (s=0x7fffc0000c80,
> pkt=0x7fffc0015a80) at libavformat/hls.c:2107
> #25 0x00007ffff075ac84 in ff_read_packet (s=0x7fffc0000c80,
> pkt=0x7fffc0015a80) at libavformat/utils.c:859
> #26 0x00007ffff075dc49 in read_frame_internal (s=0x7fffc0000c80,
> pkt=0x7fffc0015a80) at libavformat/utils.c:1586
> #27 0x00007ffff075ea61 in av_read_frame (s=0x7fffc0000c80,
> pkt=0x7fffc0015a80) at libavformat/utils.c:1788
>
>
>
> Is there a way for AVIOInterruptCB to also control the timeout of
> tcp_open()?
>
>
>
> On 18/11/19 8:44 pm, Tom Gaudasiński wrote:
>> Okay, I will try it. Thank you.
>>
>> On 18/11/19 8:43 pm, Steven Liu wrote:
>>>> 在 2019年11月18日，17:32，Tom Gaudasiński <tomg at records.headdesk.com.au> 写道：
>>>>
>>>> How do I append “#EXT-X-ENDLIST” to the end of the m3u8 using the API?
>>>> The URL I get ffmpeg to open is the m3u8 file, so I somehow need to do
>>>> it in memory, I presume?
>>>>
>>> Hello Tom,
>>>
>>> Can you try this patch please:
>>> https://patchwork.ffmpeg.org/patch/16316/
>>>> On 18/11/19 8:00 pm, Steven Liu wrote:
>>>>>> 在 2019年11月18日，16:44，Tom Gaudasiński <tomg at records.headdesk.com.au> 写道：
>>>>>>
>>>>>> That would probably work in the one case where it's looking at the next
>>>>>> entry in the list. It's actually blocking on a read, so I need some way
>>>>>> to interrupt it. I will try AVIOInterruptCB and see how I go.
>>>>> AVIOInterruptCB maybe ok if that is blocking. But is dose not useful if it always refresh m3u8.
>>>>>> On 18/11/19 7:36 pm, Steven Liu wrote:
>>>>>>>> 在 2019年11月18日，16:11，Tom Gaudasiński <tomg at records.headdesk.com.au> 写道：
>>>>>>>>
>>>>>>>> I essentially have a thread that I need to have exit immediately. I
>>>>>>>> thought tweaking timeouts would be one way to get this done, but I'm
>>>>>>>> starting to think that's a bad approach.
>>>>>>>>
>>>>>>>> So, let me, maybe, rephrase the question: Is there a way I can tell
>>>>>>>> ffmpeg to terminate? Like, closing its sockets, or something like that?
>>>>>>>> This is all from another thread.
>>>>>>> Just append an “#EXT-X-ENDLIST” into the m3u8 list is ok.
>>>>>>>
>>>>>>>> On 18/11/19 7:06 pm, Steven Liu wrote:
>>>>>>>>>> 在 2019年11月18日，15:37，Tom Gaudasiński <tomg at records.headdesk.com.au> 写道：
>>>>>>>>>>
>>>>>>>>>> That seems to change the call-stack a bit, but it still gets stuck on
>>>>>>>>>> something in hls.c.
>>>>>>>>>>
>>>>>>>>>> I'm looking into what it is. I've also tried connect_timeout.
>>>>>>>>>>
>>>>>>>>>> I really just need to terminate the av_read_frame call, so that it
>>>>>>>>>> returns immediately when I need it to, but av_read_frame appears to
>>>>>>>>>> reconnect internally.
>>>>>>>>> Ah, maybe i get your point, do you mean you don’t like the hls always retry
>>>>>>>>> get the m3u8 list when the last segment don’t update living mode?
>>>>>>>>>
>>>>>>>>>> On 18/11/19 6:02 pm, Steven Liu wrote:
>>>>>>>>>>>> 在 2019年11月18日，14:56，Tom Gaudasiński <tomg at records.headdesk.com.au> 写道：
>>>>>>>>>>>>
>>>>>>>>>>>> I'm trying to set a read timeout for a http/https stream through the
>>>>>>>>>>>> API. Currently I'm passing...
>>>>>>>>>>>>
>>>>>>>>>>>> av_dict_set(&options, "timeout", "100000", 0);
>>>>>>>>>>>>
>>>>>>>>>>>> ... to avformat_open_input().
>>>>>>>>>>>>
>>>>>>>>>>>> This appears to not work once the connection has been established. The
>>>>>>>>>>>> protocol call-stack appears to be
>>>>>>>>>>>>
>>>>>>>>>>>> hls.c -> http.c -> avio.c -> tcp.c -> network.c
>>>>>>>>>>> What about use rw_timeout ?
>>>>>>>>>>>> The issue, I think, is that tcp.c only gets it's timeout from a
>>>>>>>>>>>> ?key=val&key=val string at the end of the URL supplied to tcp.c, but
>>>>>>>>>>>> http.c or avio.c don't supply this. In fact, when I see what was passed
>>>>>>>>>>>> down, it's just "tcp://hostname.tld:80".
>>>>>>>>>>>>
>>>>>>>>>>>> Am I missing something? Is there a way to get ffmpeg to set tcp-timeout
>>>>>>>>>>>> from the very top-level API calls? I can't see it.
>>>>>>>>>>>>
>>>>>>>>>>>> Thank you.
>>>>>>>>>>>>
>>>>>>>>>>>> --Tom G.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> ffmpeg-devel mailing list
>>>>>>>>>>>> ffmpeg-devel at ffmpeg.org
>>>>>>>>>>>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>>>>>>>>>>>
>>>>>>>>>>>> To unsubscribe, visit link above, or email
>>>>>>>>>>>> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>>>>>>>>>>> Thanks
>>>>>>>>>>> Steven
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> ffmpeg-devel mailing list
>>>>>>>>>>> ffmpeg-devel at ffmpeg.org
>>>>>>>>>>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>>>>>>>>>>
>>>>>>>>>>> To unsubscribe, visit link above, or email
>>>>>>>>>>> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>>>>>>>>>> _______________________________________________
>>>>>>>>>> ffmpeg-devel mailing list
>>>>>>>>>> ffmpeg-devel at ffmpeg.org
>>>>>>>>>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>>>>>>>>>
>>>>>>>>>> To unsubscribe, visit link above, or email
>>>>>>>>>> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>>>>>>>>> Thanks
>>>>>>>>> Steven
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> ffmpeg-devel mailing list
>>>>>>>>> ffmpeg-devel at ffmpeg.org
>>>>>>>>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>>>>>>>>
>>>>>>>>> To unsubscribe, visit link above, or email
>>>>>>>>> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>>>>>>>> _______________________________________________
>>>>>>>> ffmpeg-devel mailing list
>>>>>>>> ffmpeg-devel at ffmpeg.org
>>>>>>>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>>>>>>>
>>>>>>>> To unsubscribe, visit link above, or email
>>>>>>>> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>>>>>>> Thanks
>>>>>>> Steven
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> ffmpeg-devel mailing list
>>>>>>> ffmpeg-devel at ffmpeg.org
>>>>>>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>>>>>>
>>>>>>> To unsubscribe, visit link above, or email
>>>>>>> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>>>>>> _______________________________________________
>>>>>> ffmpeg-devel mailing list
>>>>>> ffmpeg-devel at ffmpeg.org
>>>>>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>>>>>
>>>>>> To unsubscribe, visit link above, or email
>>>>>> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>>>>> Thanks
>>>>> Steven
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> ffmpeg-devel mailing list
>>>>> ffmpeg-devel at ffmpeg.org
>>>>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>>>>
>>>>> To unsubscribe, visit link above, or email
>>>>> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>>>> _______________________________________________
>>>> ffmpeg-devel mailing list
>>>> ffmpeg-devel at ffmpeg.org
>>>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>>>
>>>> To unsubscribe, visit link above, or email
>>>> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>>> Thanks
>>> Steven
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> ffmpeg-devel mailing list
>>> ffmpeg-devel at ffmpeg.org
>>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>>
>>> To unsubscribe, visit link above, or email
>>> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>>
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel at ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>
>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
Thanks
Steven
More information about the ffmpeg-devel
mailing list