[FFmpeg-user] Can ffmpeg record video from this kind of URL?

[Bo Berglund]

On Sat, 9 Oct 2021 03:23:52 +0300, Anatoly <anatoly at kazanfieldhockey.ru> wrote:

>> Why is the command working on one URL but not the other?

>Because as Moritz Barsnick <barsnick at gmx.net> Wed, 1 Sep 2021 15:37:21
>+0200 already told you:
>"There's often a lot of magic involved in extracting the video URLs from
>webpages - parsing JavaScript, downloading and parsing JSON files,
>re-using cookies and referrers, and so on."
>
>So this is very hard, nearly to impossible to create a tool or
>receipe that will automatically extract video urls on *any* arbitrary
>page. Last time I just took a quick look at www.livenewsmag.com and
>found a simple solution for you, because that page is simple enough.
>Now let's do the same:
>
>#!/bin/sh
>#resolution here may be: prem, 720, 480, 240
>resolution="480"
>sdate=`date "+%Y%m%d-%H%M"`
>filename="rec-msnbc1-${sdate}.mp4"
>msn1url="https://msnbcdailyshows.com/"
>ggid=`curl $msn1url | grep -o -e "https://goodgame\.ru/player?[0-9]\+"
>| grep -o -e "[0-9]\+"`
>m3u8url="https://hls.goodgame.ru/hls/${ggid}_${resolution}.m3u8"
>ffmpeg -user_agent "Mozilla" -i $m3u8url -c copy $filename
>
>Also this page contains url of youtube stream of same program, which
>can be extracted with
>curl "https://msnbcdailyshows.com/" | grep -o -e
>"https://youtu.be/[0-9a-zA-Z]\+"
>and then fed to youtube-dl


Oh boy! Sorry to be back here again!

Now I am caught again!

They have changed the format of the stream such that my tools fail again.

There appears to be no more any m3u8 stream....
So it is no longer possible to extract by looking for m3u8...

If I hit F12 when I display the webpage https://msnbcdailyshows.com/ I see a lot
of stuff streaming by on the Networking tab and it seems to be instead of m3u8
some <changing_text>.dash, like this:

https://vsd79.mycdn.me/dash/stream_1146644400798/stream.manifest/sig/RcfEI7wlP-c/expires/1636066246262/srcIp/158.174.104.130/clientType/0/srcAg/GECKO/mid/2125507010206/frag-high-6827906.dash

Is it possible to find a working URL for ffmpeg to use in such a situation too?

I have tried to check the site using this command:
youtube-dl -F https://msnbcdailyshows.com/
After some thinking it returns:

[info] Available formats for 2125507010206:
format code  extension  resolution note
hls-109      mp4        864x488     109k
hls-175      mp4        640x360     175k
hls-485      mp4        428x240     485k
hls-782      mp4        1280x720    782k  (best)

So I tried to use youtube-dl to download the video:
youtube-dl -f hls-175 -o test.mp4 https://msnbcdailyshows.com/

And it seems like it is doing what it should as if it was a regular youtube
streaming video, however I cannot stop it successfully like I can with normal
youtube streams by using ctrl-C. And Youtube-dl does not have any argument to
set the duration of the download either as far as I can tell.

On normal streaming videos youtube-dl closes the video download on receiving
Ctrl-C and then exits.
And the interrupted video is fixed so it can be played.
So I am using a this command structure in my at job to download a specific
video:

timeout --signal=2 64m youtube-dl -f hls-175 -o xxx.mp4 some-youtube-stream-url

And this causes youtube-dl to run for 64 minutes from when it was started and
then cleanly close the video and exit.

Not so with this one site, here youtube-dl will just abort the download and the
xxx.part file remains and cannot be viewed.

Is there any possibility to use ffmpeg itself to download from this kind of
stream and if so what would the argument for the stream be?


-- 
Bo Berglund
Developer in Sweden