[FFmpeg-user] Can ffmpeg record video from this kind of URL?

[Bo Berglund]

On Sat, 9 Oct 2021 03:23:52 +0300, Anatoly <anatoly at kazanfieldhockey.ru> wrote:

>On Thu, 07 Oct 2021 10:27:51 +0200
>Bo Berglund <bo.berglund at gmail.com> wrote:
>
>> On Wed, 1 Sep 2021 16:30:00 +0300, anatoly at kazanfieldhockey.ru wrote:
>> 
>> >Right, you need m3u8.
>> >F12->Network->Enter "m3u8" in the textbox above the list to filter
>> >out.  
>> >> Is there a command line call I can make to extract the m3u8 URL
>> >> automatically so
>> >> it can be used in a script only supplied the main page URL?  
>> >To get url of the first m3u8 from the page (this page actually has
>> >only one) curl "https://www.livenewsmag.com/msnbc-news-live-stream/"
>> >| grep -o -e "https://.\+m3u8" | head -n 1  
>>
 
>> Coming back to this issue again since the command you suggested has
>> stopped working...
>> I am trying this command and I get an empty result:
>> 
>> $ curl -s "https://msnbcdailyshows.com/" | grep -o -e
>> "https://.\+m3u8" | head -n 1
>> 
>> The page above does have an embedded video player but your command for
>> extraction of the m3u8 URL returns an *empty* output...
>> 
>> But it does work on other such pages like this:
>> 
>> $ curl -s "https://www.livenewsmag.com/msnbc-news-live-stream" | grep
>> -o -e "https://.\+m3u8" | head -n 1
>> https://1420543146.rsc.cdn77.org/Z5XPzYjFisC9SrZHwPgDUg==,1633615580/LS-ATL-54548-10/index.m3u8
>> 
>> Why is the command working on one URL but not the other?

>Because as Moritz Barsnick <barsnick at gmx.net> Wed, 1 Sep 2021 15:37:21
>+0200 already told you:
>"There's often a lot of magic involved in extracting the video URLs from
>webpages - parsing JavaScript, downloading and parsing JSON files,
>re-using cookies and referrers, and so on."
>
>So this is very hard, nearly to impossible to create a tool or
>receipe that will automatically extract video urls on *any* arbitrary
>page. Last time I just took a quick look at www.livenewsmag.com and
>found a simple solution for you, because that page is simple enough.
>Now let's do the same:
>
>#!/bin/sh
>#resolution here may be: prem, 720, 480, 240
>resolution="480"
>sdate=`date "+%Y%m%d-%H%M"`
>filename="rec-msnbc1-${sdate}.mp4"
>msn1url="https://msnbcdailyshows.com/"
>ggid=`curl $msn1url | grep -o -e "https://goodgame\.ru/player?[0-9]\+"
>| grep -o -e "[0-9]\+"`
>m3u8url="https://hls.goodgame.ru/hls/${ggid}_${resolution}.m3u8"
>ffmpeg -user_agent "Mozilla" -i $m3u8url -c copy $filename
>
>Also this page contains url of youtube stream of same program, which
>can be extracted with
>curl "https://msnbcdailyshows.com/" | grep -o -e
>"https://youtu.be/[0-9a-zA-Z]\+"
>and then fed to youtube-dl

I return again to this subject because the extraction I have used for a long
time now has failed again, probably because part of it is a russian site so they
changed to something else...

With the war in Ukraine I would very much like to record the MSNBC stream
nightly using ffmpeg and it seems impossible now.
I have tried my two ffmpeg formats using the page url:

https://msnbcdailyshows.com/

and the m3u8 URL I find if I use F12 in the browser when it plays:

https://cdn.plt.one/cdn/premium327/chunks.m3u8

But my ffmpeg command does not work with these...
Example (on one line):

ffmpeg -hide_banner -referer \"https://msnbcdailyshows.com/\" -i
https://cdn.plt.one/cdn/premium327/chunks.m3u8  -vf scale=w=-4:h=480 -c:v
libx264 -preset fast -crf 26 -c:a copy -t 120 test1.mp4

It fails to actually get any data, so much tghat the timeout does not even
trigger...

What can be done to get this working again?


-- 
Bo Berglund
Developer in Sweden