[FFmpeg-user] How to extract m3u8 stream URL from webpage with 2 players?

[Reino Wijnsma]

On 2022-01-08T10:25:48+0100, Bo Berglund <bo.berglund at gmail.com> wrote:
> On Fri, 07 Jan 2022 14:05:02 +0100, Bo Berglund <bo.berglund at gmail.com> wrote:
>> If I open the source page and use the F12 function key to activate browser debug
>> mode I can see the stream URL for both players.
>> Right now they are:
>> #1 = https://hlss.goodgame.ru/hls/188429.m3u8
>> #2 = https://1111296894.rsc.cdn77.org/LS-ATL-54548-10/tracks-v1a1/mono.m3u8
>>
>> What I would like to have is a way to extract the xxx/mono.m3u8 URL #2
>> automatically in a script.
>>
>> The URL:s seem to change irregularly so I will use any method found to extract
>> the URL minutes before the download starts by launching the extractor script via
>> cron.
> It turns out that shortly after I posted my question the second player stopped
> working altogether on the webpage, so there is nothing to test anymore...
> :(
Both work for me. They're actually one and the same.

$ xidel -s "https://msnbcdailyshows.com/" -e '//iframe' --output-node-format=html
<iframe src="https://goodgame.ru/player?188429" width="800" height="480" frameborder="0" allowfullscreen></iframe>
<iframe src="//2checksitea.blogspot.com/2012/07/099msnbc1.html" width="100%" height="430" frameborder="0" scrolling="no" allowfullscreen></iframe>

The second url refers to the exact same "goodgame.ru"-url:

$ xidel -s "https://msnbcdailyshows.com/" -f '(//iframe)[2]' -e '//div[@class="post-body entry-content"]/iframe' --ignore-namespaces --output-node-format=html
<iframe src="https://goodgame.ru/player?188429" width="600" height="400" frameborder="0" allowfullscreen></iframe>

The HLS-manifest-url:

$ xidel -s "https://msnbcdailyshows.com/" -e 'x"https://hls.goodgame.ru/manifest/{substring-after((//iframe)[1]/@src,"?")}_master.m3u8"'
#or
$ xidel -s "https://msnbcdailyshows.com/" -f '(//iframe)[2]' -e 'x"https://hls.goodgame.ru/manifest/{//div[@class="post-body entry-content"]/substring-after(iframe/@src,"?")}_master.m3u8"'
https://hls.goodgame.ru/manifest/188429_master.m3u8

$ ffmpeg -hide_banner -user_agent "Mozilla Firefox" -i "$(xidel -s "https://msnbcdailyshows.com/" -e 'x"https://hls.goodgame.ru/manifest/{substring-after((//iframe)[1]/@src,"?")}_master.m3u8"')"
[hls @ 03105100] Skip ('#EXT-X-VERSION:3')
[hls @ 03105100] Opening 'https://hls.goodgame.ru/hls/188429.m3u8' for reading
[hls @ 03105100] Skip ('#EXT-X-VERSION:3')
[hls @ 03105100] Opening 'https://hls.goodgame.ru/hls/188429-279700.ts' for reading
[hls @ 03105100] Opening 'https://hls.goodgame.ru/hls/188429-279701.ts' for reading
Input #0, hls, from 'https://hls.goodgame.ru/manifest/188429_master.m3u8':
  Duration: N/A, start: 436.194944, bitrate: N/A
  Program 0
    Metadata:
      variant_bitrate : 2330000
  Stream #0:0: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 1280x720, 30 tbr, 90k tbn
    Metadata:
      variant_bitrate : 2330000
  Stream #0:1: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp
    Metadata:
      variant_bitrate : 2330000
At least one output file must be specified

-user_agent is required!

-- 
Reino