[FFmpeg-user] How can I extract m3u8 URL for use with ffmpeg on this site?

[Bo Berglund]

On Sat, 9 Apr 2022 13:14:18 +0200, Reino Wijnsma <rwijnsma at xs4all.nl> wrote:

>By looking at the network-traffic you can see (or at least I can) this POST request:
>
>$ curl --data-raw "chname=bXNuYmNfbGl2ZQ%3D%3D&ch=http%3A%2F%2Fwww.freeintertv.com%2Fexternals%2Ftv-russia%2Fsmotret-tv3-online&html5=11" \
>    "http://www.freeintertv.com/myAjax/get_item_m3u8/"
>
>This can be replicated/automated with the HTML/XML/JSON parser Xidel, which I've showed you earlier.

I did install it on one of my Ubuntu 20.04 machines the last time you suggested
it, here is what it tells me about its version:

$ xidel --version
Xidel 0.9.8
(20180421.6162.1f357eaaf5f3)

http://www.videlibri.de/xidel.html
by Benito van der Zander <benito at benibela.de>


On the following lines I am not sure I understand what you write...
There seems to be a lot of line wraps either by the news reader I use (Free
Agent) or the commands are not meant to be entered in a terminal...


>
>The value to be used for "chname":
>
>$ xidel -s "http://www.freeintertv.com/view/id-2565" -e 'extract(//meta[@property="og:image"]/@content,"([a-z_]+)\.jpg",1)' msnbc_live
>

I tried this, as a one-line command from xidel and ending with msnbc_live (news
reader wrapping text) but got the following output (in Ubuntu 20.04):

$ xidel -s "http://www.freeintertv.com/view/id-2565" -e
'extract(//meta[@property="og:image"]/@content,"([a-z_]+)\.jpg",1)' msnbc_live
msnbc_live

So it prints the word "msnbc_live" as a single word and nothing gets stored on
the disk either...


>"chname" base64 encoded:
>
>$ xidel -s "http://www.freeintertv.com/view/id-2565" -e '
>  string-to-base64Binary(
>    extract(//meta[@property="og:image"]/@content,"([a-z_]+)\.jpg",1)
>  )
>'
>bXNuYmNfbGl2ZQ==

Here I got (again command wrapped by news reader):

$ xidel -s "http://www.freeintertv.com/view/id-2565" -e
'string-to-base64Binary(extract(//meta[@property="og:image"]/@content,"([a-z_]+)\.jpg",1))'
bXNuYmNfbGl2ZQ==

So this seems to be the same output as you showed.


>The string to POST:
>
>$ xidel -s "http://www.freeintertv.com/view/id-2565" -e '
>  {
>    "chname":string-to-base64Binary(
>      extract(//meta[@property="og:image"]/@content,"([a-z_]+)\.jpg",1)
>    ),
>    "ch":"http://www.freeintertv.com/externals/tv-russia/smotret-tv3-online",
>    "html5":"11"
>  }
>'
>{
>  "chname": "bXNuYmNfbGl2ZQ==",
>  "ch": "http://www.freeintertv.com/externals/tv-russia/smotret-tv3-online",
>  "html5": "11"
>}
>
>$ xidel -s "http://www.freeintertv.com/view/id-2565" -e '
>  request-combine(
>    (),
>    {
>      "chname":string-to-base64Binary(
>        extract(//meta[@property="og:image"]/@content,"([a-z_]+)\.jpg",1)
>      ),
>      "ch":"http://www.freeintertv.com/externals/tv-russia/smotret-tv3-online",
>      "html5":"11"
>    }
>  )/substring(url,2)
>'
>chname=bXNuYmNfbGl2ZQ%3D%3D&ch=http%3A%2F%2Fwww.freeintertv.com%2Fexternals%2Ftv-russia%2Fsmotret-tv3-online&html5=11

I tried this after making it unwrapped into a single command line, but got an
error:

xidel -s "http://www.freeintertv.com/view/id-2565" -e
'{"chname":string-to-base64Binary(extract(//meta[@property="og:image"]/@content,"([a-z_]+)\.jpg",1)),"ch":"http://www.freeintertv.com/externals/tv-russia/smotret-tv3-online","html5":"11"}'{"chname":
"bXNuYmNfbGl2ZQ==","ch":
"http://www.freeintertv.com/externals/tv-russia/smotret-tv3-online","html5":
"11"}
Error:
err:XPST0003: Unknown or unexpected operator: { (possible missing comma , or
closing parentheses)}] )
in:
{"chname":string-to-base64Binary(extract(//meta[@property="og:image"]/@content,"([a-z_]+)\.jpg",1)),"ch":"http://www.freeintertv.com/externals/tv-russia/smotret-tv3-online","html5":"11"}
[<- error occurs before here] {chname:

Did not try the following command but skipped down a bit:

>And the actual POST-request:
>
>$ xidel -s "http://www.freeintertv.com/view/id-2565" -e '
>  x:request({
>    "post":request-combine(
>      (),
>      {
>        "chname":string-to-base64Binary(
>          extract(//meta[@property="og:image"]/@content,"([a-z_]+)\.jpg",1)
>        ),
>        "ch":"http://www.freeintertv.com/externals/tv-russia/smotret-tv3-online",
>        "html5":"11"
>      }
>    )/substring(url,2),
>    "url":"http://www.freeintertv.com/myAjax/get_item_m3u8/"
>  })/raw
>'
>playlist[0]['file']='http://1128480543.rsc.cdn77.org/SlTNwu1CSVEq9yA3i_J9Zg==,1649504141/1128480543/index.m3u8'; get_item.showPlayer();


To here:

>Extract the m3u8-url:
>
>$ xidel -s "http://www.freeintertv.com/view/id-2565" -e '
>  x:request({
>    "post":request-combine(
>      (),
>      {
>        "chname":string-to-base64Binary(
>          extract(//meta[@property="og:image"]/@content,"([a-z_]+)\.jpg",1)
>        ),
>        "ch":"http://www.freeintertv.com/externals/tv-russia/smotret-tv3-online",
>        "html5":"11"
>      }
>    )/substring(url,2),
>    "url":"http://www.freeintertv.com/myAjax/get_item_m3u8/"
>  })/extract(raw,"http.+m3u8")
>'
>http://1128480543.rsc.cdn77.org/SlTNwu1CSVEq9yA3i_J9Zg==,1649504141/1128480543/index.m3u8


So next I tried this last big command again after trying to get it onto a single
line (which the newsreader has wrapped...):

$ xidel -s "http://www.freeintertv.com/view/id-2565" -e
'x:request({"post":request-combine((),{"chname":string-to-base64Binary(extract(//meta[@property="og:image"]/@content,"([a-z_]+)\.jpg",1)),"ch":"http://www.freeintertv.com/externals/tv-russia/smotret-tv3-online","html5":"11"})/substring(url,2),"url":"http://www.freeintertv.com/myAjax/get_item_m3u8/"})/extract(raw,"http.+m3u8")

Now there is only a single > output and no command prompt until I hit Ctrl-C


I am sorry if I do not understand what you have posted but to me it seems like I
am either hit by the newsreader or by totally misunderstanding what you are
posting.

Seems like the examples you have given are not meant to be typed into the Linux
terminal (since they seem to be multi-line).

Have I totally misunderstood it all? (How xidel works)


-- 
Bo Berglund
Developer in Sweden