[FFmpeg-user] How can I extract m3u8 URL for use with ffmpeg on this site?

[Reino Wijnsma]

On 2022-04-11T12:25:17+0200, Bo Berglund <bo.berglund at gmail.com> wrote:
> $ xidel --version
> Xidel 0.9.8
> (20180421.6162.1f357eaaf5f3)

A couple of months ago I already stated to use the latest binary from the development branch:

On 2022-01-09T23:07:55+0100, Reino Wijnsma <rwijnsma at xs4all.nl> wrote:
> I'm not sure why you're trying to compile Xidel from (the wrong) source when you can easily grab binaries from https://sourceforge.net/projects/videlibri/files/Xidel/ (binaries from the development map are recommended).

On 2022-04-11T12:25:17+0200, Bo Berglund <bo.berglund at gmail.com> wrote:
> There seems to be a lot of line wraps either by the news reader I use (Free
> Agent) or the commands are not meant to be entered in a terminal...

You, as a developer, don't know the difference between a minified and prettified command?
I don't know what a "news reader" is, but https://ffmpeg.org/pipermail/ffmpeg-user/2022-April/054565.html shows my mail as intended.
And of course prettified commands can be entered in a terminal.

> So it prints the word "msnbc_live" as a single word and nothing gets stored on
> the disk either...

All Xidel commands in my previous post just print the output to stdout. I don't understand why you'd expect anything to be saved to disk.

> I tried this after making it unwrapped into a single command line, but got an
> error:
>
> xidel -s "http://www.freeintertv.com/view/id-2565" -e
> '{"chname":string-to-base64Binary(extract(//meta[@property="og:image"]/@content,"([a-z_]+)\.jpg",1)),"ch":"http://www.freeintertv.com/externals/tv-russia/smotret-tv3-online","html5":"11"}'{"chname":
> "bXNuYmNfbGl2ZQ==","ch":
> "http://www.freeintertv.com/externals/tv-russia/smotret-tv3-online","html5":
> "11"}
> Error:
> err:XPST0003: Unknown or unexpected operator: { (possible missing comma , or
> closing parentheses)}] )

On 2022-04-09T13:14:18+0200, Reino Wijnsma <rwijnsma at xs4all.nl> wrote:
> $ xidel -s "http://www.freeintertv.com/view/id-2565" -e '   *opening quote of extraction-query*
>   {
>     "chname":string-to-base64Binary(
>       extract(//meta[@property="og:image"]/@content,"([a-z_]+)\.jpg",1)
>     ),
>     "ch":"http://www.freeintertv.com/externals/tv-russia/smotret-tv3-online",
>     "html5":"11"
>   }
> '                                                           *closing quote of extraction-query*
> {
>   "chname": "bXNuYmNfbGl2ZQ==",
>   "ch": "http://www.freeintertv.com/externals/tv-russia/smotret-tv3-online",
>   "html5": "11"
> }

So, the JSON is not part of the command, but it's the *output*!

> So next I tried this last big command again after trying to get it onto a single
> line (which the newsreader has wrapped...):
>
> $ xidel -s "http://www.freeintertv.com/view/id-2565" -e
> 'x:request({"post":request-combine((),{"chname":string-to-base64Binary(extract(//meta[@property="og:image"]/@content,"([a-z_]+)\.jpg",1)),"ch":"http://www.freeintertv.com/externals/tv-russia/smotret-tv3-online","html5":"11"})/substring(url,2),"url":"http://www.freeintertv.com/myAjax/get_item_m3u8/"})/extract(raw,"http.+m3u8")
>
> Now there is only a single > output and no command prompt until I hit Ctrl-C

You forgot to copy-paste the extraction-query closing quote.

If you insist on using Xidel 0.9.8, then use:

$ xidel -s "http://www.freeintertv.com/view/id-2565" -e 'x:request({"post":request-combine({"url":()},{"chname":string-to-base64Binary(extract(//meta[@property="og:image"]/@content,"([a-z_]+)\.jpg",1)),"ch":"http://www.freeintertv.com/externals/tv-russia/smotret-tv3-online","html5":"11"})/url,"url":"http://www.freeintertv.com/myAjax/get_item_m3u8/"})/extract(raw,"http.+m3u8")'

I think this is enough off-topic for this mailinglist. If you have any more questions about Xidel, I suggest you go to:
StackOverflow - https://stackoverflow.com/questions/tagged/xidel
Mailinglist - https://sourceforge.net/p/videlibri/mailman/
(not so active) Forum - https://sourceforge.net/p/xidel/discussion

On 2022-04-11T17:46:52+0200, Robert Jeffares <jeffares.robert at gmail.com> wrote:
> A script using curl to download the site to a file then grep m3u8 will give
> you a list which you can then sort to select the_one_you_want then ffmpeg
> -I the_one_you_want etc

No it will not, because a POST request is required to get the video-url in this particular case. There's no mention of "m3u8" in the site its HTML-source.
And in addition, it's really bad practise to parse HTML with grep/sed/awk etc.

P.s. Please do not top-post on this mailinglist!

-- 
Reino