[FFmpeg-devel] [PATCH 3/3] avformat: add youtube-dl based demuxer

[Gilles Chanteperdrix]

On Wed, Apr 08, 2015 at 09:59:17PM +0200, Nicolas George wrote:
> Le nonidi 19 germinal, an CCXXIII, Gilles Chanteperdrix a écrit :
> > As I explained, I did it this way for portability. system is ANSI
> > not POSIX, redirections work with windows shell too, and av_tempfile
> > is implemented by ffmpeg, so I expect it to be portable.
> 
> Implementing pipe reading for other systems looks like a better time
> investment.
> 
> > Well, actually, it looks not so hard to me.
> 
> Every person who got it wrong thought that too.
> 
> > One argument is an URL so, we could url encode it before passing it to
> > youtube-dl
> 
> No, you can not, because it is probably already URL-encoded.
> 
> > Well, actually, vfork() + exec() is better than fork() + exec(),
> > both performance wise and because it works on machines without an
> > MMU, and the implementation of posix_spawn() in glibc is too
> > conservative and uses fork() + exec() even in situations where
> > vfork() could be used, see:
> > https://sourceware.org/bugzilla/show_bug.cgi?id=10354
> > 
> > So, better use vfork() + exec() (that is what system does for
> > instance). I did not use it simply because I thought ffmpeg could
> > have to be compiled on machines without the POSIX interface.
> 
> That was arguments for 20 years ago. Nowadays, vfork() was left to bitrot
> for compatibility reasons while fork() was insanely optimized. And in the
> meantime, threads were invented, with their own set of problems when it
> comes to spawning a child process.

That is what the doc says, but the doc is wrong. Measurements show
that fork() has a detrimental effect, not as much at the moment of
the fork, this is true, this instant is short, but later: all
private writable mappings in the parent process are made read-only
and need to take a fault and go to kernel-space to re-change the MMU
configuration. You can make a simple experiment, map a large private
anonymous area (say 512 MB), and measure the time it takes to write
one byte on each page before and after a fork. At least the
difference was huge last time I tested.

And again, on a platform without MMU, you have vfork(), not fork().
So, when you just need to run exec, it is stupid 

> 
> So the short of it: use posix_spawn().
> 
> The way it is implemented is at best not your concern, at worst premature
> optimization.
> 
> > I did not use it simply because I thought ffmpeg could have to be compiled
> > on machines without the POSIX interface.
> 
> FFmpeg requires POSIX. posix_spawn() is a POSIX option, yet, so it requires
> a configure check. But apart from that, let someone who cares implement for
> other systems.
> 
> Anyway:
> 
> > > Then I suggest you just endeavour to write a pseudo-demuxer for the output
> > > of "youtube-dl -J". No need of a shell or an external command.
> 
> Just do that, no need for any fancy system call for that, and it would serve
> as a base for a more fully automatic version anyway later.

I do not follow you here. youtube-dl -J has a uselessly verbose
output, much harder to parse than youtube-dl -g which gives you just
one URL. But you still need to run youtube-dl. So, I do not see the
upside.

-- 
					    Gilles.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 811 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20150408/33d24540/attachment.asc>