[FFmpeg-devel] [PATCHv2] avutil/file_open: avoid file handle inheritance on Windows

[wm4]

On Fri, 30 Oct 2015 16:15:30 +0100
Tobias Rapp <t.rapp at noa-archive.com> wrote:

> On 30.10.2015 15:06, wm4 wrote:
> > On Fri, 30 Oct 2015 11:30:40 +0100
> > Tobias Rapp <t.rapp at noa-archive.com> wrote:
> >  
> >> On 29.10.2015 09:38, Tobias Rapp wrote:  
> >>> Attached patch fixes file lock issues in my Windows application when a
> >>> child process is started with handle inheritance enabled (standard
> >>> input/output redirection) while a FFmpeg transcoding is running in the
> >>> parent process.
> >>>
> >>> BTW: It would be great if the patch could also be applied to the 2.7/2.8
> >>> release branches.  
> >>
> >> Forgot to add links relevant to the subject.
> >>
> >> [1] https://msdn.microsoft.com/en-us/library/w7sa2b22.aspx
> >>
> >> Describes the _wsopen() function and the O_NOINHERIT flag. File handles
> >> opened by _wsopen() are inheritable by default.
> >>
> >> [2]
> >> https://msdn.microsoft.com/en-us/library/windows/desktop/aa363858%28v=vs.85%29.aspx
> >>
> >> Describes the CreateFile() function. Not directly relevant here, often
> >> used in cases outside of C/libc. Opens file without inheritance by
> >> default (lpSecurityAttributes is NULL).
> >>
> >> [3]
> >> https://msdn.microsoft.com/en-us/library/windows/desktop/ms682425%28v=vs.85%29.aspx
> >>
> >> Describes handle inheritance when creating new processes. Handle
> >> inheritance must be enabled (bInheritHandles = TRUE) e.g. when you want
> >> to pass handles for stdin/stdout via lpStartupInfo.  
> >
> > Would be great if you could put all this stuff into the commit message.  
> 
> Have attached a more verbose version of the patch.

Thanks.

> > Would it make sense to define O_CLOEXEC to O_NOINHERIT on Windows?  
> 
> Although to my knowledge O_CLOEXEC and O_NOINHERIT cause the same 
> behavior it would be confusing to trace when O_CLOEXEC maps to the 
> original Linux definition and when it maps to the Windows override when 
> reading source code. If someone finds it useful a definition like 
> FF_O_CLOEXEC could be added (to which file?) that maps to the correct 
> flag depending on the platform.

OK, since we use it only in 1 place anyway, this is probably not needed.