[FFmpeg-devel] [PATCHv2] avutil/file_open: avoid file handle inheritance on Windows

[Michael Niedermayer]

On Fri, Oct 30, 2015 at 04:15:30PM +0100, Tobias Rapp wrote:
> On 30.10.2015 15:06, wm4 wrote:
> >On Fri, 30 Oct 2015 11:30:40 +0100
> >Tobias Rapp <t.rapp at noa-archive.com> wrote:
> >
> >>On 29.10.2015 09:38, Tobias Rapp wrote:
> >>>Attached patch fixes file lock issues in my Windows application when a
> >>>child process is started with handle inheritance enabled (standard
> >>>input/output redirection) while a FFmpeg transcoding is running in the
> >>>parent process.
> >>>
> >>>BTW: It would be great if the patch could also be applied to the 2.7/2.8
> >>>release branches.
> >>
> >>Forgot to add links relevant to the subject.
> >>
> >>[1] https://msdn.microsoft.com/en-us/library/w7sa2b22.aspx
> >>
> >>Describes the _wsopen() function and the O_NOINHERIT flag. File handles
> >>opened by _wsopen() are inheritable by default.
> >>
> >>[2]
> >>https://msdn.microsoft.com/en-us/library/windows/desktop/aa363858%28v=vs.85%29.aspx
> >>
> >>Describes the CreateFile() function. Not directly relevant here, often
> >>used in cases outside of C/libc. Opens file without inheritance by
> >>default (lpSecurityAttributes is NULL).
> >>
> >>[3]
> >>https://msdn.microsoft.com/en-us/library/windows/desktop/ms682425%28v=vs.85%29.aspx
> >>
> >>Describes handle inheritance when creating new processes. Handle
> >>inheritance must be enabled (bInheritHandles = TRUE) e.g. when you want
> >>to pass handles for stdin/stdout via lpStartupInfo.
> >
> >Would be great if you could put all this stuff into the commit message.
> 
> Have attached a more verbose version of the patch.
> 
> >Would it make sense to define O_CLOEXEC to O_NOINHERIT on Windows?
> 
> Although to my knowledge O_CLOEXEC and O_NOINHERIT cause the same
> behavior it would be confusing to trace when O_CLOEXEC maps to the
> original Linux definition and when it maps to the Windows override
> when reading source code. If someone finds it useful a definition
> like FF_O_CLOEXEC could be added (to which file?) that maps to the
> correct flag depending on the platform.
> 
> Regards,
> Tobias

>  file_open.c |    3 +++
>  1 file changed, 3 insertions(+)
> 6f492d310d48ec875ec57d09d285e2f682603524  0001-avutil-file_open-avoid-file-handle-inheritance-on-Wi.patch
> From c2b599c18a173ce3a2f053701bc4ef1f14ef7aea Mon Sep 17 00:00:00 2001
> From: Tobias Rapp <t.rapp at noa-audio.com>
> Date: Thu, 29 Oct 2015 09:11:37 +0100
> Subject: [PATCH] avutil/file_open: avoid file handle inheritance on Windows
> 
> Avoids inheritance of file handles on Windows systems similar to the
> O_CLOEXEC/FD_CLOEXEC flag on Linux.
> 
> Fixes file lock issues in Windows applications when a child process
> is started with handle inheritance enabled (standard input/output
> redirection) while a FFmpeg transcoding is running in the parent
> process.
> 
> Links relevant to the subject:
> 
> https://msdn.microsoft.com/en-us/library/w7sa2b22.aspx
> 
> Describes the _wsopen() function and the O_NOINHERIT flag. File handles
> opened by _wsopen() are inheritable by default.
> 
> https://msdn.microsoft.com/en-us/library/windows/desktop/ms682425%28v=vs.85%29.aspx
> 
> Describes handle inheritance when creating new processes. Handle
> inheritance must be enabled (bInheritHandles = TRUE) e.g. when you want
> to pass handles for stdin/stdout via lpStartupInfo.
> 
> Signed-off-by: Tobias Rapp <t.rapp at noa-audio.com>
> ---
>  libavutil/file_open.c | 3 +++
>  1 file changed, 3 insertions(+)

applied

thanks

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Rewriting code that is poorly written but fully understood is good.
Rewriting code that one doesnt understand is a sign that one is less smart
then the original author, trying to rewrite it will not make it better.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20151102/6bd06431/attachment.sig>