[FFmpeg-devel] Network IO in FFmpeg (was: abstractable threading api)

Nicolas George george at nsup.org
Thu Dec 19 17:58:16 CET 2013

Le sextidi 26 frimaire, an CCXXII, Hendrik Leppkes a écrit :
> The compat isn't complete, it just contains the functions needed by
> avcodec et al, well short of pthread_cancel.
> However, providing pthread_cancel with win32 threading api is also
> non-trivial, if not impossible to find a "drop in" compat
> functionality.
> Canceling threads from the outside is a no-no in the win32 API. Most
> blocking functions usually have the option to give it a unblock event
> that you can trigger to release it, and safely shut down the thread
> from within, however as you can imagine thats not something that ports
> well over different APIs.
> The best course of action in this case would probably be to find a way
> to re-write this worker thread without pthread_cancel, other solutions
> seem rather ugly.

Regarding the pthread_cancel thing, I am the one responsible for this bit of
code, I can explain how it is needed.

The basic idea is that we have a thread blocked inside a system call:

	read(udp_sock_fd, buffer, size);

and we want to terminate it, because reading on the socket is finished. Note
that no packet may arrive there for a long time, so waiting for the system
call to return by itself is not an option.

First, a small discussion: in this particular case, we want to stop the
whole thread. If we had a way of interrupting just the system call, stopping
the thread is just a matter of checking a flag afterwards. If the situation
were inverted: we knew how to cancel the thread but want to interrupt just
the system call, then we could start a new thread just for the system call:
more work, but that does the trick. In short: whether we cancel the system
call or the whole thread, we are good.

Now, let us see the options:

* Signals

  Using a signal to interrupt a system call is a long tradition, but
  handling signals reliably is a portability nightmare, and mixing threads
  with signals is that squared. Let us avoid it.

* pthread_cancel()

  With true POSIX threads, it is perfect: pthread_cancel() only interrupts
  at specific points of the code, so we do not need to care about atomicity,
  and reading on a socket is guaranteed to be one.

* Closing the socket

  ... does not work.

* The asynchronous I/O API aio_*

  It could work, but I am afraid it does not. This is a seldom used API, and
  thus it will have a lot of pitfalls. On GNU/Linux, aio_* is implemented
  with hidden threads; furthermore, a blocking operation is not really

* Non-blocking I/O + usleep() + retry loop / poll() with periodic timeout

  The "poll() with periodic timeout" solution was used before I changed it to
  pthread_cancel(). Both solution work, but they are universally frowned

* poll() + self-pipe

  The idea is to poll() on both the socket and a pipe internal to the
  program. Writing on the internal pipe allows to interrupt the poll() call

IMHO, the best solutions for this case, are, in decreasing order:
interrupting the system call (drawback: portability nightmare); cancelling
the thread (drawback: pthreads-specific); poll+self-pipe (drawback: two
system calls per read).

If someone (Hendrik?) knows what the non-Unix APIs allows, especially the
windows ones, please share it here.

In a next mail, I will discuss why we want this thread for UDP sockets, and
what other options are available.


  Nicolas George
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20131219/6ae5a0cc/attachment.asc>

More information about the ffmpeg-devel mailing list