[FFmpeg-devel] [PATCH] filter_design: document ownership and permissions.

[Nicolas George]

Signed-off-by: Nicolas George <nicolas.george at normalesup.org>
---
 doc/filter_design.txt |  155 ++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 154 insertions(+), 1 deletion(-)


If what is stated below is considered correct, then the next task will be to
ensure that all filters obey the rules (and I am fairly sure a lot of them
do not).


diff --git a/doc/filter_design.txt b/doc/filter_design.txt
index 9e3eb3d..2754930 100644
--- a/doc/filter_design.txt
+++ b/doc/filter_design.txt
@@ -33,7 +33,160 @@ Format negotiation
 Buffer references ownership and permissions
 ===========================================
 
-  TODO
+  Principle
+  ---------
+
+    Audio and video data are voluminous; the buffer and buffer reference
+    mechanism is intended to avoid, as much as possible, expensive copies of
+    that data while still allowing the filters to produce correct results.
+
+    The data is stored in buffers represented by AVFilterBuffer structures.
+    They must not be accessed directly, but through references stored in
+    AVFilterBufferRef structures. Several references can point to the
+    same buffer; the buffer is automatically deallocated once all
+    corresponding references have been destroyed.
+
+    The characteristics of the data (resolution, sample rate, etc.) are
+    stored in the reference; different references for the same buffer can
+    show different characteristics. In particular, a video reference can
+    point to only a part of a video buffer.
+
+    A reference is usually obtained as input to the start_frame or
+    filter_samples method or requested using the ff_get_video_buffer or
+    ff_get_audio_buffer functions. A new reference on an existing buffer can
+    be created with the avfilter_ref_buffer. A reference is destroyed using
+    the avfilter_unref_bufferp function.
+
+  Reference ownership
+  -------------------
+
+    At any time, a reference “belongs” to a particular piece of code,
+    usually a filter. With a few caveats that will be explained below, only
+    that piece of code is allowed to access it. It is also responsible for
+    destroying it, although this is sometimes done automatically (see the
+    section on link reference fields).
+
+    Here are the (fairly obvious) rules for reference ownership:
+
+    * A reference received by the start_frame or filter_samples method
+      belong to the corresponding filter.
+      
+      Special exception: for video references: it may be used internally for
+      automatic copying and must not be destroyed before end_frame; it can
+      be given away to ff_start_frame.
+
+    * A reference passed to ff_start_frame or ff_filter_samples is given
+      away and must no longer be used.
+
+    * A reference created with avfilter_ref_buffer belongs to the code that
+      created it.
+
+    * A reference obtained with ff_get_video_buffer of ff_get_audio_buffer
+      belongs to the code that requested it.
+
+    * A reference given as return value by the get_video_buffer or
+      get_audio_buffer method is given away and must no longer be used.
+
+  Link reference fields
+  ---------------------
+
+    The AVFilterLink structure has a few AVFilterBufferRef fields. Here are
+    the rules to handle them:
+
+    * cur_buf is set before the start_frame and filter_samples methods to
+      the same reference given as argument to the methods and belongs to the
+      destination filter of the link. If it has not been cleared after
+      end_frame or filter_samples, libavfilter will automatically destroy
+      the reference; therefore, any filter that needs to keep the reference
+      for longer must set cur_buf to NULL.
+
+    * out_buf belongs to the source filter of the link and can be used to
+      store a reference to the buffer that has been sent to the destination.
+      If it is not NULL after end_frame or filter_samples, libavfilter will
+      automatically destroy the reference.
+
+      If a video input pad does not have a start_frame method, the default
+      method will allocate a buffer on the first output of the filter, store
+      the reference in out_buf and push a second reference to the output.
+
+    * src_buf, cur_buf_copy and partial_buf are used by libavfilter
+      internally and must not be accessed by filters.
+
+  Reference permissions
+  ---------------------
+
+    The AVFilterBufferRef structure has a perms field that describes what
+    the code that owns the reference is allowed to do to the buffer data.
+    Different references for the same buffer can have different permissions.
+
+    For video filters, the permissions only apply to the parts of the buffer
+    that have already been covered by the draw_slice method.
+
+    The value is a binary OR of the following constants:
+
+    * AV_PERM_READ: the owner can read the buffer data; this is essentially
+      always true and is there for self-documentation.
+
+    * AV_PERM_WRITE: the owner can modify the buffer data.
+
+    * AV_PERM_PRESERVE: the owner can rely on the fact that the buffer data
+      will not be modified by previous filters.
+
+    * AV_PERM_REUSE: the owner can output the buffer several times, without
+      modifying the data in between.
+
+    * AV_PERM_REUSE2: the owner can output the buffer several times an
+      modify the data in between (useless without the WRITE permissions).
+
+    * AV_PERM_ALIGN: the owner can access the data using fast operations
+      that require data alignment.
+
+    The READ, WRITE and PRESERVE permissions are about sharing the same
+    buffer between several filters to avoid expensive copies without them
+    doing conflicting changes on the data.
+
+    The REUSE and REUSE2 permissions are about special memory for direct
+    rendering. For example a buffer directly allocated in video memory must
+    not modified once it is displayed on screen, or it will cause tearing;
+    it will therefore not have the REUSE2 permission.
+
+    The ALIGN permission is about extracting part of the buffer, for
+    copy-less padding or cropping for example.
+
+
+    References received on input pads are guaranteed to have all the
+    permissions stated in the min_perms field and none of the permissions
+    stated in the rej_perms.
+
+    References obtained by ff_get_video_buffer and ff_get_video_buffer are
+    guaranteed to have at least all the permissions requested as argument.
+
+    References created by avfilter_ref_buffer have the same permissions as
+    the original reference minus the ones explicitly masked; the mask is
+    usually ~0 to keep the same permissions.
+
+    Filters should remove permissions on reference they give to output
+    whenever necessary. It can be automatically done by setting the
+    rej_perms field on the output pad.
+
+    Here are a few guidelines corresponding to common situations:
+
+    * Filters that modify and forward their frame (like drawtext) need the
+      WRITE permission.
+
+    * Filters that read their input to produce a new frame on output (like
+      scale) need the WRITE permission on input and WRITE permission on the
+      new buffer.
+
+    * Filters that intend to keep a reference after the filtering process
+      is finished (after end_frame or filter_samples returns) must have the
+      PRESERVE permission on it and remove the WRITE permission if they
+      create a new reference to give it away.
+
+    * Filters that intend to modify a reference they have kept after the end
+      of the filtering process need the REUSE2 permission and must remove
+      the PRESERVE permission if they create a new reference to give it
+      away.
 
 
 Frame scheduling
-- 
1.7.10.4