[FFmpeg-devel] [PATCH] lavc/vvc: Mark SPS used if multiple CLVSSs use it

[Frank Plowman]

Consider the following sequence of NALUs (with some PPSs etc. omitted
for brevity):
1. SPS (ID=0, content=A)
2. IDR (SPS=0)
3. IDR (SPS=0)
4. SPS (ID=0, content=B)
5. TRAIL (SPS=0)

When decode_sps is called for NALU 3., ps->sps_id_used is cleared as
IDRs are one way of forming a CLVSS.  Then, old_sps is non-NULL
containing the result of calling decode_sps for NALU 2.  We haven't
received any SPSs between NALUs 2. and 3., therefore old_sps and rsps
are identical and the function returns.  The issue is that, at this
point, ps->sps_id_used is still zero despite the SPS being used for IDR
3.  This results in the check for conflicting SPSs not working properly
when decode_sps is called for NALU 5., allowing prediction between
pictures with different SPSs and probably all sorts of other
shenanigans.

Patch addresses the problem outlined above by also setting
ps->sps_id_used in the early return case.

Signed-off-by: Frank Plowman <post at frankplowman.com>
---
Patch is also available at https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20061
---
 libavcodec/vvc/ps.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/libavcodec/vvc/ps.c b/libavcodec/vvc/ps.c
index d9f46b219a..6eec0fe586 100644
--- a/libavcodec/vvc/ps.c
+++ b/libavcodec/vvc/ps.c
@@ -288,9 +288,10 @@ static int decode_sps(VVCParamSets *ps, AVCodecContext *c, const H266RawSPS *rsp
     }
 
     if (old_sps) {
-        if (old_sps->r == rsps || !memcmp(old_sps->r, rsps, sizeof(*old_sps->r)))
+        if (old_sps->r == rsps || !memcmp(old_sps->r, rsps, sizeof(*old_sps->r))) {
+            ps->sps_id_used |= (1 << sps_id);
             return 0;
-        else if (ps->sps_id_used & (1 << sps_id))
+        } else if (ps->sps_id_used & (1 << sps_id))
             return AVERROR_INVALIDDATA;
     }
 
-- 
2.47.0