[FFmpeg-devel] [PATCH] libavformat/vvc: Make probe more conservative

[post at frankplowman.com]

From: Frank Plowman <post at frankplowman.com>

Reduce false positives for VVC files by adding additional checks in
`vvc_probe`. Specifically, `nuh_temporal_id_plus1` is tested for valid
values in extra cases depending on the NAL unit type, as per ITU-T H.266
section 7.4.2.2.

Resolves trac #10703.

Signed-off-by: Frank Plowman <post at frankplowman.com>
---
 libavformat/vvcdec.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/libavformat/vvcdec.c b/libavformat/vvcdec.c
index 31c5ae1f14..57d2769465 100644
--- a/libavformat/vvcdec.c
+++ b/libavformat/vvcdec.c
@@ -35,11 +35,17 @@ static int vvc_probe(const AVProbeData *p)
         if ((code & 0xffffff00) == 0x100) {
             uint8_t nal2 = p->buf[i + 1];
             int type = (nal2 & 0xF8) >> 3;
+            const uint8_t nuh_temporal_id_plus1 = nal2 & 0x7;
 
             if (code & 0x80) // forbidden_zero_bit
                 return 0;
 
-            if ((nal2 & 0x7) == 0) // nuh_temporal_id_plus1
+            if (nuh_temporal_id_plus1 == 0) // nuh_temporal_id_plus1
+                return 0;
+            if (nuh_temporal_id_plus1 != 1 && (type >= VVC_IDR_W_RADL && type <= VVC_RSV_IRAP_11
+                                            || type == VVC_DCI_NUT || type == VVC_OPI_NUT
+                                            || type == VVC_VPS_NUT || type == VVC_SPS_NUT
+                                            || type == VVC_EOS_NUT || type == VVC_EOB_NUT))
                 return 0;
 
             switch (type) {
-- 
2.43.0