[FFmpeg-cvslog] vulkan_h264: filter out constrained/inter flags from the profile index

[Lynne]

ffmpeg | branch: master | Lynne <dev at lynne.ee> | Wed Jun 14 05:28:03 2023 +0200| [ca818ab51c437063b2559038aa5ab73dcffb4ff5] | committer: Lynne

vulkan_h264: filter out constrained/inter flags from the profile index

As the comment says, Vulkan signals all the constrant_set flags,
and does not want them OR'd onto the profile IDC.
So just unset them.

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=ca818ab51c437063b2559038aa5ab73dcffb4ff5
---

 libavcodec/vulkan_decode.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/libavcodec/vulkan_decode.c b/libavcodec/vulkan_decode.c
index 35e265a5b1..6ba939da46 100644
--- a/libavcodec/vulkan_decode.c
+++ b/libavcodec/vulkan_decode.c
@@ -660,7 +660,12 @@ static VkResult vulkan_setup_profile(AVCodecContext *avctx,
         dec_caps->pNext = h264_caps;
         usage->pNext = h264_profile;
         h264_profile->sType = VK_STRUCTURE_TYPE_VIDEO_DECODE_H264_PROFILE_INFO_KHR;
-        h264_profile->stdProfileIdc = cur_profile;
+
+        /* Vulkan transmits all the constrant_set flags, rather than wanting them
+         * merged in the profile IDC */
+        h264_profile->stdProfileIdc = cur_profile & ~(FF_PROFILE_H264_CONSTRAINED |
+                                                      FF_PROFILE_H264_INTRA);
+
         h264_profile->pictureLayout = avctx->field_order == AV_FIELD_UNKNOWN ||
                                       avctx->field_order == AV_FIELD_PROGRESSIVE ?
                                       VK_VIDEO_DECODE_H264_PICTURE_LAYOUT_PROGRESSIVE_KHR :