[FFmpeg-devel] [PATCH] libavutil/hwcontext_qsv: fix a bug for mapping qsv frame to vaapi

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> Wenbin Chen
> Sent: Thursday, 9 September 2021 11:13
> To: ffmpeg-devel at ffmpeg.org
> Cc: Wenbin Chen <wenbin.chen at intel.com>
> Subject: [FFmpeg-devel] [PATCH] libavutil/hwcontext_qsv: fix a bug
> for mapping qsv frame to vaapi
> 
> Command below failed.
> ffmpeg -v verbose -init_hw_device vaapi=va:/dev/dri/renderD128
> -init_hw_device qsv=qs at va -hwaccel qsv -hwaccel_device qs
> -filter_hw_device va -c:v h264_qsv
> -i 1080P.264 -vf "hwmap,format=vaapi" -c:v h264_vaapi output.264
> 
> Cause: Assign pair->first directly to data[3] in vaapi frame.
> pair->first is *VASurfaceID while data[3] in vaapi frame is
> VASurfaceID. I fix this line of code. Now the command above works.
> 
> Signed-off-by: Wenbin Chen <wenbin.chen at intel.com>
> ---
>  libavutil/hwcontext_qsv.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c
> index d431e71eab..6539cae619 100644
> --- a/libavutil/hwcontext_qsv.c
> +++ b/libavutil/hwcontext_qsv.c
> @@ -781,7 +781,7 @@ static int qsv_map_from(AVHWFramesContext *ctx,
>      case AV_HWDEVICE_TYPE_VAAPI:
>      {
>          mfxHDLPair *pair = (mfxHDLPair*)surf->Data.MemId;
> -        child_data = pair->first;
> +        child_data = (uint8_t*)(intptr_t)*(VASurfaceID*)pair->first;
>          break;
>      }
>  #endif
> --

Hi Wenbin,

please let me check. That line is from my original (and current) code
regarding QSV-D3D11 and I don't have issues using hwmap with QSV and 
VAAPI. I'll need to compare the commited D3D11 patch with my fork.
Probably something is missing at another other place.

Regards,
softworkz