[FFmpeg-cvslog] libavutil/hwcontext_dxva2: Add check for possible errors from GetAdapterDisplayModeEx

[Humberto Ribeiro]

ffmpeg | branch: release/3.4 | Humberto Ribeiro <mont3z.claro5 at gmail.com> | Wed Jan  3 15:22:15 2018 -0800| [7c3d519df90f36ffb48436861e8b8028199ee0a8] | committer: James Almer

libavutil/hwcontext_dxva2: Add check for possible errors from GetAdapterDisplayModeEx

This prevents a possible crash in CreateDeviceEx when using faulty
response from GetAdapterDisplayModeEx and allows ffmpeg to fallback to
classic d3d9.

Signed-off-by: wm4 <nfxjfg at googlemail.com>
(cherry picked from commit 59b126f92225316e0cd77bb952d630553801dc85)

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=7c3d519df90f36ffb48436861e8b8028199ee0a8
---

 libavutil/hwcontext_dxva2.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/libavutil/hwcontext_dxva2.c b/libavutil/hwcontext_dxva2.c
index 2ddd4be7b1..44ebdbca95 100644
--- a/libavutil/hwcontext_dxva2.c
+++ b/libavutil/hwcontext_dxva2.c
@@ -485,7 +485,11 @@ static int dxva2_device_create9ex(AVHWDeviceContext *ctx, UINT adapter)
     if (FAILED(hr))
         return AVERROR_UNKNOWN;
 
-    IDirect3D9Ex_GetAdapterDisplayModeEx(d3d9ex, adapter, &modeex, NULL);
+    hr = IDirect3D9Ex_GetAdapterDisplayModeEx(d3d9ex, adapter, &modeex, NULL);
+    if (FAILED(hr)) {
+        IDirect3D9Ex_Release(d3d9ex);
+        return AVERROR_UNKNOWN;
+    }
 
     d3dpp.BackBufferFormat = modeex.Format;