[FFmpeg-cvslog] ffmpeg_filter: don't try to autorotate frames with hwaccel pixel formats

[James Almer]

ffmpeg | branch: master | James Almer <jamrial at gmail.com> | Tue Sep 21 09:29:15 2021 -0300| [ed1a70469e526bd3a4122242b66d73914743094f] | committer: James Almer

ffmpeg_filter: don't try to autorotate frames with hwaccel pixel formats

The transpose, rotate, hflip, and vflip filters don't support them.
Fixes ticket #9432.

Reviewed-by: Nicolas George <george at nsup.org>
Signed-off-by: James Almer <jamrial at gmail.com>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=ed1a70469e526bd3a4122242b66d73914743094f
---

 fftools/ffmpeg_filter.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c
index da0d4faf54..b798459946 100644
--- a/fftools/ffmpeg_filter.c
+++ b/fftools/ffmpeg_filter.c
@@ -699,6 +699,7 @@ static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter,
 {
     AVFilterContext *last_filter;
     const AVFilter *buffer_filt = avfilter_get_by_name("buffer");
+    const AVPixFmtDescriptor *desc;
     InputStream *ist = ifilter->ist;
     InputFile     *f = input_files[ist->file_index];
     AVRational tb = ist->framerate.num ? av_inv_q(ist->framerate) :
@@ -756,7 +757,11 @@ static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter,
     av_freep(&par);
     last_filter = ifilter->filter;
 
-    if (ist->autorotate) {
+    desc = av_pix_fmt_desc_get(ifilter->format);
+    av_assert0(desc);
+
+    // TODO: insert hwaccel enabled filters like transpose_vaapi into the graph
+    if (ist->autorotate && !(desc->flags & AV_PIX_FMT_FLAG_HWACCEL)) {
         int32_t *displaymatrix = ifilter->displaymatrix;
         double theta;