[FFmpeg-devel] [PATCHv8] libavcodec: v4l2: add support for v4l2 mem2mem codecs

[Jorge Ramirez]

On 09/03/2017 08:06 PM, Mark Thompson wrote:
> On 03/09/17 15:36, Jorge Ramirez wrote:
>> On 09/03/2017 02:27 AM, Mark Thompson wrote:
>>>> +    }
>>>> +
>>>> +    /* 2.1 update the AVCodecContext */
>>>> +    avctx->pix_fmt = ff_v4l2_v4l2fmt_to_avfmt(capture->format.fmt.pix_mp.pixelformat, AV_CODEC_ID_RAWVIDEO);
>>>> +    capture->av_pix_fmt = avctx->pix_fmt;
>>>> +
>>>> +    /* 3. set the crop parameters */
>>>> +    selection.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
>>>> +    selection.r.height = avctx->coded_height;
>>>> +    selection.r.width = avctx->coded_width;> +    ret = ioctl(s->fd, VIDIOC_S_SELECTION, &selection);
>>>> +    if (!ret) {
>>>> +        ret = ioctl(s->fd, VIDIOC_G_SELECTION, &selection);
>>>> +        if (ret) {
>>>> +            av_log(avctx, AV_LOG_ERROR, "VIDIOC_G_SELECTION ioctl\n");
>>>> +        } else {
>>>> +            av_log(avctx, AV_LOG_DEBUG, "crop output %dx%d\n", selection.r.width, selection.r.height);
>>>> +            /* update the size of the resulting frame */
>>>> +            capture->height = selection.r.height;
>>>> +            capture->width  = selection.r.width;
>>>> +        }
>>>> +    }
>>> What is this trying to do?  I'm not sure that the coded_width|height values are something you should be using here - in many cases they are larger than the actual output (what does 1920x1080 H.264, which is coded as 1920x1088, do?).  Also, while they are often sensible for the stream if it comes from lavf and a normal file container, they need not be set at all here (0x0).
>>>
>> I am just trying to crop the decoded frame size to what I understood are the expected dimensions; if you look at buffer_ops_v4l2buf_to_avframe (used to return the dequeued buffer to the frame) we use these variables to return the frame dimensions back to ffmpeg.
>>
>> When ffplaying 1920x1080 this is what xwininfo (window information utility for X) returns:
>>
>> [jramirez at igloo ~]$ xwininfo
>>
>> xwininfo: Please select the window about which you
>>            would like information by clicking the
>>            mouse in that window.
>>
>> xwininfo: Window id: 0x560000c "Big Buck Bunny, Sunflower version - /home/linaro/Videos/avc1.bbb_sunflower_1080p_30fps_normal.mp4"
>>
>>    Absolute upper-left X:  2240
>>    Absolute upper-left Y:  204
>>    Relative upper-left X:  0
>>    Relative upper-left Y:  0
>>    Width: 1920
>>    Height: 1088
>>    Depth: 24
>>    Visual: 0xeb
>>    Visual Class: DirectColor
>>    Border width: 0
>>    Class: InputOutput
>>    Colormap: 0x560000b (not installed)
>>    Bit Gravity State: ForgetGravity
>>    Window Gravity State: NorthWestGravity
>>    Backing Store State: NotUseful
>>    Save Under State: no
>>    Map State: IsViewable
>>    Override Redirect State: no
>>    Corners:  +2240+204  -2240+204  -2240-148  +2240-148
>>    -geometry 1920x1088+2230+166
>>
>>
>> maybe it works out nicely because the frames returned by the decoder are also 32 byte aligned?
>> this is what is encoded in the video
>>
>> [jramirez at igloo Videos]$ ffprobe -show_frames avc1.bbb_sunflower_1080p_30fps_normal.mp4 | more
>>
>>         1 [FRAME]
>>         2 media_type=video
>>         3 stream_index=0
>>         4 key_frame=1
>>         5 pkt_pts=2000
>>         6 pkt_pts_time=0.066667
>>         7 pkt_dts=2000
>>         8 pkt_dts_time=0.066667
>>         9 best_effort_timestamp=2000
>>        10 best_effort_timestamp_time=0.066667
>>        11 pkt_duration=1000
>>        12 pkt_duration_time=0.033333
>>        13 pkt_pos=236997
>>        14 pkt_size=1116
>>        15 width=1920
>>        16 height=1080
>>        17 pix_fmt=yuv420p
>>        18 sample_aspect_ratio=1:1
>>        19 pict_type=I
>>        20 coded_picture_number=0
>>        21 display_picture_number=0
>>        22 interlaced_frame=0
>>        23 top_field_first=0
>>        24 repeat_pict=0
>>        25 [/FRAME]
>>
>>
>> so if this is not correct, could you suggest or hint for a proper way please?
> I think I've failed to emphasise the important point here: you can't use the coded_width/coded_height fields in AVCodecContext to do something like this because they need not be set.  It is completely valid for the user to not set them when using libavcodec, such that the decoder receives coded_width = coded_height = 0 on input.  It looks like that will then do something nasty in your code, when it tries to crop the output to 0x0.

yep.

>
> If you really need the width and height as found in the bitstream here then you will need to find it yourself.  (For a different case which does something very similar to this, see the use of parsers qsvdec.)

that is ok, I can let the driver detect the discrepancy and raise an 
event with the proper dimensions. this will be slightly different to 
reinit since I will have to reconfigure the output queue as well but not 
a major issue I think....maybe another 10 LOC or so


>
> (There is also the secondary point that coded_width/coded_height do not actually contain the output width/height because of stream cropping, but that is probably less important and easier to fix.)

ok

> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel