[FFmpeg-devel] [PATCH] VFW capture support
Michael Niedermayer
michaelni
Fri Mar 7 23:06:17 CET 2008
On Fri, Mar 07, 2008 at 02:52:18PM -0300, Ramiro Polla wrote:
> Hello Michael,
>
> I've rewritten vfwcap to use a different capture strategy. It's more
> complex but much more accurate now.
>
> [...]
>>>> Doesnt vfw provide some timestamp for each frame?
>>> The timestamps returned are incredibly unreliable (if returned at all,
>>> most times I just get 0x00000000).
>> Could you give me a hint of how they are returned, so i can check the docs
>> from M$ ?
>> Use of "get current time" is very problematic as frames will have very
>> inaccuarte timestamps.
>> Thus if that is the only kind of time you can get from VFW then i fear
>> you will need to clean them up somehow so that they are at least stable
>> to +- 1ms of the nominal frame rate of the video source.
>
> Using the grab_frame strategy no timestamps were returned. With this new
> implementation the timestamps returned are accurate. There's no more need
> to use the system time.
>
> [...]
>
>>>>> if( s->flags & AVFMT_FLAG_NONBLOCK ) {
>>>>> av_log( s, AV_LOG_ERROR, "Non blocking capture not yet
>>>>> implemented.\n" );
>>>>> return AVERROR_PATCHWELCOME;
>>>>> }
>>>> where is the problem with that?
>>> I thought we didn't want FFmpeg to block unless the user specifies
>>> AVFMT_FLAG_NONBLOCK. Isn't that the purpose of the flag?
>> Yes, what i was asking, is where the problem is with implementing support
>> for AVFMT_FLAG_NONBLOCK?
>
> The new implementation deals better with AVFMT_FLAG_NONBLOCK. It does
> pretty much the same thing v4l2 does.
>
> [...]
>
>> Also can this patch do full framerate(30/25fps 50/60fields/sec) / full
>> resolution realtime capture and encoding in realtime with audio? If not
>> its definitly not well implemented.
>> Also does VFW buffer frames? If no then a seperate thread is pretty much
>> required for video capture. OTOH if it does buffer frames then the way
>> you generate timestamps is just wrong.
>
> VFW does buffer frames. The timestamps were indeed very inaccurate using
> the system time.
>
> Mans suggested using a ring buffer, a semaphore and dropping frames instead
> of using the mutexes and events and piling up every single frame.
>
> If it is preferred to drop frames, is there an existing option that could
> be used to set the maximum amount of memory/time buffered?
Droping frames might be more user friendly. But simply droping when the buffer
is full is not optimal. Reason being that with a nearly full buffer and
some short drop in available resources (cronjob or whatever) several frames
could be droped in a row.
What should be done is that as the buffer becomes more full more frames
should be droped, so for example maybe with the buffer 0-50% full no
frame drops would happen, with 62% every 4th frame would be droped, with
75% every 2nd frame would be droped, 87% 3 out of 4 frames would be droped
...
The trick here is to keep the droped frames evenly distributed.
There are many ways this can be implemnted, the simplest i could think of is
const static uint8_t score[4]={62,87,75,100};
if(score[frame_num%4] < buffer_fullness)
drop;
Another one would be to keep track of how many have been droped/not droped
in a row ...
[...]
> static enum PixelFormat vfw_pixfmt(DWORD biCompression)
> {
> switch(biCompression) {
> case MKTAG('Y', 'U', 'Y', '2'):
> return PIX_FMT_YUYV422;
> case BI_RGB:
> return PIX_FMT_BGR24;
This isnt correct or is it? BI_RGB just says RGB but not which one or am i
remembering this wrong?
[...]
> static LRESULT CALLBACK videostream_cb(HWND hwnd, LPVIDEOHDR vdhdr)
> {
> struct vfw_ctx *ctx;
> AVPacketList *pktl, *pktl_next;
>
> ctx = (struct vfw_ctx *) GetWindowLongPtr(hwnd, GWLP_USERDATA);
>
> #ifdef VFW_DEBUG
> dump_videohdr(ctx->s, vdhdr);
> #endif
>
> WaitForSingleObject(ctx->mutex, INFINITE);
>
> pktl_next = av_mallocz(sizeof(AVPacketList));
> if(!pktl_next)
> return FALSE;
>
> if(av_new_packet(&pktl_next->pkt, vdhdr->dwBytesUsed) < 0) {
> av_free(pktl_next);
> return FALSE;
> }
What will happen with the mutex if you just return here?
>
> pktl_next->pkt.pts = vdhdr->dwTimeCaptured;
> memcpy(pktl_next->pkt.data, vdhdr->lpData, vdhdr->dwBytesUsed);
>
> for(pktl = ctx->pktl ; pktl && pktl->next ; pktl = pktl->next);
> if(!pktl)
> ctx->pktl = pktl_next;
> else
> pktl->next = pktl_next;
following is simpler:
for(plast_pktl= &ctx->pktl ; *plast_pktl ; plast_pktl= &(*plast_pktl)->next);
*plast_pktl= pktl_next
>
> SetEvent(ctx->event);
> ReleaseMutex(ctx->mutex);
>
> return TRUE;
> }
>
> static int vfw_read_header(AVFormatContext *s, AVFormatParameters *ap)
> {
> struct vfw_ctx *ctx = s->priv_data;
> AVCodecContext *codec;
> AVStream *st;
> int devnum;
> int bisize;
> BITMAPINFO *bi;
> CAPTUREPARMS cparms;
> DWORD biCompression;
> int width;
> int height;
> int ret;
>
> if(!ap->time_base.den) {
> av_log(s, AV_LOG_ERROR, "A time base must be specified.\n");
> return AVERROR_IO;
> }
>
> #ifdef VFW_DEBUG
> ctx->s = s;
> #endif
>
> ctx->hwnd = capCreateCaptureWindow(NULL, 0, 0, 0, 0, 0, HWND_MESSAGE, 0);
> if(!ctx->hwnd) {
> av_log(s, AV_LOG_ERROR, "Could not create capture window.\n");
> return AVERROR_IO;
> }
>
> /* If atoi fails, devnum==0 and the default device is used */
> devnum = atoi(s->filename);
>
> ret = SendMessage(ctx->hwnd, WM_CAP_DRIVER_CONNECT, devnum, 0);
> if(!ret) {
> av_log(s, AV_LOG_ERROR, "Could not connect to device.\n");
> return AVERROR(ENODEV);
> }
>
> SendMessage(ctx->hwnd, WM_CAP_SET_OVERLAY, 0, 0);
> SendMessage(ctx->hwnd, WM_CAP_SET_PREVIEW, 0, 0);
>
> ret = SendMessage(ctx->hwnd, WM_CAP_SET_CALLBACK_VIDEOSTREAM, 0,
> (LPARAM) videostream_cb);
> if(!ret)
> return AVERROR_IO;
>
> SetWindowLongPtr(ctx->hwnd, GWLP_USERDATA, (LONG_PTR) ctx);
>
> st = av_new_stream(s, 0);
> if(!st)
> return AVERROR_NOMEM;
>
> /* Set video format */
> bisize = SendMessage(ctx->hwnd, WM_CAP_GET_VIDEOFORMAT, 0, 0);
> if(!bisize)
> return AVERROR_IO;
> bi = av_malloc(bisize);
> if(!bi)
> return AVERROR_NOMEM;
> ret = SendMessage(ctx->hwnd, WM_CAP_GET_VIDEOFORMAT, bisize, (LPARAM) bi);
> if(!ret) {
> av_free(bi);
> return AVERROR_IO;
> }
>
> dump_bih(s, &bi->bmiHeader);
>
> width = ap->width ? ap->width : bi->bmiHeader.biWidth ;
> height = ap->height ? ap->height : bi->bmiHeader.biHeight;
> bi->bmiHeader.biWidth = width ;
> bi->bmiHeader.biHeight = height;
>
> ret = SendMessage(ctx->hwnd, WM_CAP_SET_VIDEOFORMAT, bisize, (LPARAM) bi);
> if(!ret) {
> av_log(s, AV_LOG_ERROR, "Could not set Video Format.\n");
> av_free(bi);
> return AVERROR_IO;
> }
>
> biCompression = bi->bmiHeader.biCompression;
>
> av_free(bi);
>
> /* Set sequence setup */
> ret = SendMessage(ctx->hwnd, WM_CAP_GET_SEQUENCE_SETUP, sizeof(cparms),
> (LPARAM) &cparms);
> if(!ret)
> return AVERROR_IO;
>
> dump_captureparms(s, &cparms);
>
> cparms.fYield = 1; // Spawn a background thread
> cparms.dwRequestMicroSecPerFrame =
> (ap->time_base.num*1000000) / ap->time_base.den;
> cparms.fAbortLeftMouse = 0;
> cparms.fAbortRightMouse = 0;
> cparms.fCaptureAudio = 0;
> cparms.vKeyAbort = 0;
>
> ret = SendMessage(ctx->hwnd, WM_CAP_SET_SEQUENCE_SETUP, sizeof(cparms),
> (LPARAM) &cparms);
> if(!ret)
> return AVERROR_IO;
>
> codec = st->codec;
> codec->time_base = ap->time_base;
> codec->codec_type = CODEC_TYPE_VIDEO;
> codec->width = width;
> codec->height = height;
> codec->codec_id = CODEC_ID_RAWVIDEO;
> codec->pix_fmt = vfw_pixfmt(biCompression);
>
> av_set_pts_info(st, 32, 1, 1000);
>
> if(codec->pix_fmt == -1) {
> av_log(s, AV_LOG_ERROR, "Unknown compression type."
> "Please report verbose (-v 99) debug information.\n");
> return AVERROR_PATCHWELCOME;
> }
>
> ctx->mutex = CreateMutex(NULL, 0, NULL);
> ctx->event = CreateEvent(NULL, 1, 0, NULL);
>
> ret = SendMessage(ctx->hwnd, WM_CAP_SEQUENCE_NOFILE, 0, 0);
> if(!ret)
> return AVERROR_IO;
I suspect that quite a few of the returns cause various leaks, that is
the stuff from vfw_read_close() is missing.
[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
If you really think that XML is the answer, then you definitly missunderstood
the question -- Attila Kinali
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20080307/545c2963/attachment.pgp>
More information about the ffmpeg-devel
mailing list