[FFmpeg-user] FFmpeg lib + UDP + Qt issues

[isyiwang]

Hello,
First of all, thank you to this amazing community of developers!  I have
received a tremendous amount of help both directly and indirectly thanks to
you all.

I am in need of some help regarding reading encoded video udp packets and
decoding them into a Qt display.  I have managed to read the video packets
and display them in Qt but video playback 'stutters'.  Checking the FPS of
frames transmitted via the FFMpeg library to Qt, I get ~30 frames per
second, so I'm led to believe that FFMpeg itself is sending me the same
frames multiple times a second.  Here are the steps I took:


    1. Broadcast an H.264 encoded video stream via VLC to
udp:/238.38.38.38:1234

    2. Use FFMpeg lib to decode the stream.  Here is my function (apologies
in advance, if there is a better way to display code, please let me know!):

bool QVideoDecoder::decodeSeekFrame(int after)
{
bool done=false;
while(!done)
{
// Read a frame
timeoutTimer.start();
if(av_read_frame(pFormatCtx, &packet)<0)
return false;                             // Frame read failed (e.g. end of
stream)
timeoutTimer.stop();

if(packet.stream_index==videoStream)
{
int frameFinished;
avcodec_decode_video2(pCodecCtx,pFrame,&frameFinished,&packet);

// Did we get a video frame?
if(frameFinished)
{
ffmpeg::AVRational millisecondbase = {1, 1000};
int f = packet.dts;
int t =
ffmpeg::av_rescale_q(packet.dts,pFormatCtx->streams[videoStream]->time_base,millisecondbase);
if(LastFrameOk==false)
{
LastFrameOk=true;
LastLastFrameTime=LastFrameTime=t;
LastLastFrameNumber=LastFrameNumber=f;
}
else
{
// If we decoded 2 frames in a row, the last times are okay
LastLastFrameTime = LastFrameTime;
LastLastFrameNumber = LastFrameNumber;
LastFrameTime=t;
LastFrameNumber=f;
}

// Is this frame the desired frame?
if(after==-1 || LastFrameNumber>=after)
{
// Convert the image format (init the context the first time)
int w = pCodecCtx->width;
int h = pCodecCtx->height;
img_convert_ctx = ffmpeg::sws_getCachedContext(img_convert_ctx,w, h,
pCodecCtx->pix_fmt, w, h, ffmpeg::PIX_FMT_RGB24, SWS_BICUBIC, NULL, NULL,
NULL);

if(img_convert_ctx == NULL)
{
printf("Cannot initialize the conversion context!\n");
return false;
}
ffmpeg::sws_scale(img_convert_ctx, pFrame->data, pFrame->linesize, 0,
pCodecCtx->height, pFrameRGB->data, pFrameRGB->linesize);

// Convert the frame to QImage
LastFrame=QImage(w,h,QImage::Format_RGB888);

for(int y=0;y<h;y++)
memcpy(LastFrame.scanLine(y),pFrameRGB->data[0]+y*pFrameRGB->linesize[0],w*3);

// Set the time
DesiredFrameTime =
ffmpeg::av_rescale_q(after,pFormatCtx->streams[videoStream]->time_base,millisecondbase);
LastFrameOk=true;

done = true;
}
}
}
av_free_packet(&packet);
}

return done;
}

I have a periodic timer that requests the latest frame from FFMpeg. If I set
the timer to a low rate (20 FPS), I get the video played back but at a slow
rate.  If I set the timer to a high rate (60 FPS), I get the frames as soon
as they are ready, but the video stutters.  I figured that I should request
the frames back as fast as possible..  Any insight on how to avoid the
stuttering is appreciated.

    3. Send the QImage back to Qt to be displayed -- I have tested the
frames received by my Qt callback per second and can confirm that I am
receiving ~ 30 FPS which leads me to believe the stuttering is caused by
FFMpeg lib.

    4. I also wrote an interrupting callback to be called by FFMpeg if any
of the blocking functions (i.e. av_read_frame) took too long:

Timer timeoutTimer;
static int decode_interrupt_cb(void)
{
if (timeoutTimer.getElapsedTimeInMilliSec() > 1000)
{
qDebug() << "Video Timed Out!";
return 1;
}
return 0;
}

Although I sometimes do receive the "Video Timed Out!" call, the
av_read_frame function still blocks...

Any help would be greatly appreciated!


--
View this message in context: http://ffmpeg-users.933282.n4.nabble.com/FFmpeg-lib-UDP-Qt-issues-tp3810551p3810551.html
Sent from the FFmpeg-users mailing list archive at Nabble.com.