[Libav-user] muxing h264/aac stream
Anshul
er.anshul.maheshwari at gmail.com
Fri Aug 23 07:15:57 CEST 2013
On 08/22/2013 06:21 PM, Nisar Ahmed wrote:
> Coming back to this issue, I have tested 2 scenarios
>
> a. I am simply setting pts to the timestamp received from encoder and
> dts to 0 but the resulting file is showing very large frame rate value.
>
> b. Setting pts to the timestamp received from encoder but this time
> setting dts with an increment of 1001 starting from -1001 (2002/-1001,
> 0/0, 1001/1001, 5005/2002, 3003/3003). The framerate is correct this
> time but the quicktime player displays a white frame at the start of
> the movie. I investigated this issue further by using libav demuxer
> sample and the pts/dts have been changed from what I set and
> start_time is also not 0. VLC plays fine but I also want Quicktime
> player to play the file normally as well.
>
> The movies created by the software which ships with he encoder shows
> correct frame rate and start time of 0, also the pts/dts investigated
> with the demuxing sample are in the same sequence as described in the
> second scenario.
>
> I really need to understand why my pts/dts are changed by the muxer to
> different values, may be it is trying to guess correct values based on
> the inputs I provided. Which inputs matter in this kind of scenarios
> and how they relate to each other?
>
> Thanks in advance
> Nisar
>
>
> On Sat, Aug 17, 2013 at 11:05 AM, Nisar Ahmed <nisar.med at gmail.com
> <mailto:nisar.med at gmail.com>> wrote:
>
> Thanks for your answer, when I set timestamp of decoder, the frame
> rate jumps up to a very large value
>
> pts/dts = (2002/0, 0/0, 1001/0, 5005/0, 3003/0...)
>
> am I setting the time_base to correct values 1001/30000 and
> pkt.duration = 1001? the timescale of the timestamp is also 30000
>
>
> On Sat, Aug 17, 2013 at 2:10 AM, Anshul maheshwari
> <er.anshul.maheshwari at gmail.com
> <mailto:er.anshul.maheshwari at gmail.com>> wrote:
>
> -
>
>
> On Aug 16, 2013 7:30 PM, "Nisar Ahmed" <nisar.med at gmail.com
> <mailto:nisar.med at gmail.com>> wrote:
> >
> > I am muxing h264/aac stream to mp4 coming from an encoder
> and having issues syncing audio and video streams.
> >
> > I suspect that the problem is with PTS and DTS but I am not
> sure what values should i set to properly sync both streams.
> >
> > I have set the time_base of video stream's AVCodecContext to
> 1001/30000 and each AVPacket gets a duration of 1001
> >
> > I am recieving time stamps from the encoder with each NAL
> packet which is set as PTS of the AVPacket. the sequence of
> PTS/DTS I set is (2002/0, 0/1001, 1001/2002, 5005/3003,
> 3003/4004...)
> >
> > Audio stream AVCodecContext sample rate is 48000 and PTS/DTS
> are simply multiples of 1024 starting from 0 and AVPacket
> duration is 1024.
> >
> > The output movie 's frame rate is 20fps when it should be
> 29.97fps as it is ntsc.
> >
> > Please tell me how to debug this problem and what are the
> guidelines of creating pts/dts for both video and audio in my case
> >
> > _______________________________________________
> > Libav-user mailing list
> > Libav-user at ffmpeg.org <mailto:Libav-user at ffmpeg.org>
> > http://ffmpeg.org/mailman/listinfo/libav-user
> >
> Hi naseer
>
> If you are getting timestamp with encoder, i would suggest
> "dont try to manuplate that" you may set that same timestamp
> as presentation timestamp (pts) if your video is realtime then
> try not to generate B type video frame hence no need to set dts.
> Whatever clock is set by encoder try generating the pts of
> audio according to that.
>
> @enjoy
> Anshul
>
>
> _______________________________________________
> Libav-user mailing list
> Libav-user at ffmpeg.org <mailto:Libav-user at ffmpeg.org>
> http://ffmpeg.org/mailman/listinfo/libav-user
>
>
>
>
>
> _______________________________________________
> Libav-user mailing list
> Libav-user at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/libav-user
please don't top post over here,
>I really need to understand why my pts/dts are changed by the muxer to
different values, may be it is trying to guess correct values based on
the inputs I provided. Which
>inputs matter in this kind of scenarios and how they relate to each other?
Muxer does not change the pts or dts if you have provided by your own,
may be you are unable to set pts, dts in your muxer.
please refer below code, it is not kind of tutorial but extract needed
part.
Note: your a part wont work because your h264 elementary stream contain
b frame, where dts is important part.
and your b part wont work, there might be simple reason for
that it does not calculate start time properly, so when it is 0 as with
your software
then it work.
Below is very badly written code, but its work :)
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <math.h>
#include <libavutil/mathematics.h>
#include <libavformat/avformat.h>
#include <libxml/tree.h>
#define MAX_STREAM 2
#define BUFF_SIZE ( 1920 *1080 )
#define STREAM_DURATION ( 200.0 )
#define STREAM_FRAME_RATE ( 25 )
#define AUDIO_BITRATE ( 64000 )
#define AUDIO_CHANNEL ( 2 )
#define FRAME_SIZE ( 1024 )
#define VIDEO_BITRATE ( 400000 )
#define VIDEO_WIDTH ( 1920 )
#define VIDEO_HEIGHT ( 1080 )
#define VIDEO_GOP ( 12 )
#define OUTPUT_CONTEXT_FORMAT "mpegts"
#define HLS_NUM_CALLBACKS 2
typedef int (*callbackFn)(void *);
struct mux_cfg
{
int no_of_stream;
enum AVCodecID codec_id[MAX_STREAM];
int audio_sample_rate;
int video_frame_rate;
AVStream *st[MAX_STREAM];
AVFormatContext *oc;
unsigned char *obuffer;
//user specific buffer
unsigned char uobuffer[BUFF_SIZE];
int uosize;
int64_t uopts;
int pts_generator;
};
enum MediaType {
VIDEO,
AUDIO
};
struct mediaBuffer{
int size;
unsigned char *data ;
enum MediaType mtype;
int64_t pts;
int64_t dts;
};
int init(char *xml);
int start(void);
int get_buffer(void *pOutputData);
int put_buffer(void *pInputData);
int stop(void);
int deinit(void);
struct mux_cfg glb_cfg;
static int mux_write(void* opaque, char*buf, int size);
static int parse_xml(char*xml,struct mux_cfg *cfg);
static int update_input_param(xmlNodePtr ParentNode, struct mux_cfg
*cfg,xmlDocPtr doc);
static int Muxer_init(struct mux_cfg *loc_cfg, char *xml);
static int Muxer_get_buffer(struct mux_cfg *loc_cfg,void *pOutputData);
static int Muxer_put_buffer(struct mux_cfg *loc_cfg,void *pInputData);
static int Muxer_dinit(struct mux_cfg *loc_cfg);
int init(char *xml)
{
return Muxer_init(&glb_cfg,xml);
}
int start(void)
{
return 0;
}
int getCallbackFns(int no, callbackFn* ptr)
{
int retValue = -1;
if (no != HLS_NUM_CALLBACKS)
{
printf("Error: Invalid input callback number: %d\n", no);
return retValue;
}
if (ptr)
{
ptr[0] = put_buffer;
ptr[1] = get_buffer;
retValue = 0;
}
else
{
printf("Error: Invalid pointer passed for callback!\n");
return retValue;
}
return retValue;
}
int get_buffer(void *pOutputData)
{
return Muxer_get_buffer(&glb_cfg,pOutputData);
}
int put_buffer(void *pInputData)
{
return Muxer_put_buffer(&glb_cfg,pInputData);
}
int stop(void)
{
return 0;
}
int deinit(void)
{
return Muxer_dinit(&glb_cfg);
}
static AVStream *add_stream(AVFormatContext *oc, AVCodec **codec,
struct mux_cfg *loc_cfg)
{
AVCodecContext *c;
AVStream *st;
st = avformat_new_stream(oc, *codec);
if (!st)
{
fprintf(stderr, "Could not allocate stream\n");
exit(1);
}
st->id = oc->nb_streams - 1;
c = st->codec;
switch ((*codec)->type)
{
case AVMEDIA_TYPE_AUDIO:
c->codec_id = loc_cfg->codec_id[AUDIO];
c->sample_fmt = AV_SAMPLE_FMT_S16;
c->bit_rate = AUDIO_BITRATE;
c->sample_rate = loc_cfg->audio_sample_rate;
c->channels = AUDIO_CHANNEL;
c->frame_size = FRAME_SIZE;
break;
case AVMEDIA_TYPE_VIDEO:
c->codec_id = loc_cfg->codec_id[VIDEO];
c->bit_rate = VIDEO_BITRATE;
c->width = VIDEO_WIDTH;
c->height = VIDEO_HEIGHT;
c->time_base.den = loc_cfg->video_frame_rate;
c->time_base.num = 1;
c->gop_size = VIDEO_GOP; /* emit one intra frame every
twelve frames at most */
c->pix_fmt = AV_PIX_FMT_YUV420P;
break;
default:
break;
}
return st;
}
static int Muxer_init(struct mux_cfg *loc_cfg, char *xml)
{
int i = 0;
int ret = 0;
AVCodec *codec[MAX_STREAM];
memset(loc_cfg, 0, sizeof(*loc_cfg));
/* Initialize libavcodec, and register all codecs and formats. */
av_register_all();
parse_xml(xml,loc_cfg);
/* allocate the output media context */
avformat_alloc_output_context2(&loc_cfg->oc,
NULL,OUTPUT_CONTEXT_FORMAT, NULL );
if (!loc_cfg->oc)
{
fprintf(stderr,"unable to allocate allocate context\n");
}
loc_cfg->obuffer = malloc(BUFF_SIZE);
loc_cfg->oc->pb = avio_alloc_context(loc_cfg->obuffer, BUFF_SIZE,
AVIO_FLAG_WRITE,
(void*) loc_cfg,NULL, (void*)
mux_write, NULL );
for (i = 0; i < loc_cfg->no_of_stream; i++)
{
codec[i] = avcodec_find_encoder(loc_cfg->codec_id[i]);
if (!(codec[i]))
{
fprintf(stderr, "Could not find encoder for '%s'\n",
avcodec_get_name(loc_cfg->codec_id[i]));
return -1;
}
loc_cfg->st[i] = add_stream(loc_cfg->oc, &codec[i], loc_cfg);
}
ret = avformat_write_header(loc_cfg->oc, NULL );
if (ret < 0)
{
fprintf(stderr, "Error occurred when opening output file: %s\n",
av_err2str(ret));
return 1;
}
return 0;
}
static int Muxer_get_buffer(struct mux_cfg *loc_cfg,void *pOutputData)
{
struct mediaBuffer *buf = (void*) pOutputData;
if(!loc_cfg)
{
errno = EINVAL;
return -1;
}
if(loc_cfg->pts_generator)
{
buf->pts = loc_cfg->uopts;
buf->dts = 0;
}
else
{
buf->pts = 0;
buf->dts = 0;
}
if (loc_cfg->uosize == 0)
{
return 0;
}
buf->data = loc_cfg->uobuffer;
buf->size = loc_cfg->uosize;
loc_cfg->uosize = 0;
return buf->size;
}
static int Muxer_put_buffer(struct mux_cfg *loc_cfg,void *pInputData)
{
struct mediaBuffer *buf = (void*) pInputData;
int ret;
AVPacket pkt;
av_init_packet(&pkt);
if(!loc_cfg)
return -1;
if(!loc_cfg->pts_generator)
{
pkt.pts = buf->pts;
pkt.dts = buf->dts;
}
else
{
// pkt.pts = 0;
// pkt.dts = 0;
}
pkt.data = buf->data;
pkt.size = buf->size;
pkt.stream_index = buf->mtype;
ret = av_interleaved_write_frame(loc_cfg->oc, &pkt);
if (ret < 0)
{
fprintf(stderr, "av_interleaved_write_frame : %s\n",
av_err2str(ret));
}
loc_cfg->uopts = pkt.pts;
return ret;
}
static int mux_write(void* opaque, char*buf, int size)
{
struct mux_cfg *loc_cfg = opaque;
memcpy(loc_cfg->uobuffer + loc_cfg->uosize, buf, size);
loc_cfg->uosize += size;
return 0;
}
int Muxer_dinit(struct mux_cfg *loc_cfg)
{
int i = 0;
av_write_trailer(loc_cfg->oc);
for(i = 0;i< loc_cfg->no_of_stream ;i++)
{
avcodec_close(loc_cfg->st[i]->codec);
}
av_free(loc_cfg->oc->pb);
/* free the stream */
avformat_free_context(loc_cfg->oc);
return 0;
}
static int parse_xml(char*xml,struct mux_cfg *cfg)
{
int ret = 0;
xmlDocPtr doc;
xmlNodePtr nl1;
if(!xml || !cfg)
{
ret= -1;
return ret;
}
doc = xmlParseFile(xml);
if (doc == NULL)
{
ret= -1;
return ret;
}
for (nl1 = doc->children; nl1 != NULL; nl1 = nl1->next)
{
ret = xmlStrcmp(nl1->name, (const xmlChar *) "INPUT");
if (ret == 0)
{
update_input_param(nl1,cfg,doc);
}
}
xmlFreeDoc(doc);
return ret;
}
static int update_input_param(xmlNodePtr ParentNode, struct mux_cfg
*cfg,xmlDocPtr doc)
{
xmlNodePtr nl = ParentNode->children;
char *pXmlGetString = NULL;
int ret = 0;
for (; nl != NULL; nl = nl->next)
{
ret = xmlStrcmp(nl->name, (const xmlChar *) "no_of_elem_stream");
if (ret == 0)
{
pXmlGetString = (char *) xmlNodeListGetString(doc,
nl->xmlChildrenNode, 1);
sscanf(pXmlGetString, "%d", &cfg->no_of_stream);
xmlFree(pXmlGetString);
}
ret = xmlStrcmp(nl->name, (const xmlChar *) "audio_codec");
if (ret == 0)
{
pXmlGetString = (char *) xmlNodeListGetString(doc,
nl->xmlChildrenNode, 1);
if(!strcmp(pXmlGetString,"aac"))
{
cfg->codec_id[AUDIO] = AV_CODEC_ID_AAC;
}
xmlFree(pXmlGetString);
}
ret = xmlStrcmp(nl->name, (const xmlChar *) "video_codec");
if (ret == 0)
{
pXmlGetString = (char *) xmlNodeListGetString(doc,
nl->xmlChildrenNode, 1);
if(!strcmp(pXmlGetString,"h264"))
{
cfg->codec_id[VIDEO] = AV_CODEC_ID_H264;
}
xmlFree(pXmlGetString);
}
ret = xmlStrcmp(nl->name, (const xmlChar *) "audio_sample_rate");
if (ret == 0)
{
pXmlGetString = (char *) xmlNodeListGetString(doc,
nl->xmlChildrenNode, 1);
sscanf(pXmlGetString, "%d", &cfg->audio_sample_rate);
xmlFree(pXmlGetString);
}
ret = xmlStrcmp(nl->name, (const xmlChar *) "video_frame_rate");
if (ret == 0)
{
pXmlGetString = (char *) xmlNodeListGetString(doc,
nl->xmlChildrenNode, 1);
sscanf(pXmlGetString, "%d", &cfg->video_frame_rate);
xmlFree(pXmlGetString);
}
ret = xmlStrcmp(nl->name, (const xmlChar *) "pts_generator");
if (ret == 0)
{
pXmlGetString = (char *) xmlNodeListGetString(doc,
nl->xmlChildrenNode, 1);
if(!strcmp(pXmlGetString,"TRUE") ||
!strcmp(pXmlGetString,"true"))
{
cfg->pts_generator = 1;
}
xmlFree(pXmlGetString);
}
}
return ret;
}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ffmpeg.org/pipermail/libav-user/attachments/20130823/5995dab3/attachment.html>
More information about the Libav-user
mailing list