[Libav-user] swr_convert questions

[Ryan Routon]

I was actually able to figure this one out.  swr_convert was not converting
the entire frame of data so a subsequent call was needed using null
parameters to flush out the rest of the samples.  This was done because the
output buffer was not big enough apparently.  Also the source and
destination frame size were not the same.  The source was 576, while the
destination was 1024, so I created a fifo buffer in order to store the
converted samples until there was enough to enter into a packet.

Does anyone know how the memory management works under the hood when using
av_interleaved_write_frame?  We were running tests and were wondering if
you were to interleave the packets yourself if buffering would be
mitigated.   It sounds like everything is buffered until the end, but is it
smart enough to know that if an audio packet exists that there is no need
to buffer anything previous to that?  I found that at times I would be able
to interleave by hand, but at other times the audio samples would not fit
and needed to be placed at the end (typically during the finish flush).


On Sun, Jul 13, 2014 at 2:58 AM, Ryan Routon <ryanrouton at gmail.com> wrote:

> Hello All,
>
> First of all this is my first post to the mailing list so hello all =)
>
> Recently I have been working with fmpeg in order to render an mp4 using
> still images via rgba values and a user selected audio file.  I have set up
> the video portion which encodes perfectly. I have used the
> avcodec_decode_audio4
> <https://www.ffmpeg.org/doxygen/trunk/group__lavc__decoding.html#ga834bb1b062fbcc2de4cf7fb93f154a3e> function
> to pull frames from a user selected audio file.  Once I get the frame the
> samples are then stored in AV_SAMPLE_FMT_FLTP, so at this point I either
> have to manually cast the values as a short or use the sampler
> functionality.  I did the first as a proof of concept but in order to make
> the conversion as flexible as possible I opted for the sampler route.  I
> set my sampler as follows:
>
>         /* set options */
>
>         av_opt_set_int       (swr_ctx, "in_channel_count",   frameIn->
> channels,                 0);
>
>         av_opt_set_int       (swr_ctx, "in_sample_rate",     frameIn->
> sample_rate,              0);
>
>         av_opt_set_sample_fmt(swr_ctx, "in_sample_fmt",      (
> AVSampleFormat)frameIn->format,   0);
>
>         av_opt_set_int       (swr_ctx, "out_channel_count",  codecOut->
> channels,                0);
>
>         av_opt_set_int       (swr_ctx, "out_sample_rate",    codecOut->
> sample_rate,             0);
>
>         av_opt_set_sample_fmt(swr_ctx, "out_sample_fmt",     codecOut->
> sample_fmt,              0);
>
>
> my target audio profile is as follows:
>
>
>             (*codec)->sample_fmts[0] : AV_SAMPLE_FMT_FLTP;
>
>             c->bit_rate    = 64000;
>
>             c->sample_rate = 22050;
>
>             c->channels    = 2;
>
>             c->channel_layout = AV_CH_LAYOUT_STEREO;
>
>
> I has a hell of a time debugging random black box crashes in the
> swr_convert file and it turned out to be that I was passing the uint8 ** by
> value instead of reference so when there was more than one more channel it
> would crash, but changing my parameter from ->data to &->data[0] solved
> that.  So just in case that might help anyone having that particular
> problem...
>
>
>
> ..Anyways, Now the sampler works great for a 1 channel file at the same
> sample rate.  When I change the input file to a 2 channel or change the
> sample rate i still hear the audio but it is very disjointed as if there
> are gaps in the samples during the encoding process.
>
>
> Here is the function, where the get_audio_frame() function uses the decode
> 4 function:
>
>
> static int write_audio_frame_flip(AVFormatContext *oc, OutputStream *ost)
>
> {
>
>     AVCodecContext *c;
>
>     AVPacket pkt = { 0 }; // data and size must be 0;
>
>     AVFrame *frame;
>
>     int ret;
>
>     int got_packet;
>
>     int dst_nb_samples;
>
>
>
>     av_init_packet(&pkt);
>
>     c = ost->st->codec;
>
>
>
>     //Get decoded audio frame
>
>     frame = get_audio_frame_flip(ost);
>
>
>
>     if (frame)
>
>     {
>
>         SetSampler(c, frame);
>
>
>
>         // convert samples from native format to destination codec
> format, using the resampler
>
>         if (swr_ctx)
>
>         {
>
>             if (ret < 0) {
>
>                 fprintf(stderr, "Could not allocate destination samples\n"
> );
>
>                 return 0;
>
>             }
>
>
>
>             LOGD("Scaling frame->sample_rate=%d frame->nb_samples=%d
> c->sample_rate=%d", frame->sample_rate, frame->nb_samples, c->sample_rate
> );
>
>
>
>             // compute destination number of samples
>
>             dst_nb_samples = av_rescale_rnd(swr_get_delay(swr_ctx, frame->
> sample_rate) + frame->nb_samples,
>
>                                             c->sample_rate, frame->
> sample_rate, AV_ROUND_UP);
>
>
>
>             //convert the samples
>
>             ret = swr_convert(swr_ctx, &ost->tmp_frame->data[0],
> dst_nb_samples, (const uint8_t **)&frame->data[0], dst_nb_samples);
>
>
>
>             if (ret < 0) {
>
>                 LOGD("Error while converting\n");
>
>                 return 0;
>
>             }
>
>
>
>             frame = ost->tmp_frame;
>
>
>
>         } else {
>
>             dst_nb_samples = frame->nb_samples;
>
>         }
>
>
>
>         frame->pts = av_rescale_q(samples_count, (AVRational){1, c->
> sample_rate}, c->time_base);
>
>         samples_count += dst_nb_samples;
>
>     }
>
>
>
>     ret = avcodec_encode_audio2(c, &pkt, frame, &got_packet);
>
>     if (ret < 0) {
>
>         fprintf(stderr, "Error encoding audio frame: %s\n", av_err2str
> (ret));
>
>         return 0;
>
>     }
>
>
>
>     if (got_packet) {
>
>         ret = write_frame(oc, &c->time_base, ost->st, &pkt);
>
>         if (ret < 0) {
>
>             fprintf(stderr, "Error while writing audio frame: %s\n",
>
>                     av_err2str(ret));
>
>             return 0;
>
>         }
>
>     }
>
>
>
>     return (frame || got_packet) ? 0 : 1;
>
> }
>
>
> Thank you for any insight into what I might be doing wrong, this is all
> very new to me and I am sure that I am misunderstanding some call or
> concept.
>
> Ryan
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://ffmpeg.org/pipermail/libav-user/attachments/20140716/8b9a3dcd/attachment.html>