[Libav-user] Adding AMR frames to audio stream of video file
Paul B Mahol
onemda at gmail.com
Sat Jul 4 15:07:03 CEST 2015
Dana 4. 7. 2015. 13:06 osoba "Adev Dev" <androiddevmar11 at gmail.com>
napisala je:
>
> I found something that could be the reason of the problem. When I print
frame->nb_samples of AMR sound it is 320. During encoding warning is
generated "Trying to remove 704 more samples than there are in the queue".
So I assume that AAC encoder expects that frame has 1024 samples.
>
> Encoded AAC sound is about 4 times longer than it should be. When I
skipped 3 framers per 4 frames length is correct but sound is crappy still.
>
> AAC sound recorded with the same params (sampling rate: 16000, bitrate
23600) has 1024 samples in frame. Looks that AMR sound has about 4 times
more frames but each frame has about 4 times less samples(320).
>
> I assume that AAC encoder should handle that situation if it is
configured correctly. Is there anybody who knows what is wrong in codec
configuration??? Thank you for help.
>
Try feeding encoder with nulls.
Read documentation about codec_cap_delay.
>
>
>
>
> On 3 July 2015 at 13:03, Adev Dev <androiddevmar11 at gmail.com> wrote:
>>
>> Hi all!
>>
>> I prepared android project which makes encoding from AMR to AAC to
better show the problem. It takes AMR file from resources and reencode it
to "/storage/emulated/0/OutSound.aac".
>>
>> In MainActivity INPUT_AUDIO_NAME constant specifes input file. When set
to amr.m4a strange problem described in this thread occurs. After changing
to aac.m4a rencoding is working.
>>
>> I hope somebody is able to check this project and find the reason. I
used older FFMPEG library because I do not know why project is not linking
with latest version. Project is available under link:
>>
>>
https://drive.google.com/folderview?id=0B7SEEPspZQx1fnZCZGlIVF9fbGVEYmh3UGpnMmxPNVFseUlOZ2xLa010Nk1fZVJLSXlRc2c&usp=sharing
>>
>> Thank you for help.
>>
>>
>> On 2 July 2015 at 20:43, Adev Dev <androiddevmar11 at gmail.com> wrote:
>>>
>>> I have just updated FFMPEG to latest version 2.7.1. Unfortunately
problem still occurs. No progress at all.
>>> In console I see now warnings:
>>> "AVFrame.format is not set" and "AVFrame.width or height is not set".
>>>
>>> Any ideas what is wrong? Thanks for help!
>>>
>>>
>>>
>>> On 2 July 2015 at 12:55, Adev Dev <androiddevmar11 at gmail.com> wrote:
>>>>
>>>> Sure, please download from GD:
>>>>
>>>>
https://drive.google.com/folderview?id=0B7SEEPspZQx1fnZCZGlIVF9fbGVEYmh3UGpnMmxPNVFseUlOZ2xLa010Nk1fZVJLSXlRc2c&usp=sharing
>>>>
>>>> Please also check latest result on youtube:
https://www.youtube.com/watch?v=w0BAyE14xLw
>>>>
>>>> Thanks!
>>>>
>>>> On 2 July 2015 at 12:29, Paul B Mahol <onemda at gmail.com> wrote:
>>>>>
>>>>> On 7/2/15, Adev Dev <androiddevmar11 at gmail.com> wrote:
>>>>> > AMR file which is recorded in Android is correct. It can be played
both on
>>>>> > Android and on MAC. After decoding it, reencoding to AAC and adding
to
>>>>> > video file it is damaged. This video which I uploaded to YouTube
has sound
>>>>> > encoded in AAC (reencoded from AMR file).
>>>>> >
>>>>> > This is really strange because when I record audio file using AAC
codec I
>>>>> > am doing the same steps and it is ok. First decode AAC frame from
audio
>>>>> > file, then encode it and add to audio stream of video file. Maybe
some
>>>>> > other params in codec, or audio stream is not set, or set to wrong
value??
>>>>> >
>>>>>
>>>>> Could you upload and give a link to AMR file?
>>>>>
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>> > On 2 July 2015 at 12:12, Paul B Mahol <onemda at gmail.com> wrote:
>>>>> >
>>>>> >> On 7/2/15, adev dev <androiddevmar11 at gmail.com> wrote:
>>>>> >> > I was not clear enough. Sound is not bad quality. It is damaged.
Please
>>>>> >> > have a look on video file which I uploaded to YouTube:
>>>>> >> >
>>>>> >> > https://www.youtube.com/watch?v=1UcGQwvtr9s
>>>>> >> >
>>>>> >> > Video length is 4 seconds. Adding this sound makes it longer to
17
>>>>> >> seconds.
>>>>> >> > Looks like some parameters are wrong. Yes, AMR is recorded in
mono so
>>>>> >> > sample format converting is not needed. Thanks for help.
>>>>> >>
>>>>> >> And sound is damaged when listening straight from recording?
>>>>> >>
>>>>> >> >
>>>>> >> >
>>>>> >> > On 2 July 2015 at 10:14, Paul B Mahol <onemda at gmail.com> wrote:
>>>>> >> >
>>>>> >> >>
>>>>> >> >> Dana 2. 7. 2015. 07:58 osoba "adev dev" <
androiddevmar11 at gmail.com>
>>>>> >> >> napisala je:
>>>>> >> >>
>>>>> >> >> >
>>>>> >> >> > Hi,
>>>>> >> >> > thanks for answer.
>>>>> >> >> >
>>>>> >> >> > I cannot increase sound bitrate. I am using Android
MediaRecorder
>>>>> >> >> > and
>>>>> >> >> AMR codec for recording audio. AMR is needed because I am doing
Chrome
>>>>> >> >> version where AAC codec is not working. This AMR codec at least
in
>>>>> >> >> Android
>>>>> >> >> can only record with maximum bitrate 23600. It is not much but
sound
>>>>> >> >> should
>>>>> >> >> be good. Now my result is that sound is totally crappy. There
are
>>>>> >> strange
>>>>> >> >> pulses and if I record speech it is impossible to recognise
words.
>>>>> >> >> >
>>>>> >> >> > I wonder what else could be the problem. When I am adding AAC
files
>>>>> >> >> > to
>>>>> >> >> output video it is working correctly. Decoding AMR files and
encoding
>>>>> >> >> them
>>>>> >> >> again to AAC is not working. For the first glance it looks that
AMR
>>>>> >> >> decoding is not working correctly. Or the frame is in format
(not
>>>>> >> planar)
>>>>> >> >> and this makes problem. What do you think?
>>>>> >> >> >
>>>>> >> >> > This is how I read frames and decode them:
>>>>> >> >> >
>>>>> >> >> > static void encodeSoundNext(JNIEnv * env, jobject this) {
>>>>> >> >> >
>>>>> >> >> > if (input_context == NULL)
>>>>> >> >> > return;
>>>>> >> >> >
>>>>> >> >> > int samples_size;
>>>>> >> >> >
>>>>> >> >> > frameRead = 0;
>>>>> >> >> > char index = 0;
>>>>> >> >> >
>>>>> >> >> > AVFrame *decoded_frame = NULL;
>>>>> >> >> >
>>>>> >> >> > int input_audio_stream_index = get_stream_index(input_context,
>>>>> >> >> AVMEDIA_TYPE_AUDIO);
>>>>> >> >> >
>>>>> >> >> > while (frameRead >= 0) {
>>>>> >> >> >
>>>>> >> >> > AVPacket in_packet;
>>>>> >> >> >
>>>>> >> >> > index++;
>>>>> >> >> >
>>>>> >> >> > frameRead = av_read_frame(input_context, &in_packet);
>>>>> >> >> > if (frameRead < 0) {
>>>>> >> >> > trackCompressionFinished = 1;
>>>>> >> >> > avformat_close_input(&input_context);
>>>>> >> >> >
>>>>> >> >> > } else {
>>>>> >> >> >
>>>>> >> >> > if (decoded_frame == NULL) {
>>>>> >> >> > if (!(decoded_frame = avcodec_alloc_frame())) {
>>>>> >> >> > LOGE("out of memory");
>>>>> >> >> > exit(1);
>>>>> >> >> > }
>>>>> >> >> > } else {
>>>>> >> >> > avcodec_get_frame_defaults(decoded_frame);
>>>>> >> >> > }
>>>>> >> >> > int got_frame_ptr;
>>>>> >> >> > samplesBytes = avcodec_decode_audio4(in_audio_st->codec,
>>>>> >> >> > decoded_frame, &got_frame_ptr, &in_packet);
>>>>> >> >> > if (samplesBytes < 0) {
>>>>> >> >> > LOGE("Error occurred during decoding.");
>>>>> >> >> > exit(1);
>>>>> >> >> > break;
>>>>> >> >> > }
>>>>> >> >> >
>>>>> >> >> > write_audio_frame(oc, audio_st, decoded_frame);
>>>>> >> >> > av_free_packet(&in_packet);
>>>>> >> >> >
>>>>> >> >> > }
>>>>> >> >> > }
>>>>> >> >> >
>>>>> >> >> > if (decoded_frame != NULL) {
>>>>> >> >> > av_free(decoded_frame);
>>>>> >> >> > decoded_frame = NULL;
>>>>> >> >> > }
>>>>> >> >> > }
>>>>> >> >> >
>>>>> >> >> >
>>>>> >> >> > This is how I am encoding sound to AAC:
>>>>> >> >> >
>>>>> >> >> >
>>>>> >> >> > static void write_audio_frame(AVFormatContext *oc, AVStream
*st,
>>>>> >> >> > const AVFrame *frame_to_encode) {
>>>>> >> >> > AVCodecContext *c;
>>>>> >> >> > AVPacket pkt;
>>>>> >> >> > int got_packet_ptr = 0;
>>>>> >> >> >
>>>>> >> >> > av_init_packet(&pkt);
>>>>> >> >> > c = st->codec;
>>>>> >> >> > pkt.size = 0;
>>>>> >> >> > pkt.data = NULL;
>>>>> >> >> > int ret = avcodec_encode_audio2(c, &pkt, frame_to_encode,
>>>>> >> >> &got_packet_ptr);
>>>>> >> >> > if (ret < 0) {
>>>>> >> >> > exit(1);
>>>>> >> >> > }
>>>>> >> >> > if (got_packet_ptr == 1) {
>>>>> >> >> > if (c->coded_frame && c->coded_frame->pts != AV_NOPTS_VALUE) {
>>>>> >> >> > pkt.pts = av_rescale_q(c->coded_frame->pts, c->time_base,
>>>>> >> >> > st->time_base);
>>>>> >> >> > }
>>>>> >> >> > pkt.flags |= AV_PKT_FLAG_KEY;
>>>>> >> >> > pkt.stream_index = st->index;
>>>>> >> >> > // write the compressed frame in the media file
>>>>> >> >> > if (av_interleaved_write_frame(oc, &pkt) != 0) {
>>>>> >> >> > LOGE("Error while writing audio frame.");
>>>>> >> >> > exit(1);
>>>>> >> >> > }
>>>>> >> >> > }
>>>>> >> >> > av_free_packet(&pkt);
>>>>> >> >> > }
>>>>> >> >> >
>>>>> >> >> >
>>>>> >> >> > Audio stream is added to video file in this way:
>>>>> >> >> >
>>>>> >> >> >
>>>>> >> >> > static AVStream *add_audio_stream(AVFormatContext *oc, enum
>>>>> >> >> > AVCodecID
>>>>> >> >> codec_id) {
>>>>> >> >> >
>>>>> >> >> > AVCodecContext *c;
>>>>> >> >> > AVStream *st;
>>>>> >> >> >
>>>>> >> >> > st = avformat_new_stream(oc, NULL);
>>>>> >> >> >
>>>>> >> >> > c = st->codec;
>>>>> >> >> > if (!st) {
>>>>> >> >> > LOGE("Could not alloc stream.");
>>>>> >> >> > return NULL;
>>>>> >> >> > }
>>>>> >> >> >
>>>>> >> >> > // AAC is expirimental in FFMPEG2.1
>>>>> >> >> > c->strict_std_compliance = FF_COMPLIANCE_EXPERIMENTAL;
>>>>> >> >> >
>>>>> >> >> > c->codec_id = codec_id;
>>>>> >> >> > c->codec_type = AVMEDIA_TYPE_AUDIO;
>>>>> >> >> > c->bit_rate = 23600; // bitrate of the compressed sound (must
be
>>>>> >> higher
>>>>> >> >> for stereo)
>>>>> >> >> >
>>>>> >> >> > c->sample_rate = 16000;
>>>>> >> >> > c->channels = 1;
>>>>> >> >> > c->sample_fmt = AV_SAMPLE_FMT_FLT;
>>>>> >> >> >
>>>>> >> >> > if (oc->oformat->flags & AVFMT_GLOBALHEADER){
>>>>> >> >> > c->flags |= CODEC_FLAG_GLOBAL_HEADER;
>>>>> >> >> > }
>>>>> >> >> >
>>>>> >> >> > return st;
>>>>> >> >> > }
>>>>> >> >> >
>>>>> >> >> > What I noticed so far is that when I am decoding AAC files and
>>>>> >> encoding
>>>>> >> >> them again to audio stream in video files AAC frames has format
>>>>> >> >> AV_SAMPLE_FMT_FLTP. AMR frames are in AV_SAMPLE_FMT_FLT format.
Do you
>>>>> >> >> think I have to convert some how from AV_SAMPLE_FMT_FLT to
>>>>> >> >> AV_SAMPLE_FMT_FLTP?? Thanks for all hints.
>>>>> >> >> >
>>>>> >> >>
>>>>> >> >> For mono, single channel, conversion is not needed. If
recording is of
>>>>> >> >> bad
>>>>> >> >> quality encoding you can only use some other amr encoder.
>>>>> >> >>
>>>>> >> >> >
>>>>> >> >> >
>>>>> >> >> > On 1 July 2015 at 20:57, Talgorn Franc,ois-Xavier <
>>>>> >> >> fxtalgorn-at-yahoo.fr at ffmpeg.org> wrote:
>>>>> >> >> >>
>>>>> >> >> >> Hi,
>>>>> >> >> >>
>>>>> >> >> >> I don't know about AMR codec but bitrate definitely impacts
on
>>>>> >> >> >> final
>>>>> >> >> quality.
>>>>> >> >> >> Try to increase bitrate value: I had same poor quality
problems
>>>>> >> >> >> with
>>>>> >> >> MPEG4 encoding until I set the bitrate to width * height * 4.
>>>>> >> >> >> Keep in mind that poor quality might comes from a wide bunch
of
>>>>> >> >> parameters used to initialize the codec.
>>>>> >> >> >> As for example, this is how I initialize an MPEG4 codec
(A]), for
>>>>> >> >> clarity, in_ctx is initialized via the code in (B])
>>>>> >> >> >>
>>>>> >> >> >> Concerning the delay issue: I also faced such a problem. I
solved
>>>>> >> >> >> it
>>>>> >> >> using av_packet_rescale_ts() which relies on time_base, instead
of
>>>>> >> >> setting
>>>>> >> >> timestamps myself manually.
>>>>> >> >> >>
>>>>> >> >> >> I hope this comments will help put you on the road to
success :-)
>>>>> >> >> >>
>>>>> >> >> >> Good luck.
>>>>> >> >> >>
>>>>> >> >> >> A]
>>>>> >> >> >> //codec found, now we param it
>>>>> >> >> >> o_codec_ctx->codec_id=AV_CODEC_ID_MPEG4;
>>>>> >> >> >> o_codec_ctx->bit_rate=in_ctx->picture_width *
>>>>> >> >> in_ctx->picture_height * 4;
>>>>> >> >> >>
>>>>> >> >>
>>>>> >>
o_codec_ctx->width=in_ctx->format_ctx->streams[in_ctx->video_stream_idx]->codec->width;
>>>>> >> >> >>
>>>>> >> >>
>>>>> >>
o_codec_ctx->height=in_ctx->format_ctx->streams[in_ctx->video_stream_idx]->codec->height;
>>>>> >> >> >> o_codec_ctx->time_base =
>>>>> >> >>
in_ctx->format_ctx->streams[in_ctx->video_stream_idx]->codec->time_base;
>>>>> >> >> >> o_codec_ctx->ticks_per_frame =
>>>>> >> >>
>>>>> >>
in_ctx->format_ctx->streams[in_ctx->video_stream_idx]->codec->ticks_per_frame;
>>>>> >> >> >> o_codec_ctx->sample_aspect_ratio =
>>>>> >> >>
>>>>> >>
in_ctx->format_ctx->streams[in_ctx->video_stream_idx]->codec->sample_aspect_ratio;
>>>>> >> >> >>
>>>>> >> >>
>>>>> >>
o_codec_ctx->gop_size=in_ctx->format_ctx->streams[in_ctx->video_stream_idx]->codec->gop_size;
>>>>> >> >> >> o_codec_ctx->pix_fmt=AV_PIX_FMT_YUV420P;
>>>>> >> >> >>
>>>>> >> >> >>
>>>>> >> >> >>
>>>>> >> >> >> B]
>>>>> >> >> >> // register all formats and codecs
>>>>> >> >> >> av_register_all();
>>>>> >> >> >> avcodec_register_all();
>>>>> >> >> >>
>>>>> >> >> >> // open input file, and allocate format context
>>>>> >> >> >> if (avformat_open_input(&in_fmt_ctx, filename, NULL,
NULL) < 0)
>>>>> >> >> >> {
>>>>> >> >> >> fprintf(stderr, "Could not open source file %s\n",
>>>>> >> >> >> filename);
>>>>> >> >> >> exit(1);
>>>>> >> >> >> }
>>>>> >> >> >>
>>>>> >> >> >> // retrieve stream information
>>>>> >> >> >> if (avformat_find_stream_info(in_fmt_ctx, NULL) < 0)
>>>>> >> >> >> {
>>>>> >> >> >> fprintf(stderr, "Could not find stream
information\n");
>>>>> >> >> >> exit(1);
>>>>> >> >> >> }
>>>>> >> >> >>
>>>>> >> >> >> if (open_codec_context(&video_stream_idx, in_fmt_ctx,
>>>>> >> >> AVMEDIA_TYPE_VIDEO, filename) >= 0)
>>>>> >> >> >> {
>>>>> >> >> >> video_stream = in_fmt_ctx->streams[video_stream_idx];
>>>>> >> >> >> video_dec_ctx = video_stream->codec;
>>>>> >> >> >> }
>>>>> >> >> >>
>>>>> >> >> >> if (open_codec_context(&audio_stream_idx, in_fmt_ctx,
>>>>> >> >> AVMEDIA_TYPE_AUDIO, filename) >= 0) {
>>>>> >> >> >> audio_stream = in_fmt_ctx->streams[audio_stream_idx];
>>>>> >> >> >> audio_dec_ctx = audio_stream->codec;
>>>>> >> >> >> }
>>>>> >> >> >>
>>>>> >> >> >> if (!video_stream) {
>>>>> >> >> >> fprintf(stderr, "Could not find video stream in the
input,
>>>>> >> >> aborting\n");
>>>>> >> >> >> avformat_close_input(&in_fmt_ctx);
>>>>> >> >> >> exit(0);
>>>>> >> >> >> }
>>>>> >> >> >>
>>>>> >> >> >> in_video_ctx->format_ctx=in_fmt_ctx;
>>>>> >> >> >> in_video_ctx->filename=filename;
>>>>> >> >> >> in_video_ctx->codec_name=(char *)
>>>>> >> >> in_fmt_ctx->streams[video_stream_idx]->codec->codec->long_name;
>>>>> >> >> >> in_video_ctx->video_stream_idx=video_stream_idx;
>>>>> >> >> >> in_video_ctx->audio_stream_idx=audio_stream_idx;
>>>>> >> >> >>
>>>>> >> >>
>>>>> >>
in_video_ctx->picture_width=in_fmt_ctx->streams[video_stream_idx]->codec->width;
>>>>> >> >> >>
>>>>> >> >>
>>>>> >>
in_video_ctx->picture_height=in_fmt_ctx->streams[video_stream_idx]->codec->height;
>>>>> >> >> >> in_video_ctx->nb_streams=in_fmt_ctx->nb_streams;
>>>>> >> >> >>
>>>>> >> >> >>
>>>>> >> >> >>
>>>>> >> >> >>
>>>>> >> >> >> Le 1 juil. 2015 `a 10:40, adev dev <androiddevmar11 at gmail.com>
a
>>>>> >> ecrit
>>>>> >> >> >> :
>>>>> >> >> >>
>>>>> >> >> >>> I am compressing movies from bitmaps and audio files. With
AAC
>>>>> >> >> >>> files
>>>>> >> >> it is working correctly. But when I have AMR_WB files sound is
>>>>> >> corrupted.
>>>>> >> >> I
>>>>> >> >> can recognise correct words in video file but it is delayed and
with
>>>>> >> very
>>>>> >> >> bad quality.
>>>>> >> >> >>>
>>>>> >> >> >>> My AMR files are recorded with parameters:
>>>>> >> >> >>> - sampling rate: 16000,
>>>>> >> >> >>> - bitrate: 23000.
>>>>> >> >> >>>
>>>>> >> >> >>> I am setting this parameters in audio stream which is added
to
>>>>> >> video.
>>>>> >> >> Sample format is set to AV_SAMPLE_FMT_FLT. When using other
formats
>>>>> >> >> app
>>>>> >> >> crashes with "Unsupported sample format".
>>>>> >> >> >>>
>>>>> >> >> >>> What needs to be done to correctly add AMR stream to video
file?
>>>>> >> >> >>> Do
>>>>> >> I
>>>>> >> >> have to reencode it to AAC and add as AAC audio stream?? Thank
you for
>>>>> >> >> all
>>>>> >> >> hints.
>>>>> >> >> >>> _______________________________________________
>>>>> >> >> >>> Libav-user mailing list
>>>>> >> >> >>> Libav-user at ffmpeg.org
>>>>> >> >> >>> http://ffmpeg.org/mailman/listinfo/libav-user
>>>>> >> >> >>
>>>>> >> >> >>
>>>>> >> >> >>
>>>>> >> >> >> _______________________________________________
>>>>> >> >> >> Libav-user mailing list
>>>>> >> >> >> Libav-user at ffmpeg.org
>>>>> >> >> >> http://ffmpeg.org/mailman/listinfo/libav-user
>>>>> >> >> >>
>>>>> >> >> >
>>>>> >> >> >
>>>>> >> >> > _______________________________________________
>>>>> >> >> > Libav-user mailing list
>>>>> >> >> > Libav-user at ffmpeg.org
>>>>> >> >> > http://ffmpeg.org/mailman/listinfo/libav-user
>>>>> >> >> >
>>>>> >> >>
>>>>> >> >> _______________________________________________
>>>>> >> >> Libav-user mailing list
>>>>> >> >> Libav-user at ffmpeg.org
>>>>> >> >> http://ffmpeg.org/mailman/listinfo/libav-user
>>>>> >> >>
>>>>> >> >>
>>>>> >> >
>>>>> >> _______________________________________________
>>>>> >> Libav-user mailing list
>>>>> >> Libav-user at ffmpeg.org
>>>>> >> http://ffmpeg.org/mailman/listinfo/libav-user
>>>>> >>
>>>>> >
>>>>> _______________________________________________
>>>>> Libav-user mailing list
>>>>> Libav-user at ffmpeg.org
>>>>> http://ffmpeg.org/mailman/listinfo/libav-user
>>>>
>>>>
>>>
>>
>
>
> _______________________________________________
> Libav-user mailing list
> Libav-user at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/libav-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://ffmpeg.org/pipermail/libav-user/attachments/20150704/8b2582cf/attachment.html>
More information about the Libav-user
mailing list