Gapless MP4/AAC from FFmpeg: padding derived as 0 from the edit list, so the rest of the last frame plays at each join #3489

Version

Media3 pre-release (alpha, beta or RC not in this list)

More version details

Media3 1.12.0-alpha01. Also reproduced with 1.11.1.

Devices that reproduce the issue

• Android emulators, API 30, 36 and 37 (c2.android.aac.decoder)
• Samsung Galaxy S24 Ultra running Android 16 (c2.android.aac.decoder)

Devices that do not reproduce the issue

No response

Reproducible in the demo app?

Not tested

Reproduction steps

FFmpeg's MP4 muxer (FFmpeg 9.0.2, native aac encoder, -f ipod or -f mp4) writes the gapless information of an AAC track as an edit list plus a shortened last sample:

• elst: one entry, media_time = 1024 (the priming), segment_duration = the true length N;
• stts: every frame 1024, except the last one, which gets only the samples it really holds (N mod 1024), so that the sum of stts is exactly 1024 + N;
• no iTunSMPB.

BoxParser derives the encoder padding from the edit list as max(0, sum of stts - end of the edit), which is 0 for these files. The decoder still outputs the whole last frame (1024 samples), so 1024 - (N mod 1024) extra samples play after the end of every track.

1. Cut one continuous signal into parts whose lengths are not a multiple of 1024 and encode each with ffmpeg -i part.wav -c:a aac -b:a 128k -f ipod part.m4a.
2. Play the parts as a playlist with ExoPlayer and copy the output with a TeeAudioProcessor (it sits after the trimming).
3. Align the captured PCM with the source signal.

Expected result

Each track plays as exactly N samples, as FFmpeg itself decodes these files. For AAC with its fixed frame size the padding could be frames * 1024 - media_time - segment_duration, i.e. derived from what the decoder will output rather than from the stts sum.

The same encoder output remuxed without an edit list (-use_editlist 0) and with an iTunSMPB atom already plays to the sample on the same devices.

Actual result

Track 1 has N = 132357 samples (last stts delta 261), so the join after it is late by 1024 - 261 = 763 samples. Over the five tracks the joins are late by 763, 1251, 2164 and 3085 samples in total, and the album plays 532480 samples instead of 529200: up to 23 ms of padding at each join, on all the devices above.

Related: FFmpeg tickets https://trac.ffmpeg.org/ticket/8631 and https://trac.ffmpeg.org/ticket/10458 (missing end padding information in FFmpeg's MP4 output).

Media

Synthetic test files (5 tracks of one 12 s piece at 44.1 kHz), with the expected values and the generator:

• https://github.com/LordDetson/piplayer/tree/main/app/src/androidTest/assets/gapless/aac-ffmpeg (as FFmpeg writes it)
• https://github.com/LordDetson/piplayer/tree/main/app/src/androidTest/assets/gapless/aac (the same with iTunSMPB and no edit list, plays exactly)
• https://github.com/LordDetson/piplayer/blob/main/app/src/androidTest/assets/gapless/fixture.json
• https://github.com/LordDetson/piplayer/blob/main/scripts/generate_gapless_fixtures.py

Bug Report

• You will email the zip file produced by adb bugreport to android-media-github@google.com after filing this issue.