FlacExtractor can detect spurious/garbage FLAC frame headers within encoded FLAC data

Version

Media3 1.1.0

More version details

No response

Devices that reproduce the issue

All

Devices that do not reproduce the issue

No response

Reproducible in the demo app?

Not tested

Reproduction steps

Play the media with ffmpeg extension enabled for flac.

Expected result

Either the media have 2 small glitches like when playing with VLC, or the media stops at the invalid data like with the Pixel MediaCodec decoder, or the media stays in buffering state.

Current behavior can't be properly detected by code and the user only see constant flash between buffering and playing and do not understand if there's an issue on the media or the player or network issues.

Actual result

The media get stuck and ExoPlayer keeps switching between playing and buffering.

2023-08-02 15:21:17.807 T:ExoPlayer 
state [eventTime=162.93, mediaPos=121.83, window=0, period=0, BUFFERING]
2023-08-02 15:21:17.807 T:ExoPlayer 
isPlaying [eventTime=162.93, mediaPos=121.83, window=0, period=0, false]
2023-08-02 15:21:17.810 T:ExoPlayer 
state [eventTime=162.94, mediaPos=121.83, window=0, period=0, READY]
2023-08-02 15:21:17.811 T:ExoPlayer 
isPlaying [eventTime=162.94, mediaPos=121.83, window=0, period=0, true]
2023-08-02 15:21:17.968 T:ExoPlayer 
state [eventTime=163.10, mediaPos=121.83, window=0, period=0, BUFFERING]
2023-08-02 15:21:17.975 T:ExoPlayer 
isPlaying [eventTime=163.10, mediaPos=121.83, window=0, period=0, false]
2023-08-02 15:21:17.980 T:ExoPlayer 
state [eventTime=163.10, mediaPos=121.83, window=0, period=0, READY]
2023-08-02 15:21:17.981 T:ExoPlayer 
isPlaying [eventTime=163.10, mediaPos=121.83, window=0, period=0, true]

Above logs is repeated indefinitively.

Media

Sent by mail.

Bug Report

You will email the zip file produced by adb bugreport to dev.exoplayer@gmail.com after filing this issue.

Bump, still happens on last version and still very disturbing for users.

@Tolriq

This is still happening.

While @Tolriq suggested playing back the files with "2 small glitches like when playing with VLC," he said on his support forum that he's ideologically opposed to playing back corrupted files because they could "damage people's ears" somehow. It still seems like playing the file (as all other media players do) would be ideal, but there should at least be some kind of error-out instead of the rapid play/pause events.

It's actually Google ideology here ;) But thank again for the nice bad rating on Play Store for an issue you can fix in 10 seconds with flac command line that you are well aware of ;)

@rohitjoins

Ok, it sounds like this library needs to expose some kind of error when encountering a corrupted file as opposited to stalling with the repeated play/pause events, then.

@rohitjoins Were you able to recreate the issue here? Do you need any more samples of affected files?

or the media stops at the invalid data like with the Pixel MediaCodec decoder

This sounds similar to #2587.

What behaviour do you see when playing the file in this issue with the FLAC extension?

@icbaker

@icbaker The problem is really about ffmpeg decoder here and the fact that it does not error.

In this case those files seems to always have a checksum error, so it's actually OK that it errors. Errors are something that app can detect and properly report to users.

Here ExoPlayer just constantly switch between buffering and ready and never moves or error, so impossible to properly detect and warn the user (And killing battery if they don't notice).

So failing would be good and it's what's wanted as I perfectly understand not playing invalid files. Just need a way to detect.

Just need a way to detect.

Does ffmpeg throw an error that we could detect if we were looking for it? If not, then I think (just as you point out this isn't your issue to solve in the thread linked above), this is likely an issue to report to ffmpeg. If there is an ffmpeg error thrown that we could detect, please let us know the details of that.

In the general case, ExoPlayer cannot promise to detect all possible forms of invalid file and always throw an error. The expected behaviour of playing an invalid file with ExoPlayer is undefined.

since there's never a valid outputBuffer

I'm not sure this is the case with the 2023-08-02 test file. I see these lines hit after the ffmpeg error logging (which only happens on a non-skipped output buffer afaict - and is what triggers the audio sink discontinuity):

media/libraries/decoder/src/main/java/androidx/media3/decoder/SimpleDecoder.java

Lines 295 to 299 in ecb80bb

    
           } else { 
        
             outputBuffer.skippedOutputBufferCount = skippedOutputBufferCount; 
        
             skippedOutputBufferCount = 0; 
        
             queuedOutputBuffers.addLast(outputBuffer); 
        
           }

We are clearly approaching this issue from slightly different directions :)

I have noticed that with the 2023-08-02 file there is a very out of order timestamp in the sample queue around the time of the issue, with timeUs=208143673 (3m28s) which is beyond the end of the 3m9s duration ExoPlayer thinks this file is.

Logging in FlacExtractor around the time this timestamp is queued it seems something is going wrong with finding the 'first sample number' of the current frame at this point:

Queueing sample with timeUs=208143673. currentFrameFirstSampleNumber=9179136
Prev timestamps: [116924081, 117028571, 117133061, 117237551, 117342040, 117446530, 117551020, 117655510, 117760000, 117864489]
Prev sample nums: [5156352, 5160960, 5165568, 5170176, 5174784, 5179392, 5184000, 5188608, 5193216, 5197824]

Looking in the file with a hex editor, the data that ExoPlayer's FlacExtractor uses to generate the frame with this 9179136 number starts at decimal offset 23997953 with these bytes:

FF F8 59 9C DF 88

Decoding this as a FLAC frame header/start, the frame number is encoded in DF 88, which decodes to 1992 (via UTF-8 encoding as described in RFC 9639 section 9.1.5). Multiplying by 4608 (the fixed block size) gives the problematic/unexpected sample number of 9179136.

The FLAC frame that precedes it starts at decimal offset 23997582 with these bytes (found by searching back through the file for 0xFFF8):

FF F8 59 AC D1 A8 A9 54 0C D2

The D1 A8 frame number decodes to 1128, and gives a sample number of 5197824 (which matches the logs).

The FLAC frame that follows it starts at decimal offset 24021080 with these bytes:

FF F8 59 AC D1 A9

The D1 A9 frame number decodes to 1129. So the 'intermediate' frame seems spurious, since there shouldn't be a frame between 1128 and 1129.

Consulting the output of flac -a ghi-588.flac for this file, and looking at frames 1128 and 1129 I see:

frame=1128	offset=23997582	bits=187984	blocksize=4608	sample_rate=44100	channels=2	channel_assignment=MID_SIDE
frame=1129	offset=24021080	bits=171336	blocksize=4608	sample_rate=44100	channels=2	channel_assignment=MID_SIDE

So I think what's happening is that FlacExtractor is 'finding' a frame that doesn't exist, because the 0xFFF8 pattern is found in the encoded data of frame=1128 and the bytes following this pattern "look like" a valid(ish) start to a FLAC frame. This means both the preceding frame and this frame are invalid when fed to the decoder (because the previous frame is truncated to create this spurious frame). This explains why we see two errors from the FLAC decoder inside ffmpeg. In addition, the spurious frame has a timestamp (read from totally garbage bytes) which is larger than the duration of the file.

Hacking the logic in FlacFrameReader.checkAndReadFirstSampleNumber to return false when sampleNumber == 9179136 (to ignore this spurious frame) allows the file to successfully play. It also plays with MediaCodec.

Doing a similar analysis on the file in #2587 I see these two timestamps cause the decode errors:

Marking buffer with pts=60186122 to be skipped 
Marking buffer with pts=2507755 to be skipped

And similar out of order nonsense:

Queueing sample with timeUs=2507755. currentFrameFirstSampleNumber=110592                                                    
Prev timestamps: [59245714, 59350204, 59454693, 59559183, 59663673, 59768163, 59872653, 59977142, 60081632, 60186122]        
Prev sample nums: [2612736, 2617344, 2621952, 2626560, 2631168, 2635776, 2640384, 2644992, 2649600, 2654208]                 
Timestamps after 2507755: [2507755, 60395102, 60499591, 60604081, 60708571, 60813061, 60917551, 61022040, 61126530, 61231020]

The bytes that FlacExtractor is reading the 110592 frame number from start at decimal offset 12206356 in the file. The preceding occurence is at 12190210 and the next one is at 12206590. Consulting the output of flac -a ghi-2587.flac:

frame=576	offset=12188141	bits=147592	blocksize=4608	sample_rate=44100	channels=2	channel_assignment=MID_SIDE
frame=577	offset=12206590	bits=147816	blocksize=4608	sample_rate=44100	channels=2	channel_assignment=RIGHT_SIDE

So actually the preceding occurrence is also spurious (but probably didn't pass ExoPlayer's extra checks for a valid frame header). Instead working backwards from 2654208 / 4608 = 576 - which confirms the same issue: ExoPlayer's FlacExtractor is splitting a frame into two because it seems to detect a valid FLAC header inside the encoded bytes.

Adding the same hacky "skip if sampleNumber == 110592" workaround resolves the playback issue with this file (including with MediaCodec).

So now the question becomes: How do we avoid detecting spurious/extra FLAC frames if we can't assume the encoded bytes won't look like a valid(ish) frame header. I will look into this.

Aside (with some guesswork): I suspect the reason that playback was able to continue with ffmpeg in the #2587 file and not in the file in this issue is that the spuriously generated timestamp happened to be before the surrounding timestamps, while the timestamp from the file in this issue was far in the future (beyond the end of the file). A side-effect of this was also that the player's 'buffered position' would jump forward to the end of the file as soon as we loaded this 'broken' frame (this is visible when experimenting in the demo app).

Yes sorry I should have been clear, that I was all talking about the new file sent by mail, I no more have the 2023 file as it was well a very long time ago.

So all my findings are related to that last file. This file does not even start to play due to all decode skipping.

I suppose this is now a completely different issue then sorry for the noise and not being clear enough. (Or it's the same but happens on the first frame eventually).

Anyway this is a little out of my league, can test anything you need, but I guess my inputs are irrelevant now that someone is looking into the issue. Thanks.

I was all talking about the new file sent by mail,

I haven't looked at this file until now (since i was able to repro the reported issue with the previous file).

Looking at it now, this file plays succesfully to completion with the MediaCodec FLAC decoder on a Pixel 7a at the tip of the main branch (i.e. with none of the hacks/fixes discussed in this issue).

It also plays to completion with the libflac extension.

I agree it fails with the ffmpeg decoder extension without playing any audio at all. It doesn't seem to be fixed by the fix for the other files in this issue described below.

I also see it reports ERROR, MD5 signature mismatch from flac -t (the previous file only reported a warning, and the file in #2587 reported OK). The issues with this file don't look related to the original file reported in this issue. You can file a new issue if you like, but since the FLAC command line tool confirms the file is invalid, we are unlikely to prioritize investigating it.

Back to how to fix FlacExtractor to avoid detecting extra frame headers.

From RFC 9639 section 6:

In order to allow a decoder to start decoding at any place in the stream, each frame starts with a byte-aligned 15-bit sync code. However, since it is not guaranteed that the sync code does not appear elsewhere in the frame, the decoder can check that it synced correctly by parsing the rest of the frame header and validating the frame header CRC.

Looking at the file from #2587. The spurious frame=24 frame at offset 12206356 happens to have a matching CRC-8 (which is meant to be the protection against this from the RFC quoted above):

For sampleNum=110592, crc=60, expected_crc=60. bytesForCrc=0xfff8901018

Looking back at the RFC, it looks like the byte immediately following the CRC must be a subframe header (see section 9.2).

The first bit must be zero.
The following 6 bits have some reserved values. These may be used in a future FLAC spec, but I'm inclined to use them to reject spurious frames for now.

Adding that to the checks in FlacFrameReader.checkAndReadFrameHeader makes the file in #2587 playable with MediaCodec, and the original file provided as part of this issue too.

Thanks.

I opened #2611, because it does uncover a specific ExoPlayer internal not handled case that leads to infinite buffering. (All decoded buffers are marked as skipped)

And since I see dd0afef that shows that this is something that is supposed to be handled, I think this case should be handled too.

It can happen with other decoders and I have not seen anything in the code saying do not return too much skippable buffers.

Adding that to the checks in FlacFrameReader.checkAndReadFrameHeader makes the file in #2587 playable with MediaCodec, and the original file provided as part of this issue too.

Hi icbacker, I am the reporter of [#2587], at the api level of exoplayer, how can I invoke it to enable the file to be played normally by mediacodec, or do I need to modify the underlying source code of exoplayer?

I've sent a change for internal review with these fixes. When it's submitted it will appear linked in this issue, and it will be included in a future media3 release.

Tighten FLAC header detection

This is fixed by the commit linked above.

@jacobgkau

@jacobgkau don't rejoice this is not your issue ;)

But thanks @icbaker for the fix for the initial issue.

Tighten FLAC header detection

Navigation Menu

Repository navigation

FlacExtractor can detect spurious/garbage FLAC frame headers within encoded FLAC data #558

Version

More version details

Devices that reproduce the issue

Devices that do not reproduce the issue

Reproducible in the demo app?

Reproduction steps

Expected result

Actual result

Media

Bug Report

9 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	} else {
	outputBuffer.skippedOutputBufferCount = skippedOutputBufferCount;
	skippedOutputBufferCount = 0;
	queuedOutputBuffers.addLast(outputBuffer);
	}

FlacExtractor can detect spurious/garbage FLAC frame headers within encoded FLAC data #558

Description

Version

More version details

Devices that reproduce the issue

Devices that do not reproduce the issue

Reproducible in the demo app?

Reproduction steps

Expected result

Actual result

Media

Bug Report

Activity

Tolriq commented on Jan 31, 2024

jacobgkau commented on Jul 13, 2025

Tolriq commented on Jul 13, 2025

jacobgkau commented on Jul 13, 2025

icbaker commented on Jul 14, 2025

Tolriq commented on Jul 14, 2025

icbaker commented on Jul 14, 2025

9 remaining items

icbaker commented on Jul 15, 2025

Tolriq commented on Jul 15, 2025

icbaker commented on Jul 16, 2025

Tolriq commented on Jul 16, 2025

ZhangQinglian commented on Jul 17, 2025

icbaker commented on Jul 17, 2025

icbaker commented on Jul 21, 2025

Tolriq commented on Jul 21, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions