Decoder freezes on quality switch when playing HLS LL stream

Version

Media3 main branch

More version details

main, Tested with different versions, seems to be unrelated

Devices that reproduce the issue

Seems to be device unrelated. Can be reproduced on Android 14 emulator, Pixel 7a with Android 15 and other devices.
Emulators seem to reproduce the issue faster than physical devices though.

Devices that do not reproduce the issue

No response

Reproducible in the demo app?

Yes

Reproduction steps

modify AdaptiveTrackSelection to switch qualities often, e.g. by doing

Index: libraries/exoplayer/src/main/java/androidx/media3/exoplayer/trackselection/AdaptiveTrackSelection.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/trackselection/AdaptiveTrackSelection.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/trackselection/AdaptiveTrackSelection.java
--- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/trackselection/AdaptiveTrackSelection.java	(revision 10fd1ee2704e2e8b669ae0c3120cf37b62291759)
+++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/trackselection/AdaptiveTrackSelection.java	(date 1743068827615)
@@ -482,7 +482,15 @@
     reason =
         newSelectedIndex == previousSelectedIndex ? previousReason : C.SELECTION_REASON_ADAPTIVE;
     selectedIndex = newSelectedIndex;
+
+    if(lastCalledTimeStamp + 5_000 < System.currentTimeMillis()){
+      selectedIndex = 0;
+      lastCalledTimeStamp = System.currentTimeMillis();
+    }else {
+      selectedIndex = 1;
+    }
   }
+  private long lastCalledTimeStamp = System.currentTimeMillis();
   @Override
   public int getSelectedIndex() {

play media shared via e-mail e.g. on android emulator Pixel 8 Pro Android 14
observe freeze, usually within ~1 minute

Expected result

No freeze on quality switch OR freeze is reported as player error to enable recovery.

Actual result

After a quality switch the video frame freezes while audio and the playhead continue. This happens regardless of the quality being switched from/to.

after the freeze MediaCodedAdapter.dequeueInputBufferIndex always returns -1 indicating that the decoder is stuck
overwriting the decoder reuse evaluation on quality switch to REUSE_RESULT_NO fixes the problem but drops frames on every quality change reducing the overall quality of experience

Media

Will be send via e-mail

Bug Report

You will email the zip file produced by adb bugreport to android-media-github@google.com after filing this issue.

Thanks for the sample media and perfetto traces, and I can easily reproduce this issue on emulator with more aggressive track updating frequency (changing 5_000 in your code snippet to 1_000).

The DecoderCounter.queuedInputBufferCount and DecoderCounter.renderedOutputBufferCount didn't increase from the freezing point (0:30):

With adding some logs when debugging, it looks like the MediaCodec.Callback.onInputBufferAvailable / onOutputBufferAvailable were no longer triggered for video once the playback freezes

media/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/AsynchronousMediaCodecCallback.java

Lines 214 to 236 in 51efcad

    
           public void onInputBufferAvailable(MediaCodec codec, int index) { 
        
             synchronized (lock) { 
        
               availableInputBuffers.addLast(index); 
        
               if (onBufferAvailableListener != null) { 
        
                 onBufferAvailableListener.onInputBufferAvailable(); 
        
               } 
        
             } 
        
           } 
        
           @Override 
        
           public void onOutputBufferAvailable(MediaCodec codec, int index, MediaCodec.BufferInfo info) { 
        
             synchronized (lock) { 
        
               if (pendingOutputFormat != null) { 
        
                 addOutputFormat(pendingOutputFormat); 
        
                 pendingOutputFormat = null; 
        
               } 
        
               availableOutputBuffers.addLast(index); 
        
               bufferInfos.add(info); 
        
               if (onBufferAvailableListener != null) { 
        
                 onBufferAvailableListener.onOutputBufferAvailable(); 
        
               } 
        
             } 
        
           }

I assume this may be related with the frequent reconfiguration of the codec. Assigning to my colleague for more insights.

@tianyif

Thanks @tianyif, that aligns with our observations.
It does seem to happen also if the codec reconfiguration happens less often but it takes longer - not sure if it's the same "chance" to freeze on every quality switch or if the likelihood is influenced by frequent reconfigurations.

@tonihei

Hi @tonihei , is there any update on this ticket please? Since this behaviour impacts real devices and not just emulator, will really appreciate your thoughts on probable root cause and next steps.

Thanks for brining it back to our attention :)

I found a minute to take a look and it seems that

It's a duplicate of ABR Live Streaming Stutter, Glitch using LLHLS #1285 - I will keep this one as it has more debugging info.
It has nothing to do with the codec. There is always an input buffer available and we get stuck because we don't fill in new buffers anymore.
There is something weird going on filling the buffers in from the source though. I can see that it repeatedly jumps back in time, violating some basic assumptions about how our sample queue works. This is, I think, what causes it to get stuck because we can't read the samples anymore even though they are in the queue. Need to investigate further.

@tonihei

Hi @tonihei , thanks for looking into it. Will eagerly await ant further updates/insights. Please let us know if we can provide any further data or any other assistance to root cause the issue.

There are two main issues here:

The video can get stuck entirely if a track selection switches to a new format while the last loaded chunk is one of the preload hints that hasn't been resolved to a fully published part in a playlist yet. As we switch media playlists, this gets never resolved and playback eventually hangs when trying to decode data from this preload hint (which isn't allowed to leave the option to discard the data again in case the part if removed). The fix for this is to discard the data from the preload hint if we are switching between formats.
We evaluate whether to load a new format after every single load and then do our best to splice in the new data as needed. When loading independent segments from the end of the previous item, this works well because there isn't any overlap. However, if we attempt to load a new format after loading some non-independent parts we need to go back to the beginning of the segment and splice the new data in from there. If we are playing very close to the live edge, this data has already been read and the splice operation only succeeds at the beginning of the next chunk, leaving a gap of no data, which is visible as a "freeze". The solution is not allow track updates that would result in such gaps.

Fixing both issues seems to resolve the problems reported for the streams here and in #1285. When playing them very close to the live edge, the only remaining hiccup is an occasional buffering at the track switch points if the pipeline runs out of data (which is more working as intended and goes through the normal state transitions, unlike the issue reported here).

Discard trailing preload hint data when switching tracks

Both changes are now submitted, will close the issue. Please feel free to reopen or file a new one if anything doesn't look fixed.

Thanks for the fast investigation & fix, much appreciated.

@tonihei

He @tonihei, took me a while to validate the fix but unfortunately I still see freezes happen. I'll be sending a new e-mail with a new test stream as the old one is expired, could you please have another look?

I tested on the latest main branch, at the time of testing this was 6d49937
Reproduced on Pixel 8 Pro emulator API level Level 34

Thanks - I can see the remaining problem as well. We stopped switching tracks if we already know the splice won't succeed. However, it might be that we are reading samples from the data we intend to discard, and then are prevented from discarding it later.

Avoid splice in HLS chunks if it can be fulfilled by discarding

Submitted an additional fix that makes the constantly switching track selection work well with the stream above.

@tonihei

Thanks @tonihei I'll give it another shot. 🙇

Avoid splice in HLS chunks if it can be fulfilled by discarding

Navigation Menu

Repository navigation

Decoder freezes on quality switch when playing HLS LL stream #2299

Version

More version details

Devices that reproduce the issue

Devices that do not reproduce the issue

Reproducible in the demo app?

Reproduction steps

Expected result

Actual result

Media

Bug Report

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	public void onInputBufferAvailable(MediaCodec codec, int index) {
	synchronized (lock) {
	availableInputBuffers.addLast(index);
	if (onBufferAvailableListener != null) {
	onBufferAvailableListener.onInputBufferAvailable();
	}
	}
	}

	@Override
	public void onOutputBufferAvailable(MediaCodec codec, int index, MediaCodec.BufferInfo info) {
	synchronized (lock) {
	if (pendingOutputFormat != null) {
	addOutputFormat(pendingOutputFormat);
	pendingOutputFormat = null;
	}
	availableOutputBuffers.addLast(index);
	bufferInfos.add(info);
	if (onBufferAvailableListener != null) {
	onBufferAvailableListener.onOutputBufferAvailable();
	}
	}
	}

Decoder freezes on quality switch when playing HLS LL stream #2299

Description

Version

More version details

Devices that reproduce the issue

Devices that do not reproduce the issue

Reproducible in the demo app?

Reproduction steps

Expected result

Actual result

Media

Bug Report

Activity

tianyif commented on Apr 2, 2025

strangesource commented on Apr 3, 2025

luckygoyal-bitmovin commented on May 23, 2025

tonihei commented on May 23, 2025

luckygoyal-bitmovin commented on May 23, 2025

tonihei commented on May 27, 2025

tonihei commented on May 29, 2025

strangesource commented on Jun 2, 2025

strangesource commented on Jun 12, 2025

tonihei commented on Jun 20, 2025

tonihei commented on Jun 25, 2025

strangesource commented on Jun 25, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions