CompositionPlayer stops for good when playback starts on the last frame of a sequence item (audio renderer not ready after the item transition) #3452

Version

Media3 1.11.1

More version details

All Media3 artifacts are 1.11.1. Playback uses CompositionPlayer. Pre-warming is at its default (on).

Devices that reproduce the issue

Samsung Galaxy A70 (SM-A705FN), Android 11 (API 30), Snapdragon 675 (sm6150)

Devices that do not reproduce the issue

Not tested on other devices yet.

Reproducible in the demo app?

Not tested

Reproduction steps

1. Build a Composition with one EditedMediaItemSequence (audio and video) of at least two video items. Each item is a clipped MediaItem of an H.264/AAC 30 fps MP4 (the attached repro-30fps.mp4, cut into two 2.4 s clips).
2. Prepare a CompositionPlayer with that composition and start playback.
3. While playing, call seekTo(2_366). Seeks land on the first frame at or after the target, so this lands on the last video frame of item 0 (pts 2366.7 ms), 33 ms before the item boundary at 2400 ms. seekTo(2_367) would land on the boundary and not reproduce.

Minimal code. Note: this exact snippet has not been run on its own yet; see "Notes" below.

fun clip(uri: Uri, startMs: Long, endMs: Long) =
  EditedMediaItem.Builder(
    MediaItem.Builder()
      .setUri(uri)
      .setClippingConfiguration(
        MediaItem.ClippingConfiguration.Builder()
          .setStartPositionMs(startMs)
          .setEndPositionMs(endMs)
          .build()
      )
      .build()
  ).build()

val sequence = EditedMediaItemSequence.withAudioAndVideoFrom(
  listOf(clip(uri, 0, 2_400), clip(uri, 2_400, 4_800))
)
val player = CompositionPlayer.Builder(context).build()
player.setVideoSurfaceView(surfaceView)
player.setComposition(Composition.Builder(sequence).build())
player.prepare()
player.play()

// About 1 s into playback:
player.seekTo(2_366) // lands on item 0's last frame (pts 2366.7 ms); the item boundary is at 2_400 ms

Other ways to trigger it, all with the same result:

• setComposition(composition, positionMs) at that position while playing (a "rebuild" after an edit);
• the same seek while paused, followed by play().

Expected result

Playback continues from item 0 into item 1.

Actual result

1. The first frame of item 1 renders.
2. currentPosition moves about 130-150 ms past the boundary and then stops.
3. No further frames are rendered for as long as we waited (more than 10 s; our own watchdog then rebuilds the player).
4. Any later seekTo to a different position recovers it at once.

The EventLogger shows the sequence player's audio renderer going not-ready about 140 ms after the AUTO_TRANSITION. The sequence player goes to BUFFERING, and CompositionPlayer then sets playWhenReady=false on it and pauses the AudioTrack. Nothing changes after that. Excerpt: the attached freeze-a-eventlogger.txt, with the transition at 9.60 s and BUFFERING at 9.74 s.

Measured in our test harness (automated probes, 2 repetitions each):

The trigger seems to follow how much source time is left in the outgoing item (about 20-60 ms), not timeline time.

What was in the path:

• It reproduces with a plain seekTo and with setComposition(…, positionMs), playing or paused-then-play.
• It reproduces with no custom AudioProcessors at all, and without any audio-only sequences.

Our reading of the 1.11.1 source, which may be wrong:

• AudioGraphInputAudioSink.hasPendingData() became per sequence in 1.11 (TODO b/487191706).
• MediaCodecAudioRenderer.isReady() depends on it, with a 100 ms grace period.
• When one sequence player buffers, CompositionPlayer pauses all players and the final AudioTrack, so nothing drains.
• Stuck detection is disabled for sequence players (TODO b/489733731), so the player never reports an error.

Workaround we use: never start or continue playback from within about 70 ms of source time (divided by item speed) before an item boundary; seek to the boundary instead. With that, 0 of 315 probes froze.

freeze-a-eventlogger.txt

Media

Attached: repro-30fps.mp4, a synthetic 6 s clip, 1280x720 H.264 30 fps with AAC 48 kHz stereo. Our tests used similar synthetic 30 fps and 60 fps clips; the issue does not seem to depend on the content.

Notes

• Our app also applies composition-level video effects: a custom GlEffect (colour grade) and a Presentation to 720x1280. The video output goes to a Surface from a TextureView. We have not yet confirmed the snippet above without these. We will update this issue with the result.

Bug Report

• You will email the zip file produced by adb bugreport to android-media-github@google.com after filing this issue.