Repository navigation
Playback exception on cmaf vod stream which the first segment have emsg box #1002
Description
Version
Media3 1.2.0
More version details
No response
Devices that reproduce the issue
samsung SM-S911U1 (RFCW11VVC1X) Android 13
Devices that do not reproduce the issue
No response
Reproducible in the demo app?
Not tested
Reproduction steps
Play a cmaf vod stream which the first segment contains emsg box e.g. the stream have pre-roll ads
Expected result
Start and watch this video normally.
Actual result
For media v1.0.2 it works well and no any exception.
But for media v1.2.0 sometimes we can not start the playback and throws exceiption, sometimes it start normally.
I guess this issue will occur when we have pending metadata samples within fmp4 extractor. Our video chunk contain video + emsg samples and audio chuck only contain audio samples. If the audio chuck is extracted first and then no pending metadata (when extract video chuck the timeadjuster had inited), but if the video chuck is extracted first and then we will have pending metadata samples on media3 v1.2.0 (no peding emsg on v1.0.2). I think for the former the playback will be normal but for the latter it will break the playback.
On v1.2.0 for the case video chunk is extracted first, FragmentedMp4Extractor#onEmsgLeafAtomRead() have following logic:
- commit emsg sample via
emsgTrackOutput.sampleData()and then pending emsg topendingMetadataSampleInfos. TheemsgTrackOutputjust cache the sample in local buffer, not commit sample to data queue yet. - at the time of audio chunk is extracted it will commit the audio samples, and then output all
pendingMetadataSampleInfosviaemsgTrackOutput.sampleMetadata()with offset. e.g we have 4 pending emsg, and the offset of first emsg is441.
Here I think we should pass 0 instead ofoffsetto delegate.sampleMetadata() call. It may cause the offset is set to -441 for the first sample of emsg queue and cause the exception while reading emsg samples.
Media
Our stream use ssai ad solution, cmaf drm stream, and diffcult to make it to be public for debug.
Bug Report
- You will email the zip file produced by
adb bugreportto android-media-github@google.com after filing this issue.
Activity
CarlLindqvist commentedon Apr 4, 2024
We have the exact same issue on live streams with server side midrolls. Here is an example stream which stitches 2 minute midrolls every 5 minutes.
https://csm-e-tv4se2-eb.tls1.yospace.com/csm/live/581556538.m3u8?vimondId=20315823&yo.ap=https%3A%2F%2Flive.streaming.a2d.tv%2Fysproxy%2Fhttps%3A%2F%2Fvod.streaming.a2d.tv%2Fys%2F&yo.asd=true&yo.av=3&yo.br=false&yo.fr=true&yo.me=true&yo.pdt=true&yo.t.jt=750&yo.up=https%3A%2F%2Flive.streaming.a2d.tv%2Fcmaf%2Fhttp%3A%2F%2Fvarnish-live-director.a2d.tv%2Fgroup02%2Fyospace%2Fdvrtest%2Fys.isml%2F&project=freewheel&yo.lpa=dur&t=long_form%2CTV4Play%2CNyheter+%26+debatt%2CCastens+h%C3%B6rna%2Clive%2CLive+cmaf+test+TV4%2Ctekniktest%2Csub_svod%2Cdt_mms_target_1_v3_30_days%2CSEG9791%2Cdt_mms_target_3_v3_30_days%2Cdt_mms_target_1_v3_aggressive%2Cdt_mms_target_3_v3_base%2CSEG2662%2Cdt_mms_target_1_v3_base%2Cdt_mms_target_3_v3_aggressive%2Cns_st_mv-dev%2Ctv4play.se&s=TV4Play%2FNyheter+%26+debatt%2FCastens+h%C3%B6rna%2Flive&dcid=d918a1f3-0bcc-438b-a205-b2a87ad0daf6&pid=1153490&st=0%3A0%2C20%3A1&gdpr=1&gdpr_consent=CP1YjVgP1YjVgAcABBENAaEgAAAAAAAAAChQAAAAAAGBIBgACAAFgAVAA8ACCAGQAaABEAFKAP0AxQC8wHLhAAIAGQwAIAIgAcxAAIAIgAcxwAgABAAFwAMgBfAIQAREArICkx0A8ABYAFQAQQAyADQAIgAYgBSgDRAH6ARYAxQCZAF5gMsAcuQACgAIAAyAF8ArICkyEAMABYAYgDFEoAwACAAFgBEADEAYoBeZIAMABcADIAXwCsgKTAZYUgGgALAAqACCAGQAaABEADEAKUAaIA_QCLAGKAXmA5coAFAAuABkAL4BWQDtgAAA.YAAAAAAAAAAA
etanmtv commentedon Apr 4, 2024
This issue seems to have been introduced with the Media3 1.1.0 release
marcbaechinger commentedon Apr 5, 2024
Thanks everyone for reporting! We will look into this.
From the duplicate #1203 and the information here it looks like this was introduced with 1.1.0:
From #1203:
Note: there is another test stream in #1203 for repro and to verify a fix when available.
CarlLindqvist commentedon Apr 5, 2024
The test stream in #1203 is actually the same test stream.
The midrolls trigger the issue consistently, so as soon as you get an ad you should get the index out of bounds error, most probably triggered by the emsg info.
marcbaechinger commentedon Apr 6, 2024
Thanks! We can repro this with the stream. The analysis from @guoen21 is to the point. When calling
sampleMetadataimmediately aftersampleData, theoffsetneeds to be set to0(zero). The code is there since a while and was working because it apparently never happened that multiple EMSG samples were pending. A change shipped with 1.1.0 made this more likely and together this caused the problems starting with 1.1.0.A fix for this will be referenced here when it lands on GitHub.
Use zero offset when delegating EMSG sample data
guoen21 commentedon Dec 18, 2024
This fix is released with 1.4.0 and above (not mentioned in the release note), I think we can close this ticket 😄