Repository navigation
HLS muxed audio missing format label #9608
Description
We have the following HLS (live) master playlist:
#EXTM3U
#EXT-X-VERSION:6
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-STREAM-INF:BANDWIDTH=130780,CODECS="mp4a.40.2",AUDIO="aac-primary"
../../<path>/chunklist_ao.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=130780,CODECS="mp4a.40.2",AUDIO="aac-backup"
../../<path>/chunklist_ao.m3u8
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aac-primary",NAME="Home",LANGUAGE="eng",DEFAULT=YES,AUTOSELECT=YES,CHANNELS="2"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aac-backup",NAME="Home",LANGUAGE="eng",DEFAULT=YES,AUTOSELECT=YES,CHANNELS="2"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aac-primary",NAME="Away",LANGUAGE="eng",URI="../../<path>/chunklist_ao.m3u8",DEFAULT=NO,AUTOSELECT=YES,CHANNELS="2"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aac-backup",NAME="Away",LANGUAGE="eng",URI="../../<path>/chunklist_ao.m3u8",DEFAULT=NO,AUTOSELECT=YES,CHANNELS="2"
Based on the HlsPlaylistParser (line 514), if an audio track has no URI defined, then it will be stored in the muxedAudioFormat. However in this case, whenever we query the audio tracks, the muxed one has no id/label in the Format.
See the event logger's output:
MediaCodecAudioRenderer [
Group:0, adaptive_supported=YES_NOT_SEAMLESS [
[ ] Track:0, id=0, mimeType=audio/mp4a-latm, bitrate=130909, codecs=mp4a.40.2, channels=2, sample_rate=48000, supported=YES
[ ] Track:1, id=1, mimeType=audio/mp4a-latm, bitrate=130909, codecs=mp4a.40.2, channels=2, sample_rate=48000, supported=YES
]
Group:1, adaptive_supported=YES_NOT_SEAMLESS [
[X] Track:0, id=aac-primary:Away, mimeType=audio/mp4a-latm, codecs=mp4a.40.2, channels=2, sample_rate=48000, language=en, label=Away, supported=YES
[ ] Track:1, id=aac-backup:Away, mimeType=audio/mp4a-latm, codecs=mp4a.40.2, channels=2, sample_rate=48000, language=en, label=Away, supported=YES
]
This is causing a problem for us, as we would need to select the tracks based on the labels. The question is whether the missing labels of muxed audio tracks is intentional or if it's a bug? How could we workaround this issue on our side (without changing the manifest itself)?
Thanks in advance.
Activity
ojw28 commentedon Oct 27, 2021
Do you see this with 2.15.1 (i.e., the latest release)? I took a quick look at the code, and it seems to be written so that the label should be picked up. If that's not happening then I'm probably missing something, and it's probably a bug.
If you can reproduce with 2.15.1, could you provide a complete HLS sample that we can use to easily reproduce the issue, either as a link to an accessible test stream or as a zip file containing the necessary files? The master playlist alone isn't sufficient. Thanks!
szaboa commentedon Oct 29, 2021
@ojw28 We can't really provide a clear sample stream easily, however we have found a solution that we can apply on our forked version. You can see the change in the linked pull request, feel free to give us a feedback on that.
ojw28 commentedon Nov 1, 2021
I'm not sure, but that commit doesn't seem to make much sense to me. I don't understand why you'd want to set
primaryExtractorTrackType = C.TRACK_TYPE_VIDEOfor an audio only stream.Can you provide a DRM protected stream easily, even without the corresponding DRM server information? I'm not sure, but I suspect that the tracks are determined before the player needs the DRM keys, so that might be sufficient to be able to reproduce. Thanks!
szaboa commentedon Nov 2, 2021
The
primaryExtractorTrackTypeis set toC.TRACK_TYPE_VIDEObecause otherwise themuxedAudioFormatwon't be used at all, see the logic a few lines below the change in the PR (line 1371-1381).ojw28 commentedon Nov 3, 2021
Ah, I see. Could you try applying this change locally, to see whether it also resolves the issue correctly: 6b78164
Thanks!
guoen21 commentedon Nov 8, 2021
I apply and test this commit locally, seems it also works for me, thanks very much!
Now we get two following muxed tracks, but the id is same, because we use one muxedAudioFormat, but from our master m3u8 it included two tracks which has no URI defined.
expected
HLS: Merge muxedAudioFormat into primary audio tracks
guoen21 commentedon Nov 18, 2021
I also add comment for commit 45857b5,
line 1398,
playlistFormat = playlistFormat.withManifestFormatInfo(muxedAudioFormat);In here the language attribute of muxedAudioFormat may be miss, because of the playlistFormat.sampleMimeType is null.
For above audio-only case, it will output the expected result
But for normal case which has video tracks, muxed audio tracks, and additional audio tracks, the output has only one muxed audio track as following:
In my commit I enhance it to support multiple muxed audio tracks - change the single
Format muxedAudioFormattoList<Format> muxedAudioFormats. I also merge this 45857b5 commit. Now I can get the expected outputHLS: Merge muxedAudioFormat into primary audio tracks
ojw28 commentedon Nov 19, 2021
This is a good observation. We'll take a look; thanks!
There is a TODO about this here. If you've implemented it, please feel free to send us a pull request. If you've implemented it in a different way then that might also be interesting, or you could try and adapt your approach to match what's written in the TODO.
HLS: Merge muxedAudioFormat into primary audio tracks