Skip to content

Should HLS live manifest refreshing consider http request delay? #663

Closed
Closed
Should HLS live manifest refreshing consider http request delay?#663
Assignees
tianyif
Labels
needs triagequestion

Description

@baiqindotfubotv
baiqindotfubotv
opened on Sep 20, 2023

I am using exoPlayer to play a live HLS stream. I noticed the exoPlayer doesn't take the network request delay into consideration when scheduling manifest refreshing. Here are some logs I printed in onTimelineUpdate method:

23:34:47.890  E  Manifest refreshed
23:34:52.020  E  Manifest refreshed
23:34:56.212  E  Manifest refreshed
23:35:00.507  E  Manifest refreshed
23:35:04.712  E  Manifest refreshed
23:35:08.881  E  Manifest refreshed
23:35:13.075  E  Manifest refreshed
23:35:17.258  E  Manifest refreshed

As you can see the gap between each refresh is several hundreds of ms greater than segment size (4 seconds). According to the description in 6.3.4 here: https://datatracker.ietf.org/doc/html/draft-pantos-http-live-streaming-16#section-6.3.4:

"When a client loads a Playlist file for the first time or reloads a Playlist file and finds that it has changed since the last time it was loaded, the client MUST wait for at least the target duration before attempting to reload the Playlist file again, measured from the last time the client began loading the Playlist file."

It says the next refresh time should be measured from the last time the client begin loading the Playlist file. However looks like exoPlayer is scheduling next refreshing based on the the timestamp when last loading was finished. I think this is where exoPlayer does the calculation: https://github.com/androidx/media/blob/1.1.1/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/playlist/DefaultHlsPlaylistTracker.java#L764.

When the default playback position is close to the end of live window end, this behavior will cause a lot of bufferings.

So is this expected behavior?

For comparison, DASH manifest refreshing does consider network delay:
https://github.com/androidx/media/blob/1.1.1/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/DashMediaSource.java#L931. And for this reason, when exoPlayer is playing DASH live manifest at a position close to live window end, bufferings seldom happen.

Activity

  1. baiqindotfubotv
    added
    needs triage
    question
    on Sep 20, 2023
  2. rohitjoins
    assigned
    tianyif
    on Sep 20, 2023

  3. tianyif commented on Sep 25, 2023

    @tianyif
    tianyif
    on Sep 25, 2023
    Contributor

    Hi @baiqindotfubotv,

    Thanks for the question! Sounds like using "the last time the client begin loading the Playlist file" should be more accurate.

    But even with the current implementation, we still comply the standard the client MUST wait for at least the target duration before attempting to reload the Playlist file again as the interval between the two start loading time should be slightly longer than the target duration. So I'm wondering the buffering issue is coming from the other causes. Would it be possible for you to provide more details on:

    1. If you are able to reproduce and debug with demo app by changing to use "the last time the client begin loading the Playlist file" instead of "the time when last loading was finished", does it solve your buffering issue?
    2. How long is your live window? The reason why I'm asking is based on this blog post (you can search for the section "Use a long live window"), that being too close to the start of the window might cause the loading to fall behind, which will require a repreparation of the player.

    We're looking forward to your feedback!

  4. tianyif
    added
    need more info
    and removed
    needs triage
    on Sep 25, 2023

  5. baiqindotfubotv commented on Sep 27, 2023

    @baiqindotfubotv
    baiqindotfubotv
    on Sep 27, 2023
    Author

    I tested a change at this line: DefaultHlsPlaylistTracker

    I changed this line to

    earliestNextLoadTimeMs = currentTimeMs + Util.usToMs(durationUntilNextLoadUs) - loadEventInfo.loadDurationMs;

    It does fix 95% of the buffering in my case.

    The live window size is 32 seconds in length with 8 segments but I don't think window size matters. What matters is starting playback position. The buffering issue only happens with HLS if the target playback position is close to the end of live window. For example, at 28 seconds. Let's assume each manifest refresh takes exactly 500 ms. Then the refresh sequence is something like this:

    Refresh at 0: window start time: 1000, buffer size 4000 ms
Refresh at 4500: window start time 5000, buffer size 3500 ms
Refresh at 9000: window start time 9000, buffer size 3000 ms
Refresh at 13500: window start time 13000, buffer size 2500 ms
Refresh at 18000: window start time 17000, buffer size 2000 ms
Refresh at 22500: window start time 21000, buffer size 1500 ms
Refresh at 27000: window start time 25000, buffer size 1000 ms
Refresh at 31500: window start time 29000, buffer size 500 ms

    At this time the buffer size is under 500 ms and the player needs to buffer. And the same buffering issue will happen again and again.

    This won't be an issue if the playback position is not close to the end live window end. This is because in a later manifest refresh request, the live window will jump ahead 8 seconds instead of 4 seconds, thus the previous accumulated delay will be erased. If we set the playback position to 18 seconds instead of 28 seconds, it will be something like this:

    Refresh at 0: window start time: 1000, buffer size 14000 ms
Refresh at 4500: window start time 5000, buffer size 13500 ms
Refresh at 9000: window start time 9000, buffer size 13000 ms
Refresh at 13500: window start time 13000, buffer size 12500 ms
Refresh at 18000: window start time 17000, buffer size 12000 ms
Refresh at 22500: window start time 21000, buffer size 11500 ms
Refresh at 27000: window start time 25000, buffer size 11000 ms
Refresh at 31500: window start time 29000, buffer size 10500 ms
Refresh at 36000: window start time 33000, buffer size 10000 ms
Refresh at 40500: window start time 41000, buffer size 14500 ms
Refresh at 45000: window start time 45000, buffer size 14000 ms

    Note that the refresh at 40500 returns a manifest that jumps ahead 8 seconds, thus previous accumulated manifest refresh delay is erased. If playback position is close to end of live window, it is very likely to cause a lot of buffering issues.

    The work around I am using right now is to wait for the first onTimelineUpdate event, after that I use the exoPlayer.seekTo method to seek to a target playback position, thus ensuring a good buffering size.

    A note on the default playback position. Right now there is no media3 interface to configure initial playback position / buffer size for live playback. Both DASH and HLS. There are some DASH (minBufferTime) / HLS (ext-x-start) tags that could affect the initial playback position. But a lot of times the packaging is done by third parties so the client doesn't have control over it.

    ExoPlayer does have a LiveConfiguration class, but it is not useful in a lot of cases because the live offset in it is the offset to unix now time. Given a streaming url, it is impossible to know the accurate manifest start and end until the manifest is fetched from server. So it is also impossible to calculate a target live offset ahead of time.

    Previously DashMediaSource has a livePresentationDelayMs parameter, but even that was removed in this commit: livePresentationDelayMs. If exoPlayer has an api configure initial playback position it will be helpful.

  6. google-oss-bot
    added
    needs triage
    and removed
    need more info
    on Sep 27, 2023

  7. tianyif commented on Oct 10, 2023

    @tianyif
    tianyif
    on Oct 10, 2023
    Contributor

    Thanks @baiqindotfubotv for the detailed explanation! After the internal discussion, we'll be making the changes for an accurate reload interval with considering the network delay.

    Also, as for

    ExoPlayer does have a LiveConfiguration class, but it is not useful in a lot of cases because the live offset in it is the offset to unix now time. Given a streaming url, it is impossible to know the accurate manifest start and end until the manifest is fetched from server. So it is also impossible to calculate a target live offset ahead of time.

    please keep track on the existing feature request - #652.

  8. tianyif commented on Oct 10, 2023

    @tianyif
    tianyif
    on Oct 10, 2023
    Contributor

    And would it be possible for you to provide a media sample for us to test the change?

  9. tianyif
    added
    need more info
    and removed
    needs triage
    on Oct 10, 2023
  10. google-oss-bot
    added
    needs triage
    and removed
    need more info
    on Oct 12, 2023

  11. baiqindotfubotv commented on Oct 13, 2023

    @baiqindotfubotv
    baiqindotfubotv
    on Oct 13, 2023
    Author

    And would it be possible for you to provide a media sample for us to test the change?

    Email sent. But any HLS live stream should work I think. I just hard code the "Live Akamai m3u8" here: https://ottverse.com/free-hls-m3u8-test-urls/ into media3 demo. Let it play at the end of live window, for example, seek to 118000 ms since the window size is 120000. Then I can see a buffering every couple of minutes.

  12. copybara-service
    added 2 commits that reference this issue on Oct 13, 2023

    Calculate HLS live playlist refresh interval accurately

    58a63c8

    Calculate HLS live playlist refresh interval accurately

    ed03589
  13. rohitjoins
    added a commit that references this issue on Oct 23, 2023

    Calculate HLS live playlist refresh interval accurately

    2a7d85e
  14. tianyif
    closed this as completedon Oct 23, 2023
  15. androidx
    locked and limited conversation to collaborators on Dec 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

needs triagequestion

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions