Introduce seek() and let it return a promise #553

Activity

1. zcorpan

   added

   addition/proposalNew features or enhancementsNew features or enhancements

   on Jan 25, 2016
2. foolip

   added

   topic: media

   on Feb 1, 2016
3. foolip commented on Feb 1, 2016

   

   foolip

   on Feb 1, 2016

   MemberAuthor

   More actions

   Proposal:

   enum SeekMode { "precise", "fast" };
   
   dictionary SeekOptions {
     SeekMode mode = "precise";
   };
   
   partial interface HTMLMediaElement {
     Promise<void> seek(double time, optional SeekOptions options);
   };

   This would correspond to the currentTime setter and the fastSeek() in the spec which has not been implemented AFAIK. (I believe it could also be overloaded to support the rational-time preciseSeek(), if the need should arise.)

   @padenot @jernoble @eric-carlson
4. jernoble commented on Feb 1, 2016

   

   jernoble

   on Feb 1, 2016

   More actions

   rational-time preciseSeek()

   Interesting!
5. foolip commented on Feb 1, 2016

   

   foolip

   on Feb 1, 2016

   MemberAuthor

   More actions

   The Bugzilla bug for preciseSeek() and its blocking bugs have sat idle for a very long time. Is this something you see requests for in Safari? My hunch is that rational times are most important in professional media software like video editors, and that HTML media elements as a whole are still far away from solving those use cases, even with rational times.
6. eric-carlson commented on Feb 1, 2016

   

   eric-carlson

   on Feb 1, 2016

   More actions

   the fastSeek() in the spec which has not been implemented AFAIK

   fastSeek was added to WebKit in r159208
7. foolip commented on Feb 1, 2016

   

   foolip

   on Feb 1, 2016

   MemberAuthor

   More actions

   Ah, cool! In that case we shouldn't drop it from the spec without a plan. It would be simple to keep as a (promise-less) alias for seek(), but in that form it'd be hard to convince others to implement it. What's your preferred outcome? Do you have telemetry for measuring feature usage in the wild?
8. jernoble commented on Feb 1, 2016

   

   jernoble

   on Feb 1, 2016

   More actions

   We have recurring requests / bug reports about imprecise seeking, especially when clients target frame boundaries, and the only real solution is some kind of rational time based seeking. That's something I've been interested in for a while: https://lists.w3.org/Archives/Public/public-whatwg-archive/2012Oct/0013.html

   Aside from more "professional" use cases like "web based video editors" and the like, we've recently seen these kinds of bugs crop up, e.g., when people want to synchronize the page's scroll position with a specific frame in a <video> element, and they have trouble seeking to the correct frame.
9. foolip commented on Feb 1, 2016

   

   foolip

   on Feb 1, 2016

   MemberAuthor

   More actions

   Hmm, what happens if one tries to seek to the middle of a frame instead? I realize that only rational numbers are going to really fix the problem, though.

   Above I just wanted to make sure that a new seek() doesn't ruin the possibility of rational-time seeking. If you'd like to work on that, can you create a new issue and summarize what needs to be done? (I'd then redirect the legacy bugs there.) Pull requests welcome too, of course, but feedback from some of Google, Mozilla and Microsoft would be nice first.
10. jernoble commented on Feb 2, 2016

    

    jernoble

    on Feb 2, 2016

    More actions

    Hmm, what happens if one tries to seek to the middle of a frame instead?

    You seek to somewhere near the middle of the frame, but the frame only plays for 1/2 its duration. Authors could also just add a small mantissa value to bias the rounding in the direction they want. But as you note, the only real way to solve the problem is by supporting rational time values.

    If you'd like to work on that, can you create a new issue and summarize what needs to be done?

    I can do that. Building off your SeekOptions dictionary, it may be as simple as adding a timeScale option: video.seek(30 * 1001, {mode:"precise", timeScale:30000}); to seek to the 30th frame of a 29.97 fps video. For the initial feature, the time scale for the movie would have to be provided out of band, while we work on some way of exposing the movie's natural time scale to script.
11. foolip commented on Feb 2, 2016

    

    foolip

    on Feb 2, 2016

    MemberAuthor

    More actions

    As long as we can assume or require that the whole timeline uses the same time scale, then exposing a timeScale attribute on HTMLMediaElement could do it, but let's move that discussion to a separate issue and just try to not ruin it with seek() here :)
12. cpearce commented on Feb 3, 2016

    

    cpearce

    on Feb 3, 2016

    More actions

    the fastSeek() in the spec which has not been implemented AFAIK

    Firefox 31 implemented fastSeek, primarily for the benefit of touch controls on our mobile products.

    Our experience with fastSeek() has not been good.

    We've had numerous bugs filed to the effect of "I seeked and the seek didn't go where I expected". This is particularly bad in files with dumb encodings like only a single keyframe. It's also bad because the location of sync-points is not exposed to JS.

    I'm planning on adding telemetry to track how often fastSeek gets used in Firefox and using that telemetry, along with feedback from other implementers as to whether they intend to implement fastSeek, to determine whether we remove our fastSeek implementation.

    Were fastSeek to remain in some fashion, I'd like it to be renamed to something other than "fast". People see "fast" and they think "OMG I love fast! Why would I want slow??". We should name it something that implies the trade-off being made. Like a "mode:sloppy", or "mode: prev-sync-point", in the SeekOptions dict proposed above.
13. cpearce commented on Feb 3, 2016

    

    cpearce

    on Feb 3, 2016

    More actions

    Firefox's (precise) seek implementation can seek to individual audio frame boundaries, modulo rounding issues caused by storing the seek target in a double in JS and then converting it to int64_t microseconds, which is generally the time format our media stack uses internally.

    We've had a few bugs filed over the years about seeking to video frames not working thanks to said rounding issues. We've never felt it a huge priority to address that, but I'd be happy to leave scope in the spec to handle that in future.
14. foolip

    changed the title [-]Rename fastSeek() to seek() and let it return a promise[/-] [+]Introduce seek() and let it return a promise[/+] on Feb 4, 2016
15. foolip commented on Feb 4, 2016

    

    foolip

    on Feb 4, 2016

    MemberAuthor

    More actions

    I've renamed the issue to decouple it somewhat from fastSeek(), the comment that this spawned from was about seeking generally.

    What ought to be uncontroversial is introducing a seek() which behaves like the currentTime setter but returns a promise, and whether or not to merge fastSeek() into that is not a given.