Clarify/remove 'stalled' and 'progress' events for MediaSource

[chcunningham]

Note on priority: Happy to see any spec change pushed to MSE vNext. v1 is nearing cut date and IMO this isn't critical.

'Stalled' makes good sense for non-MediaSource playbacks where the UA is responsible for the fetching the media. For MSE, the web app does the fetching and MSE is unaware that the fetch is being made and cannot know whether its progressing or stalled. The MSE spec states that "the download" or "bytes received" (in resource fetch algorithm) refer to data passed in via appendBuffer() and appendStream()". With this model, it is not possible for MSE to experience a stall (appends don't stall).

MSE spec makes no special mention of 'stalled' and when to emit (if ever) is ambiguous given the lines about "the download" and "bytes received" above. At the moment Chrome raises stalled if an MSE app goes 3 seconds without appending. This leads to confusion. I propose that we should not emit stalled for MSE (similar to the decision to not emit suspend for MSE in Issue 24).

When to emit 'progress' is also not clear. Raising this during an append is not as confusing as Chrome's behavior for 'stalled', but one could argue that raising progress is still confusing (given that app control the fetch) and also not useful for app developers. Apps know when they've called append and they can clearly know the outcome/completion of that append by other events/states (e.g. updateend event, SourceBuffer.buffered). I propose that we should also not bother to emit progress.

This is also a call for app developers to point out MSE use cases for these events that I may have overlooked.

One argument that I've heard is that some players may have UI that relies on these events dating back pre-MSE implementation. If this is a real concern then we have to decide on an implementation for these events that is meaningful. Showing a stall indicator because no append has been made in the last 3 seconds is not a great UI. Apps may append less frequently than that and still deliver a perfect playback experience. Additionally, apps would be better served basing any network UI on the network fetch they make prior-to appending to MSE.

[chcunningham]

Sorry, closed by mistake

[chcunningham]

reference: #24
fyi: @wolenetz @foolip @karlt @mounirlamouri

[wolenetz]

Thanks for filing this, @chcunningham.
I suppose 'stalled' and 'progress' might make more sense to retain for MSE if we retained appendStream(). However, for V1 MSE, appendStream() is currently marked at-risk.

@jdsmith3000 , @mwatson2 Do you agree with my evaluation that this should be triaged to VNext, especially given the current tight MSE v1 timeline and our triage process? This would also afford implementors and web authors the opportunity to chime in on perhaps some useful 'stalled' or 'progress' MSE use cases in the interim (or provide evidence that 'stalled' and 'progress' indeed are underdefined/useless in MSE HTMLMediaElements). Please update my triage result as you deem fit.

[dmlap]

One data point: in video.js we trigger synthetic progress events based on content fetching already since that corresponds more closely to the definition in the HTML spec. I don't think stopping native progress or stalled events for MediaSource content would have an impact on us.

[wolenetz]

Thank you, @dmlap - that confirms @chcunningham 's hypothesis that apps should really take over 'progress' and 'stalled' logic in MSE cases.

Regarding potential use case for MSE 'stalled' and 'progress' if we use appendStream(), linking #14 to follow-up on that.

For MSE based playback, if HTML does not emit the stalled/progress events, the use case of a third app listening on HTML events for video element, will be broken/inconsistent. I feel, there should be a notification back to HTML, mandated by the standard, upon which HTML standard should generate these events.

[chcunningham]

swapnai - under what specific conditions do you think these events should be emitted for MSE?

[wolenetz]

Also, @SwapnAI, would those conditions possibly need to differ across MSE apps such that the MSE app should instead have the opportunity to own the app-specific logic and fire their own 'progress' and 'stalled' events at the mediaElement?

Sorry for the delay. Here are my comments:

Use-case: third-party video analytics and optimization solutions

Stalled events are currently part of the quality of experience analytics features of the HTML video element spec, providing insight into the performance and reliability of its internal content downloader components.

When stalled event triggers definition is left up to downloader/MSE component’s implementors, video analytics solutions need to establish their own API contract outside of W3C specs to either enforce a proper stalled event definition or identify each implementation to allow for proper interpretation of the stalled event.

Strengthening the contract between video element and MSE specs instead would ensure reliable stalled events across content downloader implementations and maintain interoperability of video analytics solutions.