863499 - Signal for "app load"

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Performance, defect, P2)

Description

[James Burke [:jrburke]]

This started on dev-gaia:

http://groups.google.com/group/mozilla.dev.gaia/browse_thread/thread/a0faa6512f71bc94

Summary:

As apps use more async/defer-loaded JS and more async APIs for final construction, the platform cannot accurately detect when the app has finished loading and is in a usable state.

Ideally the app could signal to the platform when it considers itself loaded. This would give the following benefits:

* The gaia system app could listen for this signal and provide better transitions from app screenshots to running app (avoiding "its a web page" white flash during initial load)
* The gaia system could also use it to indicate when a screenshot could be taken, without it happening during the time the app needs to finish the load, allowing app startup to complete faster.
* For Gaia metrics purposes, this event could be used to track how long apps actually take to load to be usable.

Would it be possible for the platform to support a way for the app to signal that is has finished loading?

Perhaps a 'mozappload' (or some moz-prefixed) event that can be created via new CustomEvent by the app JS code and dispatched on window, and the platform would allow mozbrowser elements to surface this event, so that the gaia system could listen for it, as it can for the other mozbrowser events:

https://developer.mozilla.org/en-US/docs/WebAPI/Browser#Events

Comment 1

[Justin Lebar (not reading bugmail)]

I guess really all we need is a way for the page to block onload until it's "ready".  I wonder if we can do that now...

Comment 2

[Boris Zbarsky [:bzbarsky]]

We don't expose explicit APIs for it to JS; you'd have to do some sort of icky XHR polling or something.  Or document.open a subframe and then document.close() when done.  Seems pretty hacky.

Exposing explicit APIs for delaying onload to web content is a bit weird, since outside the FxOS context I'm not sure there are any reasonable use cases...

Does any of that answer the question that was asked?  I have to admit I'm not clear what I'm being asked.  ;)

Comment 3

[James Burke [:jrburke]]

About the original request, some way for the web app's JS to control a signal indicating the web app has rendered. The existing signals that indicate the page is "ready" are DOMContentLoaded/document.readyState and the window load event.

Those events and states are reached when the platform detects the state of the HTML document. However, with app JS, it is common for the HTML document to reach DOMContentLoaded and even window loaded but not actually be usable yet, and it is not clear what the platform could use to know that the app is usable without some signals from the app.

For Gaia/Firefox OS systems, this leads to users seeing a blank white screen as the app finishes loading, and it complicates Gaia screenshot heuristics. Right now, the screenshot is taken during this white flash phase. So not a great screenshot, and that screenshot logic takes time away from the app in its load phase, prolonging loading.

I am open to any way to do to support the app signaling the load state. Here is a sketch to help stimulate discussion, one that is different from the previous CustomEvent suggestion:

There could be a new document property, prefixed with "moz" for now. Just picking names for illustration purposes:

document.mozAppState that can have the following values:

* undefined: means app state is not being tracked by the web page (the default for all existing HTML documents in circulation now).
* 'loading': the app is in the loading state.
* 'complete': the app has completed loading is is ready to use.

The app developer indicates they want to use app state tracking by setting mozappstate="loading" on the HTML element (document.documentElement). This attribute must be set before DOMContentLoaded if not already in the HTML source.

If mozappstate attribute is not set by DOMContentLoaded, then document.mozAppState is not being tracked, and will return undefined.

When the app JS decides the HTML document is ready for use, it sets mozappstate="complete" on an HTML element in the DOM. When this DOM element has finished loading (all sub elements have loaded, like images, background CSS), then document.mozAppState is updated to 'complete' by the platform, and a "mozappload" event is dispatched by the platform. This event can be listened to from the mozbrowser-enabled elements similar to these events:

https://developer.mozilla.org/en-US/docs/WebAPI/Browser#Events

The mozappstate="complete" can be set on any DOM element, not just the HTML element. This is to allow signalling what element is considered the main app element that should be loaded to consider the app usable. The app may have other, hidden elements used for background tasks that do not need to be fully loaded.

"mozappstate="loading" can only be set on the documentElement.

This capability is opt-in, and containers that hold browser/mozbrowser elements, like Firefox or the Gaia system app, have a way to determine if it is in play:

Listen for DOMContentLoaded. When that event or state is reached, check document.mozAppState. If 'loading' or 'complete', then it is an app that is signalling app state. If undefined, then it is a traditional HTML document where DOMCContentLoaded/onload are enough to signal state.

Comment 4

[Anne (:annevk)]

I think something like document.delayOnloadWith(Future future) is a better API for this problem. Does not exactly address the concern bz had though with respect to the use case being fairly specific to Firefox OS.

Comment 5

[James Burke [:jrburke]]

I was hoping to get something before Futures were worked out and landed. Maybe there is some alternate that would work both before and after that happens?

Although, marking what element should be considered as the appload trigger seems to have value: by the platform detecting when the element and its children finish loading, this is more robust than just trying to do it in just the app's JS, since IIRC, that state of an element is not available to JS now. 

As it is now, the JS could trigger its own event (or fulfill a Future) once the DOM element is inserted, but if there are images/iframes, CSS background images to load, it would be difficult for the JS to detect when it is all finished loading?

As for broader applicability, I expect it could be used for any app-based browser usage, including chrome packaged apps, even web-only apps in iOS. It would avoid the user seeing a white page flash as the web app loaded -- the browser container could wait for the appload event before showing the live state of the page. 

Same for web pages that just want to manage their own "splash screen" and listen internally for the "appload" event, games in particular.

Comment 6

[Robert O'Callahan (:roc) (email my personal email if necessary)]

I think this use-case goes beyond FirefoxOS. For example, desktop Firefox supports thumbnailing. It would be nice to be able to avoid taking thumbnails of incompletely loaded applications. I think it would also make sense for browsers that display some kind of "page loading" feedback to keep displaying that UI after the document load event if the app tells us it's not complete yet.

In reftests we define a special class for the root element, <html class="reftest-wait">. The test removes this class when it's finished. This works when the reftest document is running its own process; script running in the content process listens for the mutation and signals the parent process using MessageManager. I think we could do that here too. E.g., define a 'loading' attribute for the root element, let the app specify <html loading>, and remove the attribute when done. I don't know much about the multiprocess setup on B2G but I assume there's a way to inject script into the child process. That script detect the presence of the attribute and poll or use a mutation listener to detect its removal.

I think an approach like Anne suggested in comment #4 may be better though. I don't see a need to use futures here. All we need is a way for the application to delay the document load event programmatically. You could actually kind of hack this today by putting a hidden image in the document that requests a URL that never finishes loading, and then cancelling the load (e.g. by clearing src) when you want to allow the document load event to fire. A less hacky approach would be a new JS API, e.g. document.delayLoadEvent() and document.stopDelayingLoadEvent().

Comment 7

[James Burke [:jrburke]]

(In reply to Robert O'Callahan (:roc) (Mozilla Corporation) from comment #6)
> A less hacky approach would be a new JS API, e.g.
> document.delayLoadEvent() and document.stopDelayingLoadEvent().

My main concern with a JS API was making sure nodes inserted dynamically by the app JS were completely loaded before the final "load" event was triggered, even if they finished loading/rendering after document.stopDelayingLoadEvent() was called. 

It sounds like that will be the case, so a JS API works for me. 

Anne, any thoughts on document.delayLoadEvent() and document.stopDelayingLoadEvent()?

Comment 8

[Robert O'Callahan (:roc) (email my personal email if necessary)]

(In reply to James Burke [:jrburke] from comment #7)
> My main concern with a JS API was making sure nodes inserted dynamically by
> the app JS were completely loaded before the final "load" event was
> triggered, even if they finished loading/rendering after
> document.stopDelayingLoadEvent() was called. 
> 
> It sounds like that will be the case, so a JS API works for me. 

Yes. Many things, including loads of subresources, block the document load event. Adding a new thing that blocks the document load event (such as the JS API here) would not cause the load event to fire any earlier.

Comment 9

[Anne (:annevk)]

delayLoad = true | false. Fails to be set (silently) to true if load is already dispatched or a task to dispatch load has already been queued. Once successfully set to true "delays the load event" (see HTML) and then once set to false no longer does that (load will always dispatch from a task so not immediately afterwards).

Comment 10

[Robert O'Callahan (:roc) (email my personal email if necessary)]

With that approach, only one script on the page can use the feature at a time. It might be helpful to support independent scripts independently delaying the load event.

Comment 11

[Anne (:annevk)]

Oh, the semantics of your API are to create a stack? That was not clear to me. Using futures seems somewhat cleaner to me if you want to support that.

Comment 12

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Yeah, multiple calls to document.delayLoadEvent() would be balanced by the same number of calls to document.stopDelayingLoadEvent().

I don't see how futures are relevant here. There are no results to be computed, and nothing cares about when a particular delayLoadEvent() is balanced by its corresponding stopDelayingLoadEvent(). Any observers only care about when the document's load event is eventually fired.

Comment 13

[Anne (:annevk)]

Okay. If we go ahead with this someone please ping [e-mail] / [e-mail] or file a bug on WHATWG HTML that we're doing this. (I can be that person if nobody else volunteers.)

Comment 14

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Done.

Comment 15

[Boris Zbarsky [:bzbarsky]]

What roc is proposing isn't a stack; it's just a counter.

Comment 16

[Justin Lebar (not reading bugmail)]

> I think we could do that here too. E.g., define a 'loading' attribute for the root element, let 
> the app specify <html loading>, and remove the attribute when done. I don't know much about the 
> multiprocess setup on B2G but I assume there's a way to inject script into the child process. That 
> script detect the presence of the attribute and poll or use a mutation listener to detect its 
> removal.

Just FYI, this is totally doable in b2g.  The OOP frame has a frame script injected into it that runs BrowserElementChild.js, and we can do whatever from there.

Comment 17

[James Burke [:jrburke]]

Some follow up links for this:

roc started a discussion on whatwg here:

http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2013-April/039422.html

and I just started another thread (sorry, joined whatwg list today) here trying to summarize the previous thread's feedback, in an effort to move the conversation along:

http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2013-May/039531.html

I am also willing to help do an experimental gaia branch that uses this capability if/when there is a prototype/patch that may work with b2g's v1 train / mozilla-b2g18 code.

Comment 18

[James Burke [:jrburke]]

Attachment: Start of a patch for a mozDelayLoadEvent and mozStopDelayingLoadEvent

This is a speculative patch for gecko to provide a document.mozDelayLoadEvent and document.mozStopDelayingLoadEvent. It was done against the mozilla-b2g18 tree.

I will attach 5 tests after attaching this patch. 4 of 5 pass. The last one deals with triggering onload if there is an exception in the page. I am not familiar enough with the code to know how when in nsDocument.cpp (or maybe something to do with the loadGroup) gets the error notification. The hope was that I could find that point, and if mDelayLoadCount is greater than 0, manually set it to 0 and call UnblockOnload(true).

I place good odds on that approach (even the full patch) being completely incorrect though. This is my first attempt at modifying gecko code, so very green. Pointers appreciated, as well as suggestions on who might be best ask for a review. I also will be content if someone else would prefer to do this work.

Comment 19

[James Burke [:jrburke]]

Attachment: mochitest, content/base/test/test_bug863499-1.html

Comment 20

[James Burke [:jrburke]]

Attachment: mochitest, content/base/test/test_bug863499-2.html

Comment 21

[James Burke [:jrburke]]

Attachment: mochitest, content/base/test/test_bug863499-3.html

Comment 22

[James Burke [:jrburke]]

Attachment: mochitest, content/base/test/test_bug863499-4.html

Comment 23

[James Burke [:jrburke]]

Attachment: mochitest, content/base/test/test_bug863499-5.html

Comment 24

[:Eli Perelman]

Sorry to revive an old thread, but maybe we can make some progress on this still.

Currently all of the Gaia applications emit performance markers to denote their loading progress, similar to what James had proposed, but instead using the User Timing API:

performance.mark('navigationLoaded')
application designates that its core chrome or navigation interface exists in the DOM and you have marked it as ready to be displayed

performance.mark('navigationInteractive')
application designates that the core chrome or navigation interface has its events bound and is ready for user interaction

performance.mark('visuallyLoaded')
application designates that it is visually loaded, e.g. the "above-the-fold" content exists in the DOM and you have marked it as ready to be display, again not display: none; or other hiding functionality

performance.mark('contentInteractive')
application designates that it has bound the events for the minimum set of functionality to allow the user to interact with the "above-the-fold" content made available at visuallyLoaded

performance.mark('fullyLoaded')
application has been completely loaded, for example any relevant "below-the-fold" functionality has been injected into the DOM, been marked visible, and is ready for user interaction. Any required startup background processing should be complete and should exist in a stable state barring any further user interaction

---

Since there is this established convention, and all performance entries are stored in Gecko for a particular window context, I guess the missing piece is still to expose these entries to the platform or System in a meaningful and actionable way. Thoughts?

Comment 25

[Zibi Braniecki [:zbraniecki][:gandalf]]

Yeah, James' original request from Comment 1 can be now accomplished using the marks:


> * The gaia system app could listen for this signal and provide better transitions from app screenshots to running app (avoiding "its a web page" white flash during initial load)

that should happen at navigationLoaded or visuallyLoaded (depending on how dynamic the content is - for example SMS app should probably transition at navigationLoaded so that SMSes are not part of the screenshot)

> * The gaia system could also use it to indicate when a screenshot could be taken, without it happening during the time the app needs to finish the load, allowing app startup to complete faster.

Same here. For Settings the screenshot can be taken at visuallyLoaded, but for SMS navigationLoaded would be better.

> * For Gaia metrics purposes, this event could be used to track how long apps actually take to load to be usable

We use all five marks in raptor! fullyLoaded is also used to mark when we measure memory usage.

Comment 26

[:Eli Perelman]

In bug 1177226 there is work going on to expose performance entries to the System app for use within the Developer HUD, but you could probably use this for other purposes as well, e.g. have the System delay making the app visible until it triggered visuallyLoaded.

Comment 27

[Zibi Braniecki [:zbraniecki][:gandalf]]

The latter is being discussed here: https://groups.google.com/d/msg/mozilla.dev.platform/F3Mp6dZonMA/r2giTAlSIwAJ

We may want to use a separate event because not every app will want to go for visuallyLoaded before bringing it to foreground (some apps may be loading content for a while, but want to be displayed when chrome is ready).

Comment 28

[BMO Automation]

Firefox OS is not being worked on