1225064 - Add a firstpaint marker for raptor

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Gaia::System, defect)

Description

[vnicolas]

As of today many apps use visuallyLoaded as an optimization target.
This is also what is used to determine if an app is in an acceptable state for shipping (<1s).

While visuallyLoaded is useful I suggest that 'firstpaint' *after* 'visuallyLoaded' is used instead.

Why ?
  - Even if application optimize visuallyLoaded by 500ms, it does not mean any changes is perceivable by the user.
 - If is important to takes painting into account. There are some cases where an app can not land an enhancement because raptor says visuallyLoaded has regressed, while firstpaint is clearly better.

Why *after* visuallyLoaded ?
  - Some application fires a firstpaint before visuallyLoaded. Based on what I can see on Gaia, it means the app is painted, but it was not really ready to be displayed and the UI may change while the user interact with the app - which is extremely annoying.

Comment 1

[vnicolas]

Basically the way I see that is to add a performance.mark('firstpaint@' + (new URL(this.origin).origin); in a mozbrowserdocumentfirstpaint handler for appWindow.

Ideally a mozbrowserremotedocumentfirstpaint based on MozAfterRemotePaint will be better as it fires something around 20ms before mozbrowserdocumentfirstpaint and it really represent that the compositor has tried to composite the app, but this could be done in a followup as this needs some changes in BrowserElementParent.js.

Comment 2

[vnicolas]

Eli, I seriously think visuallyLoaded, while beeing useful, is busted as the main performance metric people should look at.

You say:

"The important moment is visually loaded. This correlates directly with what the user perceives as “being ready.” As an added bonus, using the visuallyLoaded metric pairs nicely with camera-based performance verifications."

Maybe in the past visuallyLoaded was correlating nicely with camera-based performance verifications, but it differs greatly from what I see. So things may have changed, and if you want to match what the user perceive (which is great), let's use firstpaint.

Comment 3

[vnicolas]

We also need to fix raptor-cli itself. As sometimes firstpaint happens way before fullyLoaded and raptor ignore such entries...

Comment 4

[:Eli Perelman]

So my thoughts here:

The reason why we went with visuallyLoaded at the time was because we didn't seem to have a way to know when to relate the visuallyLoaded moment with the actual paint. In MDN [1] the definition for visuallyLoaded is specifically phrased to account for this deficiency (caps for emphasis):

> Emit this when your application DESIGNATES that it is visually loaded, e.g. the "above-the-fold" content exists in the DOM and you have MARKED it as ready to be display...

So visuallyLoaded is not exactly congruent to be timed exactly with a paint. Also, visuallyLoaded is only supposed to represent when it appears ready to interact with, but it is truly ready to interact at contentInteractive.

That said, if it is possible to trigger a first-paint marker after visuallyLoaded, I believe that has even greater value. We currently cannot abandon visuallyLoaded just yet as it is our branch baseline comparator, we can certainly change the goals around a new marker. We will discuss that further once this new marker becomes a possibility.

Regarding your statement about missing this entry, any *markers* between appLaunch and fullyLoaded for an app should be picked up, no matter how early in the launch cycle. If that doesn't seem to be the case, please ping me or file a bug in PerformanceTest and I'll look into it.

[1] https://developer.mozilla.org/en-US/Apps/Build/Performance/Firefox_OS_app_responsiveness_guidelines#Goals

Comment 5

[vnicolas]

(In reply to :Eli Perelman from comment #4)
> Regarding your statement about missing this entry, any *markers* between
> appLaunch and fullyLoaded for an app should be picked up, no matter how
> early in the launch cycle. If that doesn't seem to be the case, please ping
> me or file a bug in PerformanceTest and I'll look into it.

What I meant is that, for some apps, firstpaint fires after fullyLoaded. And in this case it is ignored (which makes sense based on the code of raptor-cli. But in order to support this use case, we would need to fix raptor-cli).

Comment 6

[vnicolas]

(In reply to vnicolas from comment #5)
> (In reply to :Eli Perelman from comment #4)
> > Regarding your statement about missing this entry, any *markers* between
> > appLaunch and fullyLoaded for an app should be picked up, no matter how
> > early in the launch cycle. If that doesn't seem to be the case, please ping
> > me or file a bug in PerformanceTest and I'll look into it.
> 
> What I meant is that, for some apps, firstpaint fires after fullyLoaded. And
> in this case it is ignored (which makes sense based on the code of
> raptor-cli. But in order to support this use case, we would need to fix
> raptor-cli).

Basically I did a mistake in comment #3. I meant /after/ instead of /before/ :/

Comment 7

[:Eli Perelman]

Ah, I see. If you need a temporary workaround, you can fire a performance.measure instead of performance.mark.

Comment 8

[BMO Automation]

Firefox OS is not being worked on