1468271 - Track Interactive Example time to load

Status: RESOLVED FIXED

(developer.mozilla.org Graveyard :: Performance, enhancement, P1)

Description

[John Whitlock [:jwhitlock]]

What problem would this feature solve?
======================================
A reference page is ready when the interactive example is loaded and interactive. We'd like to track how long this takes for real users of MDN.

Who has this problem?
=====================
Staff contributors to MDN

How do you know that the users identified above have this problem?
==================================================================
We can see visually that the interactive examples iframe is "ready", according to the MDN page, for a few seconds before the content inside the iframe is loaded.

How are the users identified above solving this problem now?
============================================================
We're guessing how long it takes for the interactive example to load.

Do you have any suggestions for solving the problem? Please explain in detail.
==============================================================================
A solution is written in https://github.com/mozilla/kuma/pull/4846:

The Performance API [1] can be used to get a PerformanceTiming instance [2] from the iframe, and detect when it is loaded. This can be passed via an event to the MDN page. It can then be sent to Google Analytics as a custom performance event, and made available for other tools like SpeedCurve synthetics.

[1] https://developer.mozilla.org/en-US/docs/Web/API/Performance
[2] https://developer.mozilla.org/en-US/docs/Web/API/PerformanceTiming

Is there anything else we should know?
======================================

Comment 1

[[github robot]]

Commit pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/ce258719e0c3ded2df80e50599499d0b2796a25e
bug 1468271: Add perf metrics for int. examples

Expand interactive examples message handler to expect 'Performance
Events', to measure timing events such as when the iframe is fully
loaded.

Comment 2

[[github robot]]

Commit pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/a46d3c4fa28d0f1bf29ea2bf8f5cc3bc0b0e5313
bug 1468271: Change to unique hit events

Convert from timing to hit event types, and use a timestamp so that each event will be (mostly) unique.

See https://github.com/mdn/sprints/issues/143

Comment 3

[[github robot]]

Commit pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/dae599a2ebbbb300614dfe7d93f23ecebfe99d82
bug 1468271, add handler for additional post messages from interactive examples, and a refactor

Comment 4

[[github robot]]

Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/281fc367998b239c2b47210fc7cff3530adcb3bb
add handler for interactive examples load end event, bug 1468271

https://github.com/mozilla/kuma/commit/fe247e8142df692f36a834cd0ae5ab4b9b574437
Merge pull request #4873 from schalkneethling/add-handler-for-ie-load-event-end

add handler for interactive examples load end event, bug 1468271

Comment 5

[John Whitlock [:jwhitlock]]

Remaining issue is that the "interactive-editor-loading" mark is sometimes not available when loading is complete, so the user timing measures focusing on the interactive examples results in errors. More investigation is needed to determine if there is an avoidable bug. Re-assigning to me for investigation.

Comment 6

[John Whitlock [:jwhitlock]]

Looking at Google Analytics, it may be a Chrome-specific issue.

For 99.77% of the errors, Google Chrome on the desktop is the browser. About 50% appear to be on the current version (68), 40% on the previous version (67).  Chrome is represented in the successful timing examples, at about the same proportion as site visitors. There are about 40x more errors than timing samples in a day.

The error happens to both new and returning users, but to more returning users than expected by random sampling.

These don't appear to be significant but instead look like a random sampling of our typical traffic:

* MDN page
* Operating System
* Session duration
* Screen size
* Language
* Country

Comment 7

[John Whitlock [:jwhitlock]]

I can get this error pretty reliably by reloading a page with an interactive example in Chrome, like https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/forEach. It doesn't happen every time, but at least 1 out of 3 times. I can't get it to happen in Firefox with repeated reloads.

The JavaScript that sets the interactive-editor-loading mark is in the <head> of the interactive example. It's the minified version of perf.js:

https://github.com/mdn/interactive-examples/blob/master/js/editor-libs/perf.js

That code does three things:
1. Defines function postToKuma
2. Calls postToKuma({ markName: 'interactive-editor-loading' }) (The failing code)
3. Adds a 'readystatechange' listener that fires later postToKuma calls

Parts 1. and 3. run, because 3 uses 1, and is the event that leads to the failed measurement due to the 'interactive-editor-loading' mark not existing.

A few theories:

1. Chrome is skipping the call. This is possible if the JS optimizer recognizes the javascript and decides it does nothing.
2. window.parent is wrong. If you call this outside of an iframe, you get yourself. But it is right later...

Looking at the output, the uglify-es minifier is replacing the semicolons with commas, turning it into a sequence. This may be acceptable, or it may be hinting to the optimizer to skip calling the function next time. I'll like to see if a semicolon works more reliably.

Comment 8

[John Whitlock [:jwhitlock]]

The semicolon trick didn't change the results. I got the interactive-examples server talking to my Kuma development instance so I could do some more debugging, and I think I've found the cause.

MDN's message event listener is in wiki.js, post-message-handler.js in the source:

https://github.com/mozilla/kuma/blob/3453d1a436fa00306972a62a06049ef39ec27aaa/kuma/static/js/utils/post-message-handler.js#L106-L111

If the iframe is loaded before wiki.js is processed, then the message from the iframe goes nowhere. The `interactive-editor-loading` mark is not created, and the later measure fails.

If wiki.js is loaded before the iframe is processed, then the message handler is ready for the message to set an `interactive-editor-loading` mark.

The solution would be to include the post-message-handler.js in the <head>, probably inline with the Google Analytics setup, so that it is ready to go when the iframe is loaded.

Comment 9

[[github robot]]

Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/cc217e235448cc5c1a475c520ee32d69012a34a7
Fix Bug 1468271, make ie performance metrics more reliable

https://github.com/mozilla/kuma/commit/26b889c5bfd7d445bf96ba37f14d776e46d48d32
Merge pull request #4935 from schalkneethling/bug1468271-make-ie-performance-metrics-more-reliable

Fix Bug 1468271, make ie performance metrics more reliable