1311448 - [telemetry-experiment] layout.show_previous_page on aurora channel

Status: NEW

(Core :: Layout, defect, P3)

Description

[:Harald Kirschner :digitarald]

Same experiment setup as bug 1306125 but on aurora with a focus on testing stability:

Hypothesis: Flipping the prefs layout.show_previous_page and/or nglayout.initialpaint.delay does not lead to higher crashes.

Comment 1

[:Harald Kirschner :digitarald]

Attachment: bug-1311448-experiment.patch

Previous patch, adapted from beta to aurora to run 2 weeks, starting Oct 24th.

Comment 2

[:Harald Kirschner :digitarald]

Andre: You brought up QA aspects on the mailing list. Does it sound reasonable to not block this experiment for Aurora on QA but to do more in-depth QA before we run the experiment on Beta?

Some thoughts on what to QA for: We don't expect massive rendering issues, as we only clear the page earlier to paint the background color if we have it. The combo of the 2 presets might cause visual jitter in a timeframe of 250ms, but after that it should behave similar to what we have now. The hypothesis is that this visual noise is a loading indicator that actually gives users a sense of progress compared to delaying painting.

Comment 3

[Andrei Vaida [:avaida]]

Thank you for clarifying things here, Harald. I've just followed up with Dominik on this via email, we should be able to provide a reasonable manual test coverage for this experiment by Wednesday, Oct 26th.

Paul and I worked on a testing approach [1] based on the feedback we received from you and Dominik, but if you feel we're not on the right path, don't hesitate to reach out to us.

[1] https://public.etherpad-mozilla.org/p/1311448

Comment 4

[:Felipe Gomes (needinfo for replies!)]

Comment on attachment 8802595 [details] [diff] [review]
bug-1311448-experiment.patch

Review of attachment 8802595 [details] [diff] [review]:
-----------------------------------------------------------------

::: experiments/show_previous_page-aurora/install.rdf
@@ +18,5 @@
> +
> +    <!-- Front End MetaData -->
> +    <em:name>Faster Page to Page Navigation (Aurora)</em:name>
> +    <em:description>Testing the effect of disabling paint suppression</em:description>
> +    <em:aboutURL>https://bugzilla.mozilla.org/show_bug.cgi?id=1306125</em:aboutURL>

update bug number

::: experiments/show_previous_page-aurora/manifest.json
@@ +2,5 @@
> +  "publish"     : true,
> +  "priority"    : 2,
> +  "name"        : "Faster Page to Page Navigation (Aurora)",
> +  "description" : "Testing the effect of disabling paint suppression",
> +  "info"        : "<p><a href=\"https://bugzilla.mozilla.org/show_bug.cgi?id=1306125\">Related bug</a></p>",

update bug number

Comment 5

[:Harald Kirschner :digitarald]

Attachment: bug-1311448-experiment-r2.patch

Fixed incorrect bug references

Comment 6

[:Dominik Strohmeier [:dstrohmeier]]

Paul, Andre: I saw some smaller issues that you identified from QA on the etherpad and can also confirm that I was able to reproduce them. To me, these issues are exactly the reason why we want to run the experiment, e.g. the reflow that happens on the one page mentioned at the end of the etherpad.

What's your take on this? IMO, we shouldn't have these issues block the experiment - these issues are expected and we want to get data about how often they appear.

Comment 7

[Paul Silaghi, QA [:pauly]]

Hi Dominik,
Thanks for the feedback.
We completed our testing on the experiment - https://public.etherpad-mozilla.org/p/1311448.
Although the issues we found should not be considered blockers for this experiment (the plan is, if I'm not mistaken, for this experiment to ride the trains to beta 51), the behavior they depict is not ideal and it can be easily perceived as a bad user experience. From QA perspective, we can go ahead with it and ship it to beta 51, but at this point we wouldn't advise shipping this further to release.
Please let me know if you want the potential issues to be filed separately in Bugzilla.
I've also replied on the initial email thread with our results, to keep everyone in the loop.

Comment 8

[:Dominik Strohmeier [:dstrohmeier]]

Felipe, can you help me to deploy this experiment on Aurora now that we are at the beginning of the train? It was approved in the Intend to Ship email thread by Gerry Chang on Oct 27. Can you help me figure out what I still need to do to get this into production? Thanks!

Comment 9

[:Felipe Gomes (needinfo for replies!)]

Hi!  Is there already a signed xpi generated for this?  If no, we need to get that. If there's already a signed xpi, the next step is to commit the patch, along with the xpi, in the repo here: http://hg.mozilla.org/webtools/telemetry-experiment-server/

After that's done I'll guide through the final part of tagging the release and publishing it.

Comment 10

[:Felipe Gomes (needinfo for replies!)]

Ah, note: the manifest.json file in the patch needs to be updated as Aurora is now version 52 and not 51

Comment 11

[:Dominik Strohmeier [:dstrohmeier]]

Harald, can you please look into Felipe's comments? Do we already have the signed xpi generated?

Comment 12

[:Harald Kirschner :digitarald]

:asa, just to make sure to have that pref on the list for pref-flipping experiments

Comment 13

[Asa Dotzler [:asa]]

got it.

Comment 14

[BugBot [:suhaib / :marco/ :calixte]]

The bug assignee is inactive on Bugzilla, so the assignee is being reset.