Same experiment setup as bug 1306125 but on aurora with a focus on testing stability: Hypothesis: Flipping the prefs layout.show_previous_page and/or nglayout.initialpaint.delay does not lead to higher crashes.
Created attachment 8802595 [details] [diff] [review] bug-1311448-experiment.patch Previous patch, adapted from beta to aurora to run 2 weeks, starting Oct 24th.
Attachment #8802595 - Flags: review?(felipc)
Andre: You brought up QA aspects on the mailing list. Does it sound reasonable to not block this experiment for Aurora on QA but to do more in-depth QA before we run the experiment on Beta? Some thoughts on what to QA for: We don't expect massive rendering issues, as we only clear the page earlier to paint the background color if we have it. The combo of the 2 presets might cause visual jitter in a timeframe of 250ms, but after that it should behave similar to what we have now. The hypothesis is that this visual noise is a loading indicator that actually gives users a sense of progress compared to delaying painting.
Assignee: nobody → dstrohmeier
QA Contact: andrei.vaida
Thank you for clarifying things here, Harald. I've just followed up with Dominik on this via email, we should be able to provide a reasonable manual test coverage for this experiment by Wednesday, Oct 26th. Paul and I worked on a testing approach  based on the feedback we received from you and Dominik, but if you feel we're not on the right path, don't hesitate to reach out to us.  https://public.etherpad-mozilla.org/p/1311448
QA Contact: andrei.vaida → paul.silaghi
Comment on attachment 8802595 [details] [diff] [review] bug-1311448-experiment.patch Review of attachment 8802595 [details] [diff] [review]: ----------------------------------------------------------------- ::: experiments/show_previous_page-aurora/install.rdf @@ +18,5 @@ > + > + <!-- Front End MetaData --> > + <em:name>Faster Page to Page Navigation (Aurora)</em:name> > + <em:description>Testing the effect of disabling paint suppression</em:description> > + <em:aboutURL>https://bugzilla.mozilla.org/show_bug.cgi?id=1306125</em:aboutURL> update bug number ::: experiments/show_previous_page-aurora/manifest.json @@ +2,5 @@ > + "publish" : true, > + "priority" : 2, > + "name" : "Faster Page to Page Navigation (Aurora)", > + "description" : "Testing the effect of disabling paint suppression", > + "info" : "<p><a href=\"https://bugzilla.mozilla.org/show_bug.cgi?id=1306125\">Related bug</a></p>", update bug number
Attachment #8802595 - Flags: review?(felipc) → review+
Created attachment 8804325 [details] [diff] [review] bug-1311448-experiment-r2.patch Fixed incorrect bug references
Attachment #8804325 - Flags: review?(felipc) → review+
Paul, Andre: I saw some smaller issues that you identified from QA on the etherpad and can also confirm that I was able to reproduce them. To me, these issues are exactly the reason why we want to run the experiment, e.g. the reflow that happens on the one page mentioned at the end of the etherpad. What's your take on this? IMO, we shouldn't have these issues block the experiment - these issues are expected and we want to get data about how often they appear.
Hi Dominik, Thanks for the feedback. We completed our testing on the experiment - https://public.etherpad-mozilla.org/p/1311448. Although the issues we found should not be considered blockers for this experiment (the plan is, if I'm not mistaken, for this experiment to ride the trains to beta 51), the behavior they depict is not ideal and it can be easily perceived as a bad user experience. From QA perspective, we can go ahead with it and ship it to beta 51, but at this point we wouldn't advise shipping this further to release. Please let me know if you want the potential issues to be filed separately in Bugzilla. I've also replied on the initial email thread with our results, to keep everyone in the loop.
Felipe, can you help me to deploy this experiment on Aurora now that we are at the beginning of the train? It was approved in the Intend to Ship email thread by Gerry Chang on Oct 27. Can you help me figure out what I still need to do to get this into production? Thanks!
Hi! Is there already a signed xpi generated for this? If no, we need to get that. If there's already a signed xpi, the next step is to commit the patch, along with the xpi, in the repo here: http://hg.mozilla.org/webtools/telemetry-experiment-server/ After that's done I'll guide through the final part of tagging the release and publishing it.
Ah, note: the manifest.json file in the patch needs to be updated as Aurora is now version 52 and not 51
Harald, can you please look into Felipe's comments? Do we already have the signed xpi generated?
:asa, just to make sure to have that pref on the list for pref-flipping experiments
Flags: needinfo?(hkirschner) → needinfo?(asa)
You need to log in before you can comment on or make changes to this bug.