Closed Bug 1341259 Opened 3 years ago Closed 3 years ago

Run a Telemetry experiment to enable Intersection Observer API

Categories

(Core :: DOM: Core & HTML, defect, P3)

53 Branch
defect

Tracking

()

RESOLVED FIXED
Tracking Status
firefox51 --- wontfix
firefox52 --- wontfix
firefox53 --- wontfix
firefox54 --- affected

People

(Reporter: tschneider, Assigned: tschneider)

References

Details

Attachments

(6 files, 11 obsolete files)

11.46 KB, patch
Details | Diff | Splinter Review
8.34 KB, patch
Details | Diff | Splinter Review
1.83 KB, patch
Details | Diff | Splinter Review
1.40 KB, application/zip
Details
5.30 KB, application/x-xpinstall
Details
9.97 KB, patch
Details | Diff | Splinter Review
We want to A/B test enabling the Intersection API t on all platforms. The idea is that Nightly users who qualify will have the API enabled (via pref dom.IntersectionObserver.enabled); We want to switch half of these users to compare the stability between the two different groups.

Proposed Criteria:

>      Product: Firefox Nightly
>      Version: 54.0a1
>           OS: all
>      Channel: nightly
>     Build ID: 2017-02-23+
>     Language: en-US
> Sample ratio: 50%
>   Start Date: February 23, 2017
>     End Date: March 9, 2017
Attached file experiment.xpi (obsolete) —
Assignee: nobody → tschneider
Priority: -- → P3
Felipe, any more information needed to get this started?
Flags: needinfo?(felipc)
Addressed comments on IRC.
Attachment #8839438 - Attachment is obsolete: true
Attached file experiment.xpi (obsolete) —
Updated add-on.
Attachment #8839439 - Attachment is obsolete: true
Looks good. You should get the xpi signed and do some manual testing (and of course get approval from release-drivers to run the experiment) before landing.

After you do some manual testing I'll give r+ and we can push to staging to do a quick QA and them push to prod.
Flags: needinfo?(felipc)
Jason, can you please sign the add-on for me?
Flags: needinfo?(jthomas)
Attached file experiment.xpi signed (obsolete) —
Please see attached.
Flags: needinfo?(jthomas)
Manually tested everything and can confirm it works. Still waiting for release-drivers approval.
Felipe, can you please give your r+ so we can land this as soon as I heard back from release-drivers?
Flags: needinfo?(felipc)
Comment on attachment 8840147 [details] [diff] [review]
intersection-observer-nightly54.diff

Review of attachment 8840147 [details] [diff] [review]:
-----------------------------------------------------------------

Sure. Actually, you can land the patch now, as we won't update the release_tag yet. That will put it in the staging server (https://telemetry-experiment-dev.allizom.org/) where we can ask QA to test it (usually a requirement to release an experiment.. not sure if that applies to Nightly but it wouldn't hurt to do some simple testing).
Attachment #8840147 - Flags: review+
Flags: needinfo?(felipc)
Final patch to land. Includes signed add-on.
Attached patch Manifest (obsolete) — Splinter Review
Updated manifest with new dates and sample rate.
Attached patch Manifest (obsolete) — Splinter Review
One last change to the manifest. Now includes nightly version 55a1, to avoid the experiment being disabled for some people after merge today.
Attachment #8843376 - Attachment is obsolete: true
Attached file experiment.xpi (obsolete) —
Updated add-on.
Attachment #8840149 - Attachment is obsolete: true
Attachment #8840496 - Attachment is obsolete: true
jason, can you please sign the add-on on more time? Thanks.
Flags: needinfo?(jthomas)
Attached file experiment.xpi signed (obsolete) —
Please see attached.
Flags: needinfo?(jthomas)
Updated patch with new signed add-on.
Attachment #8840147 - Attachment is obsolete: true
Attachment #8844049 - Attachment is obsolete: true
Running the experiment on 10% of the Nightly user population showed no stability issues. Lets do a final run on 50% before shipping.
Created a new copy of the experiment to make sure we get enough new activations.
Attachment #8848251 - Attachment is obsolete: true
Depends on: 1348144
Attached file experiment.xpi
Updated add-on to target FF 55.0a1. Jason, please sign.
Attachment #8844054 - Attachment is obsolete: true
Attachment #8844130 - Attachment is obsolete: true
Flags: needinfo?(jthomas)
Attached file experiment.xpi signed
Please see attached.
Flags: needinfo?(jthomas)
Running the experiment on 10% and 50% of the Nightly user population showed no stability issues. Lets enable it to 100% before shipping.
Correct patch this time.
Attachment #8852208 - Attachment is obsolete: true
Felipe, can you please land the latest patch?
Flags: needinfo?(felipc)
Depends on: 1351837
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Component: DOM → DOM: Core & HTML
You need to log in before you can comment on or make changes to this bug.