Run a Telemetry experiment to enable Intersection Observer API

NEW
Assigned to

Status

()

Core
DOM
P3
normal
2 months ago
28 days ago

People

(Reporter: tobytailor, Assigned: tobytailor)

Tracking

53 Branch
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox51 wontfix, firefox52 wontfix, firefox53 wontfix, firefox54 affected)

Details

Attachments

(6 attachments, 11 obsolete attachments)

11.46 KB, patch
Details | Diff | Splinter Review
8.34 KB, patch
Details | Diff | Splinter Review
1.83 KB, patch
Details | Diff | Splinter Review
1.40 KB, application/zip
Details
5.30 KB, application/x-xpinstall
Details
9.97 KB, patch
Details | Diff | Splinter Review
(Assignee)

Description

2 months ago
We want to A/B test enabling the Intersection API t on all platforms. The idea is that Nightly users who qualify will have the API enabled (via pref dom.IntersectionObserver.enabled); We want to switch half of these users to compare the stability between the two different groups.

Proposed Criteria:

>      Product: Firefox Nightly
>      Version: 54.0a1
>           OS: all
>      Channel: nightly
>     Build ID: 2017-02-23+
>     Language: en-US
> Sample ratio: 50%
>   Start Date: February 23, 2017
>     End Date: March 9, 2017
(Assignee)

Comment 1

2 months ago
Created attachment 8839438 [details] [diff] [review]
intersection-observer-nightly54.diff
(Assignee)

Comment 2

2 months ago
Created attachment 8839439 [details]
experiment.xpi
Assignee: nobody → tschneider
Priority: -- → P3
Blocks: 1243846
status-firefox51: --- → wontfix
status-firefox52: --- → wontfix
status-firefox53: --- → wontfix
status-firefox54: --- → affected
(Assignee)

Comment 3

2 months ago
Felipe, any more information needed to get this started?
Flags: needinfo?(felipc)
(Assignee)

Comment 4

2 months ago
Created attachment 8840147 [details] [diff] [review]
intersection-observer-nightly54.diff

Addressed comments on IRC.
Attachment #8839438 - Attachment is obsolete: true
(Assignee)

Comment 5

2 months ago
Created attachment 8840149 [details]
experiment.xpi

Updated add-on.
Attachment #8839439 - Attachment is obsolete: true
Looks good. You should get the xpi signed and do some manual testing (and of course get approval from release-drivers to run the experiment) before landing.

After you do some manual testing I'll give r+ and we can push to staging to do a quick QA and them push to prod.
Flags: needinfo?(felipc)
(Assignee)

Comment 7

2 months ago
Jason, can you please sign the add-on for me?
Flags: needinfo?(jthomas)

Comment 8

2 months ago
Created attachment 8840496 [details]
experiment.xpi signed

Please see attached.
Flags: needinfo?(jthomas)
(Assignee)

Comment 9

2 months ago
Manually tested everything and can confirm it works. Still waiting for release-drivers approval.
(Assignee)

Comment 10

2 months ago
Felipe, can you please give your r+ so we can land this as soon as I heard back from release-drivers?
Flags: needinfo?(felipc)
Comment on attachment 8840147 [details] [diff] [review]
intersection-observer-nightly54.diff

Review of attachment 8840147 [details] [diff] [review]:
-----------------------------------------------------------------

Sure. Actually, you can land the patch now, as we won't update the release_tag yet. That will put it in the staging server (https://telemetry-experiment-dev.allizom.org/) where we can ask QA to test it (usually a requirement to release an experiment.. not sure if that applies to Nightly but it wouldn't hurt to do some simple testing).
Attachment #8840147 - Flags: review+
Flags: needinfo?(felipc)
(Assignee)

Comment 12

2 months ago
Created attachment 8840578 [details] [diff] [review]
intersection-observer-experiment

Final patch to land. Includes signed add-on.
https://hg.mozilla.org/webtools/telemetry-experiment-server/rev/5ff9169cf01eab2598dd1cc05919e0efcf45c795
(Assignee)

Comment 14

2 months ago
Created attachment 8843376 [details] [diff] [review]
Manifest

Updated manifest with new dates and sample rate.
I pushed the manifest update:
https://hg.mozilla.org/webtools/telemetry-experiment-server/rev/068e2fb1e8e3229441a10ad50f1712d805dd8ab2

and tagged it as release_tag:
https://hg.mozilla.org/webtools/telemetry-experiment-server/rev/da6ccc855030760dc95a081261f70ed34f896a36
(Assignee)

Comment 16

2 months ago
Created attachment 8844049 [details] [diff] [review]
Manifest

One last change to the manifest. Now includes nightly version 55a1, to avoid the experiment being disabled for some people after merge today.
Attachment #8843376 - Attachment is obsolete: true
(Assignee)

Comment 17

2 months ago
Created attachment 8844054 [details]
experiment.xpi

Updated add-on.
Attachment #8840149 - Attachment is obsolete: true
Attachment #8840496 - Attachment is obsolete: true
(Assignee)

Comment 18

2 months ago
jason, can you please sign the add-on on more time? Thanks.
Flags: needinfo?(jthomas)
Created attachment 8844130 [details]
experiment.xpi signed

Please see attached.
Flags: needinfo?(jthomas)
(Assignee)

Comment 20

2 months ago
Created attachment 8844135 [details] [diff] [review]
intersection-observer-experiment 2

Updated patch with new signed add-on.
Attachment #8840147 - Attachment is obsolete: true
Attachment #8844049 - Attachment is obsolete: true
https://hg.mozilla.org/webtools/telemetry-experiment-server/rev/6e38436bc5d2
https://hg.mozilla.org/webtools/telemetry-experiment-server/rev/13bf28e08b4b
Depends on: 1344881
(Assignee)

Comment 22

a month ago
Created attachment 8848251 [details] [diff] [review]
intersection-observer-experiment 3

Running the experiment on 10% of the Nightly user population showed no stability issues. Lets do a final run on 50% before shipping.
(Assignee)

Comment 23

a month ago
Created attachment 8848255 [details] [diff] [review]
intersection-observer-experiment 3

Created a new copy of the experiment to make sure we get enough new activations.
Attachment #8848251 - Attachment is obsolete: true
https://hg.mozilla.org/webtools/telemetry-experiment-server/rev/40a11e1fa5c17cb934656d26728675d29eff1ee1
https://hg.mozilla.org/webtools/telemetry-experiment-server/rev/88b6307803ba6683d9000ef9d3acdc071e88e238

Now you just need to file a new deployment bug like the previous one. Please make sure you have clearance from release-drivers before doing so!
(Assignee)

Updated

a month ago
Depends on: 1348144
(Assignee)

Comment 25

29 days ago
Created attachment 8852179 [details]
experiment.xpi

Updated add-on to target FF 55.0a1. Jason, please sign.
Attachment #8844054 - Attachment is obsolete: true
Attachment #8844130 - Attachment is obsolete: true
Flags: needinfo?(jthomas)
Created attachment 8852201 [details]
experiment.xpi signed

Please see attached.
Flags: needinfo?(jthomas)
(Assignee)

Comment 27

29 days ago
Created attachment 8852208 [details] [diff] [review]
intersection-observer-experiment 4

Running the experiment on 10% and 50% of the Nightly user population showed no stability issues. Lets enable it to 100% before shipping.
(Assignee)

Comment 28

29 days ago
Created attachment 8852211 [details] [diff] [review]
intersection-observer-experiment 4

Correct patch this time.
Attachment #8852208 - Attachment is obsolete: true
(Assignee)

Comment 29

29 days ago
Felipe, can you please land the latest patch?
Flags: needinfo?(felipc)
https://hg.mozilla.org/webtools/telemetry-experiment-server/rev/3c9acc1b7f5bfbbcd0ff137bceddc9ff2820e935
https://hg.mozilla.org/webtools/telemetry-experiment-server/rev/dc0e815061ca276a9c2c5b018f5f7580b58c51a1
Flags: needinfo?(felipc)
(Assignee)

Updated

28 days ago
Depends on: 1351837
You need to log in before you can comment on or make changes to this bug.