Closed Bug 1545417 Opened 1 year ago Closed 1 year ago

Two "unenroll" events are generated for the same opt-out experiment after disabling the recipe from Normandy's stage server

Categories

(Firefox :: Normandy Client, defect, P1)

defect

Tracking

()

RESOLVED DUPLICATE of bug 1543817
Tracking Status
firefox66 --- affected
firefox67 --- affected
firefox68 --- unaffected

People

(Reporter: cfat, Unassigned)

References

(Regression)

Details

Attachments

(1 file)

Attached image normandy events.gif

[Affected versions]:

  • Firefox Release 66.0.3
  • Firefox Beta 67.0b11

[Affected platforms]:

  • Windows 10 x64
  • Ubuntu 16.04 x64
  • Mac OS X 10.14

[Prerequisites]:

[Steps to reproduce]:

  1. Open a browser profile that can be targeted by the previously set recipe.
  2. Open the Delivery Console and disable the opt-out recipe from mentioned in prerequisites.
    https://delivery-console.stage.mozaws.net
  3. Restart the browser.
  4. Navigate to the "about:telemetry" page and click the "Events" button.
  5. In the Events page, select the "dynamic" option from the top-right dropdown menu.
  6. Observe the number of unenroll events for the previously disabled opt-out experiment was generated.

[Expected result]:

  • One "unenroll" event is generated.

[Actual result]:

  • Two "unenroll" events are generated for the same experiment.

[Note]:

  • This issue is not reproducible on the latest Firefox Nightly 68.0a1.
  • Please see the screen recording attached.
OS: Unspecified → All
Hardware: Unspecified → All

The priority flag is not set for this bug.
:mythmon, could you have a look please?

For more information, please visit auto_nag documentation.

Flags: needinfo?(mcooper)

This makes it a lot harder to analyze telemetry. If all data is doubled, it's easy to tell, but if this problem isn't intermittent, this is a big problem.

Flags: needinfo?(mcooper)
Priority: -- → P1

This issue is not reproducible on the latest Firefox Nightly 68.0a1.

It seems like this issue was probably fixed between 67 and 68. This telemetry isn't supposed to depend on the channel, so it shouldn't be a Nightly vs Beta change.

I wasn't able to find any changes between 67 and 68 that looked like they would cause this. I'd be interested in a reverse bisection test to see what caused this. Since this is affects 67.0.5, I think it is worth tracking down what caused this. If it is something simple, we may be able to backport it to 67 and 67.0.5.

To be clear, the regression search here is not for what caused this bug originally, but for what fixed it so we could potentially backport that fix.

Flags: needinfo?(carmen.fat)

Answering this on Carmen's behalf as she is caught up with other work.

I've run into a lot of problems while trying to do this regression, one of which was the fact that all the signed add-ons I had on hand would not work on Firefox versions older than 67. I was almost ready to give up until I found that the remote-settings-devtools repo has a working add-on for this case. After which while doing the regression and reached mozilla-inbound and mozilla-autoland I had to start changing the channel-pref file to nightly for each donwloaded build so that the recipe could target my build.

All in all it was pretty time consuming. But I managed to finish it:
12:51.12 INFO: First good revision: be3f40425b529dae1a1fc53f7998196fb6deaf2d
12:51.12 INFO: Last bad revision: bacdb7ee8f0e2b9fa68729e90e6b2dafdb2d3209
12:51.12 INFO: Pushlog:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=bacdb7ee8f0e2b9fa68729e90e6b2dafdb2d3209&tochange=be3f40425b529dae1a1fc53f7998196fb6deaf2d

Looks like bug 1543817 has fixed this issue.

@mythmon, let me know if you want us to try and find what caused this issue in the first place.

Flags: needinfo?(carmen.fat)

Thanks for the regression range Ciprian, that was very useful. You're right about the bug that caused this. The specific patch that fixed the problem in that bug was https://phabricator.services.mozilla.com/D27363

@mythmon, let me know if you want us to try and find what caused this issue in the first place.

No need, after looking at the bug, I know exactly what caused this. It was introduced in bug 1440780, when I added an alias for the add-on studies action.

I'm going to close this as a dupe of the bug that was fixed. Unfortunately, it seems like we are too late to uplift to the Trailhead release. However the problem was fixed in the 68 train, so we won't have to live with it much longer.

Status: NEW → RESOLVED
Closed: 1 year ago
Regressed by: 1440780
Resolution: --- → DUPLICATE
Duplicate of bug: 1543817
You need to log in before you can comment on or make changes to this bug.