Bugzilla

Copied from slack for some context:

We didn't uplift this before the params aren't overridden, it just gets duplicate params appended. At the time, we thought the first params would take precedence over the duplicate params, so it wouldn't matter. We learned later that it actually seems the later params take precedence, at least with SUMO's google analytics.

So our spotlight analytics were getting messed up for the device migration spotlight experiment. but that was targeting 114, and by the time we discovered this, 114 was already shipped. So, I landed in 116 and there didn't seem any point to uplift to 115, since the audience included 114. And we were able to just treat the duplicate param values as "synonyms" for the experiment's intended param values in our analysis, because that experiment's 1 spotlight was the only thing sending users to that URL with the duplicate params.

Although google analytics picked up the wrong param values, the data was effectively the same. For that experiment there was only 1 kind of spotlight leading to that page, so there wasn't anything to disambiguate. In other cases, it could be possible to have 2 spotlights with URLs to the same page but with different UTM params, used to distinguish 1 branch from the other for example. And then it would be a problem if that experiment was targeting 115 or earlier.

Sorry if this is a bit confusing. It basically means there won't be a problem unless:
1. Multiple Spotlight experiments, or multiple branches within an experiment, contain URLs leading to the same page, but with different UTM param values;
2. And the experiment(s) in question are all targeting 115 or earlier.

If the above happens, then for example, branch 1 and branch 2 will send users to the same URL, with the same UTM params, making it impossible to distinguish one from the other in google analytics. I'm not aware of any experiments like this, but if anyone has any suspicions, send me the experiment links and I can try to figure out if it's going to be a problem.

Copied from slack for some context:

We didn't uplift this before because the params aren't overridden, it just gets duplicate params appended. At the time, we thought the first params would take precedence over the duplicate params, so it wouldn't matter. We learned later that it actually seems the later params take precedence, at least with SUMO's google analytics.

So our spotlight analytics were getting messed up for the device migration spotlight experiment. but that was targeting 114, and by the time we discovered this, 114 was already shipped. So, I landed in 116 and there didn't seem any point to uplift to 115, since the audience included 114. And we were able to just treat the duplicate param values as "synonyms" for the experiment's intended param values in our analysis, because that experiment's 1 spotlight was the only thing sending users to that URL with the duplicate params.

Although google analytics picked up the wrong param values, the data was effectively the same. For that experiment there was only 1 kind of spotlight leading to that page, so there wasn't anything to disambiguate. In other cases, it could be possible to have 2 spotlights with URLs to the same page but with different UTM params, used to distinguish 1 branch from the other for example. And then it would be a problem if that experiment was targeting 115 or earlier.

Sorry if this is a bit confusing. It basically means there won't be a problem unless:
1. Multiple Spotlight experiments, or multiple branches within an experiment, contain URLs leading to the same page, but with different UTM param values;
2. And the experiment(s) in question are all targeting 115 or earlier.

If the above happens, then for example, branch 1 and branch 2 will send users to the same URL, with the same UTM params, making it impossible to distinguish one from the other in google analytics. I'm not aware of any experiments like this, but if anyone has any suspicions, send me the experiment links and I can try to figure out if it's going to be a problem.