Closed Bug 1697793 Opened 4 years ago Closed 3 years ago

RTAMO flow is broken on Firefox Release

Categories

(Firefox :: Messaging System, defect)

Product:
Firefox
Component:
Messaging System
Type:
defect

Tracking

()

Status:
RESOLVED FIXED

People

(Reporter: acornestean, Unassigned)

References

Details

Attachments

(3 files)

2021-03-11_18h14_25.mp4
2.43 MB, video/mp4
Details
SkeletonUILock.png
13.18 KB, image/png
Details
2022-03-07_09h18_25.png
222.15 KB, image/png
Details

[Steps to reproduce]:

  1. Access AMO prod on a browser different from Firefox (example: Microsoft Edge) and afterwards the page of a recommended add-on (for example uBlock Origin)
  2. Click on the “Only with Firefox – Get Firefox Now” green button
  3. On the newly opened page, click on the blue “Download Firefox” button
    Note: Firefox will be downloaded, installed and will start after the installation is complete
  4. Observe that the Multi Stage about:welcome page is displayed and not the RTAMO about:welcome page, as it should

[Expected results]:
The RTAMO about:welcome page should be displayed.

[Actual results]:
The Multi Stage about:welcome page is displayed.

NOTES:

  1. The RTAMO flow works as expected on the latest Unbranded Release, Nightly and Beta with AMO prod, stage and dev.
  2. Tests were performed on Windows 10 x64 in accordance to the steps from here: https://gist.github.com/pdehaan/71028fb23b3f3d36f8dc55a6cfd2643e
  3. Tested Firefox Release 83 and 85.0.2 and the issue is also present on those versions as well, which most likely indicates that all Release versions are affected.
Flags: needinfo?(pdahiya)
Attached video 2021-03-11_18h14_25.mp4 —

This is expected because of set default experiment rollout in Fx86 which overrides RTAMO flow and enrolls all new users in set default experiment ,

https://experimenter.services.mozilla.com/nimbus/set-default-as-first-screen-100-roll-out

Experiment ends in a week with Fx87 launch and should reinstate RTAMO flow. Thanks!

Flags: needinfo?(pdahiya)

(In reply to Punam Dahiya [:pdahiya] from comment #2)

This is expected because of set default experiment rollout in Fx86 which overrides RTAMO flow and enrolls all new users in set default experiment ,

https://experimenter.services.mozilla.com/nimbus/set-default-as-first-screen-100-roll-out

Experiment ends in a week with Fx87 launch and should reinstate RTAMO flow. Thanks!

Is this really supposed to work like this, or maybe a better question is - could this behaviour be changed? As far as I know RTAMO should be ongoing. Hearing that RTAMO is being temporarily disabled to make way for an experiment is very unexpected.

Flags: needinfo?(pdahiya)

Couple of things:

  1. It's not clear to me from the video that something is broken. Is it possible that the RTAMO UX is present on one of the other steps in the onboarding flow? RTAMO shouldn't override other onboarding options, AFAIK.
  2. The experiment page says "This is NOT an experiment. This is turning on a previous win to 100% following positive results while we wait for it to land in production in Fx87.", so if something is broken it doesn't sound like it will be fixed by waiting.

(In reply to Stuart Colville [:muffinresearch] from comment #3)

(In reply to Punam Dahiya [:pdahiya] from comment #2)

This is expected because of set default experiment rollout in Fx86 which overrides RTAMO flow and enrolls all new users in set default experiment ,

https://experimenter.services.mozilla.com/nimbus/set-default-as-first-screen-100-roll-out

Experiment ends in a week with Fx87 launch and should reinstate RTAMO flow. Thanks!

Is this really supposed to work like this, or maybe a better question is - could this behaviour be changed? As far as I know RTAMO should be ongoing. Hearing that RTAMO is being temporarily disabled to make way for an experiment is very unexpected.

Yes we should change this behavior, currently users coming from RTAMO if enrolled in an experiment sees experiment specific onboarding UI. To avoid this in future, we should exclude RTAMO users from experiment targeting, I will file a bug for this Thanks

Flags: needinfo?(pdahiya)

Closing this bug in favor of 1698153

Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → DUPLICATE

Oh, that looks like a problem. We changed that, and if Firefox is looking for non-fx-button in utm_campaign it isn't going to find it there.

(In reply to Punam Dahiya [:pdahiya] from comment #8)

Noticed on AMO page below utm_campaign as 'amo-fx-cta-607454-current-cta' instead of 'non-fx-button', @jorge is that expected?

https://www.mozilla.org/en-US/firefox/new/?utm_campaign=amo-fx-cta-607454-current-cta&utm_content=rta:dUJsb2NrMEByYXltb25kaGlsbC5uZXQ&utm_medium=referral&utm_source=addons.mozilla.org

https://searchfox.org/mozilla-central/source/browser/components/newtab/aboutwelcome/lib/AboutWelcomeDefaults.jsm#406

Thanks for catching this. Our current plan is to look to revert the utm param changes, so we can make sure the other changes that are building on top of that are compatible with RTAMO going forward. All being well this should restore the expected behavior following the push later today.

Thanks for the heads up. I also filed bug 1707038 to look into this validation.

Flags: needinfo?(jorge)

Hello,

I checked the RTAMO flow from AMO Prod using the latest Firefox Release (97.0.1/20220216172458) and it appears to have broken again. So this is why I reopened this issue.

Following the normal STR to trigger the RTAMO flow, from the original description of this issue, the Multi Stage about:welcome page is displayed and not the RTAMO about:welcome page, as it should.

I also observed that the PostSigning data file is not created when installing the browser as part of the RTAMO flow.

I did check attribution as per the steps from https://bugzilla.mozilla.org/show_bug.cgi?id=1707038#c14 and attribution can be forcefully set. As per the specified parameters listed there, accessing about:welcome will serve the RTAMO onboarding page with the “Privacy Badger” add-on.

I’ve also tested the flow from Stage on the latest Nightly as per https://bugzilla.mozilla.org/show_bug.cgi?id=1724169#c15 , with the same results as above.

Last week everything was working.

Status: RESOLVED → REOPENED
Flags: needinfo?(pdahiya)
Resolution: DUPLICATE → ---

At AppData/Local/Mozilla/Firefox there's a file created when the browser is installed, which I haven't seen before - "SkeletonUILock-c388d246". It might be related to the RTAMO issue and why the PostSigningData file is not created.

Attached image SkeletonUILock.png —

Thanks Alex for catching, can you please verify if this issue is replicable on MacOS as well as Windows. Since RTAMO page is loading on about:welcome after force attribution, the issue can be isolated to bedrock or installer end where after download and install of Firefox attribution file is not getting created on user local.

NI Bob to help debug and provide input thanks!

Flags: needinfo?(pdahiya) → needinfo?(bob.silverberg)
Flags: needinfo?(acornestean)

Nothing has changed on AMO. Our download links look like, for example, https://www.mozilla.org/firefox/download/thanks/?s=direct&utm_campaign=amo-fx-cta-952959&utm_content=rta%3Ae2RkYzYyNDAwLWYyMmQtNGRkMy04YjRhLTA1ODM3ZGU1M2MyZX0&utm_medium=referral&utm_source=addons.mozilla.org, and we would expect that to work with RTAMO. Alex, can you check whether the attribution data is being written to the installer when one visits that link?

Flags: needinfo?(bob.silverberg)

I asked agibson whether this could be a problem in Bedrock, and he checked and here is his response:
"I’ve verified that using the above link I see stub attribution params encoded in the file download URL as expected. Decoding those params also gives me the right data. So I don’t think the breakage is in bedrock from what I can see."

Hello,

I’m going to document every step I take in the reproduction of this issue.

Windows:

  1. Opened Windows Sandbox
  2. Opened Microsoft Edge and navigated to addons.mozilla.org
  3. Accessed the product page of a random recommended add-on (Tomato Clock in this particular test)
  4. Clicked on the “Download Firefox and get the extension” blue button on the add-on product page
  5. The Firefox installer gets automatically downloaded and I run the installer to install the browser
    Note: The download link looks like this - https://www.mozilla.org/en-US/firefox/download/thanks/?s=direct&utm_campaign=amo-fx-cta-627490&utm_content=rta:amlkMS1LdDJrWVlnaTMyelB1d0BqZXRwYWNr&utm_medium=referral&utm_source=addons.mozilla.org
  6. After the browser is installed, it’s automatically launched and the multi-stage about:welcome page is displayed. No RTAMO onboarding page is present.
  7. I go through all the stages of the default multi-stage about:welcome page, but there’s still no signs of the RTAMO onboarding page.
  8. I then check \AppData\Local\Mozilla\Firefox for the PostSigningData file - which has not been created and is not present at that location

Then I check forced attribution:

  1. I access about:config and I enable browser.newtabpage.activity-stream.asrouter.devtoolsEnabled
  2. I open a new tab and click on the “wrench” icon
  3. Select “Targeting” from the left sidebar and scroll to the bottom of the page
  4. I force attribution with the below parameters (which I get from the above link from where the browser download was initiated)
  • Source: addons.mozilla.org
  • Medium: referral
  • Campaign: amo-fx-cta-627490
  • Content: rta:amlkMS1LdDJrWVlnaTMyelB1d0BqZXRwYWNr
  1. Check \AppData\Local\Mozilla\Firefox for the PostSigningData file – It was just created and is in the local folder
  2. Access about:welcome in the browser – the RTAMO onboarding page is displayed and it’s serving “Tomato Clock”
    Note: The add-on can be installed without issues from the RTAMO onboarding page.

MacOS:

  1. Created a new user (to make sure no prior Firefox data is on the system)
  2. Opend Safari and navigated to addons.mozilla.org
  3. Accessed the product page of a random recommended add-on (Tomato Clock in this particular test)
  4. Clicked on the “Download Firefox and get the extension” blue button on the add-on product page
  5. The Firefox .dmg gets automatically downloaded and I install the browser when the download has finished.
    Note 1: The download link looks like this - https://www.mozilla.org/en-US/firefox/download/thanks/?s=direct&utm_campaign=amo-fx-cta-627490&utm_content=rta:amlkMS1LdDJrWVlnaTMyelB1d0BqZXRwYWNr&utm_medium=referral&utm_source=addons.mozilla.org
    Note 2: I install then browser in a custom folder on Desktop
  6. After the browser is installed, I launch it and the multi-stage about:welcome page is displayed. No RTAMO onboarding page is present.
  7. I go through all the stages of the default multi-stage about:welcome page, but there’s still no signs of the RTAMO onboarding page.
  8. I then go to about:support – “Update Folder” section and click on Show in Finder. This opens the local folders with a “Firefox” folder at the end of the path which contains the “macAttributionData” file – which has been created with the browser install. When I check the file contents, it’s empty.

Then I check forced attribution:

  1. I access about:config and I enable browser.newtabpage.activity-stream.asrouter.devtoolsEnabled
  2. I open a new tab and click on the “wrench” icon
  3. Select “Targeting” from the left sidebar and scroll to the bottom of the page
  4. I force attribution with the below parameters (which I get from the above link from where the browser download was initiated)
  • Source: addons.mozilla.org
  • Medium: referral
  • Campaign: amo-fx-cta-627490
  • Content: rta:amlkMS1LdDJrWVlnaTMyelB1d0BqZXRwYWNr
  1. I re-check the ““macAttributionData” file – it has the appropriate content now
  2. Access about:welcome in the browser – the RTAMO onboarding page is displayed and it’s serving “Tomato Clock”
    Note: The add-on can be installed without issues from the RTAMO onboarding page.

@Punam
Both macOS and Windows are affected as I described above

@Bob

For Windows, I’ve opened another instance of Windows Sandbox to ensure a clean slate, opened Edge and pasted the link you provided in Comment 16.
The browser gets downloaded, I install it and it launches. No RTAMO page is displayed, just the default multi-stage one.
Checking \AppData\Local\Mozilla\Firefox reveals that no PostSigningData file is present.

For macOS, I’ve again created a new user to ensure a clean slate. I’ve opened Safari and pasted the link you provided.
The browser gets downloaded and I extract it to a folder on Desktop and launch it. No RTAMO page is displayed, just the default multi-stage one.
Checking the macAttributionData file reveals that it’s empty.

Flags: needinfo?(acornestean)

I investigated a bit. Alex's STRs are correct and this is very likely a bug between AMO and the "download service"/CDN, so not a bug in Firefox.

Alex shared a Windows FF installer he downloaded on a fresh Windows VM and there was no attribution data in it (attribution data should be written in a special MOZCUSTOM location in the .exe binary).

I also tried on my own Windows machine. I confirmed that AMO was passing the right data to bedrock. From there, bedrock called the "stubattribution service" to get an attribution code. There is no problem with AMO or bedrock at this stage. I eventually downloaded the generic Windows FF installer, without any attribution data, but that wasn't expected. I looked at the network panel and we don't pass the referer header between bedrock and the download service. We implemented a referer header check in https://github.com/mozilla-services/stubattribution/issues/105 last year.

Copying the exact same request from the network panel (as cURL) and adding the referer header manually (-H "Referer: https://www.mozilla.org/') allowed me to downloaded the customized Windows FF installer, i.e. the installer with the attribution data. Executing this installer should write the "post signing data" and that should allow the RTAMO flow to be displayed.

Now the reason why we don't send the referer header is unclear to me. It worked in the past and I verified it many times. I've noticed that there has been some changes around the referer header in Chromium browsers, see: https://web.dev/referrer-best-practices/#default-referrer-policies-in-browsers

Looking at the referer-policy in the network panel, the policy seems to be same-origin for the request to the download service. This would explain why we don't have a referer header. Why we have this policy remains unclear to me.

At this point, I don't think there is a bug in the RTAMO code in Firefox.

Will suggested I try Internet Explorer to check the flow since he believed it should work when starting the flow from that browser.
He was right. Just checked the RTAMO flow using IE and as soon as FF got installed and launched, the RTAMO onboarding page was displayed.

I've also checked the flow from Chrome and the same behavior is observed as in the case of Edge (which I tested earlier in Comment 18) - the multi-stage a:w is displayed instead of the RTAMO one.

It looks like a referrer policy has been set as part of a Django (3.1) update (in bedrock), more information about the new behavior at: https://docs.djangoproject.com/en/3.2/ref/settings/#std:setting-SECURE_REFERRER_POLICY.

The bedrock team landed a fix in https://github.com/mozilla/bedrock/pull/11280, which should very likely fix this bug.

Verified the RTAMO flow from AMO Prod on the latest Release (starting the flow from Microsoft Edge) and it’s been fixed. Once the browser is installed and it launches, the RTAMO onboarding page is displayed as intended.
The served add-on can be properly installed and the PostSigningData file is properly created on browser install.
For more details, see the attached screenshot.

I’ve also tested the flow from AMO Stage and Dev on the latest Nightly (using the workaround from here https://bugzilla.mozilla.org/show_bug.cgi?id=1724169#c15 OR https://docs.google.com/document/d/1X2D42ra2rmkYGTBELu7uvH1wL5Tr8PChmRPV8VPpHzk/edit – which is our QA doc for testing the entire flow from Stage/Dev) and it also performs as intended.

However, I’ve noticed something out of order with the onboarding page i.e. the served add-on name is replaced by a placeholder - {$addon-name}.

Only Nightly is affected by this regardless of starting the flow from Prod, Stage or Dev. Beta is not affected.

As such, I’ve submitted a new bug for this: https://bugzilla.mozilla.org/show_bug.cgi?id=1758340 and it most likely involves https://bugzilla.mozilla.org/show_bug.cgi?id=1726426.

Although this does not belong here, I just wanted to let you know.

I'll mark this issue as RESOLVED.

Status: REOPENED → RESOLVED
Closed: 4 years ago → 3 years ago
Resolution: --- → FIXED
Attached image 2022-03-07_09h18_25.png —
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: