1758645 - The environment cannot be switched anymore between "Prod" and "Stage" using the "XPCSHELL_TEST_PROFILE_DIR=foo" variable

Status: VERIFIED FIXED

(Firefox :: Remote Settings Client, defect)

Description

[Marius Coman [:mcoman], Ecosystem QA]

Attachment: rec of the issue.gif

[Notes]:

• We received a workaround for this issue, more exactly we need to enter the XPCOMUtils.defineLazyPreferenceGetter(ChromeUtils.import("resource://services-settings/Utils.jsm").Utils, "SERVER_URL", "services.settings.server") code in the "Browser Console" before changing the environment. However, this workaround does not help us in the case of first-run experiments where we need to change the environment before opening the browser.

[Affected versions]:

• Firefox Beta 99.0b1 - Build ID: 20220308094525

[Affected Platforms]:

• Windows 10 x64
• macOS 11.6.2
• Linux Mint 20.2 x64

[Steps to reproduce]:

1. Open a "Command Prompt" window ("Terminal" on macOS and Linux).
2. Enter the `set XPCSHELL_TEST_PROFILE_DIR=foo" variable ("export XPCSHELL_TEST_PROFILE_DIR=foo" on macOS and Linux) and press the "Enter" key.
3. Open the browser from the above "Command Prompt/Terminal".
4. Install the "Remote Settings Devtools" addon.
5. Click the "Remote Settings Devtools" toolbar button.
6. Select the "Stage" option from the "Environment" dropdown placed in the top-right part of the page.
7. Observe the behavior.

[Expected result]:

• The "Stage" environment is successfully selected.

[Actual result]:

• The "Prod" environment is still selected.

[Regression Window]:

• According to Ed Lee it seems that bug 1272255 has caused this behavior.

[Additional Notes]:

• Attached a screen recording of the issue:

Comment 1

[Marius Coman [:mcoman], Ecosystem QA]

Julian, could you please take a look over this issue?

Comment 2

[Julian Descottes [:jdescottes]]

Right, I mentioned this in https://bugzilla.mozilla.org/show_bug.cgi?id=1757741

The workaround to set XPCSHELL_TEST_PROFILE_DIR in order to be able to override the URL was not documented anywhere, and we switched to another environment variable: MOZ_DISABLE_NONLOCAL_CONNECTIONS and there is no workaround possible.

IMO we need another environment variable, here with a proper name and documentation.

Comment 3

[Julian Descottes [:jdescottes]]

Attachment: Bug 1758645 - Check new env variable MOZ_REMOTE_SETTINGS_DEVTOOLS to allow overriding the remote settings server URL

Comment 4

[Julian Descottes [:jdescottes]]

Heads up, I am landing a fix which introduces a new env variable called MOZ_REMOTE_SETTINGS_DEVTOOLS.

If you have existing documentation which refers to "set XPCSHELL_TEST_PROFILE_DIR=foo" or similar, it should be updated to say "set MOZ_REMOTE_SETTINGS_DEVTOOLS=1".

We will need to uplift the patch to Beta 99 before you can use it for testing though.

Comment 5

[Marius Coman [:mcoman], Ecosystem QA]

Great news! Thank you, Julian! We will update our documentation and we will verify this issue as soon as it will be uplifted to Beta 99.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f43486fc8a95
Check new env variable MOZ_REMOTE_SETTINGS_DEVTOOLS to allow overriding the remote settings server URL r=Gijs,leplatrem

Comment 7

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/f43486fc8a95

Comment 8

[Julian Descottes [:jdescottes]]

Cross posting here: a workaround applicable to FF99 was shared at https://bugzilla.mozilla.org/show_bug.cgi?id=1757741#c6

(In reply to Ed Lee :Mardak from Bug 1757741 comment #6)

With bug 1757744 fixed, "Browser Toolbox" now shows up as "XPCShell Toolbox" when setting XPCSHELL_TEST_PROFILE_DIR. To get Remote Settings Devtools to switch to "Stage" environment on beta/release, here's a Browser Console workaround:

XPCOMUtils.defineLazyPreferenceGetter(
  ChromeUtils.import("resource://services-settings/Utils.jsm").Utils,
  "SERVER_URL",
  "services.settings.server"
)

https://github.com/mozilla-extensions/remote-settings-devtools/issues/44#issuecomment-1062060893

That workaround seems to be sufficient for testing with stage remote settings, but if not, we can file a new Remote Settings Client bug.

Comment 9

[Julian Descottes [:jdescottes]]

Marius, just want to check before requesting the uplift. Since there was a workaround shared that works with 99 (cf previous comment), do you think an uplift here is worth it? Or are you fine with using the workaround when testing with 99?

Comment 10

[Marius Coman [:mcoman], Ecosystem QA]

Hi Julian, I think we will need an uplift to Beta since we are using the "Stage" environment to test experiments and if a first-run experiment will arrive in Fx 99 we will need to change the environment before opening the browser. Thanks for all the help!

Comment 11

[Julian Descottes [:jdescottes]]

Makes sense! Will do

Comment 12

[Julian Descottes [:jdescottes]]

Comment on attachment 9267006 [details]
Bug 1758645 - Check new env variable MOZ_REMOTE_SETTINGS_DEVTOOLS to allow overriding the remote settings server URL

Beta/Release Uplift Approval Request

• User impact if declined: QA is unable to use the Remote Settings DevTools extension with Firefox 99.

I don't think it needs extra verifications since it's going to be used by QA when it lands, but the STRs are in the initial comment, just replace XPCSHELL_TEST_PROFILE_DIR=foo with MOZ_REMOTE_SETTINGS_DEVTOOLS=1

• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: No
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Small javascript change.

The only risky part is that the code checks the RELEASE_OR_BETA flag. I manually tested it on Nightly by "flipping" the RELEASE_OR_BETA check in my patch and it worked as expected.

• String changes made/needed:

Comment 13

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1272255

Comment 14

[Donal Meehan [:dmeehan]]

Comment on attachment 9267006 [details]
Bug 1758645 - Check new env variable MOZ_REMOTE_SETTINGS_DEVTOOLS to allow overriding the remote settings server URL

Approved for 99.0b4. Thanks.

Comment 15

[Donal Meehan [:dmeehan]]

https://hg.mozilla.org/releases/mozilla-beta/rev/b4abd4958cd1

Comment 16

[Marius Coman [:mcoman], Ecosystem QA]

I have verified that this issue is no longer reproducible with the latest Firefox Beta (99.0b4 Build ID - 20220315185755) installed on Windows 10 x64, macOS 11.6.4, and Ubuntu 20.04 x64. Now the environment can be successfully changed between "Prod" and "Stage" if the MOZ_REMOTE_SETTINGS_DEVTOOLS=1 variable is used.