2020769 - Reinstate FXNimbus Live Test

Status: RESOLVED FIXED

(Firefox for Android :: Experimentation and Telemetry, task)

Description

[Olivia Hall [:olivia]]

Because bug 2020683 needs default: out of mozilla central, because it will become an invalid stanza on the Nimbus side, it was proposed to be removed in bug 2012763.

We should reinstate these tests because it confirms that Nimbus is correctly parsing and identifying Gecko Preference experiments after bug 2020683 lands and the default:stanza can be removed.

We cannot remove the default stanza alone because of:

Validating manifest for different channels:
❎ 
release.............invalid
: Validation Error at features/gecko-nimbus-validation.test-preference: Invalid value null for type String
Error: Manifest contains error(s) in 4 channels
❎ 
beta................invalid
: Validation Error at features/gecko-nimbus-validation.test-preference: Invalid value null for type String
❎ 
nightly.............invalid
: Validation Error at features/gecko-nimbus-validation.test-preference: Invalid value null for type String
❎ 
developer...........invalid
: Validation Error at features/gecko-nimbus-validation.test-preference: Invalid value null for type String

What should be reinstated is:

nimbus.fml.yaml:
  gecko-nimbus-validation:
    description: "Feature for validating Nimbus Gecko functionality"
    variables:
      test-preference:
        description: "A test gecko preference"
        type: String
        gecko-pref:
          pref: "gecko.nimbus.test"
          branch: "default"

NimbusGeckoPrefHandler.kt:
fun `test nimbusGeckoPreferences has appropriate values`() {
        val handler = makeHandler()
        assertNotNull(handler.nimbusGeckoPreferences["gecko-nimbus-validation"])
        assertNotNull(
            handler.nimbusGeckoPreferences["gecko-nimbus-validation"]?.get(
                "test-preference",
            ),
        )
    }

    @Test
    fun `preferenceList has appropriate values`() {
        val handler = makeHandler()
        assertTrue(handler.preferenceList.contains(TEST_PREF))
    }

See D285806. Need to get rid of mockking after reinstating.

Comment 1

[Olivia Hall [:olivia]]

Attachment: Bug 2020769 - Remove Object Mocking for FxNimbus

With the valid FML stabilized, we can now reintroduce the tests.

This patch:

• Removes static mocking
• Adds back the gecko-pref nimbus.fml.yaml
• Adds a second pref to nimbus.fml.yaml
• General adjustments for multi-prefs / specific test scenarios

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/2e56eb8bbdee
https://hg.mozilla.org/integration/autoland/rev/3d779526dcf4
Remove Object Mocking for FxNimbus r=android-reviewers,mcarare

Comment 3

[rperta]

https://hg.mozilla.org/mozilla-central/rev/3d779526dcf4