We know from bug 1066621 that some w-p-t will fail when they hit Beta due to feature disablings. Currently, there's no way to selectively annotate a test or to pref on a feature at runtime within that harness. As a result, we'll need to include a manifest update at the time of the merge to avoid the perma-fail. I'm attaching a patch here that should take care of it.
Can we disable the test based on environment variable MOZ_DEV_EDITION set to 1?
(In reply to Rail Aliiev [:rail] from comment #2) > Can we disable the test based on environment variable MOZ_DEV_EDITION set to > 1?
Flags: needinfo?(ryanvm) → needinfo?(james)
Currently no. Disabling the tests seems strictly worse than running the tests with the correct branch-specific expectations in any case.
(In reply to James Graham [:jgraham] from comment #4) > Currently no. > > Disabling the tests seems strictly worse than running the tests with the > correct branch-specific expectations in any case. Whatever the right thing to do is, it needs to be do-able via mozconfigs. We're landing the patch from this bug on this cycle as an exception, but going forward we need to be able to enable/disable things with our automation. Landing random patches is not how to deal with changes at uplift.
So I don't really know how that's possible. We take a bunch of tests from upstream that may depend on any platform feature. In general we don't know up-front which of these tests will be affected by differences between nightly and aurora or aurora and beta. Therefore, in general, we will have to either land a patch to change the expectation for any tests that do turn out to be affected, or land a patch to disable the tests that will be affected.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.