Bugzilla

Background: in bug 1549548 I'm wanting to make a number of browser-chrome mochitests run once with a pref set to `true` and once with the pref set to `false`. I figured I'd leverage the `prefs` support we now have in test manifests for this, and create 2 manifests that set the relevant pref to the 2 values, and then have both include a "common" manifest that doesn't set any pref value.

This kind of set up is helpful to ensure that, while nightly/beta might have different values, or while you might want to run studies that toggle the pref, automation ensures that things work as designed irrespective of the pref value.

Unfortunately, it seems this doesn't work, specifically you get this error before the tests have even run:

``
0:01.61 ERROR The 'prefs' key must be set in the DEFAULT section of a manifest. Fix the following manifests: path/to/included-only-manifest.ini
```

from https://searchfox.org/mozilla-central/rev/cc280c4be94ff8cf64a27cc9b3d6831ffa49fa45/testing/mochitest/runtests.py#1529 .

From IRC:

```
15:09:32 <Gijs> Who knows about " 0:01.61 ERROR The 'prefs' key must be set in the DEFAULT section of a manifest. Fix the following manifests: path/to/included-only-manifest.ini
15:09:52 <Gijs> (both of the including manifests have a `prefs` key, the included one doesn't have any)
15:18:34 <ahal> Gijs: you might need to dupe the prefs key to the included one
15:18:48 <ahal> guess the DEFAULT section doesn't propagate down to included manifests
15:18:51 <Gijs> ahal: but I don't want to.
15:19:00 <Gijs> ahal: specifically, I want 1 set of tests, run with 2 values for a pref
15:19:08 <Gijs> so I put all the tests in a shared manifest
15:19:10 <ahal> ah, I see
15:19:13 <Gijs> created one with pref=true and one with pref=false
15:19:20 <Gijs> and made them both include the shared one
15:19:23 <Gijs> I guess that's... bad?
15:20:20 <ahal> I think I understand why it's happening, though not 100% sure what to do about it
15:20:30 <Gijs> :(
15:20:38 <Gijs> is what I want just not easily possible right now?
15:21:23 <ahal> I don't think so :(
15:21:56 <ahal> I think we'd want the DEFAULTS to propagate down like you thought it would, but that will probably require refactoring manifests across the tree
15:24:00 <Gijs> 😱
15:24:05 <Gijs> ok, I'll just duplicate things
15:24:13 <Gijs> thanks for confirming I'm not mad though!
```

Background: in bug 1549548 I'm wanting to make a number of browser-chrome mochitests run once with a pref set to `true` and once with the pref set to `false`. I figured I'd leverage the `prefs` support we now have in test manifests for this, and create 2 manifests that set the relevant pref to the 2 values, and then have both include a "common" manifest that doesn't set any pref value.

This kind of set up is helpful to ensure that, while nightly/beta might have different values, or while you might want to run studies that toggle the pref, automation ensures that things work as designed irrespective of the pref value.

Unfortunately, it seems this doesn't work, specifically you get this error before the tests have even run:

```
0:01.61 ERROR The 'prefs' key must be set in the DEFAULT section of a manifest. Fix the following manifests: path/to/included-only-manifest.ini
```

from https://searchfox.org/mozilla-central/rev/cc280c4be94ff8cf64a27cc9b3d6831ffa49fa45/testing/mochitest/runtests.py#1529 .

From IRC:

```
15:09:32 <Gijs> Who knows about " 0:01.61 ERROR The 'prefs' key must be set in the DEFAULT section of a manifest. Fix the following manifests: path/to/included-only-manifest.ini
15:09:52 <Gijs> (both of the including manifests have a `prefs` key, the included one doesn't have any)
15:18:34 <ahal> Gijs: you might need to dupe the prefs key to the included one
15:18:48 <ahal> guess the DEFAULT section doesn't propagate down to included manifests
15:18:51 <Gijs> ahal: but I don't want to.
15:19:00 <Gijs> ahal: specifically, I want 1 set of tests, run with 2 values for a pref
15:19:08 <Gijs> so I put all the tests in a shared manifest
15:19:10 <ahal> ah, I see
15:19:13 <Gijs> created one with pref=true and one with pref=false
15:19:20 <Gijs> and made them both include the shared one
15:19:23 <Gijs> I guess that's... bad?
15:20:20 <ahal> I think I understand why it's happening, though not 100% sure what to do about it
15:20:30 <Gijs> :(
15:20:38 <Gijs> is what I want just not easily possible right now?
15:21:23 <ahal> I don't think so :(
15:21:56 <ahal> I think we'd want the DEFAULTS to propagate down like you thought it would, but that will probably require refactoring manifests across the tree
15:24:00 <Gijs> 😱
15:24:05 <Gijs> ok, I'll just duplicate things
15:24:13 <Gijs> thanks for confirming I'm not mad though!
```