I'm running the gaia-ui-tests on MacOSX and the bluetooth tests can't work there, because I can't install pybluez on MacOSX. So I need to run the tests without the tests that use bluetooth. In that cases I can add -bluetooth to type. However, when I do that, e.g: GAIATEST_SKIP_WARNING=1 gaiatest --address localhost:2828 --profile $HOME/B2G/gaia/profile --testvars /Users/mwargers/B2G/gaia/tests/python/gaia-ui-tests/gaiatest/testvars.json --restart --type smoketest-bluetooth /Users/mwargers/B2G/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/manifest.ini Then it's only running 1 test, namely test_settings_dsds_cell_data.py TestDSDSCellData.test_cell_data_for_two_sims And it skips the other 127 smoketests. It seems to me something is going wrong here. http://mxr.mozilla.org/gaia/source/tests/python/gaia-ui-tests/gaiatest/tests/functional/settings/manifest.ini#3 In that manifest.ini file, bluetooth = false Is it perhaps the case that when bluetooth is not set at all in the manifest that bluetooth = true by default or something?
I think you need --type=b2g+smoketest-bluetooth
(In reply to Dave Hunt (:davehunt) from comment #1) > I think you need --type=b2g+smoketest-bluetooth I did that, that's still running only that 1 test I mentioned.
Ok, when linking to: http://mxr.mozilla.org/gaia/source/tests/python/gaia-ui-tests/gaiatest/tests/manifest.ini It goes well, that's because there's a bluetooth = false setting in there. So I guess it's indeede the case that when something is not set, it is automatically set to true.
I guess it goes wrong here somewhere: http://mxr.mozilla.org/mozilla-central/source/testing/marionette/client/marionette/runner/base.py#742
Created attachment 8498970 [details] [diff] [review] manifest.diff Adding bluetooth = false to every manifest.ini would be a possibility, but I don't think it's really sensible. It doesn't really scale. Also, there are more of these variables that could be added. This is a git diff, I don't know why /.idea/modules.xml has changes to it. I'm not sure how to make a PR (=pull request) of this. Anyway, I don't think this patch is sensible anyway. Btw, I noticed this doesn't have smoketest set at all: http://mxr.mozilla.org/gaia/source/tests/python/gaia-ui-tests/gaiatest/tests/unit/manifest.ini That means running with --type=b2g-smoketest doesn't run anything in that direcotry at all. GAIATEST_SKIP_WARNING=1 gaiatest --address localhost:2828 --profile $HOME/B2G/gaia/profile --testvars /Users/mwargers/B2G/gaia/tests/python/gaia-ui-tests/gaiatest/testvars.json --restart --type=b2g-smoketest /Users/mwargers/B2G/gaia/tests/python/gaia-ui-tests/gaiatest/tests/unit/manifest.ini I don't think that makes sense.
I guess just accepting that using "-" on type is not working always as expected, except when using it on the top level manifest. Feel free to close this bug.
QA Whiteboard: [fxosqa-auto-backlog-]
This was very likely fixed by bug 1099041.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Depends on: 1099041
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.