Running gaia-ui-test with "--type smoketest-bluetooth" runs only 1 test on my device

RESOLVED FIXED

Status

Firefox OS
Gaia::UI Tests
RESOLVED FIXED
4 years ago
3 years ago

People

(Reporter: Martijn Wargers (zombie), Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Reporter)

Description

4 years ago
I'm running the gaia-ui-tests on MacOSX and the bluetooth tests can't work there, because I can't install pybluez on MacOSX.
So I need to run the tests without the tests that use bluetooth.
In that cases I can add -bluetooth to type.

However, when I do that, e.g:
GAIATEST_SKIP_WARNING=1 gaiatest --address localhost:2828 --profile $HOME/B2G/gaia/profile --testvars /Users/mwargers/B2G/gaia/tests/python/gaia-ui-tests/gaiatest/testvars.json --restart --type smoketest-bluetooth /Users/mwargers/B2G/gaia/tests/python/gaia-ui-tests/gaiatest/tests/functional/manifest.ini

Then it's only running 1 test, namely test_settings_dsds_cell_data.py TestDSDSCellData.test_cell_data_for_two_sims
And it skips the other 127 smoketests.

It seems to me something is going wrong here.

http://mxr.mozilla.org/gaia/source/tests/python/gaia-ui-tests/gaiatest/tests/functional/settings/manifest.ini#3
In that manifest.ini file, bluetooth = false

Is it perhaps the case that when bluetooth is not set at all in the manifest that bluetooth = true by default or something?
I think you need --type=b2g+smoketest-bluetooth
Flags: needinfo?(martijn.martijn)
(Reporter)

Comment 2

4 years ago
(In reply to Dave Hunt (:davehunt) from comment #1)
> I think you need --type=b2g+smoketest-bluetooth

I did that, that's still running only that 1 test I mentioned.
Flags: needinfo?(martijn.martijn)
(Reporter)

Comment 3

4 years ago
Ok, when linking to:
http://mxr.mozilla.org/gaia/source/tests/python/gaia-ui-tests/gaiatest/tests/manifest.ini
It goes well, that's because there's a bluetooth = false setting in there.

So I guess it's indeede the case that when something is not set, it is automatically set to true.
(Reporter)

Comment 5

4 years ago
Created attachment 8498970 [details] [diff] [review]
manifest.diff

Adding bluetooth = false to every manifest.ini would be a possibility, but I don't think it's really sensible. It doesn't really scale. Also, there are more of these variables that could be added.

This is a git diff, I don't know why /.idea/modules.xml has changes to it.
I'm not sure how to make a PR (=pull request) of this.

Anyway, I don't think this patch is sensible anyway.

Btw, I noticed this doesn't have smoketest set at all:
http://mxr.mozilla.org/gaia/source/tests/python/gaia-ui-tests/gaiatest/tests/unit/manifest.ini
That means running with --type=b2g-smoketest doesn't run anything in that direcotry at all.
GAIATEST_SKIP_WARNING=1 gaiatest --address localhost:2828 --profile $HOME/B2G/gaia/profile --testvars /Users/mwargers/B2G/gaia/tests/python/gaia-ui-tests/gaiatest/testvars.json --restart --type=b2g-smoketest /Users/mwargers/B2G/gaia/tests/python/gaia-ui-tests/gaiatest/tests/unit/manifest.ini
I don't think that makes sense.
(Reporter)

Comment 6

4 years ago
I guess just accepting that using "-" on type is not working always as expected, except when using it on the top level manifest.
Feel free to close this bug.
QA Whiteboard: [fxosqa-auto-backlog-]
(Reporter)

Comment 7

3 years ago
This was very likely fixed by bug 1099041.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Depends on: 1099041
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.