872321 - Make it possible to disable individual gaia unit tests

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia, defect)

Description

[Jonathan Griffin (:jgriffin)]

It currently isn't possible to disable individual gaia unit tests, but we can add some sort of blacklist support to the Python testrunner used by buildbot.  This would allow us to green up the tests by excluding perma-failures that aren't addressed soon, so we can roll out the tests to m-c, m-i, etc.

Comment 1

[Jonathan Griffin (:jgriffin)]

Attachment: Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/9885

Pointer to Github pull-request

Comment 2

[Malini Das [:mdas] - Away, not checking bugmail]

Comment on attachment 751827 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/9885

Cool, if anyone unfamiliar with python errors is going to add to  disabled.json it might be helpful to add a try/catch around line 170: "disabled = json.loads(f.read())" in case their json is mangled and an unclear exception is thrown.

Comment 3

[Jonathan Griffin (:jgriffin)]

(In reply to Malini Das [:mdas] from comment #2)
> Comment on attachment 751827 [details]
> Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/9885
> 
> Cool, if anyone unfamiliar with python errors is going to add to 
> disabled.json it might be helpful to add a try/catch around line 170:
> "disabled = json.loads(f.read())" in case their json is mangled and an
> unclear exception is thrown.

Good point, I kept the exception but then print out a more human-friendly message, "Error while decoding disabled.json; please make sure this file has valid JSON syntax."

Comment 4

[Jonathan Griffin (:jgriffin)]

https://github.com/mozilla-b2g/gaia/commit/002322f147e5761b1244a808f6a82e3c7b85947f