As shown in bug 996003, it's currently possible for tests to slip through the cracks since the subsuite definition lives at the manifest level. It would be nice if we had a way to define the subsuite at a higher level so, for example, all tests found in browser/devtools can get that definition whether each manifest has the declaration or not.
I am not sure if there is a good solution here. If browser/devtools/browser.ini had a bunch of include:manifest directives, we could make it work, but right now we just include all the directories in the master manifest. I agree this is a problem in general, I just don't know the best solution.
Could a test manifest not just inherit any subsuite deinitions that were defined in a manifest found in a directory at a higher up level in the tree?
ie: browser/devtools/browser.ini: [DEFAULT] subsuite = devtools browser/devtools/debugger/test/browser.ini: [DEFAULT] # Unnecessary since inherits from the first manifest that specifies a subsuite, when traversing parent directories. # subsuite = devtools
we don't have a concept of directories in the manifests, but we do have a hierarchy when it comes to the includes: mochitest.ini: [include:devtools.ini] [include:browser/.../browser.ini] devtools.ini: subsuite = devtools [include:browser/devtools/.../browser.ini] [include:toolkit/devtools.../browser.ini] In the above case all the includes in devtools would automatically be part of the subsuite devtools.
You need to log in before you can comment on or make changes to this bug.