[moztest] Tests are not uniquely identified
Categories
(Testing :: Mozbase, task, P3)
Tracking
(Not tracked)
People
(Reporter: ahal, Unassigned)
Details
Sometimes developers wish to land a pattern like this:
parent1.ini
[DEFAULT]
prefs = foo=bar
[include:child.ini]
parent2.ini
[DEFAULT]
prefs = foo=baz
[include:child.ini]
child.ini
[test1.html]
[test2.html]
This way, the same set of tests can be run twice with different configurations but only defined in a single spot. The problem is that the tests are not uniquely identified, as test harnesses typically use the file system path of the test as an identifier.
The real unique identifier, would be something like path/to/parent.ini:path/to/child.ini:path/to/test1.html. We should update our test metadata structures to account for this.
This bug will likely break the world however, as we'll need to update everything that consumes test metadata from test harnesses, to mozlog, to the build system and even ActiveData / external consumers.
|
Reporter | |
Comment 1•5 years ago
|
||
An alternative solution would be to remove the ability to include manifests and figure out alternative ways to solve the use cases that those are currently being used for. Though, thinking about it even then the same test file could be registered by two different manifests. So the test path would still not be a truly unique identifier.
Description
•