585112 - [manifests] restart tests should be noted by manifests

Status: RESOLVED FIXED

(Testing Graveyard :: Mozmill, defect)

Description

[Jeff Hammel]

Currently, we denote running restart tests via `mozmill-restart -t path/to/tests`.  However, since the path/to/tests is already restart tests, the `-restart` is redundant since the test is inherently a restart test (or to put it another way, currently by its position within a directory structure it is already marked as a restart test).

Instead of having this redundancy, manifests can note which tests are restart tests or not.  Similarly, instead of the current structure of `test1.js`, `test2.js`, etc, the manifest may indicate the order of running.  

Note that this is not about changing any of the infrastructure of how Mozmill runs tests.  There will still be Mozmill and MozmillRestart which will run tests how they do now.  This is just about how to select which class to use for particular test.

As a side benefit, this may allow a single command line to run both restart tests and the normal tests.  Notably, the current infrastructure may have problems with this (as Mozmill is currently "run once and you're done"), but this will probably have to be fixed (or at least improved) for manifests anyways.

Comment 2

[Jeff Hammel]

fixed with bug 585106

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

This has already been fixed for Mozmill 2.