860005 - next round of work for manifestdestiny

Status: RESOLVED WONTFIX

(Testing :: Mozbase, defect)

Description

[Jeff Hammel]

ManifestDestiny has been implemented as a working proof of concept
which is currently used by xpcshell as well as several other
out-of-m-c harnesses for tracking test files.  The form of the
solution as supporting our desire for test manifests (see
https://mozbase.readthedocs.org/en/latest/manifestdestiny.html#why-have-test-manifests
) has proven a good match for the problem, key functionality should be
extended in the renewed effort to get more harnesses to use manifests.
This tracking bug notes the next round of effort towards this end.

Key issues:

* functionality to compare/diff the test files noted from a manifest
  with what is on disk [bug 683660 , bug 857966]
  - what files are not included in the manifest?
  - what files are in the manifest that are not on disk?

* TestManifest condition validation: bug 857081

* tracking test failures (e.g. bug 729098): it is often desirable to
  note what tests have failed from a particular run.  The sought
  answer is the output of a manifest of failed tests.

Since ManifestDestiny is just about handling manifests, some pieces of
these issues will need to be handled by the harnesses.  The vector of
this bug points towards
- needed API (etc) enhancements to manifestparser.py
- development of a skeletal (moztest) harness to illustrate and
  implement standard mozbase usage practice
- improved documentation (etc) about how all of this fits together

Comment 1

[Jeff Hammel]

Other issues not triaged here:
- updating to python 2.7 (e.g. deprecation of read_ini for ConfigParser)
  - we don't *actually* have to wait for Talos
  (though note https://bugzilla.mozilla.org/show_bug.cgi?id=858084#c4 )
- breaking up ManifestDestiny into separate files: the existing all-in-manifestparser.py was chosen for reasons that are now obselete.  we should modularize this

Comment 2

[Geoff Brown [:gbrown]]

Mass closing bugs with no activity in 2+ years. If this bug is important to you, please re-open.