1453707 - Allow TPS to run through Mach

Status: NEW

(Firefox :: Sync, enhancement, P3)

Description

[Thom Chiovoloni [:tcsc] (ex-moco)]

:davehunt mentioned in IRC (log at end) he'd be interested in doing this, so I'm filing a bug for it.


It would be nice, and would avoid breakage due to things like bug 1453496, and would make running TPS tests much more straightforward. We could probably get rid of a good amount of cruft around the config as well.

Additionally, it would be nice if it could run as a tier 2 job on taskcluster, and I think this is needed for that to happen. If it did happen, it would help avoid inadvertent breakage for other reasons, allow us to more easily run TPS on patches in mozreview, and would probably allow us to be more comfortable relying on it for test coverage.

Anyway, I don't plan on doing this any time soon (If I were to spend time on TPS it would be for bug 1451527) but would be thrilled to review patches for it.

The biggest caveat is that care should be taken to keep https://github.com/mozilla-services/sync-tps-setup working.

IRC log from #sync this morning:

<davehunt> tcsc: saw your comments on the mozprofile bug. I believe tps could run in taskcluster, it just won't be tier 1 if it depends on external resources
<davehunt> I think it would make a lot of sense having it run from mach, and to run on taskcluster rather than jenkins
<davehunt> but of course, there would be some work involved
<tcsc> oh yeah its a big pain as is
<davehunt> would it be worthwhile? I wouldn't mind working on it, depending on other priorities
<davehunt> I also think it would be neat to have a tps.xpi artifact from taskcluster, which the ios sync tests could use, rather than duplicating the source code
<tcsc> well, up to you. theres things that need doing more urgently. like tps also needs to be a webextension, but thats also might not be something you find actionable
<davehunt> yeah, I'm less familiar with that area
<tcsc> you also might need to talk to rpapa to ensure things stay working for automation, possibly. i guess taskcluster would replace that though? idk
<tcsc> yeah, lots of fiddly desktop js code required for that, i don’t blame you
<davehunt> yeah, that would be my thinking, though it would be important not to disrupt what rpapa has
<davehunt> it may not be needed at the moment, but it certainly feels like technical debt :)

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Attachment: Bug 1453707 - Add mach command for running TPS

This is a work in progress, and not ready to land. I just wanted to see what would be needed to introduce a mach command for running TPS. This will run the TPS tests against the local build, unless --binary is specified with a valid path. It still requires a configuration file, and some values are currently hard-coded, which should probably be configurable. It uses the in-tree mozbase dependencies to avoid issues with downloading from and publishing these packages to/from PyPI.

Review commit: https://reviewboard.mozilla.org/r/238366/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/238366/

Comment 2

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 8969596 [details]
Bug 1453707 - Add mach command for running TPS

tcsc: This is my working patch. I'd be happy to continue working on this later, but if someone else decides to pick this up, maybe this will help them to get started.

Comment 3

[Thom Chiovoloni [:tcsc] (ex-moco)]

Comment on attachment 8969596 [details]
Bug 1453707 - Add mach command for running TPS

This looks good to me. One comment I have is does this allow running firefox headlessly? I guess that would probably still work with MOZ_HEADLESS the way it currently does, but it might be a good option to allow.