1384210 - Add support to run-task for checking out comm-central branches in addition to mozilla-central branches.

Status: RESOLVED FIXED

(Taskcluster Graveyard :: Docker Images, enhancement)

Description

[Tom Prince [:tomprince]]

I'm working on porting comm-central, and as part of that I'd like to reuse m-c's docker images; Thunderbird doesn't have any dependencies beyond what Firefox needs, so it doesn't seem to make sense to maintain separate images.

I'd like to add an additional option to `run-task` to allow checking out a comm-* tree in addition to mozilla-*.

Comment 1

[Tom Prince [:tomprince]]

Attachment: Bug 1384210 - Add comm-central checkout support to taskcluster's run-task;

Review commit: https://reviewboard.mozilla.org/r/161022/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/161022/

Comment 2

[Tom Prince [:tomprince]]

Comment on attachment 8889975 [details]
Bug 1384210 - Add comm-central checkout support to taskcluster's run-task;

https://reviewboard.mozilla.org/r/161022/#review166342

I'm not sure if it makes sense to factor out the common logic between the `GECKO_*` and `COMM_*` checkouts. I'd be happy to do that if it makes sense.

Also, this unconditionally checkout c-c after m-c. The builds I currently have running have c-c inside of m-c and there is [talk](https://bugzilla.mozilla.org/show_bug.cgi?id=648979) of doing that generally. But I'm not yet sure if that is going to work. It would be possible to make `run-task` smart enough to figure out if one path is the parent of another, and check them out in the appropriate order. But I think that change can wait until there is a need to do that.

Comment 3

[Tom Prince [:tomprince]]

Comment on attachment 8889975 [details]
Bug 1384210 - Add comm-central checkout support to taskcluster's run-task;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/161022/diff/1-2/

Comment 4

[Tom Prince [:tomprince]]

Here is a build that is using the new option: https://tools.taskcluster.net/task-inspector/#FRHdpgRzQZ69v0YQcQ5pow

Comment 5

[Gregory Szorc [:gps]]

Comment on attachment 8889975 [details]
Bug 1384210 - Add comm-central checkout support to taskcluster's run-task;

https://reviewboard.mozilla.org/r/161022/#review167526

Comment 6

[Tom Prince [:tomprince]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=c9ae228ecb9c6b565bd70745bd482092c815b023

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/bdad4ba95b6d
Add comm-central checkout support to taskcluster's run-task; r=gps

Comment 8

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

This patch apparently caused a webRTC test to crash in WebGL code: https://treeherder.mozilla.org/logviewer.html#?job_id=118758122&repo=autoland


https://hg.mozilla.org/integration/autoland/rev/d587eeda28f3504d63b49f04f4e4e8af6f8113d0


¯\_(ツ)_/¯



https://treeherder.mozilla.org/#/jobs?repo=autoland&fromchange=a9029a587fec3ad8f2d2ea7586b4d54a7c3265e5&noautoclassify&filter-searchStr=android%20mda3&group_state=expanded&tochange=d587eeda28f3504d63b49f04f4e4e8af6f8113d0

Comment 9

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Sorry, that backout got pushed into a bunch of build bustage, so can't be used to see if the backout fixed it.

Hopefully this range has green builds at the top: https://treeherder.mozilla.org/#/jobs?repo=autoland&fromchange=a9029a587fec3ad8f2d2ea7586b4d54a7c3265e5&noautoclassify&filter-searchStr=android%20mda3&group_state=expanded&tochange=c7f314bfb6f12775e54b75e0915a56bb8e77825e

Comment 10

[Gregory Szorc [:gps]]

There's a 0% chance this patch introduced a crash in Firefox.

Comment 11

[Tom Prince [:tomprince]]

I suspect that it only did so because it triggered a rebuilding of the docker image.

Here is a build that should fail in the same way:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=07f575c3b1bd7363bdb16dc103aecf7b055cc263

Comment 12

[Gregory Szorc [:gps]]

Non-determinism is a killer :/

Comment 13

[Tom Prince [:tomprince]]

It does look like triggering a new build of an identical docker image is enough to provoke a failure.

Comment 14

[Chris Cooper [:coop] (he/him)]

I spoke to gps about this yesterday. 

Now that task configuration is in-tree, job definitions and configs for Thunderbird should live in the comm-central repo. The are already mechanisms for layering/vendoring/merging the two repos together, and unless there's an extreme bootstrapping problem in play, we should actively avoid adding comm-central-specific code to mozilla-central.

Comment 15

[Gregory Szorc [:gps]]

The non-determinism in Docker image building is described in 1289812. Essentially, since we don't pin system packages, every time we generate new Docker images there's a chance a new system package will break something. This puts whoever the unfortunate person was to trigger the image rebuild in the position of hacking around it by using a known-good base image, pinning the offending package, or fixing the underlying issue.

Comment 16

[Tom Prince [:tomprince]]

I definitely want to put as much as possible of the Thunderbird-specific configs for in comm-central. On the other hand, I do want to avoid reinventing things that are already handled in mozilla-central.

I've been experimenting with getting taskcluster to run builds on comm-central, and have got a decision task that is picking up task definitions from comm-central (https://hg.mozilla.org/users/mozilla_hocat.ca/comm-taskcluster/file/tip/ci) using the `--root` option to `mach taskgraph`.

However, I do need some way to get both comm-central and mozilla-central checked out to be able to run `mach taskgraph` while pointing at that task definitions from comm-central. I'd like to reuse the docker images used by mozilla-central, since Thunderbird doesn't have any dependencies beyond what is needed for firefox.

I think this and Bug 1385055 are most of what I need to be able to get comm-central working on taskcluster. I know there are a couple of other places that reference GECKO_{BASE,HEAD}_{REPOSITORY,REF,REV}.

I'm sure as there will be other issues that come up, but I do definitely want to avoid comm-central specific changes to m-c. I do think this and Bug 1385055 are minimally invasive changes to get comm-central bootstraped.

Comment 17

[Tom Prince [:tomprince]]

The bug that caused the backout has been mitigated by https://hg.mozilla.org/integration/autoland/rev/f269855c3c1c

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f7c8940e634b
Add comm-central checkout support to taskcluster's run-task; r=gps

Comment 19

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/f7c8940e634b