1231618 - Set MOZ_SOURCE_{REPO,CHANGESET} for taskcluster builds because tc-vcs breaks autodetection

Status: RESOLVED FIXED

(Taskcluster :: Services, defect)

Description

[Mike Hommey [:glandium]]

The symptom is the following: try builds from taskcluster are getting wrong information about the repository they are built from. For instance, if you download a firefox tarball from a TC build, start it, and go to about:buildconfig, it will claim to have been built from mozilla-central instead of try.

How this information ends up in Firefox itself is essentially coded in config/makefiles/rcs.mk in m-c, and the core of it is just this:

> hg showconfig paths.default

And since when tc-vcs "clones" try, it starts with mozilla-central, what's left as paths.default is mozilla-central. I argue that tc-vcs should switch paths.default so that it reflects the final repo it's cloning, not the base.

This might be true of git clones too, I don't know.

Comment 1

[(not currently active) Ted Mielczarek]

Yeah, this is bad. This impacts:
* about:buildconfig
* The info in application.ini
* The source links in symbol files
* The source info files we upload next to each build (https://public-artifacts.taskcluster.net/LRLjlqenTQOS7O3yEoURPA/0/public/build/target.txt, https://public-artifacts.taskcluster.net/LRLjlqenTQOS7O3yEoURPA/0/public/build/target.json)

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

This is important to fix before we make desktop builds in TC tier 1.

Git uses

getSourceRepo = \
  $(shell cd $(topsrcdir) && $(GIT) rev-parse --verify HEAD)

which produces a sha1, not a repo.. so I guess there's nothing to fix there?

Comment 3

[(not currently active) Ted Mielczarek]

We don't actually ship builds out of git, so I'm not particularly worried about that. catlee suggested that alternately we could fix this by explicitly passing the repo + changeset in as parameters, and stop making the build system query hg directly.

Comment 4

[John Ford [:jhford] CET/CEST Berlin Time]

I'm going to be working on a replacement for tc-vcs this quarter and if we can ensure that we never query the vcs directly, we can make a pretty large optimization.  If we don't do hg commands directly, we don't need to actually put the .hg or .git repository on the workers, which is a huge savings in transfer times and makes caching a lot easier.

Comment 5

[Chris AtLee [:catlee]]

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #3)
> We don't actually ship builds out of git, so I'm not particularly worried
> about that. catlee suggested that alternately we could fix this by
> explicitly passing the repo + changeset in as parameters, and stop making
> the build system query hg directly.

let's just do this. Can we define MOZ_BUILD_CHANGESET / MOZ_BUILD_REPO or something and have the build system use that instead of calling hg directly?

Comment 6

[Dustin J. Mitchell [:dustin] (he/him)]

testing/docker/desktop-build/bin/checkout-sources.sh has that data in variables already.  Perhaps if those are set, the build system could just use them, and if not, query hg?

Glandium, is that something you could whip up a patch for?

Comment 7

[(not currently active) Ted Mielczarek]

There's a patch in bug 1241111 that fixes this in one place for another reason--Ubuntu's Firefox builds don't build from a Hg repo and so they were missing info that Telemetry needed. I asked the patch author if they could fix the remaining locations but haven't heard back yet.

Comment 8

[Mike Hommey [:glandium]]

cf. comment 7.

Comment 9

[Armen [:armenzg]]

This does not block tier2 tests. Removing dep.

Comment 10

[Selena Deckelmann :selenamarie :selena]

Based on bug 124111 is this a WONTFIX?

Comment 11

[Selena Deckelmann :selenamarie :selena]

Oops bug 1241111

Comment 12

[Dustin J. Mitchell [:dustin] (he/him)]

I think once that bug is complete, we'll set those vars in build-linux.sh.  Please let me know if I've misunderstood!

Comment 13

[Dustin J. Mitchell [:dustin] (he/him)]

Attachment: MozReview Request: Bug 1231618: set MOZ_SOURCE_{REPO,CHANGESET}; r?ted.mielczarek

Review commit: https://reviewboard.mozilla.org/r/34191/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/34191/

Comment 14

[(not currently active) Ted Mielczarek]

Comment on attachment 8717517 [details]
MozReview Request: Bug 1231618: set MOZ_SOURCE_{REPO,CHANGESET}; r?ted.mielczarek

https://reviewboard.mozilla.org/r/34191/#review31697

Thanks! We'll still need to get that other bug fixed about fixing our Breakpad symbols to fix this 100%, but that's basically just a build system bug.

Comment 15

[Dustin J. Mitchell [:dustin] (he/him)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/0bc377ee611a7311b7565969aceaa26b3c2d79ea
Bug 1231618: set MOZ_SOURCE_{REPO,CHANGESET}; r=ted.mielczarek

Comment 16

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/0bc377ee611a