1199128 - Exporting esr38 to gecko-dev does not export BUILD tags

Status: NEW

(Developer Services :: General, task)

Description

[Georg Koppen]

We messed up our 38.2.1 based Tor Browser build as it was not obvious to find the right commit to build from using gecko-dev. glandium suggested that it may be worth to at least export the BUILDn tags to git to make things easier.

Comment 1

[Mike Hommey [:glandium]]

Ideally, the _RELEASE tags should be there, but we unfortunately change them, so at the very least the _BUILDn would be nice. This is especially important because the way we release things, the tip of the "default" branch on a repository is *never* what is released. But, the git branches on gecko-dev are following the "default" branch, and since there are no essentially no tags on gecko-dev, good luck finding the right head (which you can, if you know the name of the GECKO_*_RELENG branch used, but you shouldn't have to know).

Comment 2

[Mike Hommey [:glandium]]

Note the summary for this bug could well be "gecko-dev has (almost) no tags"

Comment 3

[Gregory Szorc [:gps]]

How about Tor use the canonical Mercurial repositories instead? You can find pre-generated bundles at https://hg.cdn.mozilla.net/, which should download as fast as your internet connection supports.

We can and should add hashes and sign a manifest of said hashes for the paranoid. But Mercurial verifies the SHA-1 as part of reading repository data. If you query https://hg.mozilla.org/releases/mozilla-esr38/json-tags you can get the SHA-1 of the changeset you need. Or you can `hg pull` that repo and parse the output of `hg tags`.

Because the Git repos are non-canonical and our tags situation is a mess, I'm tempted to wontfix this because getting tags agreement in gecko-dev is going to be a PITA.

Comment 4

[Nicolas Vigier]

If it's not possible to get the tags in git, as a workaround I think we can get the hg changeset using the tag, then use the hg -> git mappings to find the corresponding git commit:
https://wiki.mozilla.org/ReleaseEngineering/Applications/Mapper#GET_Routes

Comment 5

[Georg Koppen]

(In reply to Gregory Szorc [:gps] from comment #3)
> How about Tor use the canonical Mercurial repositories instead? You can find
> pre-generated bundles at https://hg.cdn.mozilla.net/, which should download
> as fast as your internet connection supports.
> 
> We can and should add hashes and sign a manifest of said hashes for the
> paranoid. But Mercurial verifies the SHA-1 as part of reading repository
> data. If you query https://hg.mozilla.org/releases/mozilla-esr38/json-tags
> you can get the SHA-1 of the changeset you need. Or you can `hg pull` that
> repo and parse the output of `hg tags`.

This does actually fit poorly to our otherwise fully git-integrated workflow with Gitian which we use for our deterministic builds. So, I'd rather avoid that if possible.

> Because the Git repos are non-canonical and our tags situation is a mess,
> I'm tempted to wontfix this because getting tags agreement in gecko-dev is
> going to be a PITA.

Hmm... I am probably missing a lot of pieces but what is tricky about exporting at least the BUILD tags which are not changing as far as glandium points out?

Comment 6

[Gregory Szorc [:gps]]

Mercurial stores tags in a .hgtags file and then resolves the current changeset a tag refers to by traversing all heads and applying a last-write-wins type of scenario. This behavior is somewhat wonky. Further complicating matters is that for historical reasons we maintain multiple Mercurial repositories for Firefox. The gecko-dev Git repo is an amalgamation of all these repos.

From the perspective of someone who knows how the hg->git replication works, adding tags to the mix scares me: there's just a lot that can go wrong.

I know it is more complexity for you, but I would recommend a) using Mercurial because it is canonical (preferred) b) using the aforementioned json-tags API + the mapper service from comment #4 to resolve the Git SHA-1 for a Mercurial tag.