Open Bug 1890435 Opened 10 months ago Updated 9 months ago

searchfox hg.mozilla.org / mercurial mozilla-central retirement to-do tracker

Categories

(Webtools :: Searchfox, task)

Product:
Webtools
Component:
Searchfox
Type:
task

Tracking

(Not tracked)

People

(Reporter: asuth, Unassigned)

References

Details

This bug is to track work that might need to be done when hg.mozilla.org's mozilla-central tree becomes non-canonical. The motivation is I did a little jumpscare when I realized my personal cinnabar m-c checkout (entirely from unified) does not match the gecko-dev rev hashes but this is expected; you need to start from gecko-dev to end up with gecko-dev compatible hashes. :glandium's blog under "So, what's next" at https://glandium.org/blog/?p=4346 covers how to get future-proof gecko-dev hashes (and would probably be the way to reboot the searchfox m-c tarball if necessary).

The status:

Known transition steps:

Relevant bugs / discussions that got us here:

There is no action to take at this time, I just wanted to memorialize my understanding for my future self and others.

Note there will be new sha1s in the new repo, so you'll have to think about that.

(In reply to Mike Hommey [:glandium] from comment #1)

Note there will be new sha1s in the new repo, so you'll have to think about that.

Can you elaborate on that? Do you mean:

  1. Searchfox needs to make sure to cut over to fetching directly from gecko-dev on github by the time the transition happens because anything mirrored from canonical github to hg.mozilla.org and then fetched via cinnabar from hg.mozilla.org will inherently have a different hash from the canonical gecko-dev hash.
  2. The transition plan calls for abandoning the existing gecko-dev hashes entirely?

Please feel free to point me at any existing docs rather than having to write up something new here. Thanks!

From my blog you linked above, at the end of the "So, what's next" section: "Now, what the announcement didn't say is that the Git repository WILL NOT be gecko-dev, doesn't exist yet, and WON'T BE COMPATIBLE". So, the latter. I'm not sure there's an existing doc about that, but if there is, Liz would probably know.

Flags: needinfo?(ehenry)

Thanks! I think I switched to scanning that section when I originally read the post when I got to the shell invocations and my brain didn't switch back to full reading until the next header.

The most pragmatic option is probably then that when we get to phase two from the announcement we:

  • create a new searchfox tree we'll speculatively call "gecko" and point it at the new github repo. It is configured to not be aware of mercurial as a thing. Presumably we'd create "gecko-beta" and "gecko-release" as well when the branches are cut and then "gecko-esrNNN", etc.
  • leave mozilla-central running like it is, fetching from hg.mozilla.org using cinnabar. This would stop updating when hg.mozilla.org gets turned off, although some minor shell script fixes might be required if treeherder's "mozilla-central" tree is changed rather than a new git-using tree with a new name brought online.
    • A minor enhancement we can make is to be able to configure the mozilla-central search box at the top of any m-c pages to reference the new gecko tree so that people don't end up accidentally searching in a stale tree since the m-c tip would eventually be a footgun. This could also affect the file path breadcrumbs too. Probably the "go to latest revision" link would also do well to bounce to the new tree too. (Go to latest revision is not currently able to follow renames/copies; I have some WIPs working towards that, but the "legacy tree forwarding" feature would probably disable that or just tunnel the information to the new tree if we ever get a hash translation layer.)

This avoids needing to complicate searchfox to be able to understand revisions as having an hg rev, a git rev, and a different git rev while not breaking existing permalinks. Also, we can avoid having to get clever in any of our shell scripts; we just fork everything in mozsearch-mozilla/shared for the new tree scripts to "gecko-shared". Also the new trees can be brought up experimentally in a decoupled fashion.

There isn't really documentation for this because the mapping hasn't been worked out yet. Some of that will get tracked in Bug 1888169. But essentially glandium is right, it won't be in the current gecko-dev and will end up being in a new repo and probably a new organization.

Flags: needinfo?(ehenry)

Thanks for the bug links!

If abandoning gecko-dev is a given, then it definitely sounds like the right plan for searchfox is to just introduce a new searchfox "gecko" tree that does not know about hg at all and have the legacy searchfox hg-based trees do the UX redirects I mention above to minimize the footguns as described in comment 4.

In the longer term, my WIP hyperblame/history/timeline work is intended to support us ingesting and storing revision "sidecar" data from https://buildhub.moz.tools/ so that we can provide better context for revisions and query/filter on the basis of things in terms of "things that changed in the last N releases/since build X/between build X and Y". This sidecar storage would let us provide an efficient revision translation layer so we could eventually replace the searchfox mozilla-central tree with a rewrite rule on the proxy or just have the searchfox "gecko" tree directly handle the URLs as a known alias. Rewrite maybe works better, as something like /gecko/map/hg/rev/HASH/PATH and /gecko/map/gecko-dev/rev/HASH/PATH would make other potential mappings like /gecko/map/build-id/rev/BUILDID/PATH more discoverable.

Moving this out from under the main hg-to-git bug 1863519 that tracks phase 1 of the migration; while this work is related it doesn't actually block phase 1.

No longer blocks: hg-to-git
See Also: → hg-to-git
You need to log in before you can comment on or make changes to this bug.