1317525 - stop-gap VCS "syncing" service from mozilla-central and servo/servo to the incubator/stylo repo

Status: RESOLVED FIXED

(Developer Services :: Servo VCS Sync, defect, P1)

Description

[Chris Peterson [:cpeterson]]

The goals here are to:

1. merge upstream changes from mozilla-central and servo/servo into the incubator/stylo repository more frequently

2. and have better visibility into regressions via more frequent CI runs

Comment 1

[Gregory Szorc [:gps]]

This is on me.

Comment 2

[Gregory Szorc [:gps]]

Attachment: vcssync: synchronize upstream changes to stylo repository (bug 1317525)

THIS COMMIT IS INCOMPLETE.

Missing pieces:

* Push the result
* Integrating Mercurial changes
* MOAR tests
* Ansible/Terraform provisioning
* Docs

------

Over the past few years, several groups have also wanted some form or
another of version control "syncing." Example use cases include:

* Mirroring a Mercurial repository to Git[Hub]
* Mirroring a Git[Hub] repository to Mercurial
* Exporting a subdirectory of mozilla-central to Git[Hub] to
  facilitate development on Git[Hub]
* Importing content from a Git repository into mozilla-central
* Converting GitHub Pull Requests into MozReview review requests

VCS Sync facilitates #1. And we have haphazard implementations of
some of the other bullet points. Other items are pipe dreams.

As part of vendoring Servo and more tightly integrating the Servo
and Firefox CI systems, we'll be rolling out a somewhat complicated
change to how Servo and Firefox commits are synchronized and landed.
This requires writing some VCS syncing code. This is a known
consumer of a somewhat more complicated VCS syncing solution that
is a few weeks to months off.

In the short term, the Stylo team wants a stop-gap mechanism to
"synchronize" changes from Servo and Firefox upstreams to their
"stylo" incubator repository. This commit implements said mechanism.

This commit provides a daemon specialized for the Stylo team's
immediate needs. The daemon connects to Pulse and listens for
messages saying there has been a push to the GitHub Servo
upstream or the hgmo Firefox upstream. When a push occurs, the
changes are imported/merged/integrated into the "stylo" repository.

Because we have future needs for more advanced VCS syncing
functionality and because testing and code reuse is important,
I've factored the new code as a standalone "vcs sync" tool. There
is some generic code for importing a Git commit into a subdirectory
of a Mercurial repository. The bulk of the remaining code is
specific to the immediate needs of the Stylo team. No sense
being an architect astronaut until the future needs actually
manifest after all.

Review commit: https://reviewboard.mozilla.org/r/93330/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/93330/

Comment 3

[Gregory Szorc [:gps]]

Here's my proposal for the stop gap:

* We establish a new head on the stylo repository to track "upstream"
* The new head will be labeled with a bookmark. Tentatively named "upstream." If using cinnabar, that bookmark should get converted to a Git branch automatically.
* When there is a push to mozilla-central, mozilla-central is merged into the "upstream" head/bookmark and the result is pushed, triggering CI.
* Where there is a push to servo/servo@master, the contents of the servo/ directory in the "upstream" head/bookmark are replaced by the contents from servo/servo@master and committed and pushed, triggering CI.
* Periodically, a Stylo developer can merge a suitable commit from the "upstream" head into the main development head.

bholley: is this proposal satisfactory?

Comment 4

[Bobby Holley (:bholley)]

I think we'll need to regenerate bindings in there, but generally I think this sounds great!

CCing the rest of the team. Anyone else have thoughts here?

Comment 5

[Gregory Szorc [:gps]]

Where exactly do we need to regenerate bindings?

And how does the work to integrate rust-bindgen into servo upstream's build system change our needs to check bindings into the tree and regenerate them?

Comment 6

[Bobby Holley (:bholley)]

(In reply to Gregory Szorc [:gps] from comment #5)
> Where exactly do we need to regenerate bindings?

Any time Servo gets paired up with a different rev of m-c. So pretty much on every push to 'upstream'.

> And how does the work to integrate rust-bindgen into servo upstream's build
> system change our needs to check bindings into the tree and regenerate them?

The bindings need into servo/servo until servo/servo CI is able to run builds that generate them automatically (so presumably post-autolander).

Comment 7

[Manish Goregaokar [:manishearth]]

> check bindings into the tree and regenerate them?

We need to check bindings into the tree as long as Servo CI doesn't do m-c builds (i.e. until the autolander magic happens)

But when doing automatic CI on the m-c side we should use build-time bindgen.

Comment 8

[Bobby Holley (:bholley)]

(In reply to Manish Goregaokar [:manishearth] from comment #7)
> But when doing automatic CI on the m-c side we should use build-time bindgen.

That's a good point. If we land support-for build-time bindgen, we don't need any of the incubator machinery to regenerate bindings.

Comment 9

[Gregory Szorc [:gps]]

I've got everything in comment #3 working locally. That excludes bindgen foo for now.

I need to work out where to run this from and what SSH key to use. I'll likely stand up a cheap EC2 instance in the devservices AWS account.

Comment 10

[Gregory Szorc [:gps]]

Comment on attachment 8811108 [details]
vcssync: synchronize upstream changes to stylo repository (bug 1317525)

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/93330/diff/1-2/

Comment 11

[Cameron McCormack (:heycam)]

This sounds great, thanks Greg.

It does seem like build-time bindgen will simplify things a fair bit.  Without it, this syncing service would need to do a build with the new m-c/servo pair (at least until the "export" phase of the build is done, i.e. when headers have been generated in the objdir), run the bindgen script, check in its changes, then do a real build.

As Manish says in comment 7, we still need to update the checked-in bindings in servo/servo whenever the main development head of incubator/stylo moves.  Is there any way we can make that process more automated?  Maybe we could we have the syncing service push a Servo branch to GitHub somewhere that includes the updated bindings, which the human could make a pull request out of when deciding to update the incubator/stylo development head?

Comment 12

[Cameron McCormack (:heycam)]

What happens when the automated mozilla-central merge fails with conflicts?  We have some local changes in the branch, like updated taskcluster files or test descriptions, and these do sometimes conflict.  (Maybe we could move those changes to different files so they're less likely to conflict.)

Comment 13

[Gregory Szorc [:gps]]

(In reply to Cameron McCormack (:heycam) from comment #12)
> What happens when the automated mozilla-central merge fails with conflicts? 
> We have some local changes in the branch, like updated taskcluster files or
> test descriptions, and these do sometimes conflict.  (Maybe we could move
> those changes to different files so they're less likely to conflict.)

That's the point of having multiple heads. The "upstream" head will be pristine and there should be 0 merge conflicts because there is no drift.

However, there could be merge conflicts when merging "upstream" into the other head. Only a human can resolve those.

Comment 14

[Gregory Szorc [:gps]]

Of course, to have accurate CI results, the "upstream" head needs to be running servo/stylo automation. That means we'll want to get the automation changes to support this landed in mozilla-central. We'll likely want to limit those to running on commits that have a "servo" root directory so they don't waste automation resources running outside of stylo and stylo-try.

Comment 15

[Gregory Szorc [:gps]]

I have the syncing running on my local machine and configured to push to https://hg.mozilla.org/users/gszorc_mozilla.com/stylo in case anyone wants to have a look. I'll likely add some polish tomorrow and hopefully get this submitted for review by EOD.

Comment 16

[Bobby Holley (:bholley)]

(In reply to Gregory Szorc [:gps] from comment #14)
> Of course, to have accurate CI results, the "upstream" head needs to be
> running servo/stylo automation. That means we'll want to get the automation
> changes to support this landed in mozilla-central. We'll likely want to
> limit those to running on commits that have a "servo" root directory so they
> don't waste automation resources running outside of stylo and stylo-try.

I don't follow this last part, can you elaborate?

For servo changes, we can cull automation for anything that doesn't touch ports/geckolib or components/style. But we'll need to run on every m-c change.

Comment 17

[Xidorn Quan [:xidorn] UTC+11]

FWIW, build-time bindgen work is currently on-going in https://github.com/servo/servo/pull/14244. That requires mach to pass in MOZ_OBJDIR via environment variable, which I think should be trivial to add to gecko's build system.

Comment 18

[Gregory Szorc [:gps]]

(In reply to Bobby Holley (:bholley) (busy with Stylo) from comment #16)
> (In reply to Gregory Szorc [:gps] from comment #14)
> > Of course, to have accurate CI results, the "upstream" head needs to be
> > running servo/stylo automation. That means we'll want to get the automation
> > changes to support this landed in mozilla-central. We'll likely want to
> > limit those to running on commits that have a "servo" root directory so they
> > don't waste automation resources running outside of stylo and stylo-try.
> 
> I don't follow this last part, can you elaborate?

There is code currently unique to the stylo repository that hacks up CI configs to run the automation you want. Those hacks need to make their way to mozilla-central. But we likely want to change the code such that things are only running/special if running on the "stylo" or "stylo-try" repository. Since I'm a huge proponent of not keying behavior off repository names (the names don't mean anything: the content does), I think we should key that behavior off the presence (or lack thereof) of a "servo" directory in the revision being built.

Comment 19

[Bobby Holley (:bholley)]

Oh I see - sounds good. Getting local modifications out of our tree sounds great.

Comment 20

[Gregory Szorc [:gps]]

I think I have this running in production on a one-off EC2 instance [that only I can access]. But I need to wait for a push to mozilla-central and servo/servo to verify the daemon is working. In the interim, I manually ran some commands and pushed a bunch of updates to the "upstream" bookmark on the stylo repo. As part of that, I discovered that a change to mozilla-central broke stylo builds. Down the rabbit hole I go...

Comment 21

[Gregory Szorc [:gps]]

This is definitely running in production. I've seen several servo commits and a handful of mozilla-central merges go through. The automation on the "upstream" bookmark is busted. I'll need to continue iterating on that. (I've definitely spent more time on the Firefox automation side of this problem than the VCS side.)

Comment 22

[Gregory Szorc [:gps]]

This is done.

Commits to servo/servo are now automatically landing in https://hg.mozilla.org/integration/autoland. Latency looks to be ~20s under normal conditions. More info at https://mozilla-version-control-tools.readthedocs.io/en/latest/vcssync/servo.html and in an email I sent around to people most impacted by this.