Closed Bug 861283 Opened 10 years ago Closed 10 years ago

initial build support for Servo

Categories

(Release Engineering :: General, defect)

x86_64
Linux
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: bhearsum, Assigned: bhearsum)

References

Details

This bug is to track initial setup of builds of Servo. Basic requirements are:
* 64-bit Linux and Mac builds
* Builds per commit to the Servo repo (https://github.com/mozilla/servo)
* TBPL or other dashboard for results

Other information:
* Prerequisites are here: https://github.com/mozilla/servo/blob/master/README.md
* RoR is on github, likely need to mirror into our own Mercurial servers
** Including the submodules.
** vcs2vcs might need support for submodules
* Current build pulls down a known good rust to build rust with, we may need to preinstall or rehost that to avoid external dependencies.

Much more to come, this is just an initial braindump.
Just for anyone's interest, I just built servo on my Ubuntu 12.04 (x86_64) according to instructions at https://github.com/mozilla/servo and had to perform some additional steps:

sudo apt-get install libfreetype6-dev
sudo apt-get install libfontconfig1-dev
sudo apt-get install libglib2.0-dev
sudo ln -s /usr/lib/x86_64-linux-gnu/libpango-1.0.so.0 /usr/lib/libpango-1.0.so

Of course YMMV but might be worth confirming these and adding them to README if proved relevant.
Braindump of conversation with Graydon, Brian, and Jack today:
* Main developer interface is github pull requests. Current workflow is:
** When a new PR is created, bors creates a new commit for it
** When the PR is r+'ed, bors creates a new commit for it, merged against the latest master, and pushes it to a repository
** Buildbot notices the commit and runs a set of jobs against it
** If all jobs complete successfully and the commit can be fastforwarded to the latest master, the commit is landed in the master repository
** If any job fails an error is reported into the pull request with a link to the build on buildbot.

An example of this is: https://github.com/mozilla/rust/pull/5726.

Other notes:
* All of the above is done by a script that Graydon wrote called "bors".
* The Buildbot that does all the tests is publicly accessible. The parts of it that are used are:
** log viewing
** retriggering jobs
** triggering "try" builds with properties
* We should focus exclusively on Servo automation for now, but it's important that Rust+Servo automation stays in sync. This may mean moving rust automation onto RelEng infrastructure later
* Future considerations to take in mind: rust/servo want to be able to offer CI services for rust based projects.
Brian and I chatted some more yesterday, here's the plan we're going to move forward with:
* RelEng will setup buildbot+bors on a machine within our network
* RelEng will dedicate some slaves for running the jobs (currently thinking 2 mac, and on-demand EC2 instances for Linux)
* Barring no security or other blocking issues, we'll somehow make the Buildbot WebStatus available publicly. This is an interim step until we can have a TBPL or similar job status display.
* Some developers will have direct access to the master + slaves to enable poking at them if something goes wrong. This will be reevaluated in the future, particularly if we get to a place where Servo builds are happening on our primary pool of masters/slaves.
* Fixes/enchancements to the Servo buildbot master + bors will primarily be done by Servo people for now. Generally, deployment  of those changes will happen through RelEng.

Brian, does this match your understanding of what we talked about yesterday? Is there anything I missed?
Flags: needinfo?(andersrb)
Ben, yes I am in agreement.

I will be holding off on adding a mac bot to our current buildbot setup in anticipation of switching to RelEng infrastructure. Next week I do intend to see if I can set up bors on the existing buildbot, as I don't have any experience with that script yet and I imagine it will need to be adapted for Servo.
Flags: needinfo?(andersrb)
Depends on: 863276
Depends on: 863278
I had an initial conversation with opsec about the security considerations. They'll be getting back to me on Monday.
Depends on: 863277
Depends on: 863836
Depends on: 863839
Depends on: 863842
Still working with opsec on how we're going to organize some things. Hope to get that figured out today/tomorrow. I've also been working on some of the puppet manifests for deploying the buildbot master + slaves. Should have a better timeline on when this will be done after things are firmed up with opsec.
Depends on: 760093
Status update: We've settled on requirements with opsec and are now waiting for the new vlans to be created (bug 866212). Once those are ready we can start bringing up machinery and getting buildbot/bors running. I'm PTO Wed-Fri this week, so this definitely won't be completed this week.
Depends on: 873480
Depends on: 874091
Depends on: 874211
Depends on: 874637
Depends on: 874954
Depends on: 875127
Depends on: 875490
Depends on: 875559
No longer depends on: 863842
No longer depends on: 874091
No longer depends on: 875490
It's been awhile, but Bors + Buildbot are setup and working fine. We've seen many changes landed in the past week or two.

Still to do, all tracked in dep bugs:
* Bring mac machines online. This is blocked on re-imaging of the machines, which is possibly blocked on some network stuff.
* Security review of the Servo networks. This is waiting on OpSec, and we need to reverify that everything still works after it's been completed.
* Proxy a read-only WebStatus to a public location. This is blocked on a security review of Buildbot.
Not blocked on 760093 anymore. The mac machines are already reimaged. I enabled mac as one of the builders that Bors requires before merging to master in https://hg.mozilla.org/users/dmitchell_mozilla.com/puppet320/rev/56f3b98a6aab
No longer depends on: 760093
The remaining two bugs aren't real blockers. The Servo CI has been in use for weeks now, and is working very well as far as I know.

I do expect the public interface (bug 886356) to be available soon, but I don't know when the sec review (bug 874637) is happening.
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
No longer depends on: 874637, 886356
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
Component: General Automation → General
You need to log in before you can comment on or make changes to this bug.