Whenever the buildbot master is reconfig-ed, the DependentL10n scheduler decouples from the nightly scheduler and we stop getting l10n builds. We need to figure out a more robust way to start this scheduler.
Assignee: nobody → ccooper
Status: NEW → ASSIGNED
Priority: -- → P2
Can you replicate this with a dummy setup? I.e., is this a buildbot bug?
Yeah, it's sortof a Buildbot bug. Because of the way our configuration is organized Buildbot recreates all of the Schedulers on a reconfig. This causes the Dependent Scheduler to lose its upstream IIRC.
FWIW, Axel filed a similar buildbot ticket upstream a while ago: http://buildbot.net/trac/ticket/35 Given the longevity of the ticket, I don't expect it to be fixed upstream any time soon.
According to that bug (oops, who filed that, me?), we can be hit by this for triggerable schedulers, too. I'd love to see a test case, if this is biting strongly enough, I'd ponder tackling it upstream.
To get us back to the state where we can at least reconfig the master for other changes, we're going to switch from a dependent scheduler to a nightly scheduler for l10n. We'll give it an offset time from the en-US nightly so that we'll have a good chance of being as up-to-date as possible. Not ideal, but better than the current situation I think.
Switching to a nightly scheduler seems to be working fine in staging. I'll have a patch ready shortly. I'm also going to schedule two rounds of nightlies/day for l10n to give us some better coverage until we get build-on-change.
Created attachment 356536 [details] [diff] [review] Switch from dependent to nightly scheduler for l10n
Attachment #356536 - Flags: review?(armenzg)
Comment on attachment 356536 [details] [diff] [review] Switch from dependent to nightly scheduler for l10n The only comment is if you could change "tip" for "default"
Attachment #356536 - Flags: review?(armenzg) → review+
Attachment #356536 - Flags: checkedâ€‘in+ checked‑in+
I've scheduled downtime for tomorrow morning.
Now working in production.
Status: ASSIGNED → RESOLVED
Last Resolved: 9 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.