472197 - Define builders for l10n repacks, nightly, onchange l10n, onchange en-US

Status: RESOLVED FIXED

Description

[Axel Hecht [:Pike]]

We need to spec out what which l10n builds need to do. There's a dump of my thoughts on https://wiki.mozilla.org/User:AxelHecht/L10n_build_deliverables on what the different deliverables would be, and what should be run for each.

Each of those need to be run per project branch with l10n builds.

Mapping those specs to builder impls should balance pros and cons along these axes:

Different builders require disk space. Source dirs, logs, builder stati etc.

Making a single builder do different kinds of builds adds complexity like property-dependent steps etc.

All code paths should be getting as much continuous testing as possible, to prevent a release build from failing because its code wasn't run for a while, if possible.


I'm personally rather happy with running property-dependent steps, thus I vote for having a single builder per platform.

It might make sense to have the builder per platform per project branch, but that's 50% a reporting question.

Comment 1

[Chris Cooper [:coop] pronoun: he]

(In reply to comment #0)
> It might make sense to have the builder per platform per project branch, but
> that's 50% a reporting question.

This is how it *would* have been setup if we did have both m-c and 1.9.1 l10n running on the same master (vs. having 1.9.1 on bsmedberg's setup).

Armen: this seems like a facet of the build-on-change work, or at the very least will influence your implementation. Given that we discussed various builder scenarios yesterday in our l10n-releng meeting, mind if I reassign to you?

Comment 2

[Axel Hecht [:Pike]]

(In reply to comment #1)
> (In reply to comment #0)
> > It might make sense to have the builder per platform per project branch, but
> > that's 50% a reporting question.
> 
> This is how it *would* have been setup if we did have both m-c and 1.9.1 l10n
> running on the same master (vs. having 1.9.1 on bsmedberg's setup).

Well, there isn't really a build-process reason to do so. In my local setup, I run fx31 and fx32 on the same builder. They don't share a build dir, though, I have the "tree" as part of the workdir for my builder.

Comment 3

[Axel Hecht [:Pike]]

We're currently running a builder per branch and OS, but a shared builder for nightlies and depends, both en-US and l10n, on a single master and common slave pools for 1.9.1 and central.

My recent thinking is that we should drop doing repacks on en-US landings, and just update detailed compare-locale stats on a separate cross-platform builder. That one would build on l10n-changes, too.

Right now, the build factory isn't set up to use properties to control steps, but we'll get there I guess. Not sure if we'll end up merging the 1.9.1 and central builders again then.

So much for the status update here.

Comment 4

[Armen [:armenzg]]

Through the work I am doing for bug 480081 and as mentioned in email thread, I am going to separate the builders by using a parameter "nightly" for the l10n factory. This choice, even though it has the inconvenience of having more code checked out on the slaves, is easier to maintain from the relEng side since it behaves the same way that the enUS factories.

Comment 5

[Axel Hecht [:Pike]]

I don't buy that ad-hoc, and it will bloat the output. I'm not sure of the practical implications.

Note that splitting the builders will result in potentially more parallel l10n repacks as they nightly l10n builders will not block the depend builders from doing repacks.

Comment 6

[Armen [:armenzg]]

(In reply to comment #5)
> Note that splitting the builders will result in potentially more parallel l10n
> repacks as they nightly l10n builders will not block the depend builders from
> doing repacks.
>
Do you mean that the repack-on-change would not be at the end of the nightly l10n builder but in its own queue and therefore being able to be processed before the whole nightly batch gets done. Is that what you mean?

Comment 7

[Axel Hecht [:Pike]]

Yes.

Comment 8

[[e-mail] (back in 2019Q1)]

Does this bug have any use at this point? We have most of the things listed in the summary...

Comment 9

[Chris Cooper [:coop] pronoun: he]

This bug has served its purpose, and I think I'm fine with closing it now. 

We tried to merge the builders as much as possible to share source. If we want to merge things further (or break them out), we can file a specific bug.