531873 - maemo multi-locale factory adds more steps to itself every time

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P2)

Description

[[e-mail] (:bhearsum)]

Because of the overriding of newBuild() in MaemoBuildFactory (http://hg.mozilla.org/build/buildbotcustom/file/tip/process/factory.py#l4057), every time a maemo nightly fires it re-adds the multi locales steps to itself, and we end up with builds that run the multi locale steps 2, 3, 4, or more times. This gets reset after a reconfig when the Factory is recreated.

We can't just have addMultiLocaleSteps run once, because we'll ignore any updates to maemo-locales in subsequent builds.

A clean way to fix this would be do multi locale builds in a makefile target in the mobile repo...which already has access to maemo-locales, and eliminates the need for Buildbot to retrieve things at runtime.

I'm no expert here though, so I'll defer to those that regularly work on this code.

Comment 1

[Armen [:armenzg]]

(In reply to comment #0)
> We can't just have addMultiLocaleSteps run once, because we'll ignore any
> updates to maemo-locales in subsequent builds.
> 
We probably we can check if the locales list has changed from the previous run. Taking for now and I will prioritize in the next few days.

Comment 2

[Axel Hecht [:Pike]]

Doing a make target is not a clean way to do this.

Note that it's highly unlikely that we'll use maemo-locales for the release factory.

The clean way to fix this is to create Builds with a list of given steps and not with self.steps, which I've always found rather peculiar in the releng factories.

Comment 3

[[e-mail] (:bhearsum)]

(In reply to comment #2)
> Doing a make target is not a clean way to do this.
> 
> Note that it's highly unlikely that we'll use maemo-locales for the release
> factory.

What about the nightly factories? If we're running multi locale builds there we should ensure that they're running properly.

> The clean way to fix this is to create Builds with a list of given steps and
> not with self.steps, which I've always found rather peculiar in the releng
> factories.

If that works, let's do it.

Comment 4

[Aki Sasaki (not active)]

Attachment: only add new steps for new locales

Not tested yet, will do so shortly.
This should only add new steps for new locales. We can't remove locales without a reconfig, but that should be rare.

Comment 5

[Aki Sasaki (not active)]

Comment on attachment 415710 [details] [diff] [review]
only add new steps for new locales

Ok, this won't work... it'll append steps after the upload.

Comment 6

[Aki Sasaki (not active)]

(In reply to comment #2)
> The clean way to fix this is to create Builds with a list of given steps and
> not with self.steps, which I've always found rather peculiar in the releng
> factories.

Ok, I'm going to do this... in init.
That'll mean that *adding* new locales will require a reconfig, as well... is that acceptable?  Is there an alternative, other than the makefile?

Comment 7

[Axel Hecht [:Pike]]

The thing is, in __init__ you don't really have async network, and I'd frown on sync network requests. That's why we pulled the stunt of putting the locale list into a passed-in build property.

I wonder, could you store the offset and locales=[] list in __init__, and insert the steps for new locales at that offset instead of appending?

Comment 8

[Aki Sasaki (not active)]

Heh. Yes, I could, and I'll think about going down that route.

On the other hand, if we were to use a makefile, the objection is that the reporting won't be great.

However, wouldn't the single-locale repack for that locale break as well, and report to the right place?

If we had contact emails in the locales file somewhere that could work as well ;-)

Comment 9

[Aki Sasaki (not active)]

Attachment: drop&recreate all multiLocale steps each build

> I wonder, could you store the offset and locales=[] list in __init__, and
> insert the steps for new locales at that offset instead of appending?

There isn't an insert method in the factory, and these steps are all at the end, so rather than directly massage self.steps, I just truncate self.steps and add the multiLocale steps each build.  This hopefully isn't too expensive (it's definitely less expensive than leaving it alone) since it's only for nightly and release builds currently.

Currently running test builds in s-m2; a second build is queued so there should be results in an hour+. Not sure if I'll be awake at that point to check.

Comment 10

[Aki Sasaki (not active)]

Comment on attachment 415842 [details] [diff] [review]
drop&recreate all multiLocale steps each build

2nd nightly also has 66 steps =)

Comment 11

[Armen [:armenzg]]

Comment on attachment 415842 [details] [diff] [review]
drop&recreate all multiLocale steps each build

hahaha I can only laugh; this is great way of solving it!

Comment 12

[Axel Hecht [:Pike]]

Patch looks good, but can you set useProgress=False for multi-locale builds? That's breaking up when you remove steps.

Comment 13

[Aki Sasaki (not active)]

Attachment: set useProgress to False

I think this is what you mean?

Comment 14

[Axel Hecht [:Pike]]

Comment on attachment 415938 [details] [diff] [review]
set useProgress to False

yep, exactly.

Comment 15

[Aki Sasaki (not active)]

Comment on attachment 415938 [details] [diff] [review]
set useProgress to False

http://hg.mozilla.org/build/buildbotcustom/rev/bf18fb42ade4

Comment 16

[Armen [:armenzg]]

This builds adds the set of steps 5 times.
http://production-master02.build.mozilla.org:8010/builders/Maemo%20mozilla-1.9.2%20nightly/builds/139

Comment 17

[Aki Sasaki (not active)]

Unless someone was reconfiging pm02 all over Xmas weekend, this isn't happening anymore.