550940 - re-enable l10n mobile repacks

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P1)

Description

[Armen [:armenzg]]

We currently have them disabled (bug 548178) because one of the dependent builders keeps on re-spawning a job.

In the past we have restarted the master to fix this (bug 548467) and now the master is again in a weird state (can't re-schedule Maemo nightly builds).

I propose doing four things:
* scheduled restart of the pm02
* run purge_events.py to get the master in a better shape
* write a patch to disable repacks-on-change for mobile (to be able to disable repacks-on-change in case the re-spawning job appears again)
* enable the nightly mobile repacks

We need to get the nightly repacks going ASAP and we can debug the re-spawning problem in the future assuming that we find a way to reproduce it.

Comment 1

[Armen [:armenzg]]

Attachment: use "enable_l10n_onchange" flag across configuration files and have it disabled for mobile on pm02

I can reduce this patch to *just* mobile l10n if that makes it less scary.

Comment 2

[Armen [:armenzg]]

Attachment: use "enable_l10n_onchange" for misc.py

Comment 3

[Aki Sasaki (not active)]

Comment on attachment 431184 [details] [diff] [review]
use "enable_l10n_onchange" for misc.py

>-    if config['enable_l10n']:
>+    if config['enable_l10n_onchange']:

I think the behavior should only happen if both enable_l10n and enable_l10n_onchange are set.

You're checking that correctly everywhere but here, line 496.  I think you want

    if config['enable_l10n'] and config['enable_l10n_onchange']:

Have you tested this? It looks good to me otherwise at this level.

Comment 4

[Aki Sasaki (not active)]

Comment on attachment 431184 [details] [diff] [review]
use "enable_l10n_onchange" for misc.py

r+ with my nit in comment 3.

jhford just noticed that nightlies were missing, so looks like this patch is just in time. Thanks Armen!

Comment 5

[Aki Sasaki (not active)]

Comment on attachment 431184 [details] [diff] [review]
use "enable_l10n_onchange" for misc.py

http://hg.mozilla.org/build/buildbotcustom/rev/d67604a1a0e9

Comment 6

[Aki Sasaki (not active)]

Comment on attachment 431183 [details] [diff] [review]
use "enable_l10n_onchange" flag across configuration files and have it disabled for mobile on pm02

http://hg.mozilla.org/build/buildbot-configs/rev/462c397333bb

Comment 7

[Aki Sasaki (not active)]

Landed and both pm* reconfiged.  However, pm02 is not updating twistd.log as of Friday morning =(

Comment 8

[Armen [:armenzg]]

HAHAHAHA This is great!!
"Write the code and they shall take care of the rest" ;)

I can verify that all Maemo l10n nightly builds have been triggered and are back to normal.

I am declaring this FIXED as wrt to enabling again the Maemo builds. I will file another bug for the actual downtime for pm02 and purging of events.