HgAllLocalesPoller pulls locales' list from Hg raw index page instead of all-locales

RESOLVED FIXED

Status

Mozilla Messaging
Release Engineering
RESOLVED FIXED
8 years ago
6 years ago

People

(Reporter: standard8, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [l10n], URL)

(Reporter)

Description

8 years ago
The gd locale isn't in all-locales for trunk. It is in there for 1.9.2 branch.

For some reason the nightly builds on trunk are triggering nightly repacks for gd.

This probably isn't good that buildbot is not doing what we're expecting it to.

A brief look doesn't show anything obvious:

http://hg.mozilla.org/build/buildbot-configs/annotate/e79d95629787/thunderbird/master.cfg#l167

looks fine, but we should probably try and figure out what's happening as it may affect us later.

Comment 1

8 years ago
We want to start building gd on trunk, too, that's bug 619071.

Let's first figure out the possible hickup in the infrastructure before we wallpaper the symptoms?

gd is the only locale that differs in all-locales on the branch and central, fwiw.
Blocks: 619071
Looking at buildbot's logs for last night's nightly, I can see the trigger action going and grabbing the correct all-locales file and no gd.

2011-01-11 06:21:08-0800 [-] L10nMixin:: A list of locales is going to be requested
2011-01-11 06:21:08-0800 [-] L10nMixin:: Getting locales from: http://hg.mozilla.org/comm-central/raw-file/1ddc6fa330c51bfd5846c1fc173cfecf5b7b9416/mail/locales/all-locales
2011-01-11 06:21:08-0800 [-] releaseLocks(<buildbotcustom.steps.base.C instance at 0xce7938c>): []
2011-01-11 06:21:08-0800 [-]  step 'trigger' complete: success
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] L10nMixin:: loaded locales' list
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted el locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted zh-TW locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted af locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted ta-LK locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted tr locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted ca locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted it locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted cs locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted pa-IN locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted ar locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted bn-BD locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted nb-NO locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted eu locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted et locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted gl locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted es-ES locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted id locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted en-GB locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted ru locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted sr locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted nl locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted ja-JP-mac locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted lt locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted vi locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted ro locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted is locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted pl locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted be locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted fr locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted bg locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted sv-SE locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted sl locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted de locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted hu locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted pt-BR locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted fi locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted da locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted ja locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted he locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted ka locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted pt-PT locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted es-AR locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted sq locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted ko locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted nn-NO locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted ga-IE locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted sk locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted zh-CN locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted si locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted uk locale
2011-01-11 06:21:08-0800 [HTTPPageGetter,client] Submitted fy-NL locale

Comment 3

8 years ago
If I read the "l10n build" right, that's actually a depend build running?

If so, I can reproduce this with asturian, http://tinderbox.mozilla.org/showbuilds.cgi?tree=Mozilla-l10n-ast&hours=36.

Seems like the depend scheduler for l10n-depend builds doesn't pay attention to all-locales?
Allright, this was not nightly triggered, but l10n push triggered. IIRC, the l10n pollers just scrape /l10n-central/ for any locale in there, and don't actually trim anything out based on an all-locale file, Axel ?

Comment 5

8 years ago
I don't know jack about the current code base anymore, the time when that used to be shared code are long gone :-( .

Armen, do you know?
HgAllLocalesPoller just polls all locales, and the dependent scheduler is:
        c['schedulers'].append(Scheduler(
            name="%s l10n" % name,
            branch=poller_index,
            treeStableTimer=None,
            builderNames=depLocaleBuilders,
            fileIsImportant=lambda c: isImportantL10nFile(c, l10n_modules) and shouldBuild(c),
            properties={
                'app': product,
                'en_revision': 'default',
                'l10n_revision': 'default',
                }
        ))

I don't see anything in there (on the Firefox side too) that would restrict things to only the locales contained in all-locales, except for the nightly triggers.
http://hg.mozilla.org/build/buildbotcustom/file/tip/misc.py#l783

> hg_all_locales_poller = HgAllLocalesPoller(hgURL = config['hgurl'],
>                              repositoryIndex = config['l10n_repo_path'],
>                              pollInterval=l10nPollInterval)

It seems that HgAllLocalesPoller determines the list of locales from a page rather than all-locales:

http://hg.mozilla.org/build/buildbotcustom/file/d9e6e02b146f/changes/hgpoller.py#l378

http://hg.mozilla.org/releases/l10n-mozilla-1.9.1?style=raw

It seems that we have a bug.
Summary: Figure out why trunk nightly builds are triggering gd repacks when gd isn't in all-locales → HgAllLocalesPoller pulls locales' list from Hg raw index page instead of all-locales
Whiteboard: [l10n]

Comment 8

8 years ago
I'd disagree, as the poller determines which changes make it into the master, not which locales make it into the tree for which build.

Example would be (if we had them) fennec dep builds, we'd only want those for the locales on fennec, but still poll all of firefox' locale repos on the same branch.

The all-locales stuff should really be handled in the scheduler.

Updated

8 years ago
No longer blocks: 619071
I believe this is fixed now that we're sharing Firefox automation.  Please reopen if you disagree (and provide new details).
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.