562042 - l10n nightlies don't upload due to missing previous nightly

Status: RESOLVED DUPLICATE of bug 573999

(Release Engineering :: General, defect, P5)

Description

[Axel Hecht [:Pike]]

Didn't find this bug on file.

When we're adding new locales, the nightlies are busted as they don't find the previous nightly.

The bad part is, they're busted before uploading the current nightly, so without manual action to actually upload something, they never actually upload anything.

Found that while adding 4 new locales today.

Gonna add more locales soon, too.

Comment 1

[Chris Cooper [:coop] (he/him)]

This isn't necessarily l10n-specific. If we were to turn on partial updates for a new branch or platform without first running with just complete updates for at least one nightly cycle, it would break the same way for en-US.

nthomas and I tossed around a bunch of ideas about how to get around this when I was developing the updates-on-slaves patches, but decided not to block on it. 

There are a few options:

* (current workaround) manually copy and rename existing complete mars (usually en-US) as placeholders for the new locale so the nightly automation can download it on the first pass. No one will have an existing build for the new locale yet, so the partial update will be meaningless anyway. This is what I did today for ast, gd, ilo, and mai.

* upload packages as they are generated. `make l10n-upload` uploads everything in one go at the end, so we'd need to make some code changes there.

* make the partial mar generation steps in buildbot conditional on downloading the previous mar. Mark a download failure of the previous mar as a WARNING, and continue on to the upload step. What version of buildbot do we need to be running to be able to use conditional steps? IIRC, it's property-based, so we could key off the previousMarFilename.

Other suggestions?

Comment 2

[Axel Hecht [:Pike]]

I'd personally favour incremental uploads, I guess.

Comment 3

[Chris AtLee [:catlee]]

(In reply to comment #1)
> * make the partial mar generation steps in buildbot conditional on downloading
> the previous mar. Mark a download failure of the previous mar as a WARNING, and
> continue on to the upload step. What version of buildbot do we need to be
> running to be able to use conditional steps? IIRC, it's property-based, so we
> could key off the previousMarFilename.

I prefer this version.  Could we have a script that does the download/generate updates/upload process?  The logic for skipping the updates if it can't download the previous mar could then be encoded into the script.  It should turn the build orange in any case, since it should only happen the first time you're spinning up a branch or locale.

Comment 4

[Nick Thomas [:nthomas] (UTC+12)]

Fixed by bug 573999.