Closed Bug 510080 Opened 15 years ago Closed 15 years ago

Mac l10n nightly repacks do not update

Categories

(Release Engineering :: General, defect)

x86
macOS
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: armenzg, Unassigned)

References

Details

We turned on the l10n nightly repacks this Tuesday and on Wednesday we realized that the Mac installers do not upload because the URL they check shows as <update></update>

For instance, the URL
https://aus2.mozilla.org/update/3/Firefox/3.6a2pre/20090810031114/Darwin_Universal-gcc3/af/nightly/Darwin%209.8.0/default/default/update.xml?force=1
I believe matches to
/opt/aus2/incoming/2/Firefox/mozilla-central/Darwin_Universal-gcc3/20090810031114/af
in aus2-staging which contains the snippets (complete.txt and partial.txt) but the update is not provided. If we switch "af" for "en-US" we can see that the URL should be correct.

morgamic is right now looking into this doing some AUS debugging; he might be able to drop some light into it.
No longer blocks: 480081
Blocks: 480081
Transferring notes from email thread.

Morgamic wrote:
###############
22:57 < morgamic> nthomas: joduinn: that anomoly that armenz was seeing is caused by the fact that there is not an ar locale in mozilla-central
22:58 < morgamic> the logic for nightlies is to always offer the latest build
22:58 < morgamic> the latest build according to our system is the second to last build directory
22:59 < morgamic> in our case,:
22:59 < morgamic> [morgamic@dm-ausstage01 Darwin_Universal-gcc3]$ cd 20090811103133/
22:59 < morgamic> [morgamic@dm-ausstage01 20090811103133]$ ls -l
22:59 < morgamic> total 4
22:59 < morgamic> drwxr-xr-x 2 ffxbld users 4096 Aug 11 12:21 en-US
22:59 < morgamic> that means, if you request with /update/3/Firefox/3.6a2pre/20090810031114/Darwin_Universal-gcc3/en-US/nightly/Darwin 
                  9.8.0/default/default/update.xml?force=1
23:00 < morgamic> it'll show up null because it's look at 20090811103133 for the latest updates and that has no /ar/ directory

So, question is why does 20090811103133 not have anything but an en-US build?  Do we need to change the logic in AUS?

Right now AUS is set to do this:
check the explicit dir (the ones we use for directories) -- if something exists, use that data
check partner fallback -- if something exists, use that data
check the _second to last_ build in the platform dir to offer the latest nightly -- if that exists, use it
if nothing found, offer null updates
If that default behavior should change, let's discuss what we should change it to.  I can write the accompanying tests to verify, etc.  Logic is described here:
AUS tests - http://mxr.mozilla.org/mozilla/source/webtools/aus/tests/
Screenshot of said tests - http://www.grabup.com/uploads/7392af50f2ebcf2339de1466288f5338.png
Actual test runner - http://khan.mozilla.org:8080/VerifyAus?test
Mike
Nick Thomas wrote:
Ok, so I bet what happened to get that en-US only 20090811 was the Tuesday morning downtime pushing the en-US was 7 hours later than normal, so the l10n nightlies for that day used the 20090810 build (ie they ran before the new en-US completed). This should just come right tomorrow when we get a proper set of nightlies for both en-US and l10n. 

We could still hit this problem if there is bustage in the future, but I'm not sure how much time and effort is worth expending to fix it.
I wrote:
This is not specific to one locale but to all locales for Mac (I rename the subject of this thread).

What Nick says it is right, the en-US nightly for Windows and Mac happened after the locales were finished due to the downtime and they were repackage off the 10th's en-US installers.
We will have to wait a couple of nights since today no l10n repackage succeeded due to version bumping.

I was wondering why on Wednesday (11th) the l10n updates for Windows did work but not for Mac even though both of them their nightlies happened later than l10n repackages.
What I think is the reason is that we never ended up having a nightly on the 11th for Windows even though we reschedule and forced it (builds #142 and #143) meaning that we did not end up with a directory with only "en-US" in it:
-bash-3.2$ ls Darwin_Universal-gcc3/ | grep 200908
...
20090810031114
20090811103133
20090812031634
20090813031939
-bash-3.2$ ls WINNT_x86-msvc/ | grep 200908
...
20090810050215
20090812045356
20090813042749

This problem of en-US happening after the l10n repackages happen should be fixed the day that we will have a TrigerableL10n scheduler rather than triggering both independently
Nick Thomas wrote:
> Ok, so I bet what happened to get that en-US only 20090811 was the
> Tuesday morning downtime pushing the en-US was 7 hours later than
> normal, so the l10n nightlies for that day used the 20090810 build
> (ie they ran before the new en-US completed). This should just come
> right tomorrow when we get a proper set of nightlies for both en-US
> and l10n.

Unfortunately this didn't happen due to the version bump on trunk. Windows and Linux have (a reduced subset of) locales dirs on dm-ausstage01 for today, but still no Mac.

We *should* get more partials tomorrow, but that still doesn't explain why we are not even getting complete update offers for Mac l10n nightlies.

> We could still hit this problem if there is bustage in the future,
> but I'm not sure how much time and effort is worth expending to fix
> it.

Agreed, but only if we can make sure complete updates are always working as the fallback.

> If that default behavior should change, let's discuss what we should
> change it to. I can write the accompanying tests to verify, etc.
> Logic is described here:
> 
> * AUS tests -
> http://mxr.mozilla.org/mozilla/source/webtools/aus/tests/
> * Screenshot of said tests -
> http://www.grabup.com/uploads/7392af50f2ebcf2339de1466288f5338.png
> Actual test runner - http://khan.mozilla.org:8080/VerifyAus?test

Mike: do we need to add some l10n tests to that matrix, or would that not be testing anything new? I would be glad help with writing those tests if they would be useful.
(In reply to comment #4)
Comment 4 was written by Chris Cooper. My only comments are comment 0 and comment 3.
Marking FIXED, just updated a German 3.7a1pre on the mac with an incremental update from the 15th to the 16th.
Status: NEW → RESOLVED
Closed: 15 years ago
Resolution: --- → FIXED
Confirmed with es-MX.
Yay!
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.