We turned on the l10n nightly repacks this Tuesday and on Wednesday we realized that the Mac installers do not upload because the URL they check shows as <update></update> For instance, the URL https://aus2.mozilla.org/update/3/Firefox/3.6a2pre/20090810031114/Darwin_Universal-gcc3/af/nightly/Darwin%209.8.0/default/default/update.xml?force=1 I believe matches to /opt/aus2/incoming/2/Firefox/mozilla-central/Darwin_Universal-gcc3/20090810031114/af in aus2-staging which contains the snippets (complete.txt and partial.txt) but the update is not provided. If we switch "af" for "en-US" we can see that the URL should be correct. morgamic is right now looking into this doing some AUS debugging; he might be able to drop some light into it.
Transferring notes from email thread. Morgamic wrote: ############### 22:57 < morgamic> nthomas: joduinn: that anomoly that armenz was seeing is caused by the fact that there is not an ar locale in mozilla-central 22:58 < morgamic> the logic for nightlies is to always offer the latest build 22:58 < morgamic> the latest build according to our system is the second to last build directory 22:59 < morgamic> in our case,: 22:59 < morgamic> [morgamic@dm-ausstage01 Darwin_Universal-gcc3]$ cd 20090811103133/ 22:59 < morgamic> [morgamic@dm-ausstage01 20090811103133]$ ls -l 22:59 < morgamic> total 4 22:59 < morgamic> drwxr-xr-x 2 ffxbld users 4096 Aug 11 12:21 en-US 22:59 < morgamic> that means, if you request with /update/3/Firefox/3.6a2pre/20090810031114/Darwin_Universal-gcc3/en-US/nightly/Darwin 9.8.0/default/default/update.xml?force=1 23:00 < morgamic> it'll show up null because it's look at 20090811103133 for the latest updates and that has no /ar/ directory So, question is why does 20090811103133 not have anything but an en-US build? Do we need to change the logic in AUS? Right now AUS is set to do this: check the explicit dir (the ones we use for directories) -- if something exists, use that data check partner fallback -- if something exists, use that data check the _second to last_ build in the platform dir to offer the latest nightly -- if that exists, use it if nothing found, offer null updates If that default behavior should change, let's discuss what we should change it to. I can write the accompanying tests to verify, etc. Logic is described here: AUS tests - http://mxr.mozilla.org/mozilla/source/webtools/aus/tests/ Screenshot of said tests - http://www.grabup.com/uploads/7392af50f2ebcf2339de1466288f5338.png Actual test runner - http://khan.mozilla.org:8080/VerifyAus?test Mike
Nick Thomas wrote: Ok, so I bet what happened to get that en-US only 20090811 was the Tuesday morning downtime pushing the en-US was 7 hours later than normal, so the l10n nightlies for that day used the 20090810 build (ie they ran before the new en-US completed). This should just come right tomorrow when we get a proper set of nightlies for both en-US and l10n. We could still hit this problem if there is bustage in the future, but I'm not sure how much time and effort is worth expending to fix it.
I wrote: This is not specific to one locale but to all locales for Mac (I rename the subject of this thread). What Nick says it is right, the en-US nightly for Windows and Mac happened after the locales were finished due to the downtime and they were repackage off the 10th's en-US installers. We will have to wait a couple of nights since today no l10n repackage succeeded due to version bumping. I was wondering why on Wednesday (11th) the l10n updates for Windows did work but not for Mac even though both of them their nightlies happened later than l10n repackages. What I think is the reason is that we never ended up having a nightly on the 11th for Windows even though we reschedule and forced it (builds #142 and #143) meaning that we did not end up with a directory with only "en-US" in it: -bash-3.2$ ls Darwin_Universal-gcc3/ | grep 200908 ... 20090810031114 20090811103133 20090812031634 20090813031939 -bash-3.2$ ls WINNT_x86-msvc/ | grep 200908 ... 20090810050215 20090812045356 20090813042749 This problem of en-US happening after the l10n repackages happen should be fixed the day that we will have a TrigerableL10n scheduler rather than triggering both independently
Nick Thomas wrote: > Ok, so I bet what happened to get that en-US only 20090811 was the > Tuesday morning downtime pushing the en-US was 7 hours later than > normal, so the l10n nightlies for that day used the 20090810 build > (ie they ran before the new en-US completed). This should just come > right tomorrow when we get a proper set of nightlies for both en-US > and l10n. Unfortunately this didn't happen due to the version bump on trunk. Windows and Linux have (a reduced subset of) locales dirs on dm-ausstage01 for today, but still no Mac. We *should* get more partials tomorrow, but that still doesn't explain why we are not even getting complete update offers for Mac l10n nightlies. > We could still hit this problem if there is bustage in the future, > but I'm not sure how much time and effort is worth expending to fix > it. Agreed, but only if we can make sure complete updates are always working as the fallback. > If that default behavior should change, let's discuss what we should > change it to. I can write the accompanying tests to verify, etc. > Logic is described here: > > * AUS tests - > http://mxr.mozilla.org/mozilla/source/webtools/aus/tests/ > * Screenshot of said tests - > http://www.grabup.com/uploads/7392af50f2ebcf2339de1466288f5338.png > Actual test runner - http://khan.mozilla.org:8080/VerifyAus?test Mike: do we need to add some l10n tests to that matrix, or would that not be testing anything new? I would be glad help with writing those tests if they would be useful.
(In reply to comment #4) Comment 4 was written by Chris Cooper. My only comments are comment 0 and comment 3.
Marking FIXED, just updated a German 3.7a1pre on the mac with an incremental update from the 15th to the 16th.
Status: NEW → RESOLVED
Last Resolved: 9 years ago
Resolution: --- → FIXED
Confirmed with es-MX. Yay!
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.