Looks like it has been busted for two days. http://nightly.mozilla.org/
Created attachment 459705 [details] [diff] [review] stopgap nightly.m.o patch This patch builds a dictionary of builds, and only includes the first of each file it encounters. Since the parsing url is sorted by date (descending), this only grabs the most recent build. It also links to the url sorted by date, which ameliorates the multiple-file issue in the index. This doesn't solve the root cause, however. I think the most elegant solution there is to have a latest- directory per branch per platform (e.g. latest-mozilla-central-win32) which would allow us to remove the directory/link and recreate it per upload, which sidesteps any per-file logic/regexes. More on that here: http://drkscrtlv.livejournal.com/320620.html (--link-to-latest would be bug 580495; there is no bug open to upload to a platform-specific latest- dir for firefox builds atm.) I'm willing to do that work if there's buy-in, but I won't be able to in the next few weeks. In the meantime, this patch should keep nightly.m.o from experiencing these symptoms.
Attachment #459705 - Flags: review?
Comment on attachment 459705 [details] [diff] [review] stopgap nightly.m.o patch http://hg.mozilla.org/webtools/nightly/rev/cbcac1390d56 How does this get pushed live?
Attachment #459705 - Flags: checked-in+
I removed the old builds, too.
Ok, the directory is cleaned up + the new script is live. Morphing the bug for the longer term fix.
Depends on: 580495
Priority: -- → P3
Summary: 2 copies of nightly builds since we changed the name → upload firefox nightlies to latest-BRANCH-PLATFORM directories
(In reply to comment #4) > Ok, the directory is cleaned up + the new script is live. > Morphing the bug for the longer term fix. Grabbing, as I *think* this is addressed by the proposal http://oduinn.com/blog/2011/01/11/change-to-fennec-firefox-xulrunner-directories-on-ftp-m-o/ (cross-posted in dev.planning https://groups.google.com/group/mozilla.dev.planning/browse_thread/thread/6a58551742033faa ) If I am missing something, and this proposal does *not* address the problem, please give details.
Assignee: nobody → joduinn
OS: Mac OS X → All
This does not address the problem. Comment 1 has details.
found in triage.
Assignee: joduinn → nobody
Assignee: nobody → coop
Component: Release Engineering → Release Engineering: Automation (General)
QA Contact: release → catlee
Whiteboard: [cleanup][automation] → [cleanup][automation][postupload]
Mass re-assign of post_upload.py bugs to Tomcat.
Assignee: coop → cbook
Product: mozilla.org → Release Engineering
We're moving to s3, this doesn't make sense anymore! We also have bouncer links that point at the latest version, eg: https://download.mozilla.org/?product=firefox-aurora-latest-ssl&os=win64&lang=en-US
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → WONTFIX
11:41 <Callek> wow we ``still`` have Firefox 57 in https://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-central/ This does make sense, until we get rid of latest-BRANCH dirs.
Status: RESOLVED → REOPENED
Resolution: WONTFIX → ---
Bug 1457816 for automated cleanup of the latest directories. That'll always have some sort of buffer for closed trees and needing to serve a nightly so this bug might still be a good plan. FWIW, we no longer have post_upload's --link-to-latest available. With some special set of AWS credentials we might build a list of keys in platform-specific latest, push in the new files, and delete any keys that should no longer be present. Bonus points for doing a cache invalidation using bug 1393990.
My vote is for killing the latest- dirs in favor of indexes.
Component: General Automation → General
Product: Release Engineering → Release Engineering
You need to log in before you can comment on or make changes to this bug.