Closed Bug 397876 Opened 14 years ago Closed 14 years ago

Nightly updates are not available for builds later than 20070927


(Release Engineering :: General, defect, P1)



(Not tracked)



(Reporter: nthomas, Assigned: preed)




(2 files)

This is fallout from the stage migration project, as we didn't have sufficient time to test partial update generation with the change in directory layout. Updates past the 20070927 nightlies are not available for the moment, but this bug is marked with our highest priority for getting fixed.
Blocks: 394069
Alright, someone sanity check my process to fix this:

The first step is to make the ftp area look like it moved too... so I basically move the mars locally, then rsync them, so when we rsync back to the server, all the times will be the same; then we delete everything older than 5 days, just like the nightly patcher would:

pushd firefox/nightly/ && mkdir -pv 2007/09 && mv -v 2007-09* 2007/09/ && popd
pushd thunderbird/nightly/ && mkdir -pv 2007/09 && mv -v 2007-09* 2007/09/ && popd
pushd calendar/sunbird/nightly/ && mkdir -pv 2007/09 && mv -v 2007-09* 2007/09/ && popd

rsync -av  --exclude=*.zip --exclude=*.dmg --exclude=*.xpi --exclude=*.exe --exclude=*.gz --exclude=*.bz2 --exclude='*.complete.mar' -e ssh ./firefox/nightly/2007/09/

rsync -av  --exclude=*.zip --exclude=*.dmg --exclude=*.xpi --exclude=*.exe --exclude=*.gz --exclude=*.bz2 --exclude='*.complete.mar' -e ssh ./calendar/sunbird/nightly/2007/09/

rsync -av  --exclude=*.zip --exclude=*.dmg --exclude=*.xpi --exclude=*.exe --exclude=*.gz --exclude=*.bz2 --exclude='*.complete.mar' -e ssh ./thunderbird/nightly/2007/09/

find -depth -mtime +5 -type f -exec rm -fv {} \;
find -depth -type d -empty -exec rmdir -v {} \;

Then, after running that, I run the nightly patcher with the patch I'm about to attach; the patch basically mungs up FTP and URL paths, so that the snippets it generates points are the new directory structure.

I'm running a test version of this right now that won't upload any of the results (runs ryncs with -n), so I can verify that this process works.
I will obviously take the -n's out of the rsync when this goes into production.
Attachment #282799 - Flags: review?(nrthomas)
Attachment #282799 - Flags: review?(rhelmer)
Comment on attachment 282799 [details] [diff] [review]
Munge us up some paths!

>+    if ($to_url =~ m|^http://ftp\.mozilla\.org/pub/mozilla\.org/(\w+)/nightly/(\d{4})\-(\d{2})\-(\d{2})\-(\d{2})\-(\w+)/(.+)|) 

Looks like there are a couple of problems with this
 * the first (\w+) won't match "calendar/sunbird"
 * the second (\w+) won't match milestones like mozilla1.8.0

I'm doing a test run now with this instead:

if ($to_url =~ m|^http://ftp\.mozilla\.org/pub/mozilla\.org/([\w\/]+)/nightly/(\d{4})\-(\d{2})\-(\d{2})\-(\d{2})\-([\w\d\.]+)/(.+)|)

for all three cases. That's in /builds/nightly-partial-generation-testing-cf, logging to test-fixed-regexp.log if you're curious. I blew away build/, aus/, and ftp/ for a clean start.

Otherwise looks fine, but as I don't know the code very well so I'm mostly looking at results.
Attachment #282799 - Flags: review?(nrthomas) → review-
Attached patch Production patchSplinter Review
Paul's patch, with my regexp tweak and the testing changes removed.
This the full patcher log,
See in particular the end where there are rsync dry runs for pushing the snippets back to the update server & partial mars to ftp.

* All the mars are in the ../nightly/2007/09/200... style, except for a couple of Sunbird from newref tbox (where updates were left on by accident)
* a collection of update snippets, which I grepped over to make sure they all use the right style of url (excepting those 2 odd balls)

If you guys are happy with the regexp tweak, then I think we can deploy the previous attachment.
Attachment #282799 - Flags: review?(rhelmer)
Landed attachment 282919 [details] [diff] [review] as revision: 1.15, with "... if ($verbose);" on the print's to STDERR.
The nightly update catch-up has run, and I checked with some QA peeps; they just got an update to the 1007100104 build.

I'll start the nightly-updater running again (from cron) and I'll leave this bug open until we have reports of good updates to tomorrow's build.
Thunderbird trunk was updated successfully from 2007100103 to 2007100123

Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9a9pre) Gecko/2007100123 Thunderbird/3.0a1pre ID:2007100123
I got a Firefox update this morning myself, so I'll mark this closed.
Closed: 14 years ago
Resolution: --- → FIXED
Has anyone had a successful update in Linux
(In reply to comment #10)
> Has anyone had a successful update in Linux
> See:

I just did.
Verified FIXED; this has been fine since the change.
Product: → Release Engineering
You need to log in before you can comment on or make changes to this bug.