635486 - Upload logs for release builds to ftp

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P2)

Comment 1

[Chris AtLee [:catlee]]

Attachment: Don't upload win32 logs to unsigned directory, print out urls for release-to-dated

Comment 2

[Chris AtLee [:catlee]]

Attachment: Add log uploader for release and l10n builds

Required a bunch of work in the log uploader to get stuff into the right places.

For l10n work (bug 625189), we pass in --l10n or --l10n --nightly to log_uploader.py. The has the effect of appending the locale name to the buildername in the log filename, e.g. http://staging-stage.build.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-l10n/mozilla-central-macosx64-l10n-dep-eu-build20.txt.gz. Makes it easier to find the log you're looking for!

Logs for l10n dep builds go into the tinderbox-builds/<branch>-l10n/ directory, along with the dep repack binaries. I initially had the logs going into the directory of the en-US binary, which has the nice effect of not cluttering up the l10n dir with lots of log files. However, I think that having the logs in the l10n dir makes them easier to find. The downside is that we'll have a month's worth of logs in there (or whatever the retention policy is).

Logs for nightly l10n jobs go into the dated directory along with the repacks, e.g. http://staging-stage.build.mozilla.org/pub/mozilla.org/firefox/nightly/2011/02/2011-02-22-03-mozilla-central-l10n/mozilla-central-linux-l10n-nightly-tr-build6.txt.gz

Logs for release builds go into a new 'logs' directory in the candidates directory, e.g. http://staging-stage.build.mozilla.org/pub/mozilla.org/firefox/nightly/3.6.14-candidates/build2/logs/

I think the signing rsync exclude list and push to mirrors exclude list will need to be updated to ignore this directory. Any other ones you know about?

Comment 3

[Chris AtLee [:catlee]]

Comment on attachment 514183 [details] [diff] [review]
Add log uploader for release and l10n builds

http://hg.mozilla.org/build/buildbotcustom/rev/4c84f7acd230

Comment 4

[Chris AtLee [:catlee]]

Comment on attachment 514180 [details] [diff] [review]
Don't upload win32 logs to unsigned directory, print out urls for release-to-dated

http://hg.mozilla.org/build/tools/rev/fb1a62d3a053

Comment 5

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to comment #2)
> I think the signing rsync exclude list and push to mirrors exclude list will
> need to be updated to ignore this directory. Any other ones you know about?

Can't see any sign of this in the two patches here, did it happen elsewhere ?

Comment 6

[Chris AtLee [:catlee]]

(In reply to comment #5)
> (In reply to comment #2)
> > I think the signing rsync exclude list and push to mirrors exclude list will
> > need to be updated to ignore this directory. Any other ones you know about?
> 
> Can't see any sign of this in the two patches here, did it happen elsewhere ?

whoops! not yet

Comment 7

[Chris AtLee [:catlee]]

Attachment: rsync excludes

Comment 8

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 519589 [details] [diff] [review]
rsync excludes

Should fix up l10n verify too:
 http://mxr.mozilla.org/build/source/buildbotcustom/process/factory.py#5529

Comment 9

[Chris AtLee [:catlee]]

Attachment: more rsync excludes

Comment 10

[Chris AtLee [:catlee]]

Comment on attachment 519589 [details] [diff] [review]
rsync excludes

http://hg.mozilla.org/build/tools/rev/e0859b978817

Comment 11

[Chris AtLee [:catlee]]

Comment on attachment 519660 [details] [diff] [review]
more rsync excludes

http://hg.mozilla.org/build/buildbotcustom/rev/48a72b08bb38