Closed Bug 1217223 Opened 9 years ago Closed 9 years ago

Try binaries and logs being uploaded to separate directories on archive.mozilla.org

Categories

(Release Engineering :: General, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: whimboo, Assigned: nthomas)

References

Details

Attachments

(1 file)

As I have seen today while trying to get a try build via treeherder, we point to the wrong folder with the short rev, while the build is located in a folder with the full rev:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=a4e305468a92

Dir 	mdeboer@mozilla.com-a4e305468a92/ 		
Dir 	mdeboer@mozilla.com-a4e305468a92516fa01d518da152b8e383d61b6b/ 	

We should that clean-up and always use a single folder for everything. Also treeherder has to show this URL. I assume the full rev should be used then.
Hope you don't mind me tweaking the summary/adding more info, but it took a couple of reads to figure out what the issue was here (and that it wasn't Treeherder's fault).

STR:
1) Look at the raw log URLs for the builds/tests for a try push (eg either browse for it on http://archive.mozilla.org/pub/firefox/try-builds/ or use the raw log links on Treeherder)

Expected:
All logs and binaries are in the same directory.

Actual:
Logs are in the directory with a short SHA:
http://archive.mozilla.org/pub/firefox/try-builds/mdeboer@mozilla.com-a4e305468a92/try-linux64-debug/
Whereas binaries are in a separate directory that uses the long SHA:
http://archive.mozilla.org/pub/firefox/try-builds/mdeboer@mozilla.com-a4e305468a92516fa01d518da152b8e383d61b6b/try-linux64-debug/
Summary: Mixed folders for try builds on archive.mozilla.org → Binaries and logs being uploaded to separate directories on archive.mozilla.org
(Treeherder uses the job log URL path to work out the directory, so once the binaries and logs are back in the same directory as each other, the Treeherder links will work regardless of whether that "same directory" is the long or short SHA :-))
Thanks for tweaking Ed! It was indeed a bit hard to understand.
obviously this has something to do with ftp->s3 migration but I suspect https://bugzil.la/1175655 is related
Depends on: 1213721
Pretty sure this is fallout from bug 1175655, fix coming up.
Assignee: nobody → nthomas
Blocks: 1175655
(In reply to Byron Campen [:bwc] from comment #6)

Please rebase your patch onto the current tip of m-c/m-i, to pick up config changes for where the upload of the builds goes to. Your files can be found in 
  http://ftp-origin-scl3.mozilla.org/pub/firefox/try-builds/rjesup@wgate.com-bb91c0031bfd/try-win32/
because they went to the old upload host.
(In reply to Nick Thomas [:nthomas] from comment #7)
> Pretty sure this is fallout from bug 1175655, fix coming up.

The S3 migration did regress things, the re-implementation of post_upload doesn't have this truncation of the revision:
http://hg.mozilla.org/build/tools/file/default/stage/post_upload.py#l518
Comment on attachment 8677746 [details] [diff] [review]
[buildbotcustom] Don't truncate the revision when uploading the log

r+ from IRC

https://hg.mozilla.org/build/buildbotcustom/rev/7927cf9b5417
https://hg.mozilla.org/build/buildbotcustom/rev/495d7dd40f29

Deployed to try build masters, to fix uploads from now on. I'm not proposing to fix older directories, because try turns over so fast anyway.
Attachment #8677746 - Flags: review?(catlee)
Attachment #8677746 - Flags: review+
Attachment #8677746 - Flags: checked-in+
Thanks Nick for the fix. It works pretty well. So I think we can close this bug? Or does it miss something else?
I don't have anything else, just hadn't had time to verify the fix.

FYI, there is one other try-upload oddity being tracked in bug 1217742. And other places we use the short rev to be fixed in bug 1217745.
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Blocks: 1213721
No longer depends on: 1213721
Summary: Binaries and logs being uploaded to separate directories on archive.mozilla.org → Try binaries and logs being uploaded to separate directories on archive.mozilla.org
Component: General Automation → General
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: