Closed Bug 417633 Opened 12 years ago Closed 12 years ago

talos needs to support new upload dir structure

Categories

(Release Engineering :: General, defect, P3)

defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: rhelmer, Unassigned)

References

Details

Currently, Talos hits two bugs that the old test infrastructure also has:

1) there is a race condition between the start time on tinderbox and the actual depend build available, so sometimes the start time is reported incorrectly
2) clobbers are not tested, but results are reported as if they were

The core problem is that each depend build is uploaded to a dir like:
firefox/tinderbox-builds/hostname/build.exe

Each clobber build is sent to:
firefox/nightly/2008-02-15-01/build.exe

Cause of #1: Buildbot asks Tinderbox for the latest depend build, and queues the result. Since the former build.exe is overwritten each time, it's very likely that a new build has been uploaded in the meantime, so the start time is not correct

Cause of #2: Talos only knows to check the former style of URL, not the latter.

What I'd like to do to fix this is to land bug 291167, as soon as possible after we've switched to Talos from the old bl-bld* machines. This means that the depend builds will be uploaded to a URL like:

firefox/tinderbox-builds/1203036542/build.exe

To fix #1 Talos will need to use the start time derived from the TinderboxPoller Change in order to find the correct build to download.

To fix #2 Talos will need to detect if binaryURL is set in the Change from TinderboxPoller, and follow that URL to download the correct clobber build. TinderboxPoller does not currently honor this field, it will need to be made to do this.

I think that it would be better to have #1 fixed and Talos skipping clobber builds, than the current situation (getting the start time wrong and falsely reporting that it tests clobber builds), so fixing #1 seems like the more critical bug here.
Blocks: 291167
from bug 291167's attachment (303341):

   # Note that we traverse this code for "hourlies" as well as nightlies/clobbers
+  # $package_name, and hence $short_ud,  would be (eg) 
+  # "fx-linux-tbox-trunk/1202546936" if DependToDated is set, 
+  # "fx-linux-tbox-trunk/" othersise, and "2008-02-09-00-trunk" for clobbers.

so, I'm reading this as unixtime on the hourly builds and full time for clobbers. We should also add some logic to detect clobber builds in quickparse.txt and extract the "binaryUrl" included there.
Blocks: 393044
Component: Testing → Build & Release
Product: Core → mozilla.org
QA Contact: testing → build
Version: unspecified → other
Status: NEW → ASSIGNED
Assignee: nobody → rcampbell
Blocks: 421611
Status: ASSIGNED → NEW
Depends on: 417094
moz2 working is taking precedence right now so this will be on hold for a week or so.
Component: Build & Release → Release Engineering
Component: Release Engineering → Release Engineering: Projects
QA Contact: build → release
still working on moz2 stuff which, is taking longer than expected. Will pick up again if time permits.
Assignee: rcampbell → nobody
Priority: -- → P3
From the work in bug 291167 talos now handles downloading from dated directories.  Bug 433639 has been filed to have talos correctly handle nightly builds.  From my reading of the bug there is also a question of how to handle clobber builds - is this still an issue?  If so, can we rewrite this bug to only handle the missing case and not the problems that are being fixed elsewhere?
From talks with nthomas talos should handle clobber builds the same as nightlies - so issues with testing clobbers will be covered with bug 433639.

Otherwise this bug is fixed.
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Moving closed Future bugs into Release Engineering in preparation for removing the Future component.
Component: Release Engineering: Future → Release Engineering
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.