Closed Bug 710461 Opened 8 years ago Closed 6 years ago

Be smarter about downloading mar files in update verify

Categories

(Release Engineering :: Release Automation: Other, defect, P3)

x86
All
defect

Tracking

(Not tracked)

RESOLVED DUPLICATE of bug 862910

People

(Reporter: nthomas, Unassigned)

Details

Bug 710427 highlighted that we do a lot of repeated work when performing an update verify. Suppose, for the sake of argument, that you have a long lived branch "3.6" which has shipped N releases (N=31 currently). N-1 of those releases will have the same update.xml, as only the latest release is offered a real partial.

So, for every platform-locale pair you will end up downloading the complete.mar N times to verify the hash is correct, because all the old versions get the same complete. We can shortcut this by 
* building a cache of hashes & filesizes for urls we've retrieved, and comparing the cached values with the xml instead of downloading
* building a cache of verified xml content

With the current parallelization this may not result in a high hit rate, but we could break things up by locale to improve that. It should be a very nice speed improvement too.
found in triage.
Component: Release Engineering → Release Engineering: Automation
QA Contact: release → catlee
Bulk move of bugs to Release Automation component.
Component: Release Engineering: Automation (General) → Release Engineering: Automation (Release Automation)
Mass move of bugs to Release Automation component.
No longer blocks: hg-automation
Product: mozilla.org → Release Engineering
Duplicate of bug 862910. I'm currently working on this.
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 862910
You need to log in before you can comment on or make changes to this bug.