1636886 - Delays up to 4 minutes when trying to find a conditioned profile for download for each and every cycle

Status: RESOLVED FIXED

(Testing :: Raptor, defect, P2)

Description

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Sometimes it can take up to 4 minutes before a conditioned profile can be found to download:

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=301543680&repo=autoland&lineNumber=1650-1703

I wonder under which condition that can happen.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Also why are we downloading the same conditioned profile again and again for each and every page cycle? What I can see is that the taskid is always wrong for the first tries, and finally we get a profile from a different task.

I assume the download is based on the task id? If we have an update, we should make sure to update the task id for the current job, so we will not re-download the conditioned profile.

Comment 2

[Greg Mierzwinski [:sparky]]

Those failures were caused by a silent failure in condprof on windows, tarek landed a patch saturday to prevent this issue: https://phabricator.services.mozilla.com/D74532

That said, I agree that we shouldn't be downloading the condprof each time and it should only be downloaded once - this sounds like a bug.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to Greg Mierzwinski [:sparky] from comment #2)

Those failures were caused by a silent failure in condprof on windows, tarek landed a patch saturday to prevent this issue: https://phabricator.services.mozilla.com/D74532

It's not clear to me (what means settled here) if that is a temporary solution or a permanent fix.

Lets fix the summary to make the bug more descriptive.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+2] from comment #3)

It's not clear to me (what means settled here) if that is a temporary solution or a permanent fix.

We clarified that in yesterdays triage meeting. And yes we should get this fixed.

Greg, is it something you can work on? Or do we need Tarek?

Comment 5

[Greg Mierzwinski [:sparky]]

Attachment: Bug 1636886 - Only download conditioned-profiles once in Raptor. r?#perftest,tarek

This patch makes it so that we only download and extract the conditioned-profile once and have tests use copies of that original copy. It also adds some debugging logs to the condprof download_file function to debug an issue where we seem to hit a cache-miss when we have already downloaded the artifact.

Comment 6

[Greg Mierzwinski [:sparky]]

:whimboo, I made some changes that have us only download the conditioned-profile once and use copies of it in the tests. The directories are also cleaned up afterwards.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4dd20d7e4ed3
Only download conditioned-profiles once in Raptor. r=tarek,perftest-reviewers,Bebe

Comment 8

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/4dd20d7e4ed3