Closed Bug 1234416 Opened 4 years ago Closed 3 years ago

Frequent b2g hazard and device build "timed out after 600 seconds of no output" running and rerunning config.sh

Categories

(Release Engineering :: General, defect)

defect
Not set

Tracking

(firefox46 fixed)

RESOLVED WONTFIX
Tracking Status
firefox46 --- fixed

People

(Reporter: philor, Unassigned)

Details

Attachments

(1 file)

https://treeherder.mozilla.org/logviewer.html#?job_id=18860428&repo=mozilla-inbound
https://treeherder.mozilla.org/logviewer.html#?job_id=18854765&repo=mozilla-inbound
https://treeherder.mozilla.org/logviewer.html#?job_id=18857974&repo=mozilla-inbound
https://treeherder.mozilla.org/logviewer.html#?job_id=18859002&repo=mozilla-inbound
https://treeherder.mozilla.org/logviewer.html#?job_id=18859463&repo=mozilla-inbound
https://treeherder.mozilla.org/logviewer.html#?job_id=18863098&repo=mozilla-inbound
https://treeherder.mozilla.org/logviewer.html#?job_id=18863220&repo=mozilla-inbound
https://treeherder.mozilla.org/logviewer.html#?job_id=3599478&repo=b2g-inbound
https://treeherder.mozilla.org/logviewer.html#?job_id=3599477&repo=b2g-inbound
https://treeherder.mozilla.org/logviewer.html#?job_id=3599244&repo=b2g-inbound
https://treeherder.mozilla.org/logviewer.html#?job_id=3599468&repo=b2g-inbound
https://treeherder.mozilla.org/logviewer.html#?job_id=3598873&repo=b2g-inbound
https://treeherder.mozilla.org/logviewer.html#?job_id=3598130&repo=b2g-inbound
https://treeherder.mozilla.org/logviewer.html#?job_id=3592844&repo=b2g-inbound
https://treeherder.mozilla.org/logviewer.html#?job_id=2937868&repo=mozilla-central
https://treeherder.mozilla.org/logviewer.html#?job_id=2929818&repo=mozilla-central
https://treeherder.mozilla.org/logviewer.html#?job_id=2928800&repo=mozilla-central
https://treeherder.mozilla.org/logviewer.html#?job_id=2929649&repo=mozilla-central
this seems to have come out of the blue- the code for poolmanager was added on Nov 30 from:
https://hg.mozilla.org/mozilla-central/rev/d0e882fcbbf7

Now 3 weeks later we are having issues?

:catlee, you had reviewed the code that added the reference to pool manager, is there a reason why this package isn't available for these jobs?
Flags: needinfo?(catlee)
catlee is on pto, maybe coop would have another lead
Flags: needinfo?(catlee) → needinfo?(coop)
I've reopened the bug 1207688. The python + openssl hackery there seems to have been more fragile than expected.
Flags: needinfo?(coop)
The traceback about being unable to import requests is a warning -- all that code is intended to be optional and shouldn't impact the rest of the job. The errors related to git later in the log look more relevant.
No longer depends on: 1207688
Yeah, the poolmanager change isn't related, nor the python+openssl hacks. This looks like the normal 'repo' suckage. Something is causing 'HEAD' in some of the git repos to not exist or be invalid.

We have some minor cleanup to do in the post-run handling to remove the spurious exception there.
Comment on attachment 8704362 [details]
MozReview Request: Bug 1234416 - Check that userconfig exists before trying to remove. r=jlund

https://reviewboard.mozilla.org/r/29647/#review26499

lgtm :)

::: testing/mozharness/scripts/b2g_build.py:1150
(Diff revision 1)
> +        if os.path.exists(userconfig_path):

another option is to use self.rmtree

https://dxr.mozilla.org/mozilla-central/rev/29258f59e5456a1a518ccce6b473b50c1173477e/testing/mozharness/mozharness/base/script.py#162

this will check for path existance for you, then, if it's not a dir, it will use os.remove but throw in some mh logging and retry logic to boot
Attachment #8704362 - Flags: review?(jlund) → review+
https://hg.mozilla.org/mozilla-central/rev/18fedb75f8b0
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
we've updated the git tarballs used by automation - hopefully that helps this issue.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Status: REOPENED → RESOLVED
Closed: 4 years ago3 years ago
Resolution: --- → WONTFIX
Component: General Automation → General
You need to log in before you can comment on or make changes to this bug.