If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

Perma failure when 56 merges to beta in browser_service_workers_multi_content_process.js | Test timed out -

RESOLVED FIXED in Firefox 56

Status

()

Firefox
Developer Tools: about:debugging
P3
normal
RESOLVED FIXED
2 months ago
2 months ago

People

(Reporter: aryx, Assigned: jdescottes)

Tracking

(Blocks: 2 bugs, {intermittent-failure})

unspecified
Firefox 57
intermittent-failure
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox56 fixed, firefox57 fixed)

Details

MozReview Requests

()

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(1 attachment)

Merge is tomorrow (August 2nd)

Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=119734413&repo=try

TEST-UNEXPECTED-FAIL | devtools/client/aboutdebugging/test/browser_service_workers_multi_content_process.js | Test timed out -

TEST-UNEXPECTED-FAIL | devtools/client/aboutdebugging/test/browser_service_workers_push.js | The service-workers url appears in the list: http://example.com/browser/devtools/client/aboutdebugging/test/service-workers/empty-sw.js - Got false, expected true
Flags: needinfo?(jdescottes)
Summary: Perma failure when 56 merges to beta in → Perma failure when 56 merges to beta in browser_service_workers_multi_content_process.js | Test timed out -
(Assignee)

Comment 1

2 months ago
Are you sure about the perma failure? 

From what I see in the central as beta push https://treeherder.mozilla.org/#/jobs?repo=try&revision=b47de49ad58a9f315a1a6ddcbe3c8fbd5db1a38d&selectedJob=119734009, there are successful jobs where this test doesn't fail. Given that this test is already intermittent, are we sure it's a blocker?
Flags: needinfo?(jdescottes) → needinfo?(aryx.bugmail)
Comment hidden (mozreview-request)
(Assignee)

Comment 3

2 months ago
This patch should normally address intermittents from Bug 1340199 and Bug 1349121
Assignee: nobody → jdescottes
Blocks: 1340199, 1349121
Status: NEW → ASSIGNED
Priority: -- → P3

Comment 4

2 months ago
mozreview-review
Comment on attachment 8892832 [details]
Bug 1386412 - fix race conditions in browser_service_workers_multi_content_process.js;

https://reviewboard.mozilla.org/r/163830/#review169184

Thanks for chasing this intermittent.

It looks like there migh be other suspicious usages of waitForMutation:
http://searchfox.org/mozilla-central/source/devtools/client/aboutdebugging/test/browser_page_not_found.js#24
http://searchfox.org/mozilla-central/source/devtools/client/aboutdebugging/test/browser_service_workers_push_service.js#97
http://searchfox.org/mozilla-central/source/devtools/client/aboutdebugging/test/browser_service_workers_status.js#46
http://searchfox.org/mozilla-central/source/devtools/client/aboutdebugging/test/browser_service_workers_timeout.js#66

That makes me wonder if we should warn from eslint usages of "yield waitForXXXX"...
but unfortunately, some usages are valid.

::: devtools/client/aboutdebugging/test/browser_service_workers_multi_content_process.js:30
(Diff revision 1)
> +  let onMutation = waitForMutation(serviceWorkersElement, { childList: true });
> +
> +  let swTab = yield addTab(TAB_URL, { background: true });
>  
> -  yield waitForMutation(serviceWorkersElement, { childList: true });
> +  info("Wait for service worker to appear in the list");
> +  yield onMutation;

If that's what caused this intermittent, it looks like there is other potential intermittents:
http://searchfox.org/mozilla-central/source/devtools/client/aboutdebugging/test/browser_service_workers.js#20
http://searchfox.org/mozilla-central/source/devtools/client/aboutdebugging/test/browser_service_workers_fetch_flag.js#18
http://searchfox.org/mozilla-central/source/devtools/client/aboutdebugging/test/browser_service_workers_timeout.js#23
Attachment #8892832 - Flags: review?(poirot.alex) → review+
(In reply to Julian Descottes [:jdescottes] from comment #1)
> Are you sure about the perma failure? 
The test failed on Linux x64 and Windows 7, both on opt and pgo (which is an opt). These are at least 5 failures for this in pne test run: https://treeherder.mozilla.org/#/jobs?repo=try&revision=b47de49ad58a9f315a1a6ddcbe3c8fbd5db1a38d&filter-searchStr=devtools%20e10s&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-resultStatus=retry&filter-resultStatus=usercancel&filter-resultStatus=running&filter-resultStatus=pending&filter-resultStatus=runnable&selectedJob=119781861
Flags: needinfo?(aryx.bugmail)
(Assignee)

Comment 6

2 months ago
Thanks for the review! I forgot to mention but I did a few try pushes to confirm if this was useful.

First the central as beta simulation without my patch: https://treeherder.mozilla.org/#/jobs?repo=try&revision=221c12eb40bfd6e9d04936c5b2aea085fa39aba1
we can see frequent dt9 failures, for the timeout issue mentioned.

Then same simulation with a first version of my patch: https://treeherder.mozilla.org/#/jobs?repo=try&revision=ffea93d1a821c5bd2ab567dc548c033941514a52 
still a dt9 failure (but less frequent) and failing later in the test

Finally a last test with a patch also trying to fix the issue found in the second try push:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=482f6a8b794cc4558e9bb6274a8b9fbb6b28d9f6
(I did this last one based off central, instead of the beta simulation, so the chunk was dt8 instead of dt9)

(In reply to Alexandre Poirot [:ochameau] from comment #4)
> Comment on attachment 8892832 [details]
> Bug 1386412 - fix race conditions in
> browser_service_workers_multi_content_process.js;
> 
> https://reviewboard.mozilla.org/r/163830/#review169184
> 
> Thanks for chasing this intermittent.
> 
> It looks like there migh be other suspicious usages of waitForMutation:
> http://searchfox.org/mozilla-central/source/devtools/client/aboutdebugging/
> test/browser_page_not_found.js#24
> http://searchfox.org/mozilla-central/source/devtools/client/aboutdebugging/
> test/browser_service_workers_push_service.js#97
> http://searchfox.org/mozilla-central/source/devtools/client/aboutdebugging/
> test/browser_service_workers_status.js#46
> http://searchfox.org/mozilla-central/source/devtools/client/aboutdebugging/
> test/browser_service_workers_timeout.js#66
> 
> That makes me wonder if we should warn from eslint usages of "yield
> waitForXXXX"...
> but unfortunately, some usages are valid.

Do you have any example in mind?

> 
> :::
> devtools/client/aboutdebugging/test/
> browser_service_workers_multi_content_process.js:30
> (Diff revision 1)
> > +  let onMutation = waitForMutation(serviceWorkersElement, { childList: true });
> > +
> > +  let swTab = yield addTab(TAB_URL, { background: true });
> >  
> > -  yield waitForMutation(serviceWorkersElement, { childList: true });
> > +  info("Wait for service worker to appear in the list");
> > +  yield onMutation;
> 
> If that's what caused this intermittent, it looks like there is other
> potential intermittents:
> http://searchfox.org/mozilla-central/source/devtools/client/aboutdebugging/
> test/browser_service_workers.js#20
> http://searchfox.org/mozilla-central/source/devtools/client/aboutdebugging/
> test/browser_service_workers_fetch_flag.js#18
> http://searchfox.org/mozilla-central/source/devtools/client/aboutdebugging/
> test/browser_service_workers_timeout.js#23

I'll file a follow up to cleanup other about:debugging tests. The pattern is racy anyway, I guess depending on the test they are more or less likely to fail, but fixing it shouldn't hurt.

Comment 7

2 months ago
Pushed by jdescottes@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/0bf2574ad9ff
fix race conditions in browser_service_workers_multi_content_process.js;r=ochameau
(Assignee)

Updated

2 months ago
Blocks: 1386613

Comment 8

2 months ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/0bf2574ad9ff
Status: ASSIGNED → RESOLVED
Last Resolved: 2 months ago
status-firefox57: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → Firefox 57

Comment 9

2 months ago
bugherderuplift
https://hg.mozilla.org/releases/mozilla-beta/rev/00ead85af43b
status-firefox56: --- → fixed

Comment 10

2 months ago
5 failures in 888 pushes (0.006 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* try: 5

Platform breakdown:
* linux64: 3
* windows7-32: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1386412&startday=2017-07-31&endday=2017-08-06&tree=all
You need to log in before you can comment on or make changes to this bug.