Intermittent browser_sourceeditor_initialization.js | application timed out after 330 seconds with no output

RESOLVED FIXED

Status

()

Firefox
Developer Tools
RESOLVED FIXED
6 years ago
5 years ago

People

(Reporter: msucan, Assigned: msucan)

Tracking

({intermittent-failure})

Trunk
x86
Linux
intermittent-failure
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [test which aborts the suite][test disabled])

Attachments

(2 attachments)

(Assignee)

Description

6 years ago
Test runs fine but the application crashes.

s: talos-r3-fed-018
TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/browser/devtools/sourceeditor/test/browser_sourceeditor_initialization.js | application timed out after 330 seconds with no output
PROCESS-CRASH | chrome://mochitests/content/browser/browser/devtools/sourceeditor/test/browser_sourceeditor_initialization.js | application crashed (minidump found)
Thread 0 (crashed)


I never saw this crasher and I have no clue why it might happen.
Whiteboard: [orange]
Comment hidden (Treeherder Robot)
That's an intentional crash following the previous line, which is the significant part: "application timed out after 330 seconds with no output." The test hangs, so we crash in the (usually vain) hope that the stack from the crash will tell you what was hung.
Blocks: 438871
Summary: application crashed after running test browser_sourceeditor_initialization.js → Intermittent browser_sourceeditor_initialization.js | application timed out after 330 seconds with no output
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
(Assignee)

Comment 11

6 years ago
Created attachment 553183 [details] [diff] [review]
[in-fx-team] proposed fix

Proposed fix. Just open a tab and remove it, as in a usual mochitest. I believe the failure might be caused by a test harness bug.
Assignee: nobody → mihai.sucan
Status: NEW → ASSIGNED
Attachment #553183 - Flags: review?(rcampbell)
Comment on attachment 553183 [details] [diff] [review]
[in-fx-team] proposed fix

let's try it!
Attachment #553183 - Flags: review?(rcampbell) → review+
(Assignee)

Comment 13

6 years ago
Comment on attachment 553183 [details] [diff] [review]
[in-fx-team] proposed fix

Landed:

http://hg.mozilla.org/integration/fx-team/rev/36dc4ffaaaae

Let's hope this fixes the intermittent failures. Rob, thanks for the r+!
Attachment #553183 - Attachment description: proposed fix → [in-fx-team] proposed fix
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
And that last one is from the push after your fix landed, so I'm afraid that wasn't it.
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Whiteboard: [orange] → [test which aborts the suite][orange]
(Assignee)

Comment 23

6 years ago
I suggest disabling the test and go for try server runs, and see why it fails.

Will look into this more, tomorrow.
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
I disabled the test because it was practically perma-orange!
Whiteboard: [test which aborts the suite][orange] → [test which aborts the suite][orange][test disabled]
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
(In reply to Mihai Sucan [:msucan] from comment #11)
> Proposed fix. Just open a tab and remove it, as in a usual mochitest. I
> believe the failure might be caused by a test harness bug.

I don't see how this could have made a difference - a new tab in the opening window shouldn't affect anything. Isn't it more likely that your window load listener is running before the opened window's load listener, or something like that?
(Assignee)

Comment 50

6 years ago
(In reply to Gavin Sharp from comment #49)
> (In reply to Mihai Sucan [:msucan] from comment #11)
> > Proposed fix. Just open a tab and remove it, as in a usual mochitest. I
> > believe the failure might be caused by a test harness bug.
> 
> I don't see how this could have made a difference - a new tab in the opening
> window shouldn't affect anything. Isn't it more likely that your window load
> listener is running before the opened window's load listener, or something
> like that?

Not really, because the test runs fine. I believe it was a matter of focus. That assumption still holds true - I mean I still believe it's a problem of focus.

I talked to Tim Taubert on #fxteam and also believes the same. Between tests the test harness waits for focus on the main browser window and here the main browser window is never focused once the test ends (once the SourceEditor window is closed). This happens because (1) the SE window closes and the window manager picks the "running test" window instead of focusing the main browser window, and (2) my attempt to focus the main window fails because all code runs too fast.

Locally I was able to reproduce the problem under "special circumstances". The solution:
http://tbpl.mozilla.org/?tree=Try&rev=fb0045334099

... still waiting for results. What it does: it waits for the SE window to get focus, then run tests, then close, and properly focus the main window. This works locally.

Thoughts?
(Assignee)

Comment 51

6 years ago
Results are green:

http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/mihai.sucan@gmail.com-fb0045334099

... manually checked the logs, because the TBPL page is now broken - I don't know why.

(still, on Mac *many*/almost all devtools tests failed due to bug 679301)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
(Assignee)

Comment 60

6 years ago
Created attachment 553737 [details] [diff] [review]
[in-fx-team] another fix

Another attempt to fix the intermittent failures. This patch now enables the test.

This had green try results. http://tbpl.mozilla.org/?tree=Try&rev=fb0045334099
Attachment #553737 - Flags: review?(rcampbell)
Comment on attachment 553737 [details] [diff] [review]
[in-fx-team] another fix

I don't really have a great sense that this is going to fix the problem. It feels like we're starting to get into circular patch cycles to try to make this work without really fixing the problem.

I'd like to understand what's actually causing this failure, but can give this patch a try. If it doesn't work, we'll need to try something more radical.
Attachment #553737 - Flags: review?(rcampbell) → review+
Comment hidden (Treeherder Robot)
(Assignee)

Comment 63

6 years ago
(In reply to Rob Campbell [:rc] (robcee) from comment #61)
> Comment on attachment 553737 [details] [diff] [review]
> another fix
> 
> I don't really have a great sense that this is going to fix the problem. It
> feels like we're starting to get into circular patch cycles to try to make
> this work without really fixing the problem.
> 
> I'd like to understand what's actually causing this failure, but can give
> this patch a try. If it doesn't work, we'll need to try something more
> radical.

Please see comment 50 for details.
(Assignee)

Comment 64

6 years ago
Comment on attachment 553737 [details] [diff] [review]
[in-fx-team] another fix

Thanks for the r+!

Patch landed:
http://hg.mozilla.org/integration/fx-team/rev/75eeb6cb7050

If this still fails, then (1) Try server is not reliable :( and (2) we really need a more radical approach, as pointed out.
Attachment #553737 - Attachment description: another fix → [in-fx-team] another fix
(In reply to comment #64)
> If this still fails, then (1) Try server is not reliable :(

When writing a fix for an intermittent variable, it's often a good idea to run the affected test suite on your try server job multiple times (you can do that from TBPL.)  For example, if you're dealing with a test which fails 3 out of 5 times, running the Moth tests 20 times can give you a pretty good idea on whether the problem has been really fixed, or whether you're just being lucky on the first run.
Comment on attachment 553737 [details] [diff] [review]
[in-fx-team] another fix

http://hg.mozilla.org/mozilla-central/rev/75eeb6cb7050
Status: ASSIGNED → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
Keywords: intermittent-failure
Whiteboard: [test which aborts the suite][orange][test disabled] → [test which aborts the suite][test disabled]
You need to log in before you can comment on or make changes to this bug.