Closed Bug 1134307 Opened 9 years ago Closed 8 years ago

Intermittent browser_bug592641.js | uncaught exception - Error: operation not possible on dead CPOW at :0

Categories

(Firefox :: General, defect, P5)

Product:
Firefox
Component:
General
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Milestone:
Firefox 51
Tracking Flags:
Tracking Status
e10s + ---
firefox47 --- wontfix
firefox48 --- wontfix
firefox49 --- fixed
firefox50 --- fixed
firefox51 --- fixed

People

(Reporter: RyanVM, Assigned: kevchan85, Mentored)

References

(Blocks 1 open bug)

Details

(Keywords: intermittent-failure)

Attachments

(1 file, 4 obsolete files)

bug1134307_Dead_CPOW.diff
1.66 KB, patch
mconley
: review+
Details | Diff | Splinter Review 
The dead CPOW error doesn't look good.

09:21:13 INFO - 66 INFO TEST-START | dom/html/test/browser_bug592641.js
09:21:15 INFO - JavaScript error: , line 0: Error: operation not possible on dead CPOW
09:21:15 INFO - 67 INFO checking window state
09:21:15 INFO - 68 INFO Console message: [JavaScript Warning: "unsafe CPOW usage" {file: "resource:///modules/sessionstore/TabState.jsm" line: 96}]
09:21:15 INFO - 69 INFO Console message: [JavaScript Warning: "unsafe CPOW usage" {file: "resource:///modules/sessionstore/TabState.jsm" line: 96}]
09:21:15 INFO - 70 INFO TEST-PASS | dom/html/test/browser_bug592641.js | Title should be correct on load #1
09:21:15 INFO - 71 INFO Console message: [JavaScript Warning: "unsafe CPOW usage" {file: "chrome://mochitests/content/browser/dom/html/test/browser_bug592641.js" line: 33}]
09:21:15 INFO - 72 INFO Console message: [JavaScript Warning: "unsafe CPOW usage" {file: "chrome://mochitests/content/browser/dom/html/test/browser_bug592641.js" line: 33}]
09:21:15 INFO - 73 INFO TEST-UNEXPECTED-FAIL | dom/html/test/browser_bug592641.js | uncaught exception - uncaught exception: undefined at :0
09:21:15 INFO - Stack trace:
09:21:15 INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:simpletestOnerror:1476
09:21:15 INFO - null:null:0
09:21:15 INFO - JavaScript error: , line 0: uncaught exception: undefined
09:21:15 INFO - 74 INFO Console message: [JavaScript Warning: "unsafe CPOW usage" {file: "chrome://mochitests/content/browser/dom/html/test/browser_bug592641.js" line: 52}]
09:21:15 INFO - 75 INFO Console message: [JavaScript Error: "uncaught exception: undefined"]
09:21:15 INFO - 76 INFO Console message: [JavaScript Error: "Error: operation not possible on dead CPOW"]
09:21:54 INFO - 77 INFO Console message: 1424280114489 Toolkit.GMP WARN GMPInstallManager.parseResponseXML got node name: html, expected: updates
09:21:54 INFO - 78 INFO Console message: [JavaScript Error: "1424280114491 Toolkit.GMP ERROR GMPInstallManager.simpleCheckAndInstall Could not check for addons: {"target":{},"message":"got node name: html, expected: updates"}" {file: "resource://gre/modules/Log.jsm" line: 749}]
09:21:54 INFO - 79 INFO Console message: 1424280114711 Services.HealthReport.HealthReporter WARN Saved state file does not exist.
09:22:43 INFO - Xlib: extension "RANDR" missing on display ":0".
09:22:44 INFO - TEST-INFO | screentopng: exit 0
09:22:44 INFO - 80 INFO TEST-UNEXPECTED-FAIL | dom/html/test/browser_bug592641.js | Test timed out - expected PASS
09:22:44 INFO - 81 INFO MEMORY STAT vsize after test: 21991423696896
09:22:44 INFO - 82 INFO MEMORY STAT residentFast after test: 494071808
09:22:44 INFO - 83 INFO TEST-OK | dom/html/test/browser_bug592641.js | took 90129ms
09:22:44 INFO - 84 INFO TEST-UNEXPECTED-FAIL | dom/html/test/browser_bug592641.js | Found a tab after previous test timed out: http://mochi.test:8888/browser/dom/html/test/bug592641_img.jpg - expected PASS
09:22:44 INFO - 85 INFO TEST-UNEXPECTED-FAIL | dom/html/test/browser_bug592641.js | Found a tab after previous test timed out: http://mochi.test:8888/browser/dom/html/test/bug592641_img.jpg - expected PASS
Not really sure as to the best component for this, but Core::IPC probably isn't.
tracking-e10s: --- → ?
Component: IPC → General
Product: Core → Firefox
Summary: Intermittent browser_bug592641.js | uncaught exception - uncaught exception: undefined at :0 → Intermittent browser_bug592641.js | uncaught exception - Error: operation not possible on dead CPOW at :0
Intermittent e10s test failure
Priority: -- → P5
Hey kevin!

So the good news is that we get stack information for dead CPOW exceptions, which really helps to narrow down where the problem is. (This is true as of about ~20 days ago since bug 1249698 landed).

Clicking on the "brasstacks" link from one of the OrangeFactor comments, I was able to find the test log, which has this in it:

01:45:14     INFO -  255 INFO TEST-UNEXPECTED-FAIL | dom/html/test/browser_bug592641.js | uncaught exception - Error: operation not possible on dead CPOW at load2Done@chrome://mochitests/content/browser/dom/html/test/browser_bug592641.js:52:7
01:45:14     INFO -  testScope/test_executeSoon/<.run@chrome://mochikit/content/browser-test.js:993:9
01:45:14     INFO -  Stack trace:
01:45:14     INFO -  chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:simpletestOnerror:1609
01:45:14     INFO -  JavaScript error: chrome://mochitests/content/browser/dom/html/test/browser_bug592641.js, line 52: Error: operation not possible on dead CPOW
01:45:14     INFO -  256 INFO Console message: [JavaScript Error: "Error: operation not possible on dead CPOW" {file: "chrome://mochitests/content/browser/dom/html/test/browser_bug592641.js" line: 52}]
01:45:14     INFO -  load2Done@chrome://mochitests/content/browser/dom/html/test/browser_bug592641.js:52:7
01:45:14     INFO -  testScope/test_executeSoon/<.run@chrome://mochikit/content/browser-test.js:993:9

So the problem appears to be in dom/html/test/browser_bug592641.js on line 52:

http://searchfox.org/mozilla-central/rev/f80822840bc5f3d2d3cae3ece621ddbce72e7f54/dom/html/test/browser_bug592641.js#52

And what we've got here is a test attempting to access the title of a loaded page by way of the contentWindow CPOW. It should instead ask the tab2Browser object directly, via

ctx.tab2Browser.contentTitle

Can you make that change and make sure the test passes? If so, we can push this one to try. After that, if you're starting to get the hang of it, maybe you can go on a spree and start taking care of some of the rest of the (many) dead CPOW exceptions[1]: I know our tree sheriffs would be pretty happy about that!

[1]: https://bugzilla.mozilla.org/buglist.cgi?quicksearch=dead%20CPOW&list_id=13186804
Assignee: nobody → kevchan85
Mentor: mconley
(In reply to Mike Conley (:mconley) - (Needinfo me!) from comment #78)
> And what we've got here is a test attempting to access the title of a loaded
> page by way of the contentWindow CPOW. It should instead ask the tab2Browser
> object directly, via
> 
> ctx.tab2Browser.contentTitle

Can't help but wonder if this pattern is frequent enough that a mass find & replace would be in order. And yes, ++ to fixing dead CPOW failures!
Not too many hits:

https://dxr.mozilla.org/mozilla-central/search?q=contentWindow.document.title&=mozilla-central

But this did point out that there's a _2nd_ usage within this file that should probably be updated as well.
Attached patch bug1134307_Dead_CPOW.diff (obsolete) — — Splinter Review 
I finished this one. I have the other line in another file.
Attachment #8785358 - Flags: review?(mconley)
Mike, I have the other bug ready to upload Should I create a new bug and post it there? I cannot find anything referencing this file from a quick search beside this thread.
Flags: needinfo?(mconley)
This patch has changes to lines referencing contentTitle for tabs 1 and two.
Attachment #8785375 - Flags: review?(mconley)
Attached patch bug1134307_Dead_CPOW_1.diff (obsolete) — — Splinter Review 
This file has a change to line 33
I think fixing both of these CPOW uses in browser_bug592641.js in this bug makes sense. However, I think we can probably safely have the changes occur in the same file. Can you please combine your patches?
Flags: needinfo?(mconley) → needinfo?(kevchan85)
Attached patch bug1134307_Dead_CPOW.diff (obsolete) — — Splinter Review 
Here the file with both patches. I will start looking at other CPOW stuff as you suggested.
Attachment #8785358 - Attachment is obsolete: true
Attachment #8785358 - Flags: review?(mconley)
Attachment #8785375 - Attachment is obsolete: true
Attachment #8785375 - Flags: review?(mconley)
Thank you for removing the other patches I was trying to figure out how to do it on this thread.

        Attachment #8785376 -
        Attachment is obsolete: true

        Attachment #8785381 -
        Attachment is obsolete: true

        Attachment #8785384 -
        Flags: review?(mconley)

    
    

    
  
Comment on attachment 8785384 [details] [diff] [review]
bug1134307_Dead_CPOW.diff

Review of attachment 8785384 [details] [diff] [review]:
-----------------------------------------------------------------

This looks great to me! I'm going to push this to try.

In the meantime, take a look at the list of bugs and feel free to needinfo? me on any that you want. I'll assign you and help you where you need.

Thanks for your work!

        Attachment #8785384 -
        Flags: review?(mconley) → review+
Flags: needinfo?(mconley)

    
    

    
  
Mike what are the flags for? I was watching the testing it says its done. Is there anything else I can do?

    
    

    
  
Hrm, not sure why the more recent push wasn't put in the bug. The one in comment 90 was obsolete (I forgot to request retriggers).

So here's the right one: https://treeherder.mozilla.org/#/jobs?repo=try&revision=cc09e297fff69d67caa4853909b9aa91bfc09d73

What I've done is pushed a request to our automation to run all of the tests under dom/html/test (including the one you've fixed). I've requested 20 retriggers each to see if the intermittent failure shows up in any of them.

Each of our test "types" is represented by a symbol in that link, and there's even further categorization after that. For example, "1" and "bc1" are part of the M suite. In this case, M stands for mochitest, which is the name of the testing framework that we're using. 1 and bc1 are flavours of mochitest, and each flavour has different capabilities and powers. Your test is a "browser" mochitest, so it's running in the ones that start with "bc".

Checking the test log of one of the Linux 64 bc1s here: https://archive.mozilla.org/pub/firefox/try-builds/mconley@mozilla.com-cc09e297fff69d67caa4853909b9aa91bfc09d73/try-linux64/try_ubuntu64_vm_test-mochitest-browser-chrome-1-bm51-tests1-linux64-build473.txt.gz

If I do a Find in Page on that log, I can find browser_bug592641.js, so the test you patched is running in that suite, and it appears to be passing.

Green means pass. Orange means one or more tests failed. In this case, I'm seeing failures on OS X 10.10 debug for "1" (Also called "Mochitest 1"). Clicking on that orange will show a readout of the failure in the bottom pane. Each one is reporting this failure:

TEST-UNEXPECTED-FAIL | leakcheck | default process: 10216 bytes leaked (BasicContainerLayer, BasicImplData, BasicLayerManager, BasicPaintedLayer, CompositableClient, ...)

Which is a pre-existing very frequent orange, and is almost certainly unrelated to your patch, so we can ignore it. The red "1" in the M-e10s group means that the test resulted in the browser crashing, but this one says that this is an intermittent assertion failure for the dom/html/test/test_fullscreen-api.html test, so again, unrelated.

So so far, we're looking okay here. The intermittents that we just saw in the try push are annoying because they're not related, but we have to check on them anyway to see if we've broken anything. That's precisely why your work here is so useful - the more intermittents we get rid of, the more we can trust our test results the first time! :)

I've left the needinfo on myself to check the try build once all of the tests are done (likely on the weekend or on Monday), after which, if we're all green, we should be good to land.

In the meantime, let me know if you want any suggestions on another bug like this you can fix.
Flags: needinfo?(kevchan85)

    
    

    
  
This looks good to me on try, despite the other (unrelated) oranges.
Flags: needinfo?(mconley)

    
    

    
  
Author: kevin <kevchan85@yahoo.com>
Bug number: 1134307
Commit message: Bug 1134307 - Remove a CPOW from browser_bug592641.js to avoid a dead CPOW intermittent. r=mconley
Keywords: checkin-needed
OS: Linux → Unspecified
Hardware: x86_64 → Unspecified

    
    

    
      
  

    
    

    
  
I tweaked the commit message to be a bit more descriptive.

    
    

    
  
Pushed by ryanvm@gmail.com:
https://hg.mozilla.org/integration/fx-team/rev/89ea6ea53888
Access the title of the loaded page by using ctx.tab2Browser.contentTitle in browser_bug592641.js to avoid CPOW usage. r=mconley
Keywords: checkin-needed

    
    

    
  
https://hg.mozilla.org/mozilla-central/rev/89ea6ea53888
Status: NEW → RESOLVED
Closed: 8 years ago

          status-firefox51:
          --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → Firefox 51

    
    

    
  
Pushed by ryanvm@gmail.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/cea23a652dcf
Remove some CPOWs from browser_windowName.js to avoid a dead CPOW intermittent. r=mconley

    
    

    
      
  

          You need to log in
          before you can comment on or make changes to this bug.
        






  

    
  







  

    

      
Attachment

      

      
      
      
      
    

    

      

        

          

            
General

            

              Creator: 



            

            
Created: 

            
Updated: 

            
Size: