Devtools doesn't open in artifact builds - resource://devtools/client/framework/about-devtools-toolbox.js, line 32: NS_ERROR_XPC_CANT_MODIFY_PROP_ON_WN: Cannot modify properties of a WrappedNative

RESOLVED WORKSFORME

Status

()

Firefox
Developer Tools
P1
normal
RESOLVED WORKSFORME
7 months ago
7 months ago

People

(Reporter: bgrins, Unassigned)

Tracking

(Depends on: 1 bug)

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(4 attachments)

(Reporter)

Description

7 months ago
Created attachment 8864926 [details]
toolbox-open-error.png

I'm seeing this after I pulled down Bug 1319111.  The call to `aLoadInfo.resultPrincipalURI = aURI;` is resulting in the following error locally:

`NS_ERROR_XPC_CANT_MODIFY_PROP_ON_WN: Cannot modify properties of a WrappedNative  about-devtools-toolbox.js:32`

This is happening on a new profile with a local build that has https://hg.mozilla.org/mozilla-central/rev/f9abb9c83452 applied. If I try/catch that statement the browser toolbox opens again.
(Reporter)

Comment 1

7 months ago
Hi Honza, are you able to open the devtools toolbox after Bug 1319111?  If so, is there some reason this would throw only in some environments?  I have clobbered and tested with a clean profile on osx.
Flags: needinfo?(honzab.moz)
(Reporter)

Comment 2

7 months ago
(In reply to Brian Grinstead [:bgrins] from comment #1)
> Hi Honza, are you able to open the devtools toolbox after Bug 1319111?  If
> so, is there some reason this would throw only in some environments?  I have
> clobbered and tested with a clean profile on osx.

Expanding the ni? since I've had at least one other person confirm the inability to open devtools.  I'm not sure if Honza will be available at this time of day and I don't want this to leak to a Nightly build.

Is it at least safe to try/catch this line? https://hg.mozilla.org/mozilla-central/rev/f9abb9c83452#l8.15
Flags: needinfo?(bzbarsky)
(Reporter)

Comment 3

7 months ago
Just checked, and the binary built for m-c does not have the problem: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&selectedJob=96924926.  So maybe this is related to doing an artifact build locally.
Sounds like the binary parts of the artifact build involved aren't in sync with the JS parts.  That error basically means that in the build involve there is no resultPrincipalURI property on nsILoadInfo, so xpconnect thinks you're trying to add an expando.

try-catching that line is probably a security bug if it throws, at least in a build where the IDL property exists.  An exception there would indicate null aLoadInfo, which we _very_ much want to throw in this case.
Flags: needinfo?(bzbarsky)
(Reporter)

Updated

7 months ago
Summary: Devtools doesn't open - resource://devtools/client/framework/about-devtools-toolbox.js, line 32: NS_ERROR_XPC_CANT_MODIFY_PROP_ON_WN: Cannot modify properties of a WrappedNative → Devtools doesn't open in artifact builds - resource://devtools/client/framework/about-devtools-toolbox.js, line 32: NS_ERROR_XPC_CANT_MODIFY_PROP_ON_WN: Cannot modify properties of a WrappedNative
Just out of curiousity, what does about:buildconfig claim for your build rev?  (I don't know exactly what it does for artifact builds, but...)

What does "mach artifact last" say, if that's actually a supported thing?
Flags: needinfo?(bgrinstead)
(Reporter)

Comment 6

7 months ago
about:buildconfig
Build platform
target
x86_64-apple-darwin15.6.0
Build tools
Compiler 	Version 	Compiler flags
			
Configure options

--enable-artifact-builds MAKE=/Applications/Xcode.app/Contents/Developer/usr/bin/make

As for `./mach artifact last`, even though it is supposed to be there here's what I see when running it: "The artifact command does not accept the arguments: last".. However, if I clobber and re-build then I see:

 0:26.65 Installing from remote pushhead 0b255199db9d6a6f189b89b7906f99155bde3726 on mozilla-central
 0:26.65 Installing from https://queue.taskcluster.net/v1/task/UKwPqB0bT7Sv__eK7PXgDQ/artifacts/public/build/firefox-55.0a1.en-US.mac.common.tests.zip
Flags: needinfo?(bgrinstead)
I can confirm a regular build works on my machine, even though I faced the same issue as Brian when using an artifact build.
(Reporter)

Comment 8

7 months ago
This definitely looks like a case where artifact builds are doing the wrong thing.

My checkout: https://hg.mozilla.org/mozilla-central/rev/23fe0b76a018a5077a0f7234cff91c41e4b6af64 (push date 2017-05-05 15:05 +0000)
Artifact pushhead: https://hg.mozilla.org/mozilla-central/rev/0b255199db9d6a6f189b89b7906f99155bde3726 (push date	2017-05-04 15:48)

Nick, you appear to be the contact person on https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/Artifact_builds. Any ideas what's going on here?
Flags: needinfo?(honzab.moz) → needinfo?(nalexander)
% hg debugancestor 0b255199db9d6a6f189b89b7906f99155bde3726 f9abb9c83452
356528:0b255199db9d6a6f189b89b7906f99155bde3726

So that artifact pushead is definitely not including the patch for bug 1319111 (which is rev f9abb9c83452).
I think bz has identified this correctly: looks like a binary change happened locally and there's no green build upstream with the binary change that is included entirely in your current tree.  That is, you have locally:

3:binary-change
|
2:local-change
|
1:mozilla-central

|mach artifact| is walking backward from 3:binary-change until it finds a revision that has a green push (in this case, 1:mozilla-central).  As binary-change lands upstream, there will be artifacts available... but you still won't get them since the "candidate" will still be 1:mozilla-central, which will be _behind_ the upstream pushes that have the binary-change landed.

This is just a weakness of artifact builds.  You'll need to wait for upstream to incorporate your change, and then rebase on top.
Flags: needinfo?(nalexander)
Depends on: 901840
(Reporter)

Comment 11

7 months ago
To be clear, in this case there's no local changes - I see this in a fresh m-c checkout.

I just pulled again from m-c to e7bf9443be2c4a5187c37440e35f3526148d7fa8, and see the same problem.  Here's the log from the artifact build:

 0:20.18 Attempting to find a pushhead containing e7bf9443be2c4a5187c37440e35f3526148d7fa8 on mozilla-central.
 0:20.18 Attempting to find a pushhead containing e7bf9443be2c4a5187c37440e35f3526148d7fa8 on integration/mozilla-inbound.
 0:20.78 Attempting to find a pushhead containing e7bf9443be2c4a5187c37440e35f3526148d7fa8 on releases/mozilla-beta.
 0:20.98 Retrieving the last 50 pushheads starting with id 31770 on mozilla-central
 0:24.50 Installing from remote pushhead 0b255199db9d6a6f189b89b7906f99155bde3726 on mozilla-central

I don't understand why it is falling all the way back to 0b255199db9d6a6f189b89b7906f99155bde3726. I see that currently the e7bf9443be2c4a5187c37440e35f3526148d7fa8 build isn't done [0], but the previous push at 23fe0b76a018a5077a0f7234cff91c41e4b6af64 is [1].  So I expect that the artifact build would use 23fe0b76a018a5077a0f7234cff91c41e4b6af64.

[0]: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=e7bf9443be2c4a5187c37440e35f3526148d7fa8
[1]: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=23fe0b76a018a5077a0f7234cff91c41e4b6af64
(In reply to Nick Alexander :nalexander from comment #10)
> I think bz has identified this correctly: looks like a binary change
> happened locally and there's no green build upstream with the binary change
> that is included entirely in your current tree.  That is, you have locally:
> 
> 3:binary-change
> |
> 2:local-change
> |
> 1:mozilla-central
> 

This seems to be affecting all of the DevTools team and is happening without any local binary changes... it happens on a new clone of m-c.

@Nick: Are you saying that the only thing we can do is wait? It is 3 days now :/
Flags: needinfo?(nalexander)
(In reply to Michael Ratcliffe [:miker] [:mratcliffe] from comment #12)
> (In reply to Nick Alexander :nalexander from comment #10)
> > I think bz has identified this correctly: looks like a binary change
> > happened locally and there's no green build upstream with the binary change
> > that is included entirely in your current tree.  That is, you have locally:
> > 
> > 3:binary-change
> > |
> > 2:local-change
> > |
> > 1:mozilla-central
> > 
> 
> This seems to be affecting all of the DevTools team and is happening without
> any local binary changes... it happens on a new clone of m-c.
> 
> @Nick: Are you saying that the only thing we can do is wait? It is 3 days
> now :/

It sounds like something is wrong here; one shouldn't expect to wait 3 days.  I'm going to dig into this right now.
Flags: needinfo?(nalexander)
(In reply to Nick Alexander :nalexander from comment #13)
> (In reply to Michael Ratcliffe [:miker] [:mratcliffe] from comment #12)
> > (In reply to Nick Alexander :nalexander from comment #10)
> > > I think bz has identified this correctly: looks like a binary change
> > > happened locally and there's no green build upstream with the binary change
> > > that is included entirely in your current tree.  That is, you have locally:
> > > 
> > > 3:binary-change
> > > |
> > > 2:local-change
> > > |
> > > 1:mozilla-central
> > > 
> > 
> > This seems to be affecting all of the DevTools team and is happening without
> > any local binary changes... it happens on a new clone of m-c.
> > 
> > @Nick: Are you saying that the only thing we can do is wait? It is 3 days
> > now :/
> 
> It sounds like something is wrong here; one shouldn't expect to wait 3 days.
> I'm going to dig into this right now.

Early investigation suggests folks are seeing https://bugzilla.mozilla.org/show_bug.cgi?id=1362949, which we think is the result of divergence between the artifact fetching code and buildbot/Task Cluster indexing.  We're digging in.
(Reporter)

Comment 15

7 months ago
Even after Bug 1362949 has been resolved I'm still seeing the original WrappedNative error when opening devtools on m-c tip.

Local checkout: https://hg.mozilla.org/mozilla-central/rev/ebbcdaa5b580 (push date 2017-05-10 13:07 +0000)

Log from artifact build:
 0:23.04 Attempting to find a pushhead containing ebbcdaa5b5802ecd39624dd2acbdda8547b8384d on mozilla-central.
 0:23.60 Attempting to find a pushhead containing ebbcdaa5b5802ecd39624dd2acbdda8547b8384d on integration/mozilla-inbound.
 0:23.71 Attempting to find a pushhead containing ebbcdaa5b5802ecd39624dd2acbdda8547b8384d on releases/mozilla-beta.
 0:23.94 Retrieving the last 50 pushheads starting with id 90127 on integration/mozilla-inbound
 0:24.16 Retrieving the last 50 pushheads starting with id 31792 on mozilla-central
 0:26.69 Installing from remote pushhead 11f3f002c7f4b8bc55678ddf9a902d8d36e1015b on integration/mozilla-inbound
For me artifact builds have been working on and off today, on clean checkouts of central.

Latest central (ebbcdaa5b5802ecd39624dd2acbdda8547b8384d) is not working for me either. 
> Installing from remote pushhead ebbcdaa5b5802ecd39624dd2acbdda8547b8384d on mozilla-central

An older central (120d8562d4a53e4f78bd86c6f5076f6db265e5a3) is fine. 
> Installing from remote pushhead 120d8562d4a53e4f78bd86c6f5076f6db265e5a3 on mozilla-central

Both with 0 local changes, on OSX.
(In reply to Julian Descottes [:jdescottes] from comment #16)
> For me artifact builds have been working on and off today, on clean
> checkouts of central.
> 
> Latest central (ebbcdaa5b5802ecd39624dd2acbdda8547b8384d) is not working for
> me either. 
> > Installing from remote pushhead ebbcdaa5b5802ecd39624dd2acbdda8547b8384d on mozilla-central
> 
> An older central (120d8562d4a53e4f78bd86c6f5076f6db265e5a3) is fine. 
> > Installing from remote pushhead 120d8562d4a53e4f78bd86c6f5076f6db265e5a3 on mozilla-central
> 
> Both with 0 local changes, on OSX.

bgrins, jdescottes: can we get some more information about how this is failing?  Based on https://bugzilla.mozilla.org/show_bug.cgi?id=1362458#c16 it seems that maybe something has really broken with artifact builds and we're not just witnessing an issue fetching libraries?  That deserves more information.

Can I get:

1) the output of |hg tip|
2) the complete output of |mach clobber && mach build && mach package|
3) as much runtime logging as you can provide

It would be surprising if we have finally hit a true artifacts problem, but of course it's possible...
Flags: needinfo?(jdescottes)
Flags: needinfo?(bgrinstead)
Sure, here is my info:

jdescottes$ hg tip

changeset:   405171:bd2ca8ca74cc
tag:         tip
parent:      405045:120d8562d4a5
user:        Ruturaj K. Vartak <ruturaj@gmail.com>
date:        Wed May 10 17:25:04 2017 +0200
summary:     Bug 1354504 - Add autocomplete to network monitor search box. r=jdescottes, ntim, rickychien


(I am also adding `hg parent` because hg tip obviously just points to a changeset I recently rebased for a review. Can this really have an impact?)

jdescottes$ hg parent

changeset:   405170:ebbcdaa5b580
fxtree:      central
parent:      405169:8ad533808102
parent:      405110:11f3f002c7f4
user:        Carsten "Tomcat" Book <cbook@mozilla.com>
date:        Wed May 10 15:07:28 2017 +0200
summary:     merge mozilla-inbound to mozilla-central a=merge

I will add the other two as attachments as they are pretty big
Flags: needinfo?(jdescottes)
Created attachment 8866430 [details]
jdescottes_clobber_build_package.log
Created attachment 8866432 [details]
jdescottes_mach_run.log

(I just tried opening the toolbox after Nightly opened)
(Reporter)

Comment 21

7 months ago
I've also had them working on and off today.  It wasn't working earlier, but I just pulled and it is working now:

> hg tip
changeset:   357490:ce2218406119
tag:         tip
user:        Wes Kocher <wkocher@mozilla.com>
date:        Wed May 10 10:01:18 2017 -0700
Flags: needinfo?(bgrinstead)
(Reporter)

Comment 22

7 months ago
Created attachment 8866515 [details]
build-log-ce2218406119.txt

A working clobber/build/package call from ce2218406119
After investigation with bgrins in #fx-team today, I conclude that:

1) artifact builds are working as expected, post https://bugzilla.mozilla.org/show_bug.cgi?id=1362949#c18.

2) the ticket bgrins filed (https://bugzilla.mozilla.org/show_bug.cgi?id=1363871) is a legitimate issue.  A little investigation reveals that we are getting honest-to-goodness nsINullHttpChannel/NullHttpChannel instances, which throw when accessing channelId.  That ticket can address that issue.

So I'm going to close this as RESO WORKSFORME until we see additional artifact-specific problems.

bgrins, jdescotte: thanks for helping dig into the artifacts side of this!
Status: NEW → RESOLVED
Last Resolved: 7 months ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.