Closed Bug 1160837 Opened 9 years ago Closed 9 years ago

Beacons from other tabs appear in the network panel

Categories

(DevTools :: Netmonitor, defect)

37 Branch
defect
Not set
minor

Tracking

(firefox38 wontfix, firefox38.0.5 wontfix, firefox39 affected, firefox40 fixed, firefox41 fixed)

RESOLVED FIXED
Firefox 41
Tracking Status
firefox38 --- wontfix
firefox38.0.5 --- wontfix
firefox39 --- affected
firefox40 --- fixed
firefox41 --- fixed

People

(Reporter: mozilla, Assigned: past)

References

Details

(Keywords: regression)

Attachments

(3 files, 10 obsolete files)

14.43 KB, application/zip
Details
11.30 KB, patch
Details | Diff | Splinter Review
9.16 KB, patch
Details | Diff | Splinter Review
User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0
Build ID: 20150415140819

Steps to reproduce:

1) Start FireFox with no tabs opened
2) Open a new tab and open the developer-tools, switch to the "network"-tab
3) Being in the just opened tab, visit any website, e.g. "mozilla.org"
4) Now open another tab and visit "tinypic.com"
5) After waiting a few seconds (sometimes i had to move my mouse and hover a few advertisments at the tinypic.com-website), you should see in the developer tools, that the first website starts a few HTTP-requests to a foreign domain

I also tried it on a friend's PC running FireFox on an UNIX operation system (I currently use Windows 7) and it worked too, malware therefore can be excepted)



Actual results:

A website opened in a tab started an HTPP-request initiated by another tab.
The advertisments on the "tinypic.com"-site seem to use an UXSS-bug in FireFox, they did not in the Internet Explorer and Google Chrome.


Expected results:

Nothing.
Or it could just be the Dev Tools network tab putting the request in the wrong list.

I did twice see the following POST happen in one unrelated site's network tab when playing with the ad-infested tinypic site. Both the request and response had a 0 content-length. The response was basically just setting cookies, the request headers are below:

Host: lax1.ib.adnxs.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:40.0) Gecko/20100101 Firefox/40.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
DNT: 1
Referer: http://lax1.ib.adnxs.com
Cookie: sess=1; uuid2=3674857803336055050; anj=dTM7k!M4.NDunaTDgEREg3jgQrP$=aO4%9.@fbqYnJ!7+!!%m`$4!<P; icu=ChII7f4cEAoYASABKAEwxYSXqgUQxYSXqgUYAA..
Connection: keep-alive
Content-Length: 0

Note the Referer header -- that's content in the other tab, so a fairly good clue the request wasn't actually injected into the tab where it's showing up in the Dev Tools list.
Component: Untriaged → Developer Tools: Netmonitor
By "ad-infested" I mean Ghostery is reporting 60-80+ blockable items (varies on different reloads). That's a lot of trackers! Obviously you have to disable adblockers like Ghostery or our own Tracking Protection in order to see this bug in action.
Joe: please get someone from devtools to look into whether this is a security problem for firefox or just devtools getting confused about where to display the data.
Flags: needinfo?(jwalker)
Flags: needinfo?(jwalker)
Thanks Dan.

We're fixing bugs in the network monitor, so it's most likely that this is a bug in devtools. Possibly a regressions, possibly fixed already, because I failed to reproduce this.

CCed several people, assigning James because someone should be assigned to a possible security bug.

If we find that it's a netmonitor bug, lets remove the 'Security-Sensitive Core Bug' flag.
Assignee: nobody → jlong
Is this with e10s disabled?
This is most likely just our devtools.

I think this is a regression from bug 1045229. For some reason we *always* log the request is it's a "beacon" type: https://github.com/mozilla/gecko-dev/blob/master/toolkit/devtools/webconsole/network-monitor.js#L755

Beacons are initiated from this: https://developer.mozilla.org/en-US/docs/Web/API/Navigator/sendBeacon

They seem to be mainly used for advertising, which is why you need to interact with the site a little bit and you'll see these.

I don't know how to fix this, but I'm 99% sure that's the problem. Why aren't beacons associated with windows (which is why we unconditionally log them)?
Flags: needinfo?(dougt)
With this change the network panel only matches beacons from the inspected page in both e10s and non-e10s mode. I expect a similar change is needed for b2g, but I haven't tried it yet. Doug, does that look reasoable?
This is a devtools bug, not a security bug, but I don't have the necessary rights to clear the flag.
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Severity: normal → minor
Summary: Universal-Cross-Site-Scripting vulnerability → Beacons from other tabs appear in the network panel
Apparently topFrameElement.contentPrincipal doesn't exist on b2g, but it doesn't look like nodePrincipal is what I need either. Alex or Doug, any ideas?
Attachment #8607027 - Attachment is obsolete: true
Attachment #8607460 - Flags: feedback?(dougt)
Attachment #8607460 - Flags: feedback?
Attachment #8607460 - Flags: feedback? → feedback?(poirot.alex)
Comment on attachment 8607460 [details] [diff] [review]
The network panel should only record beacons from the monitored page

Review of attachment 8607460 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good if you are able to merge e10s and b2g checks.
Otherwise a small comment about what beacon is would be helpful.

::: toolkit/devtools/webconsole/network-monitor.js
@@ +769,5 @@
> +      let nonE10sMatch = this.window &&
> +                         aChannel.loadInfo.loadingDocument === this.window.document;
> +      let e10sMatch = this.topFrame &&
> +                      (this.topFrame.contentPrincipal &&
> +                       this.topFrame.contentPrincipal.equals(aChannel.loadInfo.loadingPrincipal));

frame.contentPrincipal seems to be defined only for xul:browser frames.
I would prefer seeing something more generic, like the check used for b2g.

http://mxr.mozilla.org/mozilla-central/source/toolkit/content/widgets/browser.xml#471

Doesn't b2gMatch subsumes e10sMatch on firefox-e10s?
Attachment #8607460 - Flags: feedback?(poirot.alex) → feedback+
I think this is what is invoked in e10s:
https://dxr.mozilla.org/mozilla-central/source/toolkit/content/widgets/remote-browser.xml#158

Unfortunately the b2g check in my patch doesn't work on b2g nor on e10s.
What is wrong with topFrame.nodePrincipal? It looks good to me if I manually inspect one app frame nodePrincipal.
Would it be about aChannel.loadInfo.loadingPrincipal being wrong??
Group: core-security
The problem is that topFrame.nodePrincipal doesn't match aChannel.loadInfo.loadingPrincipal in either b2g or e10s, which leads me to believe that it points to the wrong thing. It matches topFrame.ownerDocument.nodePrincipal in both cases, which leads me to believe that it corresponds to the document that contains the <iframe> or <browser>, which is not what we are looking for.
topFrame.nodePrincipal is correct from what I can see, nodePrincipal.origin reflect the app://xxx.gaiamobile.org
and nodePrincipal.appId is correct.
I imagine the beacon request comes with an unexpected principal... Do you have its origin/appId?

But also it sounds like we could load in a tab, something from another principal, like in an iframe.
The iframe will be using another principal. Also in the <iframe> or <xul:browser>, the frame principal is static and set to the app principal, but I *think* we can load a document from another origin and then the document will be having another principal.
You got me thinking that I may not be trying the most common case on b2g. What I'm using in all cases for testing is this page:

http://htmlpad.org/console-beacon/

I expect that the patch lets the request generated from clicking the button appear in the network panel, but it doesn't on b2g.
It finally dawned on me that we could use the appId on b2g, since that's what it's there for. This patch makes the test page from comment 15 work in all 3 environments, as well as an app I made to test sendBeacon from an actual app on b2g.

Debugging this issue highlighted another problem however: various NetworkHelper methods seem to no longer work in either e10s or b2g. In particular getRequestLoadContext() fails, which causes getTopFrameForRequest() and presumably getAppIdForRequest() to fail as well. If these worked as expected, I don't think we would have needed any special path for TYPE_BEACON. Here are some error messages I got after dump()ing the exceptions:

e10s:
Handler function NM_observeActivity threw an exception: [Exception... "Component returned failure code: 0x80004002 (NS_NOINTERFACE) [nsIInterfaceRequestor.getInterface]"  nsresult: "0x80004002 (NS_NOINTERFACE)"  location: "JS frame :: resource://gre/modules/commonjs/toolkit/loader.js -> resource://gre/modules/devtools/toolkit/webconsole/network-monitor.js :: NM__matchRequest :: line 800"  data: no]
getRequestLoadContext: [Exception... "Component returned failure code: 0x80004002 (NS_NOINTERFACE) [nsIInterfaceRequestor.getInterface]"  nsresult: "0x80004002 (NS_NOINTERFACE)"  location: "JS frame :: resource://gre/modules/commonjs/toolkit/loader.js -> resource://gre/modules/devtools/toolkit/webconsole/network-helper.js :: NH_getRequestLoadContext :: line 256"  data: no]
getRequestLoadContext2: TypeError: aRequest.loadGroup is null
getTopFrameForRequest: TypeError: this.getRequestLoadContext(...) is null

b2g:
getRequestLoadContext: [Exception... "Component returned failure code: 0x80004002 (NS_NOINTERFACE) [nsIInterfaceRequestor.getInterface]"  nsresult: "0x80004002 (NS_NOINTERFACE)"  location: "JS frame :: resource://gre/modules/commonjs/toolkit/loader.js -> resource://gre/modules/devtools/toolkit/webconsole/network-monitor.js :: getInterface :: line 82"  data: no]
getRequestLoadContext2: TypeError: aRequest.loadGroup is null
getTopFrameForRequest: TypeError: this.getRequestLoadContext(...) is null

Alex, any ideas about these?
Attachment #8607460 - Attachment is obsolete: true
Attachment #8607460 - Flags: feedback?(dougt)
Attachment #8608157 - Flags: feedback?(poirot.alex)
Stealing.
Assignee: jlong → past
Does it fail with such exceptions only for beacon requests?
If that's the case, I imagine there is something wrong with the channel created for this request, missing the loadinfo:
http://mxr.mozilla.org/mozilla-central/source/dom/base/Navigator.cpp#1127
NS_NewChannel from sendBeacon:
  http://mxr.mozilla.org/mozilla-central/source/dom/base/Navigator.cpp#1128
doesn't pass any loadGroup.

Whereas we can see some being passed for xhr, for example:
  http://mxr.mozilla.org/mozilla-central/source/dom/base/nsXMLHttpRequest.cpp#1760
Or some other random DOM code:
  http://mxr.mozilla.org/mozilla-central/source/dom/base/nsDocument.cpp#1343

You could give it a try, it seems like you can fetch the loadgroup via:
  nsIDocument* doc = mWindow->GetExtantDoc();
  nsCOMPtr<nsILoadGroup> loadGroup = doc->GetDocumentLoadGroup();
And then, create the request like that:
  rv = NS_NewChannel(getter_AddRefs(channel),
                     uri,
                     doc,
                     nsILoadInfo::SEC_NORMAL,
                     nsIContentPolicy::TYPE_BEACON,
                     loadGroup,
                     nullptr);
Attachment #8608157 - Flags: feedback?(poirot.alex) → feedback+
I tried this idea and it seems to work fine for desktop (e10s and single-process), but b2g asserts:

Assertion failure: !loadContext, at /Users/past/src/gecko/netwerk/base/PrivateBrowsingChannel.h:37
#01: mozilla::dom::NavigatorBinding::sendBeacon(JSContext*, JS::Handle<JSObject*>, mozilla::dom::Navigator*, JSJitMethodCallArgs const&)[/Users/past/src/gecko/obj-b2g/dist/B2GDebug.app/Contents/MacOS/XUL +0x1287bf4]
#02: mozilla::dom::GenericBindingMethod(JSContext*, unsigned int, JS::Value*)[/Users/past/src/gecko/obj-b2g/dist/B2GDebug.app/Contents/MacOS/XUL +0x19cf344]
#03: js::CallJSNative(JSContext*, bool (*)(JSContext*, unsigned int, JS::Value*), JS::CallArgs const&)[/Users/past/src/gecko/obj-b2g/dist/B2GDebug.app/Contents/MacOS/XUL +0x379c3fa]
#04: js::Invoke(JSContext*, JS::CallArgs, js::MaybeConstruct)[/Users/past/src/gecko/obj-b2g/dist/B2GDebug.app/Contents/MacOS/XUL +0x3743fe2]
#05: Interpret(JSContext*, js::RunState&)[/Users/past/src/gecko/obj-b2g/dist/B2GDebug.app/Contents/MacOS/XUL +0x375d6dd]
#06: js::RunScript(JSContext*, js::RunState&)[/Users/past/src/gecko/obj-b2g/dist/B2GDebug.app/Contents/MacOS/XUL +0x3752c3e]

My tests included both running a HelloWorld-type packaged app on b2g desktop that uses sendBeacon() and visiting htmlpad.org/console-beacon from the Browser app. In both cases it fails here:

https://dxr.mozilla.org/mozilla-central/source/netwerk/base/PrivateBrowsingChannel.h#37

...which is called from here:

https://dxr.mozilla.org/mozilla-central/source/dom/base/Navigator.cpp#1144

I don't know much about this code, so I am hoping that Richard or Doug could let us know if this is a feasible approach.
Flags: needinfo?(rlb)
Attached file Beacon app
Here is my sample app in case anyone wants to try it.
This is the first (working) approach with tests for this fix and also fo bug 1045229, which didn't include any. This is what I intend to land if I can't make the second approach work.
Attachment #8608157 - Attachment is obsolete: true
Attachment #8612884 - Flags: review?(poirot.alex)
Comment on attachment 8612327 [details] [diff] [review]
Set the loadGroup in navigator.sendBeacon

Review of attachment 8612327 [details] [diff] [review]:
-----------------------------------------------------------------

I think at this point you shouldn't ni? but just ask for review for this patch!

::: dom/base/Navigator.cpp
@@ +1131,5 @@
>                       doc,
>                       nsILoadInfo::SEC_NORMAL,
> +                     nsIContentPolicy::TYPE_BEACON,
> +                     loadGroup,
> +                     nullptr);

I think you got this error because of this nulltptr defined here. I think it ends up being the load context.

Again, I'm not an expert of this code, I'm just basing my suggestions on existing code.
But may be we should pass the docshell, like this code:
http://mxr.mozilla.org/mozilla-central/source/dom/base/nsScriptLoader.cpp#280

Few line afters we already query the context on the docshell, so that is most likely it:
  nsIDocShell* docShell = mWindow->GetDocShell();
  nsCOMPtr<nsIInterfaceRequestor> callbacks(do_QueryInterface(docshell));

  NS_NewChannel(..., loadGroup, callbacks);
Comment on attachment 8612884 [details] [diff] [review]
The network panel should only record beacons from the monitored page

Review of attachment 8612884 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good, but please keep up fixing the underlying code, that is so much better than workarounds!
Attachment #8612884 - Flags: review?(poirot.alex) → review+
(In reply to Alexandre Poirot [:ochameau] from comment #25)
> I think at this point you shouldn't ni? but just ask for review for this
> patch!

I would have, but it's a patch known to not work, so it seems a bit premature.

> But may be we should pass the docshell, like this code:
> http://mxr.mozilla.org/mozilla-central/source/dom/base/nsScriptLoader.cpp#280

This doesn't work unfortunately, I still get the same assertion. I'll try to dig further.
Or may be you need this:
http://mxr.mozilla.org/mozilla-central/source/dom/base/nsDocument.cpp#1341
  nsCOMPtr<nsIInterfaceRequestor> req = nsContentUtils::SameOriginChecker();
  NS_NewChannel(..., loadGroup, req);
That's the only alternative I can see in our codebase. Most just pass null loadgroup/callbacks.
And some pass either the docshell or this SameOriginChecker as callbacks.
That doesn't work either unfortunately. I haven't landed the first patch because one of the tests fail on e10s and I need to fix it. Amusingly enough it doesn't fail with the second patch applied.
The test was too good, so it caught an actual bug! In the previous iteration the check was still broad enough to show beacons from other tabs that had the same origin as the current tab. This version fixes that and also fixes another problem on b2g where gDevTools isn't present.
Attachment #8612884 - Attachment is obsolete: true
Attachment #8616706 - Flags: review?(poirot.alex)
Attached patch Interdiff (obsolete) — Splinter Review
Here are the changes from what you approved last time.
Comment on attachment 8616706 [details] [diff] [review]
The network panel should only record beacons from the monitored page

Review of attachment 8616706 [details] [diff] [review]:
-----------------------------------------------------------------

::: toolkit/devtools/webconsole/network-monitor.js
@@ +14,5 @@
>  loader.lazyImporter(this, "NetUtil", "resource://gre/modules/NetUtil.jsm");
>  loader.lazyServiceGetter(this, "gActivityDistributor",
>                           "@mozilla.org/network/http-activity-distributor;1",
>                           "nsIHttpActivityDistributor");
> +loader.lazyGetter(this, "testing", () => {

Having such name for a global is disturbing.
What about gTesting, gIsTesting, IsTesting?

@@ +791,5 @@
>        }
>      }
>  
> +    if (aChannel.loadInfo &&
> +        aChannel.loadInfo.contentPolicyType == Ci.nsIContentPolicy.TYPE_BEACON) {

May be a note about this bug could help,
as there is some followup to be done to have a sane check.
Attachment #8616706 - Flags: review?(poirot.alex) → review+
Comment on attachment 8616706 [details] [diff] [review]
The network panel should only record beacons from the monitored page

Review of attachment 8616706 [details] [diff] [review]:
-----------------------------------------------------------------

::: toolkit/devtools/webconsole/network-monitor.js
@@ +18,5 @@
> +loader.lazyGetter(this, "testing", () => {
> +  let testing = false;
> +  try {
> +    const { gDevTools } = require("resource:///modules/devtools/gDevTools.jsm");
> +    testing = gDevTools.testing;

In bug 1171654, I am planning to move |testing| onto DevToolsUtils instead, so it would then be available in B2G.  Fine to land this now, just a note for the future.
(In reply to Alexandre Poirot [:ochameau] from comment #33)
> Having such name for a global is disturbing.
> What about gTesting, gIsTesting, IsTesting?

I like gTesting!

> May be a note about this bug could help,
> as there is some followup to be done to have a sane check.

Will add a reference to the tentative platform patch.

(In reply to J. Ryan Stinnett [:jryans] (use ni?) from comment #34)
> In bug 1171654, I am planning to move |testing| onto DevToolsUtils instead,
> so it would then be available in B2G.  Fine to land this now, just a note
> for the future.

Good to know, thanks.
Addressed comments and added a null check to fix a harmless error that was logged on test shutdown.
Attachment #8616706 - Attachment is obsolete: true
Attachment #8616713 - Attachment is obsolete: true
sorry had to back this out for test failures like https://treeherder.mozilla.org/logviewer.html#?job_id=3389325&repo=fx-team
Flags: needinfo?(past)
This is going to be fun: I can't reproduce it locally on either Mac or Linux, opt or debug builds, running the single failing test alone, or every devtools test.
Flags: needinfo?(past)
(In reply to Panos Astithas [:past] from comment #44)
> This is going to be fun: I can't reproduce it locally on either Mac or
> Linux, opt or debug builds, running the single failing test alone, or every
> devtools test.

FWIW I tried reproducing on linux opt and windows debug builds (single test + whole suite) and also can't reproduce locally.
The test was failing because it was trying to modify gDevTools.testing and failing, as the lazy loader I had added in network-monitor.js was using the original cached value. This patch avoids that problem at the expense of a slightly increased overhead every time gTesting is checked, but jryans is going to fix all of that in bug 1171654 anyway.
Attachment #8617301 - Attachment is obsolete: true
Attachment #8620906 - Flags: review?(poirot.alex)
Attached patch Interdiff (obsolete) — Splinter Review
This is the only change.
Comment on attachment 8620906 [details] [diff] [review]
The network panel should only record beacons from the monitored page

Review of attachment 8620906 [details] [diff] [review]:
-----------------------------------------------------------------

I think you just need defineProperty, no need of any setter.

::: toolkit/devtools/webconsole/network-monitor.js
@@ +766,5 @@
>      // content.
>      // TODO: one particular test (browser_styleeditor_fetch-from-cache.js) needs
>      // the gDevTools.testing check. We will move to a better way to serve its
>      // needs in bug 1167188, where this check should be removed.
> +    if (!gTesting && aChannel.loadInfo &&

I don't get it, the only usage of gTesting I can see is this one. We should only need a getter?!?

I imagine this issue isn't about having a setter,
but is the precise behavior of loader.lazyGetter, which is different from defineProperty+getter.
lazyGetter will only fetch the value once, the first time we access the value, then, we are going the cached value on all next accesses.
defineProperty+getter is always going to fetch gDevTools.testing value on any access.
Attachment #8620906 - Flags: review?(poirot.alex) → review+
Yes, you are right, the setter is not needed.
Now without the setter.
Attachment #8620906 - Attachment is obsolete: true
Attachment #8620908 - Attachment is obsolete: true
Backed out for making browser_projecteditor_contextmenu_02.js permafail on at least Win7 debug (not sure about other Windows variants at this point in time).
https://treeherder.mozilla.org/logviewer.html#?job_id=3421174&repo=fx-team

https://hg.mozilla.org/integration/fx-team/rev/b3a00a0a6ef4
(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #53)
> Backed out for making browser_projecteditor_contextmenu_02.js permafail on
> at least Win7 debug (not sure about other Windows variants at this point in
> time).
> https://treeherder.mozilla.org/logviewer.html#?job_id=3421174&repo=fx-team
> 
> https://hg.mozilla.org/integration/fx-team/rev/b3a00a0a6ef4

Ugh, this is the second time that test has permafailed due to seemingly unrelated changes.  My guess last time was that the patch caused the test to be run in a different chunk where the clipboard state was different.  I'll see if I can reproduce locally.
Comment on attachment 8612327 [details] [diff] [review]
Set the loadGroup in navigator.sendBeacon

Following the SendBeacon implementation history and digging through old bugs I discovered that the loadGroup is not passed on purpose. From bug 987387 comment 0: "sendBeacon clears the loadgroup in order to make the request survive the user leaving the page"

So this approach is not going to work and we need to keep doing the complicated thing.
Attachment #8612327 - Attachment is obsolete: true
Flags: needinfo?(rlb)
Flags: needinfo?(dougt)
Status: ASSIGNED → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 41
Comment on attachment 8620991 [details] [diff] [review]
The network panel should only record beacons from the monitored page

Approval Request Comment
[Feature/regressing bug #]: bug 1045229, which was uplifted up to version 36.
[User impact if declined]: users will be confused by seeing network requests not originating from the website they are inspecting. As this bug shows, sometimes they fear a security compromise has occurred.
[Describe test coverage new/current, TreeHerder]: automated tests added.
[Risks and why]: low to medium risk for the network monitor and web console, but low risk for Firefox overall, as the change is contained in devtools code.
[String/UUID change made/needed]: none
Attachment #8620991 - Flags: approval-mozilla-beta?
Attachment #8620991 - Flags: approval-mozilla-aurora?
Comment on attachment 8620991 [details] [diff] [review]
The network panel should only record beacons from the monitored page

Liz will decide for beta. Taking it at least for aurora (40).
Attachment #8620991 - Flags: approval-mozilla-aurora? → approval-mozilla-aurora+
Needs rebasing for Aurora uplift.
Flags: needinfo?(past)
Attached patch Patch for auroraSplinter Review
Here is a rebased patch for aurora.
Flags: needinfo?(past)
Comment on attachment 8620991 [details] [diff] [review]
The network panel should only record beacons from the monitored page

This would be nice to fix asap, but it's late to uplift to beta. We only have this last beta cycle before RC next Monday. Since this has been a regression since Firefox 36 and it isn't an actual security issue, I think we can live with it for one more release.
Attachment #8620991 - Flags: approval-mozilla-beta? → approval-mozilla-beta-
Product: Firefox → DevTools
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: