"Edit and resend" network requests doesn't appear to work

UNCONFIRMED
Unassigned

Status

defect
P3
normal
UNCONFIRMED
2 years ago
11 months ago

People

(Reporter: josh, Unassigned)

Tracking

(Blocks 1 bug)

54 Branch
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

Reporter

Description

2 years ago
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:54.0) Gecko/20100101 Firefox/54.0
Build ID: 20170418004027

Steps to reproduce:

1. Open debugging console.
2. Load a webpage that performs XHR requests.
3. Click network tab, locate a request and right click + select "Edit and Resend".
4. Click the "send" button.


Actual results:

Nothing appears to happen.


Expected results:

New network request should appear in the Network tab, with updated details from the edited original request.
Component: Untriaged → Developer Tools: Netmonitor

Comment 1

2 years ago
I've been having the same issue with 52.0 on my work machine, but not with 53.0 on my home machine. Did some digging and found that disabling FoxyProxy solved the issue. Updating FoxyProxy from 4.5.6 to 4.6.5 permanently solved it after that.

Comment 2

2 years ago
Actually, I take that back. It's working randomly on and off now every time I enable/disable/remove add-ons, not tied to any one add-on in particular.

Comment 3

2 years ago
I find I can reproduce very consistently on the login page of a vanilla Nextcloud 12 instance running on Nginx. I think I can also reproduce consistently on Github. Slightly different from the attached video, the send button doesn't collapse the side panel at all if the bug is triggered.

1. Open network monitor
2. Load https://github.com
3. Select the first request (`GET /`)
4. Click Edit and Resend
5. Click Send (without any changes)
--- nothing happens?



Tried to narrow down the regression window:

First bad: https://hg.mozilla.org/integration/autoland/rev/bdc79139892e

> app_name: firefox
> build_date: 2017-02-09 17:07:47.618000
> build_file: ~\.mozilla\mozregression\persist\bdc79139892e--autoland--firefox-54.0a1.en-US.win64.zip
> build_type: inbound
> build_url: https://queue.taskcluster.net/v1/task/bnMCUHtrSR2WIFBfWwLeFQ/runs/0/artifacts/public%2Fbuild%2Ffirefox-54.0a1.en-US.win64.zip
> changeset: bdc79139892e9417108f2653af5a3fd9c37d1b62
> pushlog_url: https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=cef58e7e09619097a3d614a497a81f345469fa03&> > tochange=bdc79139892e9417108f2653af5a3fd9c37d1b62
> repo_name: autoland
> repo_url: https://hg.mozilla.org/integration/autoland
> task_id: bnMCUHtrSR2WIFBfWwLeFQ


> 2017-06-21T14:24:57: DEBUG : Found commit message:
> Bug 1308449 - Implement custom request view;r=Honza,rickychien
> 
> move send/cancel func to component
> use thunk to handle sendHTTPRequest
> use single updateRequest method to update all form values
> 
> MozReview-Commit-ID: Ed2hhSdCFpW
> 
> 2017-06-21T14:24:57: INFO : The bisection is done.
Flags: needinfo?(gasolin)

Comment 4

2 years ago
Hm... on closer look, that change only caused the panel to not close. The requests weren't sent in the previous build either.



I think this is where the resend actually stopped working: https://hg.mozilla.org/integration/autoland/rev/f9c16c7e72ef

> app_name: firefox
> build_date: 2016-09-12 14:27:29.927000
> build_file: ~\.mozilla\mozregression\persist\f9c16c7e72ef--autoland--firefox-51.0a1.en-US.win64.zip
> build_type: inbound
> build_url: https://queue.taskcluster.net/v1/task/VdFuZgHFSTqc4aCSTykNuw/runs/0/artifacts/public%2Fbuild%2Ffirefox-51.0a1.en-US.win64.zip
> changeset: f9c16c7e72efaed1129bb949732ca0ec1936f387
> pushlog_url: https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=402dbb3419f489906677bace2be85c7326bd2763&tochange=f9c16c7e72efaed1129bb949732ca0ec1936f387
> repo_name: autoland
> repo_url: https://hg.mozilla.org/integration/autoland
> task_id: VdFuZgHFSTqc4aCSTykNuw

> 2017-06-21T15:31:50: DEBUG : Found commit message:
> Bug 1270096 - Netmonitor request replay: disable CORS checks, set request headers accurately r=Honza
> 
> MozReview-Commit-ID: Ew7zPkrhOHd
> 
> 2017-06-21T15:31:50: INFO : The bisection is done.
Flags: needinfo?(gasolin) → needinfo?(jsnajdr)

Comment 5

2 years ago
I can confirm that I am also experiencing this issue using 56.0b3 (64-bit) on MacOS and have been experiencing it for a few months now.
I tested on a simpler site https://www.ltg.ed.ac.uk/~ht/xhr.html and it works fine.

But when testing with https://github.com as comment 3 I saw same non-responding issue.


The browser toolbox console print some errors:

```
"onPacket threw an exception: Error: Server did not specify an actor, dropping packet: {"error":"unknownError","message":"error occurred while processing 'sendHTTPRequest: [Exception... \"Failed to open input source 'https://github.com/'\"  nsresult: \"0x805e0006 (<unknown>)\"  location: \"JS frame :: resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/server/actors/webconsole.js :: onSendHTTPRequest :: line 1697\"  data: yes]\nStack: onSendHTTPRequest@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/server/actors/webconsole.js:1697:5\nonPacket@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/server/main.js:1797:15\nreceiveMessage@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/transport/transport.js:761:7\nLine: 1697, column: 0"}
Stack: onPacket@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/client/main.js:944:9
send/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/transport/transport.js:570:13
exports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:109:14
exports.makeInfallible/<@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/ThreadSafeDevToolsUtils.js:109:14
Line: 944, column: 9"  ThreadSafeDevToolsUtils.js:88
```

```
error occurred while processing 'sendHTTPRequest: [Exception... "Failed to open input source 'https://github.com/'"  nsresult: "0x805e0006 (<unknown>)"  location: "JS frame :: resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/server/actors/webconsole.js :: onSendHTTPRequest :: line 1697"  data: yes]
Stack: onSendHTTPRequest@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/server/actors/webconsole.js:1697:5
onPacket@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/server/main.js:1797:15
receiveMessage@resource://gre/modules/commonjs/toolkit/loader.js -> resource://devtools/shared/transport/transport.js:761:7
Line: 1697, column: 0  main.js:1660
```

```
Content Security Policy: The page’s settings blocked the loading of a resource at https://github.com/ (“default-src 'none'”).  (unknown)
```

That might means github just block the reply from browser side. 
Honza, do you have the bandwidth to check that?
Flags: needinfo?(odvarko)

Comment 7

2 years ago
(In reply to Fred Lin [:gasolin] from comment #6)
> That might means github just block the reply from browser side. 
> Honza, do you have the bandwidth to check that?

I experience this issue with the web app I develop at work. It used to work and one day it stopped working. We don't do anything specific to block such functionality.

In reality, the browser doesn't even acknowledge or appear to show the attempt to make another request. It just disappears.
Richard, do you have any tips about what could be the problem here?

Here are STR (copied from comment #3):

1. Open network monitor
2. Load https://github.com
3. Select the first request (`GET /`)
4. Click Edit and Resend
5. Click Send (without any changes)
--- nothing happens?


The Network monitor is implementing edit & resend feature, which is based on this code.
http://searchfox.org/mozilla-central/source/devtools/server/actors/webconsole.js#1662-1708

When trying to resend root request to `https://github.com` there is an exception:

Content Security Policy: The page’s settings blocked the loading of a resource at https://github.com/ (“default-src 'none'”).

Are we using wrong `securityFlags` or `contentPolicyType` props passed into NetUtil.newChannel()?
Or maybe `channel.loadGroup` is wrong?


Honza
Flags: needinfo?(odvarko) → needinfo?(rsoderberg)

Comment 9

2 years ago
Most of the words you used in your question are new to me, so I may not be able to help much. You might want Richard Newman instead?
Flags: needinfo?(rsoderberg)
Please see comment #8
Or is there anyone else I could ask?
Honza
Flags: needinfo?(rnewman)
(In reply to Jan Honza Odvarko [:Honza] from comment #8)

> Are we using wrong `securityFlags` or `contentPolicyType` props passed into
> NetUtil.newChannel()?
> Or maybe `channel.loadGroup` is wrong?

You should probably be capturing both of those fields from the original request's channel.

I haven't done a ton of research, but I expect that `SEC_ALLOW_CROSS_ORIGIN_DATA_IS_NULL` is either going to break or be a security hole, and that `TYPE_OTHER` is wrong, given that it's documented as

> Gecko/Firefox developers: Avoid using TYPE_OTHER. Especially for
> requests that are coming from webpages.

Edit-and-resend implies that the second request should be identical where possible.

You're also specifying LOAD_ANONYMOUS, which is certainly going to break some requests.

AIUI the `loadGroup` won't be a problem, so long as you're picking it from the same document that was used for the original request.
Flags: needinfo?(rnewman)
(In reply to Richard Newman [:rnewman] from comment #11)
> You're also specifying LOAD_ANONYMOUS, which is certainly going to break
> some requests.

One thing that's hard to get right when implementing the "Edit and Resend" feature is the "Edit" part. When the user edits the request headers, we want to send the headers exactly as specified, with as few changes as possible.

For example, what if I want to resend the request without cookies, or with different cookie values? If LOAD_ANONYMOUS was not set, the browser would automatically add the Cookie headers, sending a different request.

Non-privileged webpages are not allowed to set security-sensitive headers like Cookies at all. That's why the devtools implementation cannot use regular XHR or fetch(), but needs to use the low-level nsIHttpChannel with various clever flags and options.

See bug 1270096 for more details and notice the long list of duplicate and dependent bugs that got fixed there.

What I suspect happened here is that some details of CORS or CSP implementation has changed and the "Edit and Resend" feature stopped working. The first step I'd recommend is to use mozregression to figure out what exactly that change was.
Flags: needinfo?(jsnajdr)

Comment 13

2 years ago
(In reply to Jarda Snajdr [:jsnajdr] from comment #12)
> The first step I'd recommend is to use mozregression to figure out
> what exactly that change was.

I tried to narrow down the regression range in comment #4, and ended up on https://hg.mozilla.org/integration/autoland/rev/f9c16c7e72ef
(In reply to Bob from comment #13)
> I tried to narrow down the regression range in comment #4, and ended up on
> https://hg.mozilla.org/integration/autoland/rev/f9c16c7e72ef

Oh, I apologize for not reading the comment thread carefully enough. That means that this bug is not a recent regression, but it never worked since I landed the changes in bug 1270096.

The one thing that seems special about the github.com request is that the response has a Content-Security-Policy header:

Content-Security-Policy: default-src 'none'; base-uri 'self'; child-src render.githubusercontent.com; connect-src 'self' uploads.github.com status.github.com collector.githubapp.com api.github.com www.google-analytics.com github-cloud.s3.amazonaws.com github-production-repository-file-5c1aeb.s3.amazonaws.com github-production-upload-manifest-file-7fdce7.s3.amazonaws.com github-production-user-asset-6210df.s3.amazonaws.com wss://live.github.com; font-src assets-cdn.github.com; form-action 'self' github.com gist.github.com; frame-ancestors 'none'; img-src 'self' data: assets-cdn.github.com identicons.github.com collector.githubapp.com github-cloud.s3.amazonaws.com *.githubusercontent.com; media-src 'none'; script-src assets-cdn.github.com; style-src 'unsafe-inline' assets-cdn.github.com

That means that the loaded github.com page has a non-default content security policy. The nsIHttpChannel that is being used to resend the request inherits its loadGroup from the document:

channel.loadGroup = doc.documentLoadGroup;

So the policy is applied to the resent request, too, and apparently fails.

If the loadGroup wasn't set, the resent request wouldn't appear in the Network Monitor at all, because it wouldn't be detected that it belongs to the inspected tab. That's the main and probably only reason why we're setting the loadGroup. 

Maybe there is some BYPASS_CSP_CHECK flag that could be set on the channel? In this case, it's not the untrusted content that issues the request, but rather the trusted devtools panel executing a direct command from the user.
This problem still exists on Mac OS Sierra, Firefox 64.0

Here is the video, it has little older browser, but the the issue is still there

https://player.vimeo.com/video/239093939
I can't understand what is the difference between repeating the request VS user repeats the request with the UX which fired it.
(In reply to armen88@gmail.com from comment #16)
> I can't understand what is the difference between repeating the request VS
> user repeats the request with the UX which fired it.

In one case, the request is sent by the github.com page loaded in your browser. In the other case, it's sent by the Netmonitor tool in devtools, which is internally almost identical to just another web page loaded in the Devtools area of your browser.

Both these documents have a different Content Security Policy set. The github.com one allows the request to be sent, that's why it works. The Devtools CSP doesn't allow the request, and that's why it fails.
Jarda

Currently I need this feature for local development and for local servers. Is there a way to enable it, so it does not fail?

Updated

Last year
Product: Firefox → DevTools
You need to log in before you can comment on or make changes to this bug.