Closed Bug 1756770 Opened 2 years ago Closed 26 days ago

Network Blocking in DevTools blocks responses, but not requests

Categories

(DevTools :: Netmonitor, defect, P2)

Product:
DevTools
Component:
Netmonitor
Version:
Firefox 97
Type:
defect

Tracking

(firefox129 fixed)

Status:
RESOLVED FIXED
Milestone:
129 Branch
Tracking Flags:
Tracking Status
firefox129 --- fixed

People

(Reporter: milkservice, Assigned: jdescottes)

References

(Blocks 2 open bugs)

Details

Attachments

(4 files)

Bug 1756770 - [devtools] Block requests before they are sent to the server
48 bytes, text/x-phabricator-request
Details | Review
Bug 1756770 - [devtools] Stop clearing network resources from the frontend on navigation
48 bytes, text/x-phabricator-request
Details | Review
Bug 1756770 - [devtools] Update network cors tests to expect preflight request after the flight request
48 bytes, text/x-phabricator-request
Details | Review
Bug 1756770 - [devtools] Update test helper waitForNetworkEvents to handle late blockedReason
48 bytes, text/x-phabricator-request
Details | Review

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:97.0) Gecko/20100101 Firefox/97.0

Steps to reproduce:

  • Go to a website that executes some POST action on a user interaction
  • In Firefox DevTools, block the url for that POST action
  • Execute the user interaction

Actual results:

  • Firefox shows "Blocked by DevTools" in the Network tab for the POST request, and the web application correctly shows that the request failed. BUT the server still received the POST request and modified the data.

Expected results:

  • Firefox should block the whole request, and not let it be sent to the server. Chrome correctly blocks the request.

The Bugbug bot thinks this bug should belong to the 'DevTools::Netmonitor' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Component: Untriaged → Netmonitor
Product: Firefox → DevTools

Thank you for the report.

I can easily reproduce the problem on my machine (Win10, Firefox Nightly 99)

  1. Here is the place where we abort blocked requests:
    https://searchfox.org/mozilla-central/rev/ad7ecfa618ec3a65db8405d9f1125059fe4a6a15/devtools/server/actors/network-monitor/utils/network-utils.js#251

Perhaps it's done too late?

Blocks: devtools-resource-blocking
Status: UNCONFIRMED → NEW
Ever confirmed: true
Severity: -- → S3
Priority: -- → P3
See Also: → 1803531

While working on intercept for BiDi, we are having the same issue (Bug 1848156). It turns out that those activity observers happen too late to intercept the request.

Instead, if we move the interception to http-on-modify-request (handled here), then the request will be properly blocked before being sent to the server.

However, as mentioned in Bug 1848156, doing only this will also prevent the request from being detected by our usual activity observers, which trigger the creation of the network event at the moment.

This means that in order to fix this, we need first to move the creation of the event to http-on-modify-request, and then we can start blocking network events when we detect them.

Depends on: 1849686
Depends on: 1880803
No longer depends on: 1849686

Hello,

I am also having this issue. In my case, it's happening with a DELETE request. If I block it via the network tab, it will appear as Blocked, but the DELETE action will be executed nonetheless.

Here is my config:

Name: Firefox
Version: 124.0.2
Build ID: 20240401114208
Distribution ID:
Update Channel: release
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:124.0) Gecko/20100101 Firefox/124.0
OS: Darwin 23.4.0 Darwin Kernel Version 23.4.0: Fri Mar 15 00:10:42 PDT 2024; root:xnu-10063.101.17~1/RELEASE_ARM64_T6000
Rosetta Translated: false

Whiteboard: [devtools-triage]
Priority: P3 → P2
Whiteboard: [devtools-triage]

We now have the option to detect (and therefore block) network events earlier thanks to Bug 1880803.
However enabling this mode for DevTools still triggers failures as we can see on the following try push https://treeherder.mozilla.org/jobs?repo=try&revision=6fea49b127d522cbe2517da82876b9c150def8e1

The failures are:

  • devtools/client/netmonitor/test/browser_net_cors_requests.js: this one is expected, CORS preflight are now detected after the originating request, we should adapt the test.
  • devtools/client/netmonitor/test/browser_net_headers-proxy.js
  • (http3) devtools/client/netmonitor/test/browser_net_header-dns.js
  • (http3) devtools/client/webconsole/test/browser/browser_webconsole_shows_reqs_in_netmonitor.js

Previously requests would sometimes be able to reach the server, even when blocked by devtools.
The point where we blocked the requests could either be before or after the request was sent over the network.
Now with early events, the request will consistently be blocked before the transaction is committed and reaches
the network layer.

A test is added to check the behavior.

Assignee: nobody → jdescottes
Status: NEW → ASSIGNED
Blocks: 1901504
Pushed by jdescottes@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/ae3b7a9bd850
[devtools] Block requests before they are sent to the server r=devtools-reviewers,bomsy
https://hg.mozilla.org/integration/autoland/rev/8486e4eb21ab
[devtools] Stop clearing network resources from the frontend on navigation r=devtools-reviewers,bomsy
https://hg.mozilla.org/integration/autoland/rev/056536e02242
[devtools] Update network cors tests to expect preflight request after the flight request r=devtools-reviewers,bomsy

Backed out for causing high frequency mochitest failures on browser_net_response_CORS_blocked.js

[task 2024-06-11T00:07:14.314Z] 00:07:14     INFO - TEST-START | devtools/client/netmonitor/test/browser_net_response_CORS_blocked.js
[task 2024-06-11T00:07:31.102Z] 00:07:31     INFO -  console.trace: Places - FrecencyRecalculator: "Initializing Frecency Recalculator"
[task 2024-06-11T00:07:31.102Z] 00:07:31     INFO -  resource://gre/modules/PlacesFrecencyRecalculator.sys.mjs 110 PlacesFrecencyRecalculator
[task 2024-06-11T00:07:31.102Z] 00:07:31     INFO -  D:\task_171806011108046\build\tests\mochitest\server.js 178 runServer
[task 2024-06-11T00:07:31.102Z] 00:07:31     INFO -  D:\task_171806011108046\build\tests\mochitest\server.js 71
[task 2024-06-11T00:07:31.106Z] 00:07:31     INFO -  console.trace: Places - FrecencyRecalculator: "Arm frecency recalculation"
[task 2024-06-11T00:07:31.107Z] 00:07:31     INFO -  resource://gre/modules/PlacesFrecencyRecalculator.sys.mjs 366 maybeStartFrecencyRecalculation
[task 2024-06-11T00:07:31.107Z] 00:07:31     INFO -  resource://gre/modules/PlacesFrecencyRecalculator.sys.mjs 149 PlacesFrecencyRecalculator
[task 2024-06-11T00:07:31.107Z] 00:07:31     INFO -  D:\task_171806011108046\build\tests\mochitest\server.js 178 runServer
[task 2024-06-11T00:07:31.107Z] 00:07:31     INFO -  D:\task_171806011108046\build\tests\mochitest\server.js 71
[task 2024-06-11T00:07:31.108Z] 00:07:31     INFO -  console.trace: Places - FrecencyRecalculator: "Got places-init-complete topic"
[task 2024-06-11T00:07:31.108Z] 00:07:31     INFO -  resource://gre/modules/PlacesFrecencyRecalculator.sys.mjs 422 observe
[task 2024-06-11T00:07:31.109Z] 00:07:31     INFO -  D:\task_171806011108046\build\tests\mochitest\server.js 178 runServer
[task 2024-06-11T00:07:31.109Z] 00:07:31     INFO -  D:\task_171806011108046\build\tests\mochitest\server.js 71
[task 2024-06-11T00:07:31.109Z] 00:07:31     INFO -  console.error: (new TypeError("connection not specified or invalid.", "resource://gre/modules/Sqlite.sys.mjs", 1560))
[task 2024-06-11T00:07:31.110Z] 00:07:31     INFO -  console.error: (new TypeError("can't access property \"executeBeforeShutdown\", db is undefined", "resource://gre/modules/PlacesUtils.sys.mjs", 1469))
[task 2024-06-11T00:08:03.887Z] 00:08:03     INFO - TEST-INFO | started process screenshot
[task 2024-06-11T00:08:04.071Z] 00:08:04     INFO - TEST-INFO | screenshot: exit 0
[task 2024-06-11T00:08:04.072Z] 00:08:04     INFO - Buffered messages logged at 00:07:14
[task 2024-06-11T00:08:04.072Z] 00:08:04     INFO - Entering test bound testCORSNotificationPresent
[task 2024-06-11T00:08:04.073Z] 00:08:04     INFO - Test that CORS notification is present
[task 2024-06-11T00:08:04.073Z] 00:08:04     INFO - Initializing a network monitor pane.
[task 2024-06-11T00:08:04.074Z] 00:08:04     INFO - Adding a new tab with URL: https://example.com/browser/devtools/client/netmonitor/test/html_cors-test-page.html
[task 2024-06-11T00:08:04.074Z] 00:08:04     INFO - Tab added and finished loading
[task 2024-06-11T00:08:04.075Z] 00:08:04     INFO - Net tab added successfully: https://example.com/browser/devtools/client/netmonitor/test/html_cors-test-page.html
[task 2024-06-11T00:08:04.076Z] 00:08:04     INFO - Console message: [JavaScript Error: "Content-Security-Policy: The pages settings blocked an inline style (style-src-attr) from being applied because it violates the following directive: default-src chrome: resource:" {file: "chrome://global/content/customElements.js" line: 499 column: 25 source: "display: none !important"}]
[task 2024-06-11T00:08:04.077Z] 00:08:04     INFO - Console message: [JavaScript Error: "Content-Security-Policy: The pages settings blocked an inline style (style-src-attr) from being applied because it violates the following directive: default-src chrome: resource:" {file: "chrome://global/content/customElements.js" line: 499}]
[task 2024-06-11T00:08:04.078Z] 00:08:04     INFO - Console message: [JavaScript Error: "Content-Security-Policy: The pages settings blocked an inline style (style-src-attr) from being applied because it violates the following directive: default-src chrome: resource:" {file: "chrome://global/content/elements/panel.js" line: 62}]
[task 2024-06-11T00:08:04.079Z] 00:08:04     INFO - Console message: [JavaScript Error: "Content-Security-Policy: The pages settings blocked an inline style (style-src-attr) from being applied because it violates the following directive: default-src chrome: resource:" {file: "chrome://global/content/customElements.js" line: 499 column: 25 source: "display: none !important"}]
[task 2024-06-11T00:08:04.080Z] 00:08:04     INFO - Console message: [JavaScript Error: "Content-Security-Policy: The pages settings blocked an inline style (style-src-attr) from being applied because it violates the following directive: default-src chrome: resource:" {file: "chrome://global/content/customElements.js" line: 499}]
[task 2024-06-11T00:08:04.081Z] 00:08:04     INFO - Console message: [JavaScript Error: "Content-Security-Policy: The pages settings blocked an inline style (style-src-attr) from being applied because it violates the following directive: default-src chrome: resource:" {file: "chrome://global/content/elements/panel.js" line: 62}]
[task 2024-06-11T00:08:04.081Z] 00:08:04     INFO - Network monitor pane shown successfully.
[task 2024-06-11T00:08:04.082Z] 00:08:04     INFO - Disabling cache and reloading page.
[task 2024-06-11T00:08:04.082Z] 00:08:04     INFO - Buffered messages logged at 00:07:15
[task 2024-06-11T00:08:04.083Z] 00:08:04     INFO - > Network event progress: NetworkEvent: 1/1, PayloadReady: 0/1, EventTimings: 0/1, got NetMonitor:NetworkEvent for server0.conn0.netEvent7
[task 2024-06-11T00:08:04.083Z] 00:08:04     INFO - Got marker: dom-interactive
[task 2024-06-11T00:08:04.083Z] 00:08:04     INFO - Got marker: dom-complete
[task 2024-06-11T00:08:04.084Z] 00:08:04     INFO - Got two timeline markers, done waiting
[task 2024-06-11T00:08:04.085Z] 00:08:04     INFO - > Network event progress: NetworkEvent: 1/1, PayloadReady: 1/1, EventTimings: 0/1, got NetMonitor:PayloadReady for server0.conn0.netEvent7
[task 2024-06-11T00:08:04.086Z] 00:08:04     INFO - > Network event progress: NetworkEvent: 1/1, PayloadReady: 1/1, EventTimings: 1/1, got NetMonitor:NetworkEventUpdated:EventTimings for server0.conn0.netEvent7
[task 2024-06-11T00:08:04.086Z] 00:08:04     INFO - Wait for completion of all NetworkUpdateEvents packets...
[task 2024-06-11T00:08:04.087Z] 00:08:04     INFO - Clearing the network requests in the UI
[task 2024-06-11T00:08:04.087Z] 00:08:04     INFO - making request to a origin that doesn't allow cross origin
[task 2024-06-11T00:08:04.088Z] 00:08:04     INFO - Waiting until the requests appear in netmonitor
[task 2024-06-11T00:08:04.089Z] 00:08:04     INFO - > Network event progress: NetworkEvent: 1/1, PayloadReady: 0/1, EventTimings: 0/1, got NetMonitor:NetworkEvent for server0.conn0.netEvent10
[task 2024-06-11T00:08:04.089Z] 00:08:04     INFO - > Network event progress: NetworkEvent: 2/1, PayloadReady: 0/1, EventTimings: 0/2, got NetMonitor:NetworkEvent for server0.conn0.netEvent11
[task 2024-06-11T00:08:04.090Z] 00:08:04     INFO - Console message: [JavaScript Error: "Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://example.org/browser/devtools/client/netmonitor/test/sjs_simple-test-server.sjs. (Reason: CORS header Access-Control-Allow-Origin missing). Status code: 200."]
[task 2024-06-11T00:08:04.091Z] 00:08:04     INFO - Console message: [JavaScript Error: "Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://example.org/browser/devtools/client/netmonitor/test/sjs_simple-test-server.sjs. (Reason: CORS request did not succeed). Status code: (null)."]
[task 2024-06-11T00:08:04.091Z] 00:08:04     INFO - > Network event progress: NetworkEvent: 2/1, PayloadReady: 1/1, EventTimings: 0/2, got NetMonitor:PayloadReady for server0.conn0.netEvent11
[task 2024-06-11T00:08:04.092Z] 00:08:04     INFO - > Network event progress: NetworkEvent: 2/1, PayloadReady: 1/1, EventTimings: 1/2, got NetMonitor:NetworkEventUpdated:EventTimings for server0.conn0.netEvent11
[task 2024-06-11T00:08:04.092Z] 00:08:04     INFO - Buffered messages finished
[task 2024-06-11T00:08:04.093Z] 00:08:04     INFO - TEST-UNEXPECTED-FAIL | devtools/client/netmonitor/test/browser_net_response_CORS_blocked.js | Test timed out - 
[task 2024-06-11T00:08:04.094Z] 00:08:04     INFO - GECKO(1188) | Completed ShutdownLeaks collections in process 9384
[task 2024-06-11T00:08:04.094Z] 00:08:04     INFO - TEST-START | Shutdown
Flags: needinfo?(jdescottes)

Depends on D212806

The waitForNetworkEvents helper increases the number of expected event timings when receiving a non-blocked network event.
However, a network event can receive its blocked reason in an update, not necessarily in the initial resource.
So we should update the number of expected events when we receive an update containing a blocked reason for a previous event.

Sorry about this! This seems to reproduce consistently in --verify mode. I think it highlights a weakness in our test helper waitForNetworkEvents.

Flags: needinfo?(jdescottes)
Pushed by jdescottes@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/288b3b429dfe
[devtools] Block requests before they are sent to the server r=devtools-reviewers,bomsy
https://hg.mozilla.org/integration/autoland/rev/84dcd71cc2e1
[devtools] Stop clearing network resources from the frontend on navigation r=devtools-reviewers,bomsy
https://hg.mozilla.org/integration/autoland/rev/963446da27de
[devtools] Update network cors tests to expect preflight request after the flight request r=devtools-reviewers,bomsy
https://hg.mozilla.org/integration/autoland/rev/85eb9393b8d5
[devtools] Update test helper waitForNetworkEvents to handle late blockedReason r=bomsy,devtools-reviewers
Regressions: 1902326
No longer regressions: 1902326

Perfherder has detected a devtools performance change from push 85eb9393b8d599c3d42f760192e8917281166658.

Regressions:

Ratio Test Platform Options Absolute values (old vs new)
6% damp panelsInBackground.reload.DAMP windows10-64-shippable-qr e10s fission stylo webrender 413.67 -> 437.18
4% damp panelsInBackground.reload.DAMP linux1804-64-shippable-qr e10s fission stylo webrender-sw 432.84 -> 449.19
2% damp panelsInBackground.reload.DAMP windows10-64-shippable-qr e10s fission stylo webrender 434.46 -> 443.43

As author of one of the patches included in that push, we need your help to address this regression.
Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the patch(es) may be backed out in accordance with our regression policy.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.

You can run these tests on try with ./mach try perf --alert 752

For more information on performance sheriffing please see our FAQ.

Duplicate of this bug: 1803531
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: