1613988 - Allow wildcards (*) for blocking URLs

Status: VERIFIED FIXED

(DevTools :: Netmonitor, enhancement, P3)

Description

[:Harald Kirschner :digitarald]

Same as Chrome.

https://chromedevtools.github.io/devtools-protocol/tot/Network/#method-setBlockedURLs

Comment 1

[:Harald Kirschner :digitarald]

Some of the initial work landed in https://phabricator.services.mozilla.com/D46671

URLs get blocked here: https://searchfox.org/mozilla-central/source/devtools/server/actors/network-monitor/network-observer.js#749

Comment 2

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Harald the link in comment #0 is giving "Oops you hit a 404!"
Honza

Comment 3

[:Harald Kirschner :digitarald]

Deep linking seems to be busted in CDP docs, sorry.

Raw docs would be here: https://github.com/chromium/chromium/blob/e9c901261deec6dc1c18b59b0ac881965eec6ad3/third_party/blink/public/devtools_protocol/browser_protocol.pdl#L4552

The link doesn't add too much context, just shows how wildcards are being allowed in arguments.

Comment 4

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Harald, how wildcards should be treated?

Does the following make sense?

"a*b" => everything that starts with "a" and ends with "b"
"a*" => everything that starts with "a"
"*b" => everything that ends with "b"
"*a*" => everything that has an "a" in it
"*a*b*"=> everything that has an "a" in it, followed by anything, followed by a "b", followed by anything

From:
https://stackoverflow.com/questions/26246601/wildcard-string-comparison-in-javascript

This might also be useful:
https://gist.github.com/donmccurdy/6d073ce2c6f3951312dfa45da14a420f

Honza

Comment 5

[:Harald Kirschner :digitarald]

Thanks for summarizing the technical side, Honza.

I think it all makes sense except the start/end constraint (aka we should not add ^…$ in regexp)

The rules would, for example, be "a*" => everything that contains "a" followed by 1+ characters

This is also how Chrome treats it (*.com filters down to all COM tlds). We can tweak it based on feedback and tighten it more if needed.

Comment 6

[Duncan Dean]

Hi Harald, I've been looking into this enhancement and have a working solution.

Just a few things:

• Should the regexp be case sensitive or not?
• I assume we wish to retain the old behaviour if no * is present, i.e. just check if the URL includes the query.

Thank you!

Comment 7

[Duncan Dean]

Attachment: Bug 1613988 - Allow wildcards for blocking URLs. r=honza!,harald!

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/38361b9f7581
Allow wildcards for blocking URLs. r=Honza,Harald

Comment 9

[Alexandru Michis [:malexandru]]

https://hg.mozilla.org/mozilla-central/rev/38361b9f7581

Comment 10

[:Harald Kirschner :digitarald]

(clearing ni, answered in phab)

Comment 11

[Catalin Sasca, Desktop QA [:csasca]]

This works as intended on latest Nightly 75.0a1 (2020-03-04) with these steps provided by Honza:

0. Open DevTools and select the Network panel
1. Load e.g. cnn.com
2. Open the Blocking side panel (on the left side) - click the proper button in the toolbar
3. Create a new entry in the side panel e.g: video*square that should block the following image on the pate video_buffer_square_blk.gif (note that you might see different images so, you might want to create different pattern)

The "video*square" blocking pattern should block all requests that contain "video + something + square" in their names

Thanks for the help!

Comment 12

[:Harald Kirschner :digitarald]

We should update https://wiki.developer.mozilla.org/en-US/docs/Tools/Network_Monitor/request_list#Blocking_specific_URLs