1745761 - [DNR] Hook up rule engine with the network

Status: RESOLVED FIXED

(WebExtensions :: Request Handling, enhancement, P2)

Description

[Rob Wu [:robwu]]

Enforce the following DNR actions: block, redirect, allow, upgradeScheme, modifyHeaders, allowAllRequests

Comment 1

[Rob Wu [:robwu]]

To speed up the hooking up with the network part, I've moved the non-terminal actions to separate bugs (bug 1797403 + bug 1797404).

The focus of this bug will be the following actions: block, redirect, upgradeScheme.
These actions may cancel or replace the channel of a request.

Comment 2

[Rob Wu [:robwu]]

Attachment: Bug 1745761 - Connect DNR to network stack

This patch implements the "block", "upgradeScheme" and "redirect" DNR
actions, plus a comprehensive set of unit tests that exercise relevant
scenarios and edge cases.

Comment 3

[Rob Wu [:robwu]]

Attachment: Bug 1745761 - Enforce privateBrowsingAllowed for DNR

Comment 4

[Rob Wu [:robwu]]

Attachment: Bug 1745761 - Pass tabId from channel to DNR

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5abe72701c13
Connect DNR to network stack r=rpl
https://hg.mozilla.org/integration/autoland/rev/74b33f61c4d3
Enforce privateBrowsingAllowed for DNR r=rpl

Comment 6

[Natalia Csoregi [:nataliaCs]]

Backed out for failures on test_ext_dnr_without_webrequest.js

• backout: https://hg.mozilla.org/integration/autoland/rev/4fc0603aba5df363385ec5ad3bab7cd06fe09965
• push: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&revision=74b33f61c4d3b326920d236113461a737b65ead6
• failure log: https://treeherder.mozilla.org/logviewer?job_id=397629107&repo=autoland&lineNumber=6600

[task 2022-11-24T22:58:32.474Z] 22:58:32     INFO -  TEST-START | xpcshell.ini:toolkit/components/extensions/test/xpcshell/test_ext_dnr_without_webrequest.js
[task 2022-11-24T22:58:32.514Z] 22:58:32     INFO -  adb launch_application: am startservice -W -n 'org.mozilla.geckoview.test_runner/org.mozilla.geckoview.test_runner.XpcshellTestRunnerService$i0' -a android.intent.action.MAIN --es env0 XPCOM_DEBUG_BREAK=stack-and-abort --es env1 MOZ_CRASHREPORTER=1 --es env2 MOZ_CRASHREPORTER_NO_REPORT=1 --es env3 MOZ_DISABLE_NONLOCAL_CONNECTIONS=1 --es env4 MOZ_DEVELOPER_REPO_DIR=/builds/worker/checkouts/gecko --es env5 MOZ_DISABLE_CONTENT_SANDBOX=1 --es env6 MOZ_FETCHES_DIR=/builds/worker/fetches --es env7 MOZ_DISABLE_SOCKET_PROCESS=1 --es env8 LD_LIBRARY_PATH=/data/local/tmp/test_root/xpcb --es env9 MOZ_LINKER_CACHE=/data/local/tmp/test_root/xpcb --es env10 GRE_HOME=/data/local/tmp/test_root/xpcb --es env11 XPCSHELL_TEST_PROFILE_DIR=/data/local/tmp/test_root/xpc/p/79e98579-d328-4bc7-af85-8bcf802421f8 --es env12 HOME=/data/local/tmp/test_root/xpc/p --es env13 XPCSHELL_TEST_TEMP_DIR=/data/local/tmp/test_root/xpc/tmp/a468386b-d256-4025-9e41-53ee589a7a5c --es env14 MOZ_ANDROID_DATA_DIR=/data/local/tmp/test_root/xpcb --es env15 MOZ_IN_AUTOMATION=1 --es env16 MOZ_ANDROID_CPU_ABI=x86_64 --es env17 MOZHTTP2_PORT=41654 --es env18 MOZNODE_EXEC_PORT=39704 --es env19 TMPDIR=/data/local/tmp/test_root/xpc/p/79e98579-d328-4bc7-af85-8bcf802421f8 --es env20 XPCSHELL_MINIDUMP_DIR=/data/local/tmp/test_root/xpc/minidumps/79e98579-d328-4bc7-af85-8bcf802421f8 --es arg0 -g --es arg1 /data/local/tmp/test_root/xpcb --es arg2 --greomni --es arg3 /data/local/tmp/test_root/xpcb/geckoview-test_runner.apk --es arg4 -m --es arg5 -e --es arg6 'const _HEAD_JS_PATH = "/data/local/tmp/test_root/xpc/head.js";' --es arg7 -e --es arg8 'const _MOZINFO_JS_PATH = "/data/local/tmp/test_root/xpc/p/79e98579-d328-4bc7-af85-8bcf802421f8/mozinfo.json";' --es arg9 -e --es arg10 'const _PREFS_FILE = "/data/local/tmp/test_root/xpc/tmp/a468386b-d256-4025-9e41-53ee589a7a5c/user.js";' --es arg11 -e --es arg12 'const _TESTING_MODULES_DIR = "/data/local/tmp/test_root/xpc/m";' --es arg13 -f --es arg14 /data/local/tmp/test_root/xpc/head.js --es arg15 -e --es arg16 'const _HEAD_FILES = ["/data/local/tmp/test_root/xpc/toolkit/components/extensions/test/xpcshell/head.js", "/data/local/tmp/test_root/xpc/toolkit/components/extensions/test/xpcshell/head_telemetry.js", "/data/local/tmp/test_root/xpc/toolkit/components/extensions/test/xpcshell/head_sync.js", "/data/local/tmp/test_root/xpc/toolkit/components/extensions/test/xpcshell/head_storage.js"];' --es arg17 -e --es arg18 'const _JSDEBUGGER_PORT = 0;' --es arg19 -e --es arg20 'const _TEST_CWD = "/data/local/tmp/test_root/xpc/toolkit/components/extensions/test/xpcshell";' --es arg21 -e --es arg22 'const _TEST_FILE = ["test_ext_dnr_without_webrequest.js"];' --es arg23 -e --es arg24 'const _TEST_NAME = "xpcshell.ini:toolkit/components/extensions/test/xpcshell/test_ext_dnr_without_webrequest.js";' --es arg25 -e --es arg26 '_execute_test(); quit(0);' --ez use_multiprocess True --es out_file /data/local/tmp/test_root/xpc/logs/xpcshell-4eff4043-dcf2-47f6-9fab-add63e49938c.log
[task 2022-11-24T22:58:32.754Z] 22:58:32     INFO -  remotexpcshelltests.py | xpcshell.ini:toolkit/components/extensions/test/xpcshell/test_ext_dnr_without_webrequest.js | 17239 | Launched Test App
[task 2022-11-24T22:58:50.247Z] 22:58:50     INFO -  remotexpcshelltests.py | xpcshell.ini:toolkit/components/extensions/test/xpcshell/test_ext_dnr_without_webrequest.js | 17239 | Application ran for: 0:00:17.772997
[task 2022-11-24T22:58:50.347Z] 22:58:50  WARNING -  TEST-UNEXPECTED-FAIL | xpcshell.ini:toolkit/components/extensions/test/xpcshell/test_ext_dnr_without_webrequest.js | xpcshell return code: 0
[task 2022-11-24T22:58:50.347Z] 22:58:50     INFO -  TEST-INFO took 17874ms

Comment 7

[Rob Wu [:robwu]]

This is the relevant part of the log:

TEST-UNEXPECTED-FAIL | xpcshell.ini:toolkit/components/extensions/test/xpcshell/test_ext_dnr_without_webrequest.js | redirect_request_with_dnr_to_extensionPath - [redirect_request_with_dnr_to_extensionPath : 165] undefined - Expected: moz-extension://00e41a36-432b-4c14-abc3-ad6d3e92560e/war.txt?1, Actual: jar:file:///data/local/tmp/test_root/xpc/p/11197859-091d-4de3-b93c-db883580c169/temp/generated-extension.xpi!/war.txt?1 - false == true

On Android, we run extensions in-process, i.e. extensions.webextensions.remote = false.
I can reproduce the error on desktop with:

./mach test toolkit/components/extensions/test/xpcshell/test_ext_dnr_without_webrequest.js --verbose --log-mach-verbose --tag=in-process-webextensions --sequential

I reduced the problem and filed bug 1802385. I think that for now, the best course of action is to skip that part of the test when extensions are running in-process.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b3d9f351d26b
Connect DNR to network stack r=rpl
https://hg.mozilla.org/integration/autoland/rev/e44d95849202
Enforce privateBrowsingAllowed for DNR r=rpl

Comment 9

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/b3d9f351d26b
https://hg.mozilla.org/mozilla-central/rev/e44d95849202

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fbf423f25c14
Pass tabId from channel to DNR r=rpl

Comment 11

[Serban Stanca [:SerbanS]]

Backed out for causing mochitests failures on test_ext_dnr_tabIds.html.

• Backout link
• Push with failures
• Failure Log
• Failure line: TEST-UNEXPECTED-FAIL | toolkit/components/extensions/test/mochitest/test_ext_dnr_tabIds.html | Test timed out

Comment 12

[Rob Wu [:robwu]]

(In reply to sstanca from comment #11)

Backed out for causing mochitests failures on test_ext_dnr_tabIds.html.

The test was failing because the host_permissions in the manifest were not granted despite temporarilyInstalled: true + granted_host_permissions: true. This was because of bug 1802571. I'm skipping the test on Android and then relanding it. The test will be re-enabled in bug 1802571.

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b5eb5ec25320
Pass tabId from channel to DNR r=rpl

Comment 14

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/b5eb5ec25320