Closed Bug 1775194 Opened 2 years ago Closed 1 year ago

Disable reporting API on nightly

Categories

(Core :: DOM: Security, task, P2)

task

Tracking

()

RESOLVED FIXED
109 Branch
Tracking Status
firefox109 --- fixed

People

(Reporter: hsinyi, Assigned: tschuster)

References

(Blocks 1 open bug)

Details

(Whiteboard: [domsecurity-active])

Attachments

(1 file)

Reporting API has been enabled on nightly for several years. And it is out dated that it requires work to catch up the latest spec https://bugzilla.mozilla.org/show_bug.cgi?id=1620573
It makes sense that we disable the feature on nightly. Enable it again when we have sorted out the release plan.

Blocks: 1631237

Agreed. And it probably shouldn't live under DOM: Security. I can take point to disable it myself.

Assignee: nobody → fbraun

This disables the outdated, incorrect implementation of the
Reporting API. The current implementation was only enabled
on Nightly builds, but given its current state it does not
even make sense there.

Severity: -- → S3
Status: NEW → ASSIGNED
Priority: -- → P2
Whiteboard: [domsecurity-active]

Backed out for causing mochitest failures on test_serviceworker_interfaces.html

Failure line: TEST-UNEXPECTED-FAIL | dom/serviceworkers/test/test_serviceworker_interfaces.html | false: Report should be defined on self

Push with failures

Failure log

Backout link

[task 2022-06-22T14:15:58.534Z] 14:15:58     INFO - TEST-PASS | dom/serviceworkers/test/test_serviceworker_interfaces.html | true: NetworkInformation should  NOT be defined on self 
[task 2022-06-22T14:15:58.534Z] 14:15:58     INFO - Buffered messages finished
[task 2022-06-22T14:15:58.535Z] 14:15:58     INFO - TEST-UNEXPECTED-FAIL | dom/serviceworkers/test/test_serviceworker_interfaces.html | false: Report should  be defined on self 
[task 2022-06-22T14:15:58.535Z] 14:15:58     INFO -     SimpleTest.ok@SimpleTest/SimpleTest.js:417:16
[task 2022-06-22T14:15:58.535Z] 14:15:58     INFO -     setupSW/window.onmessage@dom/serviceworkers/test/test_serviceworker_interfaces.html:31:11
[task 2022-06-22T14:15:58.536Z] 14:15:58     INFO - Not taking screenshot here: see the one that was previously logged
[task 2022-06-22T14:15:58.537Z] 14:15:58     INFO - TEST-UNEXPECTED-FAIL | dom/serviceworkers/test/test_serviceworker_interfaces.html | false: ReportBody should  be defined on self 
[task 2022-06-22T14:15:58.537Z] 14:15:58     INFO -     SimpleTest.ok@SimpleTest/SimpleTest.js:417:16
[task 2022-06-22T14:15:58.537Z] 14:15:58     INFO -     setupSW/window.onmessage@dom/serviceworkers/test/test_serviceworker_interfaces.html:31:11
[task 2022-06-22T14:15:58.537Z] 14:15:58     INFO - Not taking screenshot here: see the one that was previously logged
[task 2022-06-22T14:15:58.538Z] 14:15:58     INFO - TEST-UNEXPECTED-FAIL | dom/serviceworkers/test/test_serviceworker_interfaces.html | false: ReportingObserver should  be defined on self 
[task 2022-06-22T14:15:58.538Z] 14:15:58     INFO -     SimpleTest.ok@SimpleTest/SimpleTest.js:417:16
[task 2022-06-22T14:15:58.538Z] 14:15:58     INFO -     setupSW/window.onmessage@dom/serviceworkers/test/test_serviceworker_interfaces.html:31:11
[task 2022-06-22T14:15:58.539Z] 14:15:58     INFO - Not taking screenshot here: see the one that was previously logged
[task 2022-06-22T14:15:58.540Z] 14:15:58     INFO - TEST-UNEXPECTED-FAIL | dom/serviceworkers/test/test_serviceworker_interfaces.html | 3 === 0: The following interface(s) are not enumerated: Report, ReportBody, ReportingObserver 
[task 2022-06-22T14:15:58.540Z] 14:15:58     INFO -     SimpleTest.ok@SimpleTest/SimpleTest.js:417:16
[task 2022-06-22T14:15:58.540Z] 14:15:58     INFO -     setupSW/window.onmessage@dom/serviceworkers/test/test_serviceworker_interfaces.html:31:11
[task 2022-06-22T14:15:58.540Z] 14:15:58     INFO - TEST-PASS | dom/serviceworkers/test/test_serviceworker_interfaces.html | true: Bogus pref annotation for Module 
Flags: needinfo?(fbraun)

Since the backout was made there was another failure that was confirmed to have come from you push.

Failure line: TEST-UNEXPECTED-TIMEOUT | /html/cross-origin-embedder-policy/multi-globals/workers-coep-report.https.html | Multiple globals for Worker constructor: COEP reports - Test timed out

Push with failures

Failure log

Backout link

[task 2022-06-22T14:20:02.715Z] 14:20:02     INFO - TEST-START | /html/cross-origin-embedder-policy/multi-globals/workers-coep-report.https.html
[task 2022-06-22T14:20:02.724Z] 14:20:02     INFO - Setting pref browser.tabs.remote.useCrossOriginEmbedderPolicy to true
[task 2022-06-22T14:20:02.894Z] 14:20:02     INFO - PID 2240 | JavaScript error: https://web-platform.test:8443/html/cross-origin-embedder-policy/multi-globals/workers-coep-report.https.html, line 15: TypeError: global.ReportingObserver is not a constructor
[task 2022-06-22T14:20:12.931Z] 14:20:12     INFO - 
[task 2022-06-22T14:20:12.931Z] 14:20:12     INFO - TEST-UNEXPECTED-TIMEOUT | /html/cross-origin-embedder-policy/multi-globals/workers-coep-report.https.html | Multiple globals for Worker constructor: COEP reports - Test timed out
[task 2022-06-22T14:20:12.931Z] 14:20:12     INFO - TEST-INFO | expected FAIL
[task 2022-06-22T14:20:12.933Z] 14:20:12     INFO - TEST-UNEXPECTED-ERROR | /html/cross-origin-embedder-policy/multi-globals/workers-coep-report.https.html | Unhandled rejection: global.ReportingObserver is not a constructor
[task 2022-06-22T14:20:12.933Z] 14:20:12     INFO - observeReports@https://web-platform.test:8443/html/cross-origin-embedder-policy/multi-globals/workers-coep-report.https.html:15:20
[task 2022-06-22T14:20:12.933Z] 14:20:12     INFO - onload<@https://web-platform.test:8443/html/cross-origin-embedder-policy/multi-globals/workers-coep-report.https.html:30:7
[task 2022-06-22T14:20:12.933Z] 14:20:12     INFO - Test.prototype.step@https://web-platform.test:8443/resources/testharness.js:2590:25
[task 2022-06-22T14:20:12.933Z] 14:20:12     INFO - Test.prototype.step_func/<@https://web-platform.test:8443/resources/testharness.js:2637:35
[task 2022-06-22T14:20:12.933Z] 14:20:12     INFO - TEST-INFO took 10217ms
[task 2022-06-22T14:20:12.933Z] 14:20:12     INFO - Restarting browser for new test group
[task 2022-06-22T14:20:12.934Z] 14:20:12     INFO - PID 2240 | 1655907612933	Marionette	INFO	Stopped listening on port 55047
[task 2022-06-22T14:20:13.014Z] 14:20:13     INFO - PID 2240 | [Parent 2240, Main Thread] WARNING: ContentParent: id=15ec04400 - BlockShutdown: CanSend.: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:3607
[task 2022-06-22T14:20:13.015Z] 14:20:13     INFO - PID 2240 | [Parent 2240, Main Thread] WARNING: ContentParent: id=15ec04400 - ShutDownProcess: Sent shutdown message.: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:1808
[task 2022-06-22T14:20:13.016Z] 14:20:13     INFO - PID 2240 | [Parent 2240, Main Thread] WARNING: ContentParent: id=152712800 - BlockShutdown: CanSend.: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:3607
[task 2022-06-22T14:20:13.016Z] 14:20:13     INFO - PID 2240 | [Parent 2240, Main Thread] WARNING: ContentParent: id=152712800 - ShutDownProcess: Sent shutdown message.: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:1808
[task 2022-06-22T14:20:13.017Z] 14:20:13     INFO - PID 2240 | [Parent 2240, Main Thread] WARNING: ContentParent: id=15f495400 - BlockShutdown: CanSend.: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:3607
[task 2022-06-22T14:20:13.018Z] 14:20:13     INFO - PID 2240 | [Parent 2240, Main Thread] WARNING: ContentParent: id=15f495400 - ShutDownProcess: Sent shutdown message.: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:1808
[task 2022-06-22T14:20:13.019Z] 14:20:13     INFO - PID 2240 | [Parent 2240, Main Thread] WARNING: ContentParent: id=15ec04400 - ShutDownProcess: Closing channel.: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:1843
[task 2022-06-22T14:20:13.020Z] 14:20:13     INFO - PID 2240 | [Parent 2240, Main Thread] WARNING: ContentParent: id=15ec04400 - RemoveShutdownBlockers: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:3709
[task 2022-06-22T14:20:13.021Z] 14:20:13     INFO - PID 2240 | [Parent 2240, Main Thread] WARNING: ContentParent: id=152712800 - ShutDownProcess: Closing channel.: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:1843
[task 2022-06-22T14:20:13.022Z] 14:20:13     INFO - PID 2240 | [Parent 2240, Main Thread] WARNING: ContentParent: id=152712800 - RemoveShutdownBlockers: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:3709
[task 2022-06-22T14:20:13.023Z] 14:20:13     INFO - PID 2240 | [Parent 2240, Main Thread] WARNING: ContentParent: id=15f495400 - ShutDownProcess: Closing channel.: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:1843
[task 2022-06-22T14:20:13.024Z] 14:20:13     INFO - PID 2240 | [Parent 2240, Main Thread] WARNING: ContentParent: id=15f495400 - RemoveShutdownBlockers: file /builds/worker/checkouts/gecko/dom/ipc/ContentParent.cpp:3709
[task 2022-06-22T14:20:13.450Z] 14:20:13     INFO - Browser exited with return code 0
[task 2022-06-22T14:20:13.453Z] 14:20:13     INFO - Closing logging queue
[task 2022-06-22T14:20:13.453Z] 14:20:13     INFO - queue closed
[task 2022-06-22T14:20:13.473Z] 14:20:13     INFO - Application command: /opt/worker/tasks/task_165590695434041/build/application/Firefox Nightly.app/Contents/MacOS/firefox --marionette about:blank -foreground -profile /var/folders/xk/gftf94q92cb5r37qj2w7wtrm000014/T/tmpbwwaseuk
[task 2022-06-22T14:20:13.479Z] 14:20:13     INFO - PID 2291 | 1655907602580	Marionette	INFO	Marionette enabled
[task 2022-06-22T14:20:13.480Z] 14:20:13     INFO - PID 2291 | 1655907602609	Marionette	INFO	Listening on port 55668
[task 2022-06-22T14:20:13.481Z] 14:20:13     INFO - PID 2291 | console.warn: SearchSettings: "get: No settings file exists, new profile?" (new NotFoundError("Could not open the file at /var/folders/xk/gftf94q92cb5r37qj2w7wtrm000014/T/tmpuc0jrn0x/search.json.mozlz4", (void 0)))
[task 2022-06-22T14:20:13.481Z] 14:20:13     INFO - Starting runner
[task 2022-06-22T14:20:13.758Z] 14:20:13     INFO - PID 2291 | 2022-06-22 14:20:13.757 firefox[2291:56838] Persistent UI failed to open file file:///Users/cltbld/Library/Saved%20Application%20State/org.mozilla.nightly.savedState/window_1.data: No such file or directory (2)
[task 2022-06-22T14:20:14.076Z] 14:20:14     INFO - TEST-START | /html/editing/editing-0/autocapitalization/autocapitalize.html

I had no idea this is used in so many independent tests, which is sad. In essence, this means we need to rely on our own tests for these features (COEP, mostly).

Reminds me of a similar case, where we did not support securitypolicyviolation events (for CSP) and various CSP-unrelated tests started expecting it. That was in bug 1037335 (and its dependency tree).

Flags: needinfo?(fbraun)

Forgot to mention, that I'm on it. I'm afraid we got no choice but to mark those WPT as failing.

I had no idea this is used in so many independent tests, which is sad. In essence, this means we need to rely on our own tests for these features (COEP, mostly).

We could just enable the pref for these tests.

Attachment #9282194 - Attachment description: Bug 1775194 - disable reporting API r?ckerschb → Bug 1775194 - disable reporting API r?ckerschb,nika!
Pushed by fbraun@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/e883552f154d
disable reporting API r=ckerschb,emilio
Flags: needinfo?(fbraun)
Pushed by fbraun@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/c3682c83547e
disable reporting API r=ckerschb,emilio

Backed out changeset c3682c83547e (bug 1775194) for causing wpt failures in /deprecation-reporting/idlharness.any.html

Backout link: https://hg.mozilla.org/integration/autoland/rev/b75452cff52ce1c17ddf010504e78ad3fc9e9bb7

Push with failures

Failure log

INFO - TEST-PASS | /deprecation-reporting/idlharness.any.html | idl_test validation 
[task 2022-06-28T14:00:27.649Z] 14:00:27     INFO - TEST-UNEXPECTED-FAIL | /deprecation-reporting/idlharness.any.html | DeprecationReportBody interface: existence and properties of interface object - assert_own_property: self does not have own property "DeprecationReportBody" expected property "DeprecationReportBody" missing
[task 2022-06-28T14:00:27.649Z] 14:00:27     INFO - IdlInterface.prototype.assert_interface_object_exists@http://web-platform.test:8000/resources/idlharness.js:1309:24
Flags: needinfo?(fbraun)

Surprising these did not come up in the previous test run. OK, let's make sure these tests also get the dom.reporting.enabled set to true.

Flags: needinfo?(fbraun)
Pushed by fbraun@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/068f148f46f2
disable reporting API r=ckerschb,emilio

Backed out changeset 068f148f46f2 (Bug 1775194) for causing wpt failures on document-write-reporting-tentative.html.
Backout link
Push with failures
Failure Log

Flags: needinfo?(fbraun)
Backout by mlaza@mozilla.com:
https://hg.mozilla.org/mozilla-central/rev/82f00e347ac2
Backed out changeset 068f148f46f2 for causing wpt failures on document-write-reporting-tentative.html. CLOSED TREE

It's confusing why these test errors did not show up on try and I keep getting new ones between backout and backout.

@Hsin-Yi: It seems that a lot of APIs are using those observers for testing completely unrelated tests (e.g., COEP). I believe we need to keep this feature in some sort of reasonable state to not lose test coverage for COEP and such.

Flags: needinfo?(fbraun) → needinfo?(htsai)

(In reply to Frederik Braun [:freddy] from comment #17)

It's confusing why these test errors did not show up on try and I keep getting new ones between backout and backout.

@Hsin-Yi: It seems that a lot of APIs are using those observers for testing completely unrelated tests (e.g., COEP). I believe we need to keep this feature in some sort of reasonable state to not lose test coverage for COEP and such.

Thank you Freddy for giving it a try, and sorry that it turns out to be not as trivial as the first thought. I don't really know who has a broad view of this API and other features like COEP, document policy, etc. So, yup, I believe we should keep the status for Reporting API as is for another while, and review a plan for this.

Flags: needinfo?(htsai)
Pushed by fbraun@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/1491786f8402
disable reporting API r=ckerschb,emilio
Type: defect → task

There's a r+ patch which didn't land and no activity in this bug for 2 weeks.
:freddy, could you have a look please?
If you still have some work to do, you can add an action "Plan Changes" in Phabricator.
For more information, please visit auto_nag documentation.

Flags: needinfo?(fbraun)
Flags: needinfo?(emilio)
Flags: needinfo?(emilio)
Attachment #9282194 - Attachment description: Bug 1775194 - disable reporting API r?ckerschb,nika! → Bug 1775194 - disable reporting API r=ckerschb,emilio
Assignee: fbraun → nobody
Status: ASSIGNED → NEW
Flags: needinfo?(fbraun)
Assignee: nobody → tschuster
Pushed by tschuster@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/45318a69aa49
disable reporting API r=ckerschb,emilio,freddyb
Status: NEW → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
Target Milestone: --- → 109 Branch

Should dom.reporting.featurePolicy.enabled be disabled too? Found this from https://github.com/mdn/browser-compat-data/pull/18557.

Flags: needinfo?(fbraun)

Oh, commandeered by Tom actually.

Flags: needinfo?(tschuster)

Oh, we probably should probably disable it even in Nightly and only enable it for tests.

Flags: needinfo?(tschuster)
Flags: needinfo?(fbraun)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: