Closed Bug 1760085 Opened 4 years ago Closed 7 months ago

Intermittent toolkit/components/antitracking/test/xpcshell/test_purge_trackers.js | single tracking bug

Categories

(Core :: Privacy: Anti-Tracking, defect, P3)

Product:
Core
Component:
Privacy: Anti-Tracking
Type:
defect

Tracking

()

Status:
RESOLVED INCOMPLETE

People

(Reporter: jmaher, Unassigned)

References

Details

(Keywords: intermittent-failure, intermittent-testcase, perf-alert, Whiteboard: [stockwell disabled])

Attachments

(3 files, 2 obsolete files)

Bug 1760085 - Disable test_purge_trackers.js on linux 18.04 for frequent failures. r=#intermittent-reviewers
48 bytes, text/x-phabricator-request
Details | Review
Bug 1760085 - move test disabling rule for Linux to correct test test_purge_trackers.js. r=#intermittent-reviewers
48 bytes, text/x-phabricator-request
Details | Review
Bug 1760085 - disable test_purge_trackers.js on linux for causing frequent failures. r?#intermittent-reviewers
48 bytes, text/x-phabricator-request
Details | Review
No description provided.
Severity: -- → S3

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.

Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → INCOMPLETE

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=434151400&repo=mozilla-central

Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.

Status: REOPENED → RESOLVED
Closed: 2 years ago → 2 years ago
Resolution: --- → INCOMPLETE

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=437225793&repo=mozilla-central

Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.

Status: REOPENED → RESOLVED
Closed: 2 years ago → 2 years ago
Resolution: --- → INCOMPLETE

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=444393811&repo=autoland

Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
Attachment #9385577 - Attachment is obsolete: true

Update

There have been 35 total failures within the last 7 days:

  • 2 failures on Linux 18.04 x64 WebRender asan opt
  • 11 failures on Linux 18.04 x64 WebRender debug/opt
  • 1 failure on Linux 18.04 x64 WebRender Shippable opt
  • 21 failures on Linux 18.04 x64 WebRender tsan opt

Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=455257443&repo=mozilla-central&lineNumber=2830

[task 2024-04-20T10:39:48.388Z] 10:39:48     INFO -  TEST-START | toolkit/components/antitracking/test/xpcshell/test_purge_trackers.js
[task 2024-04-20T10:39:57.866Z] 10:39:57  WARNING -  TEST-UNEXPECTED-FAIL | toolkit/components/antitracking/test/xpcshell/test_purge_trackers.js | xpcshell return code: 0
[task 2024-04-20T10:39:57.867Z] 10:39:57     INFO -  TEST-INFO took 9468ms
[task 2024-04-20T10:39:57.867Z] 10:39:57     INFO -  >>>>>>>
[task 2024-04-20T10:39:57.868Z] 10:39:57     INFO -  (xpcshell/head.js) | test MAIN run_test pending (1)
[task 2024-04-20T10:39:57.868Z] 10:39:57     INFO -  (xpcshell/head.js) | test run_next_test 0 pending (2)
[task 2024-04-20T10:39:57.868Z] 10:39:57     INFO -  (xpcshell/head.js) | test MAIN run_test finished (2)
[task 2024-04-20T10:39:57.869Z] 10:39:57     INFO -  running event loop
[task 2024-04-20T10:39:57.869Z] 10:39:57     INFO -  toolkit/components/antitracking/test/xpcshell/test_purge_trackers.js | Starting testNotPurging
[task 2024-04-20T10:39:57.870Z] 10:39:57     INFO -  (xpcshell/head.js) | test testNotPurging pending (2)
[task 2024-04-20T10:39:57.870Z] 10:39:57     INFO -  (xpcshell/head.js) | test run_next_test 0 finished (2)
[task 2024-04-20T10:39:57.871Z] 10:39:57     INFO -  "CONSOLE_MESSAGE: (info) No chrome package registered for chrome://branding/locale/brand.properties"
[task 2024-04-20T10:39:57.872Z] 10:39:57     INFO -  PID 21467 | console.log: *** PurgeTrackerService:: "returning early, activeWithCookieBehavior: false, purgeEnabled: true"
[task 2024-04-20T10:39:57.873Z] 10:39:57     INFO -  TEST-PASS | toolkit/components/antitracking/test/xpcshell/test_purge_trackers.js | testNotPurging - [testNotPurging : 61] cookie remains. - true == true

Hi Tim, can you please assign this to someone?
Thank you.

Flags: needinfo?(tihuang)
Whiteboard: [stockwell needswork:owner]
Assignee: nobody → amadan
Flags: needinfo?(tihuang)
Attachment #9399872 - Attachment is obsolete: true

Abhishek, any updates on this one? It has reached disable recommended bugs list with 257 total failures in the last 30 days: https://treeherder.mozilla.org/intermittent-failures/bugdetails?startday=2024-04-18&endday=2024-05-18&tree=trunk&failurehash=all&bug=1760085
It will be disabled on linux if not fixed. Thank you.

Flags: needinfo?(amadan)
Pushed by csabou@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/90c06108720a Disable test_purge_trackers.js on linux 18.04 for frequent failures. r=intermittent-reviewers,jmaher
Whiteboard: [stockwell disable-recommended] → [stockwell disabled]

I was looking into this, I just got back from PTO. I will prioritize this

Flags: needinfo?(amadan)
Pushed by archaeopteryx@coole-files.de: https://hg.mozilla.org/integration/autoland/rev/2f2bf98bf4cb move test disabling rule for Linux to correct test test_purge_trackers.js. r=intermittent-reviewers,anti-tracking-reviewers,jmaher DONTBUILD

(In reply to Pulsebot from comment #139)

Pushed by archaeopteryx@coole-files.de:
https://hg.mozilla.org/integration/autoland/rev/2f2bf98bf4cb
move test disabling rule for Linux to correct test test_purge_trackers.js.
r=intermittent-reviewers,anti-tracking-reviewers,jmaher DONTBUILD

Perfherder has detected a browsertime performance change from push 2f2bf98bf4cb90c06ec3496cf3538efd8d90640a.

Improvements:

Ratio Test Platform Options Absolute values (old vs new) Performance Profiles
6% speedometer3 TodoMVC-Svelte-Complex-DOM/Adding100Items/Sync macosx1015-64-shippable-qr fission webrender 3.37 -> 3.17 Before/After
4% speedometer3 TodoMVC-Svelte-Complex-DOM/Adding100Items/Sync macosx1015-64-shippable-qr fission webrender 3.31 -> 3.19 Before/After

Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.

You can run these tests on try with ./mach try perf --alert 1222

For more information on performance sheriffing please see our FAQ.

  1. The change identified as regressor did not change product code, it only disabled a test on Linux.
  2. Based on the graph, the values increased (= regression) when bug 1898745 landed later.
  3. The improvement mentioned in comment 145 seems to have started with the landing of bug 1888756 and bug 1901064 in one push.
Assignee: abhishekmadan2002 → nobody
Flags: needinfo?(ryanvm)
Flags: needinfo?(ryanvm)

Joel, recent changes in https://hg.mozilla.org/integration/autoland/rev/1e2cf5b6ed1fbdcfacbe6a61fe96d4aa98def571#l6.14 have made the failure rate in this bug spike as seen here: https://treeherder.mozilla.org/intermittent-failures/bugdetails?startday=2025-02-10&endday=2025-03-12&tree=trunk&failurehash=all&bug=1760085
Could we reinstate that linux condition cause it looks like it fails across the board on linux?

Flags: needinfo?(jmaher)
Summary: Intermittent toolkit/components/antitracking/test/xpcshell/test_purge_trackers.js | single tracking bug → Frequent toolkit/components/antitracking/test/xpcshell/test_purge_trackers.js | single tracking bug

yes, lets just make it os == 'linux' && os_version == '18.04' && processor == 'x86_64'

Flags: needinfo?(jmaher)
Pushed by amarc@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/13d823d16348 disable test_purge_trackers.js on linux for causing frequent failures. r?#intermittent-reviewers
Pushed by amarc@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/1e8104bb5f08 Fix typo in disable patch CLOSED TREE

(In reply to Pulsebot from comment #187)

Pushed by amarc@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/13d823d16348
disable test_purge_trackers.js on linux for causing frequent failures.
r?#intermittent-reviewers

Hi Cristina! Do you think this changeset could have caused the performance regression reported below ?
Thanks!

Perfherder has detected a mozperftest performance change from push 1e8104bb5f08f6dd47bd9f323a971a58e3db8ee9.

Regressions:

Ratio Test Platform Options Absolute values (old vs new)
8% browser_ml_smart_tab_perf.js SMART-TAB-EMBEDDING-total-memory-usage linux1804-64-shippable 221.73 -> 239.17
7% browser_ml_smart_tab_perf.js SMART-TAB-EMBEDDING-total-memory-usage linux1804-64-shippable 221.79 -> 236.54

As author of one of the patches included in that push, we need your help to address this regression.
Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the patch(es) may be backed out in accordance with our regression policy.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.

You can run these tests on try with ./mach try perf --alert 44431

For more information on performance sheriffing please see our FAQ.

Flags: needinfo?(chorotan)

Alex, those changes just disable a test and prevents it from running because of a high failure rate on linux. I don't see how that could impact browser performance as it's just for CI reasons, it doesn't change behavior for users, and is not to have so many failures in Treeherder.
Most likely the regressor is somewhere else and I think this one is a very likely candidate: https://treeherder.mozilla.org/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&revision=614f785903275dba64ad243937543112b4912849

Flags: needinfo?(chorotan) → needinfo?(afinder)
Summary: Frequent toolkit/components/antitracking/test/xpcshell/test_purge_trackers.js | single tracking bug → Intermittent toolkit/components/antitracking/test/xpcshell/test_purge_trackers.js | single tracking bug

Thanks for your input! It was confirmed that Bug 1951338 most likely caused the regression mentioned by Alex Finder.

Flags: needinfo?(afinder)

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.

Status: REOPENED → RESOLVED
Closed: 2 years ago → 7 months ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: