Closed Bug 1592498 Opened 5 years ago Closed 4 years ago

Intermittent browser/components/tests/browser/browser_urlbar_matchBuckets_migration60.js | Test timed out -

Categories

(Firefox :: Address Bar, defect, P2)

Product:
Firefox
Component:
Address Bar
Type:
defect
Points:
2

Tracking

()

Status:
RESOLVED FIXED
Milestone:
85 Branch
Iteration:
84.2 - Nov 2 - Nov 15
Tracking Flags:
Tracking Status
firefox85 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: mak)

References

(Depends on 1 open bug)

Details

(Keywords: intermittent-failure, Whiteboard: [stockwell disabled][stockwell needswork:owner])

Attachments

(4 files)

Bug 1592498 - Disable browser_urlbar_matchBuckets_migration60.js on Win QR Debug. r?#intermittent-reviewers
47 bytes, text/x-phabricator-request
Details | Review
Bug 1592498 - Revert the disable on win
47 bytes, text/x-phabricator-request
Details | Review
Bug 1592498 - adjust disabling condition for browser_urlbar_matchBuckets_migration60.js on all win platforms r=jmaher
47 bytes, text/x-phabricator-request
Details | Review
Bug 1592498 - Remove UI migrations up to Firefox 59 and migration 66 for intermittent failures. r=Gijs
47 bytes, text/x-phabricator-request
Details | Review

Filed by: btara [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=273600555&repo=autoland
Full log: https://queue.taskcluster.net/v1/task/Ds_mmAKmT_mM1CPnUCtt0Q/runs/0/artifacts/public/logs/live_backing.log

[task 2019-10-30T02:27:27.396Z] 02:27:27 INFO - TEST-START | browser/components/tests/browser/browser_urlbar_matchBuckets_migration60.js
[task 2019-10-30T02:28:12.395Z] 02:28:12 INFO - TEST-INFO | started process screentopng
[task 2019-10-30T02:28:12.965Z] 02:28:12 INFO - TEST-INFO | screentopng: exit 0
[task 2019-10-30T02:28:12.965Z] 02:28:12 INFO - Buffered messages logged at 02:27:27
[task 2019-10-30T02:28:12.965Z] 02:28:12 INFO - Entering test bound migrate
[task 2019-10-30T02:28:12.965Z] 02:28:12 INFO - TEST-PASS | browser/components/tests/browser/browser_urlbar_matchBuckets_migration60.js | Pref should be cleared initially - "" == "" -
[task 2019-10-30T02:28:12.965Z] 02:28:12 INFO - TEST-PASS | browser/components/tests/browser/browser_urlbar_matchBuckets_migration60.js | Study should not be installed initially - true == true -
[task 2019-10-30T02:28:12.965Z] 02:28:12 INFO - Buffered messages finished
[task 2019-10-30T02:28:12.965Z] 02:28:12 INFO - TEST-UNEXPECTED-FAIL | browser/components/tests/browser/browser_urlbar_matchBuckets_migration60.js | Test timed out -
[task 2019-10-30T02:28:12.965Z] 02:28:12 INFO - GECKO(6450) | MEMORY STAT | vsize 2951MB | residentFast 284MB | heapAllocated 91MB
[task 2019-10-30T02:28:12.965Z] 02:28:12 INFO - TEST-OK | browser/components/tests/browser/browser_urlbar_matchBuckets_migration60.js | took 45011ms
[task 2019-10-30T02:28:12.965Z] 02:28:12 INFO - checking window state
[task 2019-10-30T02:28:13.789Z] 02:28:13 INFO - GECKO(6450) | Completed ShutdownLeaks collections in process 6750
[task 2019-10-30T02:28:13.809Z] 02:28:13 INFO - GECKO(6450) | Completed ShutdownLeaks collections in process 6715
[task 2019-10-30T02:28:13.810Z] 02:28:13 INFO - GECKO(6450) | Completed ShutdownLeaks collections in process 6645
[task 2019-10-30T02:28:13.846Z] 02:28:13 INFO - GECKO(6450) | Completed ShutdownLeaks collections in process 6635
[task 2019-10-30T02:28:13.873Z] 02:28:13 INFO - GECKO(6450) | Completed ShutdownLeaks collections in process 6682
[task 2019-10-30T02:28:13.889Z] 02:28:13 INFO - GECKO(6450) | Completed ShutdownLeaks collections in process 6618
[task 2019-10-30T02:28:13.892Z] 02:28:13 INFO - GECKO(6450) | Completed ShutdownLeaks collections in process 6519
[task 2019-10-30T02:28:13.928Z] 02:28:13 INFO - GECKO(6450) | Completed ShutdownLeaks collections in process 6502
[task 2019-10-30T02:28:13.956Z] 02:28:13 INFO - GECKO(6450) | Completed ShutdownLeaks collections in process 6578
[task 2019-10-30T02:28:14.217Z] 02:28:14 INFO - GECKO(6450) | Completed ShutdownLeaks collections in process 6450
[task 2019-10-30T02:28:14.217Z] 02:28:14 INFO - TEST-START | Shutdown

Assignee: nobody → dluca

I have no idea why that change would have made this bug spike so much. My only theory is that maybe the remote-settings changes are somehow causing issues? Makes me wonder what we would happen if we ran a Try push with just that change reverted. Also makes me wonder if this'll go away after tomorrow's update lands.

Flags: needinfo?(ryanvm)
Pushed by dluca@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/0378a2adb358
Disable browser_urlbar_matchBuckets_migration60.js on Win QR Debug. r=jmaher
Keywords: leave-open
Whiteboard: [stockwell disable-recommended] → [stockwell disabled]
Pushed by dluca@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/97a9b17dba53
Updated disable browser_urlbar_matchBuckets_migration60.js to win Debug. CLOSED TREE
Keywords: leave-open
Whiteboard: [stockwell disabled]

https://hg.mozilla.org/mozilla-central/rev/a9a91a4e774c

Status: NEW → RESOLVED
Closed: 4 years ago
status-firefox73: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → Firefox 73
Status: RESOLVED → REOPENED
status-firefox73: fixed → ---
Resolution: FIXED → ---
Target Milestone: Firefox 73 → ---

We have 15 failures on windows, in the last 4 days, (after the re-enable patch): https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?startday=2019-12-05&endday=2019-12-09&tree=trunk&bug=1592498

:jmaher, do you think we should disable it again?

Flags: needinfo?(jmaher)
Whiteboard: [stockwell disable-recommended]

that is a high enough failure rate, we should disable this on windows again.

Flags: needinfo?(jmaher)

Backed out changeset a9a91a4e774c (Bug 1592498) revert to disabled on windows debug.

Backout by dvarga@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/06fc7f9ee3fa
Backed out changeset a9a91a4e774c revert to disabled on windows debug r=jmaher.
Whiteboard: [stockwell disable-recommended] → [stockwell needswork:owner], [stockwell disabled]
Whiteboard: , [stockwell disable-recommended] → [stockwell disabled]

This bug failed 30 times in the last 7 days. Occurs on macosx1014-64-shippable, windows10-64, windows10-64-qr, windows10-64-shippable, windows10-64-shippable-qr, windows7-32, windows7-32-shippable on opt build types.

Recent log:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=290086063&repo=autoland&lineNumber=4000

Justin: Can you have someone take a look at this bug?

Flags: needinfo?(dolske)

Update:
There have been 44 failures within the last 7 days:

  • 1 failure on OS X 10.14 debug
  • 7 failures on OS X 10.14 Shippable opt
  • 4 failures on Windows 10 x64 opt
  • 2 failures on Windows 10 x64 QuantumRender opt
  • 22 failures on Windows 10 x64 QuantumRender Shippable opt
  • 1 failure on Windows 7 opt
  • 7 failures on Windows 7 opt

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=291598773&repo=mozilla-central&lineNumber=4634

[task 2020-03-04T09:38:09.538Z] 09:38:09 INFO - TEST-PASS | browser/components/tests/browser/browser_urlbar_matchBuckets_migration60.js | Pref should be cleared initially - "" == "" -
[task 2020-03-04T09:38:09.538Z] 09:38:09 INFO - TEST-PASS | browser/components/tests/browser/browser_urlbar_matchBuckets_migration60.js | Study should not be installed initially - true == true -
[task 2020-03-04T09:38:09.538Z] 09:38:09 INFO - Buffered messages finished
[task 2020-03-04T09:38:09.538Z] 09:38:09 INFO - TEST-UNEXPECTED-FAIL | browser/components/tests/browser/browser_urlbar_matchBuckets_migration60.js | Test timed out -
[task 2020-03-04T09:38:09.538Z] 09:38:09 INFO - GECKO(3136) | MEMORY STAT | vsize 2104187MB | vsizeMaxContiguous 67068347MB | residentFast 250MB | heapAllocated 75MB
[task 2020-03-04T09:38:09.538Z] 09:38:09 INFO - TEST-OK | browser/components/tests/browser/browser_urlbar_matchBuckets_migration60.js | took 45087ms

Whiteboard: [stockwell disabled] → [stockwell disabled][stockwell needswork:owner]

There are 60 failures associated to this bug in the last 7 days. These are occurring on:

  • macosx1014-64-shippable
  • windows10-64
  • windows10-64-qr
  • windows10-64-shippable
  • windows10-64-shippable-qr
  • windows7-32-shippable
Attachment #9134285 - Attachment description: Bug 1592498 - adjust disabling condition for all windows r?#intermittent-reviewers → Bug 1592498 - adjust disabling condition for browser_urlbar_matchBuckets_migration60.js on all win platforms r=jmaher
Pushed by nbeleuzu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/a9dcd1fb27f2
adjust disabling condition for browser_urlbar_matchBuckets_migration60.js on all win platforms r=jmaher

Justin, could you assign this to someone?

Flags: needinfo?(dolske)
Flags: needinfo?(dolske) → needinfo?(mdeboer)

Marco, seems this test is now disabled on all of Windows. Any idea what's going on?

Flags: needinfo?(mdeboer) → needinfo?(mak)

P5 doesn't help finding bugs in need of help, when tests are disabled they should be bumped to P2 imo...

To fail only on Windows, it must be something related with the experiment the test is trying to use. I can try some debugging in the next days, hopefully it's not too much into the Normandy details. I'll leave the needinfo open.

Flags: needinfo?(mak)
Priority: P5 → P2
Flags: needinfo?(mak)

Based on code inspection, the only things that can fail here are "shield-init-complete" never arriving, or the PreferenceExperiments API never resolving. I suspect the former and this is probably just an effect of bug 1601111.
I'm honestly not sure it's worth spending time trying to fix this bug, when this migration happened in Firefox 60 and the only downside is this failure.
I'd rather remove more UI migrations, do we expect a consistent amount of users to upgrade from a version < 60 to the current version? for normal releases I think the risk is very limited, due to the many watersheds in the middle.
For ESR we have 60, 68 and 78, I'm not sure whether every ESR is a watershed. But anyway, I'm also sure we didn't run this experiment on ESR, so this code would still be pointless.

In practice, I'd suggest to remove all the migrations up to 62 included (Firefox 59) and then I'd also remove 66 (this one, that was part of Firefox 60). At the next ESR release, we could move on removing up to migration 80.

If you're ok with that, I can do it in the next days.

Depends on: 1601111
Flags: needinfo?(mak) → needinfo?(gijskruitbosch+bugs)

We have a watershed at 72, FWIW.

(In reply to Marco Bonardo [:mak] from comment #60)

Based on code inspection, the only things that can fail here are "shield-init-complete" never arriving, or the PreferenceExperiments API never resolving. I suspect the former and this is probably just an effect of bug 1601111.

FWIW, fixing that bug does seem like we should do it "at some point". I'm also not 100% sure why removing migrations would alleviate the concern from that bug?

I'm honestly not sure it's worth spending time trying to fix this bug, when this migration happened in Firefox 60 and the only downside is this failure.
I'd rather remove more UI migrations, do we expect a consistent amount of users to upgrade from a version < 60 to the current version? for normal releases I think the risk is very limited, due to the many watersheds in the middle.
For ESR we have 60, 68 and 78, I'm not sure whether every ESR is a watershed. But anyway, I'm also sure we didn't run this experiment on ESR, so this code would still be pointless.

In practice, I'd suggest to remove all the migrations up to 62 included (Firefox 59) and then I'd also remove 66 (this one, that was part of Firefox 60). At the next ESR release, we could move on removing up to migration 80.

This seems OK to me.

Flags: needinfo?(gijskruitbosch+bugs)

(In reply to :Gijs (he/him) from comment #62)

FWIW, fixing that bug does seem like we should do it "at some point". I'm also not 100% sure why removing migrations would alleviate the concern from that bug?

Removing migrations will only alleviate this specific failure, not solve or help in any way bug 1601111.

Assignee: dluca → mak
Status: REOPENED → ASSIGNED
Iteration: --- → 84.2 - Nov 2 - Nov 15
Points: --- → 2
Component: General → Address Bar
Pushed by mak77@bonardo.net:
https://hg.mozilla.org/integration/autoland/rev/02f6c7de8259
Remove UI migrations up to Firefox 59 and migration 66 for intermittent failures. r=Gijs
Keywords: leave-open

https://hg.mozilla.org/mozilla-central/rev/02f6c7de8259

Status: ASSIGNED → RESOLVED
Closed: 4 years ago → 4 years ago
status-firefox85: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 85 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: