1654192 - Use RKV safe mode for xulstore

Status: RESOLVED FIXED

(Toolkit :: Storage, enhancement, P3)

Comment 1

[Victor Porof [:vporof][:vp]]

Attachment: Bug 1654192 - Part 1: Update RKV dependency to safe-mode, r=nanj

Comment 2

[Victor Porof [:vporof][:vp]]

Attachment: Bug 1654192 - Part 2: Update xulstore to use RKV in safe mode, r=nanj

Depends on D85314

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f1ee7be3a135
Part 1: Update RKV dependency to safe-mode, r=nanj
https://hg.mozilla.org/integration/autoland/rev/885b9b821bc0
Part 2: Update xulstore to use RKV in safe mode, r=nanj

Comment 4

[Noemi Erli[:noemi_erli]]

Backed out 2 changesets (Bug 1654192) for causing failures in browser_startup_mainthreadio.js CLOSED TREE

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&resultStatus=testfailed%2Cbusted%2Cexception&revision=885b9b821bc04c32e97f1a924170fda8858b9113&selectedTaskRun=dNg2WghbSVmm5ZsBHiIwow.0

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=311444845&repo=autoland&lineNumber=2491
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=311449853&repo=autoland&lineNumber=1783

Backout: https://hg.mozilla.org/integration/autoland/rev/2276446c81d6afe0efe07040a9f584ffc6b04792

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6657944c0b63
Part 1: Update RKV dependency to safe-mode, r=nanj
https://hg.mozilla.org/integration/autoland/rev/bad6faf26935
Part 2: Update xulstore to use RKV in safe mode, r=nanj

Comment 6

[Dorel Luca [:dluca]]

Backed out 2 changesets (bug 1654192) for Talos failures in \xulstore\data.safe.bin. CLOSED TREE

Log:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=311666256&repo=autoland&lineNumber=1748

Push with failures:
https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&revision=bad6faf26935ebcc07e602769e43286ddd5d183a

Backout:
https://hg.mozilla.org/integration/autoland/rev/2038f000aed1471bb91ff711cabb3e32b7982279

Comment 7

[BugBot [:suhaib / :marco/ :calixte]]

There are some r+ patches which didn't land and no activity in this bug for 2 weeks.
:vporof, could you have a look please?
For more information, please visit auto_nag documentation.

Comment 8

[Victor Porof [:vporof][:vp]]

Back on it now.

Comment 9

[Victor Porof [:vporof][:vp]]

Attachment: Bug 1654192 - Part 3: Use manager instead of creating unique instances every time, r=nanj

Depends on D85315

Comment 10

[Alexandru Ionescu (needinfo me) [:alexandrui]]

(In reply to Dorel Luca [:dluca] from comment #6)

Backed out 2 changesets (bug 1654192) for Talos failures in \xulstore\data.safe.bin. CLOSED TREE

Log:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=311666256&repo=autoland&lineNumber=1748

Push with failures:
https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&revision=bad6faf26935ebcc07e602769e43286ddd5d183a

Backout:
https://hg.mozilla.org/integration/autoland/rev/2038f000aed1471bb91ff711cabb3e32b7982279

:dluca, is there any way I can see that this patch no longer causes the failures before it was backed out? If there's a push to try please guide :vporof through that.

Comment 11

[Victor Porof [:vporof][:vp]]

Here's try run: https://treeherder.mozilla.org/jobs?repo=try&revision=4c53597c41c86e4de8f0a16f23998375fb2a7cf8

Comment 12

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Seems like the push is failing talos slightly different from the previous \xulstore\data.mdb
https://treeherder.mozilla.org/jobs?repo=try&revision=4c53597c41c86e4de8f0a16f23998375fb2a7cf8&searchStr=talos

Comment 13

[Victor Porof [:vporof][:vp]]

Right, bizarre that didn't show up in my try run.

So what's happening is that we've transitioned to using a different storage backend, and therefore a one-time migration needs to happen before the data can be accessed. In other words, we went from '\xulstore\data.mdb' to the newly added '\xulstore\data.safe.bin'.

We plan to remove the old storage backend fairly soon in the future. But even if we don't, to be clear, the old data is accessed only one for the one-time migration. And it won't happen if there's no 'data.mdb' available (which is the case for users on release – this has only been running in nightly and beta).

I would expect it's reasonable to keep the old '\xulstore\data.mdb' in 'xtalos/xperf_whitelist.json' in the meantime?

Comment 14

[Alexandru Ionescu (needinfo me) [:alexandrui]]

I added them because your patches were backed out because of these intermittents so I wanted to make sure the reason why they were backed out is gone.
If you think that is reasonable and will get rid of the intermittents, go ahead.

Comment 15

[Victor Porof [:vporof][:vp]]

Alright, this is ready for a re-review.
Try: https://treeherder.mozilla.org/jobs?repo=try&revision=26ddf28433b4d7f13267e16fc78ddc664b7a9d84

Comment 16

[Victor Porof [:vporof][:vp]]

Another try, still green: https://treeherder.mozilla.org/jobs?repo=try&revision=acf49fa546a9e43cfdb2fb4fe69180214fa307f1

Comment 17

[Dorel Luca [:dluca]]

I can see in the comments your question has been answered, removed NI as such.

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/26abc175808e
Part 1: Update RKV dependency to safe-mode, r=nanj
https://hg.mozilla.org/integration/autoland/rev/a92f0deb07b2
Part 2: Update xulstore to use RKV in safe mode, r=nanj,perftest-reviewers,AlexandruIonescu
https://hg.mozilla.org/integration/autoland/rev/b09ca0548784
Part 3: Use manager instead of creating unique instances every time, r=nanj

Comment 19

[Alexandru Michis [:malexandru]]

Backed out 5 changesets (Bug 1654192, Bug 1680230) for causing tsan bc failures in RWLock.

Backout link: https://hg.mozilla.org/integration/autoland/rev/821f3a55cb71c53d361d962c5060c972ef34e52b

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&fromchange=e090b0e10f5de9c444d145caeff7b166fbc17736&searchStr=linux%2C18.04%2Cx64%2Ctsan%2Copt&tochange=821f3a55cb71c53d361d962c5060c972ef34e52b&selectedTaskRun=FaBgyqyeS3iSb_SuxyUJ6Q.0

Failure log: https://treeherder.mozilla.org/logviewer?job_id=323742182&repo=autoland&lineNumber=2293

Comment 20

[Victor Porof [:vporof][:vp]]

Thanks! The error is isolated to bug 1680230, not this one, but the two should land simultaneously. On it!

Comment 21

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/54d791bb417b
Part 1: Update RKV dependency to safe-mode, r=nanj
https://hg.mozilla.org/integration/autoland/rev/16f75cdb5ea0
Part 2: Update xulstore to use RKV in safe mode, r=nanj,perftest-reviewers,AlexandruIonescu
https://hg.mozilla.org/integration/autoland/rev/15a3d7e84a53
Part 3: Use manager instead of creating unique instances every time, r=nanj

Comment 22

[Dorel Luca [:dluca]]

https://hg.mozilla.org/mozilla-central/rev/54d791bb417b
https://hg.mozilla.org/mozilla-central/rev/16f75cdb5ea0
https://hg.mozilla.org/mozilla-central/rev/15a3d7e84a53

Comment 23

[Alexandru Ionescu (needinfo me) [:alexandrui]]

== Change summary for alert #28050 (as of Fri, 11 Dec 2020 05:44:11 GMT) ==

Improvements:

Ratio	Suite	Test	Platform	Options	Absolute values (old vs new)
2%	startup_about_home_paint_realworld_webextensions	startup_about_home_paint_realworld_webextensions	linux64-shippable-qr	e10s stylo webrender	843.92 -> 823.50
2%	startup_about_home_paint_realworld_webextensions	startup_about_home_paint_realworld_webextensions	linux64-shippable-qr	e10s stylo webrender	843.25 -> 826.08

For up to date results, see: https://treeherder.mozilla.org/perfherder/alerts?id=28050

Comment 24

[Victor Porof [:vporof][:vp]]

Hey, 2% is 2%. Add that to the 4-7% from bug 1594995, at least it's something.