1614831 - Stutters/freezing when opening new tabs

Status: RESOLVED FIXED

(Core :: Permission Manager, defect, P1)

Description

[Asif Youssuff]

Reported on https://www.reddit.com/r/firefox/comments/f2eoxy/constant_microfreezing/

Basic information

Steps to Reproduce:

1. Browse reddit
2. open a new tab

Expected Results:

Fast opening of a new tab.

Actual Results:

freezes the browser for a second. Like the whole tab or even the whole browser freezes and you just gotta wait.

---

More information

User reports that disabling accessibility services doesn't make a difference.

Profile URL: https://perfht.ml/2UPXY9A

Basic systems configuration:

OS version: Windows 7

GPU model: NVIDIA GeForce GTS 450

Number of cores:

Amount of memory (RAM):

Thanks so much for your help.

Comment 1

[Asif Youssuff]

Attachment: about:support

Comment 2

[Mike Conley (:mconley) (:⚙️)]

Very interesting. We're stuck in the parent process launching a content process.

Looks like we're spending a ton of time enumerating entries in the permissions database? Hey ehsan, do you have any idea what might be happening here?

Comment 3

[microlap]

Hello, I'm the original poster.

If I need to add anything, please tell me what to do.

Edit:
Number of cores: 4 (Intel Quad 6600)
Amount of memory (RAM): 8 GB

Comment 4

[(no longer active)]

This is such a sad profile to look at. My guess as to the cause for the slowness is that the original poster, microlap, has a profile with a permissions database which contains a lot of permission entries in it. (It would be nice if we could verify that at some point.) Now let's see what this code is doing:

• In the parent process, we notice we're about to load a document, so we decide to send the corresponding content process all of the permissions that it needs. So far, so good.
• In order to do that, we start by asking for all of the "permission keys" for the principal. A permission key is computed from the origin by doing a lot of work. It's really not important what the details are, all you have to keep in mind is that this work is a bit expensive, though the cost can be expensive when doing it for one origin. To begin, we create one of these keys for each subdomain that we can create from the principal (e.g. if we start from https://www.foo.com we also create a key for https://foo.com).
• Now that we have all of these permission keys ready, we start to send the corresponding permissions down to the content process for each key one by one. The bulk of this work is done here and that's where our expensive loop is.
• In this loop, we iterate over every single permission in the database, and for each one we calculate a permission key. We use this permission key to decide whether the permission at hand should be ignored or selected to be sent down to the content process.
• As I mentioned before, creating the permission key is a bit expensive. Let's imagine the permission manager database having N permissions stored in it, and the user opening a document from a principal which we can create M subdomains from. This algorithm would cause us to compute M*(N+1) permission keys if I'm counting things right, which means it can be potentially very expensive. :-(
• To make this story sad, instead of calculating these permission keys, we could just pass down the origin directly, and do a comparison based on the origin of the permission entry.

Comment 5

[Johann Hofmann [:johannh]]

Oh man, I agree, that is just sad. This feels like a P1, we should find someone to own it.

In addition to Ehsan's idea, we should consider purging permissions that are older than a certain date (maybe excluding certain types like cookie). I know that legacy add-ons could sometimes create tens of thousands of permissions and there are so many places where we need to query those.

Comment 6

[(no longer active)]

(In reply to Johann Hofmann [:johannh] from comment #5)

In addition to Ehsan's idea, we should consider purging permissions that are older than a certain date (maybe excluding certain types like cookie). I know that legacy add-ons could sometimes create tens of thousands of permissions and there are so many places where we need to query those.

Indeed. I think in order to be able to do that effectively we should consider adding a accessTime field to the permission manager's database, and keep track of when each permission is used, and after a while purge those which e.g. have not been used in 6 months or something like that.

Comment 7

[(no longer active)]

I created an example permission manager database with 100,000 permission entries, and I can reproduce this bug with that. I have a fix based on that.

Here is a profile before: https://perfht.ml/39OxJEX. Here is a profile after: https://perfht.ml/3bV7deT.

Comment 8

[(no longer active)]

Attachment: Bug 1614831 - Speed up the transmission of permissions to content processes upon navigation;

Comment 9

[microlap]

(In reply to :ehsan akhgari from comment #4)

This is such a sad profile to look at. My guess as to the cause for the slowness is that the original poster, microlap, has a profile with a permissions database which contains a lot of permission entries in it. (It would be nice if we could verify that at some point.)

how can I do that? Or, disable/edit these permissions?

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/05fcd5b647bd
Speed up the transmission of permissions to content processes upon navigation; r=baku

Comment 11

[(no longer active)]

(In reply to microlap from comment #9)

(In reply to :ehsan akhgari from comment #4)

This is such a sad profile to look at. My guess as to the cause for the slowness is that the original poster, microlap, has a profile with a permissions database which contains a lot of permission entries in it. (It would be nice if we could verify that at some point.)

how can I do that? Or, disable/edit these permissions?

Would you be comfortable with sharing your permission manager database with me privately? With the permission manager database file I can use the exact same database as yours to ensure that this fix will indeed improve the performance in the tab opening scenario described in comment 0 sufficiently to solve the reported performance problem. Please note that this database file will contain the addresses of websites which you have saved permissions on (e.g. where you've responded to browser prompts requesting access to things like your location, notifications, camera and microphone, etc.), so please do NOT post it to the public bug.

If you are comfortable with sharing this file, please open about:support in Firefox, find "Profile Folder", click on the "Open Folder" button next to it, then close Firefox, and then find the file named permissions.sqlite in that folder, and email it to me (my first name at mozilla.com). You can compress this file if it is too large. Thanks!

Comment 12

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/05fcd5b647bd

Comment 13

[:Gijs (he/him)]

Do we have any kind of visibility into what is a "normal" number of entries in the permissions db, perhaps keyed by permission type, and/or would it make sense to get telemetry on that to guide future performance considerations in this area?

Comment 14

[microlap]

(In reply to :ehsan akhgari from comment #11)

If you are comfortable with sharing this file, please open about:support in Firefox, find "Profile Folder", click on the "Open Folder" button next to it, then close Firefox, and then find the file named permissions.sqlite in that folder, and email it to me (my first name at mozilla.com). You can compress this file if it is too large. Thanks!

I sent you the file.
It was about 30k entries. I deleted most of them (with a backup) and now everything's great. Thank you

Comment 15

[(no longer active)]

(In reply to :Gijs (he/him) from comment #13)

Do we have any kind of visibility into what is a "normal" number of entries in the permissions db, perhaps keyed by permission type,

We don't.

and/or would it make sense to get telemetry on that to guide future performance considerations in this area?

It would probably make sense to gather that data at some point, yes.

Comment 16

[(no longer active)]

(In reply to microlap from comment #14)

(In reply to :ehsan akhgari from comment #11)

If you are comfortable with sharing this file, please open about:support in Firefox, find "Profile Folder", click on the "Open Folder" button next to it, then close Firefox, and then find the file named permissions.sqlite in that folder, and email it to me (my first name at mozilla.com). You can compress this file if it is too large. Thanks!

I sent you the file.

Thanks, I'm trying to find it in my emails... Do you mind letting me know about the subject of the email and/or the account you sent it from?

It was about 30k entries. I deleted most of them (with a backup) and now everything's great. Thank you

That's great to hear!

Comment 17

[microlap]

(In reply to :ehsan akhgari from comment #16)

Thanks, I'm trying to find it in my emails... Do you mind letting me know about the subject of the email and/or the account you sent it from?

the subject is "Bug 1614831 permissions file"
if you can't find it let me know I'll send you a new one

Comment 18

[(no longer active)]

(In reply to microlap from comment #17)

(In reply to :ehsan akhgari from comment #16)

Thanks, I'm trying to find it in my emails... Do you mind letting me know about the subject of the email and/or the account you sent it from?

the subject is "Bug 1614831 permissions file"
if you can't find it let me know I'll send you a new one

It is strange, I can't find it (checked my spam folder too). FWIW my address once again is ehsan at mozilla dotcom :) I'd appreciate if you could resend this please.

Comment 19

[microlap]

(In reply to :ehsan akhgari from comment #18)

It is strange, I can't find it (checked my spam folder too). FWIW my address once again is ehsan at mozilla dotcom :) I'd appreciate if you could resend this please.

I'm sorry, I sent it to *.org and apparently it didn't return back.
Resent, same subject, email ends in 'ukr.net'

Comment 20

[Asif Youssuff]

This should probably be a leave-open until Ehsan can take a look at that db that the reporter sent.

Comment 21

[:Gijs (he/him)]

(In reply to Asif Youssuff from comment #20)

This should probably be a leave-open until Ehsan can take a look at that db that the reporter sent.

We normally keep bugs closed if a patch has landed; doing otherwise interferes with QA verifying the fix that did already happen here, and makes it hard to track which fixes land in which release cycle.

If the db that the reporter sent produces issues that need further code changes, they can happen in a separate bug. Pinging Ehsan for comment #19...

Comment 22

[Bogdan Maris, Desktop Test Engineering]

Is there a way I can get my hands on a permissions.sqlite in order to reproduce and then verify this issue, or a way I can add many entries in one?
If not, can someone who reproduced this in the first place verify that this is fixed using Firefox 75.0b10 (https://archive.mozilla.org/pub/firefox/candidates/75.0b10-candidates/)