1512385 - [Content blocking] Permissions doorhanger cannot be triggered

Status: RESOLVED INVALID

(Firefox :: Protections UI, defect)

Description

[David Olah]

[Affected versions]
Firefox 65.0a1 (2018-12-05)

[Affected platforms]
Ubuntu, Windows, Mac OS

[Description]

Trying:

- default settings in a new profile
- to set the dom.storage_access.auto_grants pref to false which should trigger the doorhanger prompt after that
- dom.storage_access.prompt.testing set to false

Actual result: The Permissions doorhanger cannot be triggered in no test circumstance.

[Note]
This bug is a blocker for QE.

Comment 1

[(no longer active)]

How are you testing exactly?  The doorhanger certainly appears, so if you don't see it, that means you're doing something wrong about how you are performing the test.

I have posted detailed instructions for testing in https://docs.google.com/document/d/11x1B85DH3Advtv3M00yjnzw6o4-TTi3PCagPf-6faPQ/edit.  Since comment 0 provides almost no details about what you've been testing, here are some generic comments to repeat what I've posted there again:

  * Go to https://senglehardt.com/test/storage_access/.
  * Follow the instructions on the page.  That means:
    * Make sure dom.storage_access.enabled is true (default in Nightly)
    * Make sure urlclassifier.trackingAnnotationTable.testEntries is set to storage-access.englehardt-tracker.com.
    * Make sure network.cookie.cookieBehavior is 4 (default in Nightly)
    * Make sure you have interacted with the tracker on the page (open https://storage-access.englehardt-tracker.com/index.html and click around on the page)
  * Make sure dom.storage_access.auto_grants is false (you've done this already)
  * Make sure dom.storage_access.prompt.testing either doesn't exist or is false (it is important for it to not be true. the pref by default doesn't exist and on a new profile you can just leave it as non-existing.)

Then, click on "Call document.requestStorageAccess()" and you will see a doorhanger.

I'm closing this bug since the bug doesn't really exist and it is very confusing when you file a bug like this and mark it as "major".  The doorhanger has many automated tests so there is zero chance for the doorhanger to just be completely broken.  ;-)

Comment 2

[David Olah]

With the steps from Comment 1, we managed to trigger the doorhanger. The steps from the kickoff meeting were not complete, that's the reason we did not manage before but our confusion still remains:
- it is important for us to know when this scenario will be encountered on a real case (the doorhanger is triggered by a website not a test page without doing manual configurations)
- can you exemplify how do we trigger the doorhanger in a real case scenario?
- please give us a functional workflow for the doorhanger?

From our perspective, this bug is still valid because if the user has no way to trigger it in a real case scenario, QE is retesting only the developer team's test scenario, which is not valid from testing point of view. In case that additional code should be landed to make this feature usable, please let us know.

Comment 3

[Johann Hofmann [:johannh]]

Isn't the above a functional workflow?

There is basically no "real case scenario", since no or few real word sites actually use the API we just implemented (yet). Even if they would use it, they first have to "beat" our internal heuristics (this doorhanger is intentionally hard to trigger because generally speaking users hate permission prompts).

As such, I'm not aware of any real word website you can use to test. I don't see why this would be a requirement for a completely new API that it not enabled in release and supported by only one other major browser (Safari).

Comment 4

[(no longer active)]

To add to what Johann said, there are two things at play here:

  * We have a doorhanger which we show sometimes, when Firefox decides that it wants to ask the user if it's OK to allow a tracker to access the user's browsing history.

  * We have a set of automated heuristics that allow Firefox to decide when to show this doorhanger prompt.  These heuristics are designed so that in most cases, users don't see the doorhanger prompt.

It is the first part that we're asking QE to help test, not the second part, as the second part has no human-visible component to interact with (besides the prefs that I've documented in the kick-off doc, which are all hidden prefs and are not intended for users to configure.)

The existence of these heuristics mean that if QE doesn't take any actions besides downloading Nightly and running our test page, the automatic heuristics will kick in and will correctly take over (and as a result prevent the doorhanger from showing up.)  This is fine for users, because users generally don't want to see permission prompts, and of course users are not trying to test this doorhanger prompt.  But this does not work for QE, since QE _is_ interested only in the doorhanger and not the heuristic.

So what has happened here is that you've seen the heuristics working and I've now described what's going on, and now you seem to be pushing back on my explanations on what is the right way to test this doorhanger prompt.  I suggest instead let's try to focus on what it is that we are trying to test here, and try to note that the feature under testing is a doorhanger which we are trying to *not show users as much as possible*, so asking for "how do we trigger the doorhanger in a real case scenario" isn't really meaningful, the answer would be "you can't, by design."

As Johann mentioned, the functional workflow is already presented in great detail.  If something somewhere is unclear I do appreciate more specific questions so that I can answer them.

As far as the doorhanger being triggered by a website not a test page, now that Johann and I have explained in detail why the manual configuration would be necessary even if there were real sites using this feature already (which we don't know of BTW) what are you hoping to get out of a real site that the test page doesn't give you?  You're more than welcome to read the documentation of the API <https://developer.mozilla.org/en-US/docs/Web/API/Storage_Access_API/Using> and create your own test pages, we were trying to save some work on your side by preparing one for you.

Comment 5

[David Olah]

I think that the underlying confusion on this issue is that there were gaps in the QE understanding the permission door-hanger and its current scope and our inability to express that concern regarding it: comment 2 is part of a functional workflow, but it lacks the details of its integration with the rest of the elements of the feature at hand which are still changing. As we came to understand, the door hanger it is not and will be not available yet for the large public, since the API implementation in Firefox is a new one. (although one could pose the question: since Safari already uses it, there should be sites in the wild which should already call it?!) - maybe at a later time, the expected business logic that involves the door hanger could be tested and validated.

Given the above and the fact that we couldn’t initially trigger the door-hanger with only the steps from the kick-off meeting minutes, I hope this clarifies the reasons for which we felt that this bug should be a qablocker and/or major severity at the time of its logging. 

At the present time, what is left to discuss is only the concern that at the current time, the QE could only test the basic functionality of the door hanger, but not generate and validate scenarios based on how/when the users will expect to see it and interact with it.

Comment 6

[(no longer active)]

(In reply to David Olah from comment #5)
> I think that the underlying confusion on this issue is that there were gaps
> in the QE understanding the permission door-hanger and its current scope and
> our inability to express that concern regarding it: comment 2 is part of a
> functional workflow, but it lacks the details of its integration with the
> rest of the elements of the feature at hand which are still changing. As we
> came to understand, the door hanger it is not and will be not available yet
> for the large public, since the API implementation in Firefox is a new one.
> (although one could pose the question: since Safari already uses it, there
> should be sites in the wild which should already call it?!)

Maybe, but please note that sites using these features may very likely be UA-detecting and only call these APIs on particular browsers.  I have yet to see sites call this API in Firefox, but it doesn't mean they haven't started to yet.  I obviously haven't personally browsed all of the web.  :-)

> - maybe at a
> later time, the expected business logic that involves the door hanger could
> be tested and validated.
> 
> Given the above and the fact that we couldn’t initially trigger the
> door-hanger with only the steps from the kick-off meeting minutes, I hope
> this clarifies the reasons for which we felt that this bug should be a
> qablocker and/or major severity at the time of its logging. 

Sure, no problem.

> At the present time, what is left to discuss is only the concern that at the
> current time, the QE could only test the basic functionality of the door
> hanger, but not generate and validate scenarios based on how/when the users
> will expect to see it and interact with it.

Yes.  What I've tried to say previously was that this level of testing is what we're requesting for from QE, since the other parts of the system that are at play here (e.g. the part which automatically determines whether permissions should be granted to a user) is tested using an automated test and isn't something we need to have manual testing on.  Sorry if I haven't been more clear on this point.

Thanks!