1680076 - Proton evaluation survey

Status: RESOLVED FIXED

(User Research :: Project Request, task, P1)

Description

[Rosanne Scholl]

Request Description
Do the changes in the Proton program increase the number of users who enjoy using Firefox? Conduct an in-product, at-scale survey before and after the changes to document user response in the areas of visual resigned and cohesion.

Deadline
Ongoing work in 2020Q4 and 2021H1

Priority Level
1

Priority Level Description
This survey is one of the main success measurements for the Proton program.

Supporting Information
https://mana.mozilla.org/wiki/display/FDPDT/Proton

Comment 1

[Rosanne Scholl]

Research plan: https://docs.google.com/document/d/13g_OV6-pPZuTKXI7cMwWcbYKhbQrUf4EQ-jz0vj2Aq4/edit?ts=5fa43063#heading=h.5m9wi5l8l1py

Comment 2

[Stephan Sokolow]

I don't have permission to access the research plan but please do take userChrome.css into consideration.

With the current state of things, my userChrome.css tweaks take my perception of Firefox's UX from "so much worse than Chrome that I'd consider switching to something like Ungoogled Chromium" to "a bit better than Chrome... thankfully, given Chrome's crippled extension APIs".

If I were presented with an A/B test, the first thing I'd do is head somewhere like /r/firefox to look for userChrome hacks for whatever I didn't like.

Comment 3

[Rosanne Scholl]

A baseline survey for user evaluations of FX85's visual design was launched today to new and existing users.

Comment 4

[Rosanne Scholl]

(In reply to Stephan Sokolow from comment #2)

I don't have permission to access the research plan but please do take userChrome.css into consideration.

With the current state of things, my userChrome.css tweaks take my perception of Firefox's UX from "so much worse than Chrome that I'd consider switching to something like Ungoogled Chromium" to "a bit better than Chrome... thankfully, given Chrome's crippled extension APIs".

If I were presented with an A/B test, the first thing I'd do is head somewhere like /r/firefox to look for userChrome hacks for whatever I didn't like.

Thank you for the feedback, Stephan Sokolow. We're glad your tweaks are working for you.

Comment 5

[Stephan Sokolow]

(In reply to Rosanne Scholl from comment #4)

Thank you for the feedback, Stephan Sokolow. We're glad your tweaks are working for you.

It's not that I want to say "Hey, I'm using userChrome.css!"

Rather, that I was trying to say "It'll be very important to make sure that the presence (and, ideally, nature) of a userChrome.css is taken into consideration when gathering data." and giving an example of how big a difference it can make in the user's perception of the browser.

Comment 6

[Rosanne Scholl]

An additional baseline survey for user evaluations of FX88's visual design was launched on April 23rd to new and existing users.

Comment 7

[icey]

I'm not sure if here's right place for giving feedback, so sorry for bothering.

My firefox developer edition updated to 89 several days ago, and my UI changed in an "unexpected" way. I was not informed of the change (partially because of my poor information sources), and I cannot revert that in my settings menu. I even didn't know the name of this new UI, "proton".

Fortunately I found a way to turn it of in the about:config page. To be honest, I like the previous condensed tabs and sharp edges. So personally I suggest that a switch might be added and let people choose "going proton or not", since this can be done.

To go further, it may be somewhat "impolite" if an important choice is made without informing and fallback options. Maybe I can control the update of software on windows, but the software packages follows upstream on some rolling linux distributions.

And as you can see, my English is poor and my modal may be harsh or offensive. Sorry for that again.

Comment 8

[Qianyue [:qian]]

I don't have access to the links above. What are the results of the survey? Instead of complaining about the proton implementation and getting "we're confident in this decision" (bug #1709425), I'd like to ask why the decision was made in the first place? If the survey supports the new design, why do so many people want to disable it? ([1], [2], [3], [4], [5])

Besides the open source code, shouldn't Firefox also make open the design documents and survey results?

Comment 9

[Qianyue [:qian]]

(In reply to Qianyue [:qian] from comment #8)

Besides the open source code, shouldn't Firefox also make open the design documents and survey results?

To quote The Mozilla Manifesto

Principle 8
Transparent community-based processes promote participation, accountability and trust.

Comment 10

[Qianyue [:qian]]

Advocacy? I might have been emotional and biased but the ask was simple:

1. Is it possible to make open the documents above?
2. Would you care to share the results of the survey?