1148460 - Create First Run builds for UserTesting baseline research

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: General, defect)

Description

[Anthony Lam (:antlam) Find me, @antlam on Slack!]

As per our conversations, the first step is to measure/test what we currently have (the Welcome Screen) so that we can have a baseline for our findings.

This test will not include the sync content that we currently show. Here's a breakdown of the 3 builds. Designs to follow. Let me know if this is still correct:

1) Control slide - Give arbitrary tasks and then ask if they can add to reading list. Get overall sentiments.

    http://cl.ly/image/2c1v3e18151y

2) Mission slide - Give arbitrary tasks and then ask if they can add to reading list. Get overall sentiments.

    http://cl.ly/image/2W0P3q2V0x2G

3) Feature slide ("this is the Reading List feature") - Give arbitrary tasks and then ask if they can add to reading list. Get overall sentiments.

    http://cl.ly/image/33261j1q0j06

Comment 1

[Anthony Lam (:antlam) Find me, @antlam on Slack!]

Attachment: prev_UT_mocks.png

WIP...

A few notes:
 - Copy is a WIP
 - Not sure about the "Tell your friends" call-to-action but I didn't know what else to put there right now
 - Should we have different colored backgrounds to the illustrations so we can keep track of the feedback or is that unnecessary? 
 - Thoughts, comments, suggestions welcome

Chenxia, how should i give you the illustration in the slide? Just as a .PNG? We can keep it simple for this, no animations necessary.

Comment 2

[Chenxia Liu [:liuche]]

PNG is fine, I can ask you for margins and sizing if you have any major changes from the existing first run.

Comment 3

[Anthony Lam (:antlam) Find me, @antlam on Slack!]

Attachment: mob_FR_UT1.png

Does this work? The panel label background above it will be #4695BD

Comment 4

[Anthony Lam (:antlam) Find me, @antlam on Slack!]

Attachment: prev_UT_custom_mock1.png

Mock of the mentioned "customize" slide.

CTA would be where the check is right now, this is the feedback/confirmation screen after the user has picked a default from the dialog. 

Copy for button " Choose my default" ?

Comment 5

[Anthony Lam (:antlam) Find me, @antlam on Slack!]

Attachment: check_addtoRL.zip

Assets.

Chenxia, keep in mind, I use the same ones for feedback/confirmation in the latest share overlay work with mcomella. :)

Comment 6

[Chenxia Liu [:liuche]]

I'll be putting the builds up at http://people.mozilla.org/~liuche/bug-1148460/ - I have the two controls up already (which took more work than expected, because of some Android annoyances; Galaxy Nexus couldn't handle the asset so I did some aliasing and resource duplication).

Comment 7

[Chenxia Liu [:liuche]]

Attachment: Screenshot: Reading list panel

I set the icon width/height to 60dp - let me know if you have a different preference for that.

It's annoying that on certain devices, the screen will be made scrollable - this is from the fix we did for bug 1128431. mcomella also gave me some ideas about how to try to get a real fix in for this (not a bandaid), so I'll investigate.

Comment 8

[Chenxia Liu [:liuche]]

Attachment: Screenshat: Panel picker dialog

This is what the dialog looks like - I can change the title, or remove it completely.

Comment 9

[Chenxia Liu [:liuche]]

Attachment: Screenshot: First run after selecting a default

The fix for bug 1128431 made the "Start browsing" link fixed height, so in this situation, the bounds on it are really big. All of that is clickable, while the layout bounds for the green checkbox are really tiny - it's very easy to dismiss the first run panel and "start browsing" by accident if you try to tap on the green arrow (where the button used to be).

I want to investigate if there's a *real* fix for bug 1128431, not just a bandaid. (The bandaid for this also causes the layout to add a scrollbar, even on xxhdpi devices, but only on these user builds. But hm, maybe there's another fix actually now that I think about it...)

Anyways, feedback request for you, antlam:
- Do you have a size request for the green checkbox, in dp?
- What do you think about the strings? I experimented with adding ! which I liked but also seemed a bit childish.
- Any other feedback?

Comment 10

[Chenxia Liu [:liuche]]

All 4 builds are up at http://people.mozilla.org/~liuche/bug-1148460/ .

1-control.apk: Control apk, where no first run experience is shown at all
2-sync.apk: "Set up Sync" apk, which is what is currently riding with the trains with 38
3-readinglist.apk: Shows the reading list icon with a tip on how to add things to the reading list.
4-customize.apk: Allows user too select their default home panel. (We don't give any suggestions on how to change the home panel.)

These should obviously be renamed before asking people to download and install them - I'd also like antlam to try them out and give some feedback. I've tested these on a 2.3 SII (which used to have the truncating problem), galaxy nexus, and nexus 7.

P.S. I fixed the giant start browsing link problem :3

Comment 11

[Chenxia Liu [:liuche]]

Attachment: Screenshot: Customize

Comment 12

[Anthony Lam (:antlam) Find me, @antlam on Slack!]

Comment on attachment 8593147 [details]
Screenshot: Reading list panel

+ for 60dp, that's the spec! well done.

The spacing on this is awkward but I'm not too worried about that for UserTesting purposes.

As a side, simply vertically centering the content in the white would be a good start (tablets only).

Comment 13

[Anthony Lam (:antlam) Find me, @antlam on Slack!]

Comment on attachment 8593685 [details]
Screenshat: Panel picker dialog

Works! thanks Chenxia.

Comment 14

[Anthony Lam (:antlam) Find me, @antlam on Slack!]

Comment on attachment 8593690 [details]
Screenshot: First run after selecting a default

(In reply to Chenxia Liu [:liuche] from comment #9)
> Anyways, feedback request for you, antlam:
> - Do you have a size request for the green checkbox, in dp?

The assets I gave you should have the check mark at 65dp x 50dp :)

> - What do you think about the strings? I experimented with adding ! which I
> liked but also seemed a bit childish.

Hm, maybe! I think we tried that before. Although I imagine we'd want Matej to look over the copy at the end of all this testing and before we ship the next iteration of this.

> - Any other feedback?

I could spec this out better for tablets. The spacing is a bit larger and more awkward but we can do that in follow up bugs. For UserTesting purposes, I'd say these are great!

Comment 15

[Anthony Lam (:antlam) Find me, @antlam on Slack!]

(In reply to Chenxia Liu [:liuche] from comment #10)
> All 4 builds are up at http://people.mozilla.org/~liuche/bug-1148460/ .
> 
> 1-control.apk: Control apk, where no first run experience is shown at all
> 2-sync.apk: "Set up Sync" apk, which is what is currently riding with the
> trains with 38
> 3-readinglist.apk: Shows the reading list icon with a tip on how to add
> things to the reading list.
> 4-customize.apk: Allows user too select their default home panel. (We don't
> give any suggestions on how to change the home panel.)
> 
> These should obviously be renamed before asking people to download and
> install them - I'd also like antlam to try them out and give some feedback.
> I've tested these on a 2.3 SII (which used to have the truncating problem),
> galaxy nexus, and nexus 7.
> 
> P.S. I fixed the giant start browsing link problem :3

Awesome! will have a look at these at the office tomorrow. In the meantime, Gemma, here are the builds!

Comment 16

[Gemma Petrie (she/her) [:gemma]]

Thanks Chenxia & Anthony!

Here is my draft of tasks/questions for usertesting.com. Please let me know if you have any questions, comments, additions, etc. I'll plan to get 2 pilot tests running tomorrow (Tuesday) morning. 

--Introduction--
Thanks for participating today. We'd like you to imagine that a friend has convinced you to try out a new web browser. We will ask you to complete a short series of tasks in which you will be asked to install and use this web browser. Please talk through your thinking process  aloud while you complete these tasks. This is not a test of you, so there are no right or wrong answers. If you do not have an immediate answer, it's okay to say that as your answer.  Please try to let the recording run even if you are just waiting for something to load. We also want to see how the browser behaves on your device. 

--Tasks--
1) Please follow this link to download a special version of the Firefox browser: URL. The download will begin automatically. (You are welcome  to uninstall Firefox when this test is over.)

2) When the download is complete, please open the browser, but don't click on anything else yet. 

3) For just moment, please stop touching your device. Please describe what you would normally do next after opening this browser for the first time.

4) Without touching your device. Please describe what you see on your screen.

5) Without touching your device, please answer the following questions. Do you normally take app tours? Why or why not? Can you remember the  last app tour you took that you liked? Why did you like it?

6) Regardless of your previous response, please go through the screens presented to you. Please talk through what you are seeing. Stop when you get to the end.

7) For just a moment, please stop touching your device. What did you think of this tour? Was anything surprising? Interesting? Confusing? Boring? 

8) Next, please use your browser to visit the BBC website: http://www.bbc.com/. Find a story that interests you and click on it. 

9) Please imagine you want to read this article later. How would you save it? 

10) Please open a new tab and search for "New York Times Japan earthquake." Find a news story on this topic and click on it. 

11) Please imagine you want to read this article later. Please use a feature *within the Firefox browser* to save it for later. 

12) Please quit Firefox and then re-open it as if it's a new day. Now bring up the earthquake article you saved using the browser feature from the previous step.

13) After using Firefox for a few minutes, what seems different from your normal browser? What seems similar?

14) Have you seen anything in Firefox that would make you switch browsers? Please explain. 

15) What Android browser do you use most of the time? What is your favorite thing about that browser? 

--Text Questions--
What words would you use to describe this Firefox browser?

How is the Firefox browser experience different from or similar to other Android web browsers you have used?

What is the make and model of your Android device? What version of Android are you running?

Comment 17

[Chenxia Liu [:liuche]]

Thanks, this looks great. I'm excited to see what you learn!

Smattering of comments inline.

> 3) For just moment, please stop touching your device. Please describe what
> you would normally do next after opening this browser for the first time.

"For just a moment"

> 5) Without touching your device, please answer the following questions. Do
> you normally take app tours? Why or why not? Can you remember the  last app
> tour you took that you liked? Why did you like it?
> 
I made one apk that doesn't have a welcome screen at all (1-control.apk), so just a note here that we could remove #5-7 for that build.

> 8) Next, please use your browser to visit the BBC website:
> http://www.bbc.com/. Find a story that interests you and click on it. 
> 
> 9) Please imagine you want to read this article later. How would you save
> it? 

Good call to action - I am curious if this will be specific enough because "bookmarking" is an equivalently good answer, but let's wait and see.

> 12) Please quit Firefox and then re-open it as if it's a new day. Now bring
> up the earthquake article you saved using the browser feature from the
> previous step.

We might want to include more specific instructions for "quitting Firefox". We no longer have a "quit" button, and the user may need to "swipe-close" from recent apps to actually quit Firefox, or if they're on Gingerbread, actually go to Settings to "Force close" Firefox. Optionally, we could have them close that tab and then close Firefox (or hide it, really).

> What is the make and model of your Android device? What version of Android
> are you running?

We could optionally include instructions on how to find the version - I'm not really sure what the tech skill level of the testers are, so I'll defer to you! It certainly would be helpful to know the Android version, esp if we run into problems if the build doesn't work for some reason.

(Instructions: Go to the Settings app on your Android device, and click on the "About phone" item. The version should be listed under "Android version".)

Comment 18

[Anthony Lam (:antlam) Find me, @antlam on Slack!]

This looks good to me - thanks Gemma!

Comment 19

[Chenxia Liu [:liuche]]

Closing this since the builds are done.

Comment 20

[Chenxia Liu [:liuche]]

Attachment: MozReview Request: bz://1148460/liuche

/r/7659 - Unify layouts
/r/7661 - fold
/r/7663 - Bug 1148460 - Control
/r/7665 - Reading list.

Pull down these commits:

hg pull -r 67f133bbb95947e645b69a924b18076cd91bc1c3 https://reviewboard-hg.mozilla.org/gecko/

Comment 21

[Chenxia Liu [:liuche]]

Comment on attachment 8597524 [details]
MozReview Request: bz://1148460/liuche

/r/7659 - Unify layouts
/r/7661 - fold
/r/7663 - Bug 1148460 - Control
/r/7665 - Customize

Pull down these commits:

hg pull -r 666a42aa439a9296da06f66b7da3ed78ba74f16f https://reviewboard-hg.mozilla.org/gecko/

Comment 22

[Chenxia Liu [:liuche]]

Realized I didn't have this code anywhere, so putting the reviewboard links here.

I branched at Control to make Reading List and Customize (both of them share a parent, Control).

Comment 24

[Chenxia Liu [:liuche]]

Attachment: MozReview Request: Customize

Comment 25

[Chenxia Liu [:liuche]]

Attachment: MozReview Request: Unify layouts

Comment 26

[Chenxia Liu [:liuche]]

Attachment: MozReview Request: fold

Comment 27

[Chenxia Liu [:liuche]]

Attachment: MozReview Request: Bug 1148460 - Control