819210 - Add UA overrides for top Brazil sites

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia, defect, P1)

Description

[Lawrence Mandel [:lmandel] (use needinfo)]

Attachment: Brazil UA override candidate list

Attached is a candidate list of 43 of the Alexa top 100 Brazil sites for the B2G UA whitelist. This list of 43 sites was determined via screenshot comparison of the sites with the Firefox for Android, Firefox OS, and Android Stock UAs.

Our goal is to create a recommended list of Brazillian sites for the UA whitelist.

There are 3 actions:

1. Smoke test each of these sites, preferably on B2G, with a spoofed UA to ensure that with the UA override the site appearance is improved and the site is mostly functional. Non primary function, like a vertically scrollable list of posters/images or function buried deep within the site, should not block a smoke test from passing.

2. Add the UA overrides to Gaia.

3. Open an evangelism bug for each site that is added to the UA override list in order to track the work required to remove a site from the list.

Legend for the attachment:
(Android UA) = This site requires the Android stock UA to get mobile content
(better content with Android UA) = This site receives mobile content with the Fennec UA but better mobile content with the Android stock UA
(nsfw) = not safe for work (this is an adult site)

Comment 1

[Gervase Markham [:gerv]]

Lawrence: are these sites which are already known to require an override, or is the purpose of this bug to test them to see if they do?

Gerv

Comment 2

[Lawrence Mandel [:lmandel] (use needinfo)]

These sites have been demonstrated to serve different content to the different UAs. The initial purpose of this bug is to test these sites to see if a UA override alone is sufficient to get the mobile version of these sites functional on B2G. Once we have the short list of sites we can add UA overrides and start evangelism for fixes.

Comment 3

[Tony Chung [:tchung]]

Attachment: changeUA.sh script

the script to make this work easier is attached.  

instructions:
$ ./changeUA.sh fennec
$ ./changeUA.sh android
$ ./changeUA.sh

The last command resets the UA to the default. The script is available at https://gist.github.com/4234696 and has one prereq that adb needs to be on your path.

Comment 4

[David Scravaglieri [:scravag]]

I don't think it's a Gaia issue. Please confirm.

Comment 5

[Lawrence Mandel [:lmandel] (use needinfo)]

AFAIK this bug should be in the Gaia component as UA overrides are added to ua-override-prefs.js. Bug 798694 has the details.

Comment 6

[David Scravaglieri [:scravag]]

Ok blocked.

Comment 7

[Tony Chung [:tchung]]

The QA testing work has landed for brazil.   Please proceed with the sites on this list.

https://docs.google.com/spreadsheet/ccc?key=0AhE7m4JB2j6tdGpIQU14NldLbXExQnZiTG1uZGhhY2c#gid=0

Comment 8

[Lawrence Mandel [:lmandel] (use needinfo)]

Attachment: UA overrides for Brazil sites based on QA feedback

Note: Where possible I've used the Fennec UA. Some sites require the Android UA in order to serve mobile content.

I haven't tested this patch (someone needs to show me how to build Gaia) but wanted to post the patch in case someone else can help out.

Comment 9

[Gervase Markham [:gerv]]

Lawrence: I think it's great that you are using the Fennec UA where possible - that's the right move. If we have to use a "stock Android browser" UA, then are you sure you've picked the right one? It looks to me like you've used the UA from a Samsung Galaxy S II. I know that some sites (wrongly, but they do) look up hardware identifiers in the UA in order to deduce capabilities. The Galaxy S II has e.g. a much larger screen than the phones we are shipping. Surely it would make most sense for us to send a UA string corresponding as closely as possible to the stock Android browser on hardware similar to what we are shipping?

There may possibly be a confidentiality leak problem there, but I think we should try and come up with something which better approximates the UA of a device in the class of the phones we are shipping. (Note: I don't actually know if it's been decided who is making the launch phone.)

Gerv

Comment 10

[Lawrence Mandel [:lmandel] (use needinfo)]

Good point about the Android UA. We should use the same UA with which UA has tested the on device override.

Aaron - Can you comment with the Android UA that was used to test the on device overrides?

Comment 11

[Tony Chung [:tchung]]

(In reply to Lawrence Mandel [:lmandel] from comment #10)
> Good point about the Android UA. We should use the same UA with which UA has
> tested the on device override.
> 
> Aaron - Can you comment with the Android UA that was used to test the on
> device overrides?

The tested UA was from the changeUA.sh script.  https://gist.github.com/4331189

Specifically: 

 echo "Changing UA to Android Stock UA"
ua="Mozilla/5.0 (Linux; Android 4.0.4; Galaxy Nexus Build/IMM76B) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.133 Mobile Safari/535.19"

 echo "Changing UA to Fennec UA"
ua="Mozilla/5.0 (Android; Mobile; rv:20.0) Gecko/20.0 Firefox/20.0"

Comment 12

[Gervase Markham [:gerv]]

Whoever wrote that script obviously didn't consider that Android doesn't have one single "stock UA", and didn't even use the stock browser anyway - that's the Chrome UA for a Galaxy Nexus! The UA for the stock browser from my CyanogenMod browser is:

Mozilla/5.0 (Linux; U; Android 2.3.7; en-gb; GT-I9100 Build/GRJ22) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1

We need to figure out how close to a completely stock UA we need to get for a site to work. If the Fennec UA (i.e. adding "Android") doesn't help, then we should next try adding the Webkit bits, to give us:

Mozilla/5.0 (Linux; Android; rv:X.X) Gecko/X.X Firefox X.X AppleWebKit/X.X (KHTML, like Gecko) Version/X.X Mobile Safari/X.X

and only after that, try removing the Firefox-related tokens, and then adding some of the other cruft like "U", languages, and some sort of hardware identifier which closely matches our phone and/or an Android version number. In no case should we need to claim to be Chrome.

The rationale here is that keeping the lying to a minimum might actually lead to better results (particularly as we are setting one UA here for all B2G devices).

Would it be best to develop a small set of UAs to test with, in preference order? Start with the least preferred, and keep changing up the list until things break.

Gerv

Comment 13

[Lawrence Mandel [:lmandel] (use needinfo)]

I wrote the script so the blame lies with me. The fact that Android doesn't have a single UA does make it harder to select one. I grabbed an Android UA - you're right that it's not really a stock UA. The comment about Android stock was to try and keep things simple for our testers who are very new to the world of site compatibility testing.

I can see the argument for following your proposal (above). I can also see the argument for the simplicity of either serving the Fennec UA or a specific Android UA. (We have new QA resources performing the compatibility testing. We should keep things as simple for them as possible.) I don't know that there is going to be a huge benefit to "optimizing" the UA that we send to the smallish list of sites that require an Android UA override other than the Fennec UA. If it's preferable, we can start with the evangelism to change the UA detection of the sites that require an Android UA.

Does that work for you?

Comment 14

[Gervase Markham [:gerv]]

If you want to stick with a single alternative (other than "Fennec UA"), I can see that. But I do think we should use one which is picked from the stock Android browser on a device similar to the device we will ship on, rather than one picked from Chrome on a Nexus.

Gerv

Comment 15

[Lawrence Mandel [:lmandel] (use needinfo)]

If we are to take a UA from an older Android device I think we'll need to retest the (In reply to Tony Chung [:tchung] from comment #7)
> The QA testing work has landed for brazil.   Please proceed with the sites
> on this list.
> 
> https://docs.google.com/spreadsheet/
> ccc?key=0AhE7m4JB2j6tdGpIQU14NldLbXExQnZiTG1uZGhhY2c#gid=0

Can you clarify the results on a couple of the entries:
1. noticias.uol.com.br - comment is No action needed. I get a mobile site in the Android stock browser and a desktop site on B2G.
2. imdb.com - I only see a pass for Android but imdb.com serves a mobile site to Fennec.

Comment 16

[Lawrence Mandel [:lmandel] (use needinfo)]

(In reply to Gervase Markham [:gerv] from comment #14)
> If you want to stick with a single alternative (other than "Fennec UA"), I
> can see that. But I do think we should use one which is picked from the
> stock Android browser on a device similar to the device we will ship on,
> rather than one picked from Chrome on a Nexus.
> 
> Gerv

We can consider a different UA as you propose (do you have a suggestion?) but I don't think it's going to make much practical difference at this point. 

In terms of device capabilities:
1. QA has already confirmed that the sites for which we are proposing overrides are functional with the Nexus UA. 
2. Device capabilities go beyond hardware characteristics to include CSS, DOM, and media capabilities. I don't think we want B2G to be equated to older Android browsers in these respects.

In terms of fingerprintability, do you think that there is a significant difference between the Android Stock and Nexus UAs?

(Note, in any case I do need to change the Android UA that's used in the patch because, as Gerv pointed out, it is currently using the untested Galaxy S2 UA.)

Comment 17

[Aaron Train [:aaronmt]]

(In reply to Lawrence Mandel [:lmandel] from comment #15)
> Can you clarify the results on a couple of the entries:
> 1. noticias.uol.com.br - comment is No action needed. I get a mobile site in
> the Android stock browser and a desktop site on B2G.

Confirming mobile on Fennec/Android, I'd put a recommendation for either.

> 2. imdb.com - I only see a pass for Android but imdb.com serves a mobile
> site to Fennec.

Confirming mobile on Fennec/Android, I'd put a recommendation for either.

Comment 18

[Lawrence Mandel [:lmandel] (use needinfo)]

Thanks Aaron. In order to ensure the quality of the rest of the results (and future runs), can you ask QAnalysts why these two sites were marked as they were?

Comment 19

[Lawrence Mandel [:lmandel] (use needinfo)]

Attachment: Updated patch with Nexus UA and previously excluded site

This is a WIP patch that updates the list to use the tested Nexus UA and adds the previously excluded noticias.uol.com.br.

Comment 20

[Lawrence Mandel [:lmandel] (use needinfo)]

Attachment: Updated patch with Nexus UA and previously excluded site, take 2

Actually attaching a patch this time.

Comment 21

[Dietrich Ayala (:dietrich)]

Is this patch ready for review?

Comment 22

[Lawrence Mandel [:lmandel] (use needinfo)]

Comment on attachment 695030 [details] [diff] [review]
Updated patch with Nexus UA and previously excluded site, take 2

I would prefer that Gerv be OK with using the UA so marking for his feedback. (See comment 16.) Any change resulting from Gerv's feedback should be trivial.

Comment 23

[Jason Smith [:jsmith]]

Comment on attachment 695030 [details] [diff] [review]
Updated patch with Nexus UA and previously excluded site, take 2

Fabrice is out. I'd recommend Vivien as a backup reviewer.

Comment 24

[Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please)]

Comment on attachment 695030 [details] [diff] [review]
Updated patch with Nexus UA and previously excluded site, take 2

The patch looks good to me.

Comment 25

[Gervase Markham [:gerv]]

Comment on attachment 695030 [details] [diff] [review]
Updated patch with Nexus UA and previously excluded site, take 2

Reluctant f+.

I would strongly prefer that bugs were filed on evangelizing these sites and the bug numbers included in the patch. I feel that if that's not done, there's a strong risk that these issues will not be followed up.

I still think it's unwise to use a Chrome UA rather than a stock Android one, and I think we should be trying to minimise the differences between our stock UA and the one we send. I understand the argument that QA has tested with this UA; I think we should be advising them to use a different one, or different set, in the future.

<general deep sigh>

Gerv

Comment 27

[Jason Smith [:jsmith]]

Someone needs to build a gaia pull request and land this.

Comment 28

[Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please)]

https://github.com/mozilla-b2g/gaia/commit/65ac627ce686be0348d53419fc702e38e5d75cbe

Comment 29

[Lawrence Mandel [:lmandel] (use needinfo)]

Vivien - Thanks for landing.

I have been working on an update for the patch that includes the relevant bug numbers (as Gerv suggested). I'll file a follow-up bug for the patch with the bug numbers.

Comment 30

[Lawrence Mandel [:lmandel] (use needinfo)]

Removing the confidential flag now that Brazil has been publicly announced as a launch market.

https://blog.mozilla.org/press/2013/02/firefox-os-expansion/