524339 - Expose device type in user agent

Status: RESOLVED WONTFIX

(Core :: Networking, defect)

Description

[Doug Turner (:dougt)]

I have been working on the improving flash performance in Fennec.  When we spoof Fennec's user agent to match Maemo Browser's user agent, we see a very noticeable performance improvement.  We believe that the Maemo Browser is served video using the optimized Sorenson [1] codec and Fennec is not.

We have asked YouTube to treat Fennec in the same way.  However, our current UA string:

"Mozilla/5.0 (X11; U; Linux armv7l; en-US; rv:1.9.2a1pre) Gecko/20090322 Fennec/1.0b4pre"

Does not give YouTube any hint as to when they can serve higher quality video.  One easy thing we can do is to start putting the device model in the user agent.  This doesn't provide all of the nice things that something that UAProf gives you, but also doesn't come with the baggage of RDF, figuring out which URL we should put in our headers, or did I already say RDF?

[1] http://en.wikipedia.org/wiki/Sorenson_codec


This patch will expose device information only on the nokia devices (the n8xx sercies and the n900).  Additional devices can be added simply by putting the right logic in nsSystemInfo.cpp.

I also converted some similar logic in the gfx platform code.

With this patch, our fennec user agent becomes:



"Mozilla/5.0 (X11; U; Linux armv7l; Nokia N900; en-US; rv:1.9.2a1pre) Gecko/20090322 Fennec/1.0b4pre"

Comment 1

[Doug Turner (:dougt)]

Attachment: patch v.1

Comment 2

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Comment on attachment 408250 [details] [diff] [review]
patch v.1

Looks fine, but if on HILDON, use the N900 as the default variables for DPI and sMaemoClassic?  I don't think that data will get set now unless there's an explicit match.

Comment 3

[Doug Turner (:dougt)]

@vlad there are matches now, right?  if you add a new nokia device, you are probably going to want to decide on the DPI and on the sMaemoClassic flag.

Comment 4

[Doug Turner (:dougt)]

http://hg.mozilla.org/mozilla-central/rev/6ced393c6387

Comment 5

[Stuart Parmenter]

you should take this to bug 517086, where our UA is actually being set for final release.

Comment 6

[Mark Finkle (:mfinkle) (use needinfo?)]

Bug 517086 is only about adding the "Firefox" part to the UA. This patch tweaks the UA at a different level.

Comment 7

[Mark Finkle (:mfinkle) (use needinfo?)]

Comment on attachment 408250 [details] [diff] [review]
patch v.1

>+    FILE *fp = fopen ("/proc/component_version", "r");
>+    if (fp) {
>+      while ((read = getline(&line, &len, fp)) != -1) {
>+        if (line) {
>+          if (strstr(line, "RX-51")) {
>+            SetPropertyAsACString(NS_ConvertASCIItoUTF16("device"), NS_LITERAL_CSTRING("Nokia N900"));
>+          } else if (strstr(line, "RX-44") ||
>+                     strstr(line, "RX-48") ||
>+                     strstr(line, "RX-32") ) {
>+            SetPropertyAsACString(NS_ConvertASCIItoUTF16("device"), NS_LITERAL_CSTRING("Nokia N8xx"));

You could break out of the while loop if you found a match

>+      if (line)
>+        free(line);
>+    }

Should you close the file?

Comment 8

[Doug Turner (:dougt)]

Attachment: close file.

Comment 9

[Doug Turner (:dougt)]

http://hg.mozilla.org/mozilla-central/rev/c01460b6b02f

We want both on 192

Comment 10

[Mike Beltzner [:beltzner, not reading bugmail]]

Comment on attachment 408250 [details] [diff] [review]
patch v.1

a192=beltzner for this and the close file

Comment 11

[Doug Turner (:dougt)]

http://hg.mozilla.org/releases/mozilla-1.9.2/rev/087573e67f1c
http://hg.mozilla.org/releases/mozilla-1.9.2/rev/b80e835e5c7f

Comment 12

[Peter Van der Beken [:peterv]]

Doesn't this make Fennec's UA not follow the spec we have for UA strings (https://developer.mozilla.org/En/User_Agent_Strings_Reference)? OS-or-CPU would be "Linux armv7l; Nokia N900", I think that might trip parsers into treating "Nokia N900" as the Localization Information.

Comment 13

[Doug Turner (:dougt)]

we probably should update the user agent reference on d.m.o.  I do not think that this is incompatible with any RFC.

Comment 14

[Boris Zbarsky [:bzbarsky]]

It's not a matter of RFC; it's a matter of how we've been telling people to write UA analyzers for years.  This change will break those analyzers.  Updating the reference does nothing without rolling out new analyzer software across the whole web to deal with the new format...

Comment 15

[Doug Turner (:dougt)]

i am open to suggestions on where this sort of information can be placed in the UA.  Maybe at the end?

Comment 16

[Boris Zbarsky [:bzbarsky]]

You can put a VendorComment after your VendorProductToken, right?  Would that do the trick?

Comment 17

[Eric Shepherd [:sheppy]]

The continuing comments here bring me to ask: is this ready to be documented, or are we planning to change how we add the device type to the user-agent string before 1.9.2 is done?

Comment 18

[Doug Turner (:dougt)]

Attachment: move Device Info to end of UA string

Comment 19

[Boris Zbarsky [:bzbarsky]]

You probably want to put it in parens there, no?

Comment 20

[Doug Turner (:dougt)]

I am not sure parens add anything.

"Mozilla/5.0 (X11; U; Linux armv7l; en-US; rv:1.9.2a1pre) Gecko/20090322 Fennec/1.0b4pre Nokia N900"

Comment 21

[Boris Zbarsky [:bzbarsky]]

Your call.  I was looking at the 

  Mozilla/9.876 (X11; U; Linux 2.2.12-20 i686, en; rv:2.0) Gecko/25250101
  Netscape/5.432b1 (C-MindSpring)

example in the wiki, which seems to fit best here.

Comment 22

[Doug Turner (:dougt)]

Attachment: what bz said

probably can't hurt.

Comment 23

[Boris Zbarsky [:bzbarsky]]

Comment on attachment 408908 [details] [diff] [review]
what bz said

gerv's a better reviewer for this

Comment 24

[Gervase Markham [:gerv]]

What exactly are we trying to tell people here? Are we trying to tell them the hardware? Or the OS version? Which two versions of Fennec are we trying to distinguish between (i.e. which should get the high quality video and which the low quality video)?

The UA spec has:
( Platform ; Security  ; OS-or-CPU ;   Localization information ; GeckoVersion )
and the Fennec user agent originally had:
( X11;       U;          Linux armv7l; en-US;                     rv:1.9.2a1pre)

Mac and Windows use the first field to say Mac and Windows, but Unix all says X11 with the proper Unix version in field 3. So that part's right.

Would the problem be solved if "Linux armv7l" became "Maemo armv7l" or "Linux Maemo armv7l"?

Is this problem (of web content negotiation) better solved with the Accept header?

Gerv

Comment 25

[Doug Turner (:dougt)]

Hey Gerv,

We want to expose the hardware type and CPU is enough.  For example, on some devices, there may be be hw accelerated graphics and video.  Sites would like to know.  Of course this isn't a complete solution (something like UAProf is, but wow... i don't want to go there).

"Maemo armv71" isn't enough because there may be devices that fit that but have drastically different features/characteristics.

I am not sure how an Accept header would help.

Comment 26

[Gervase Markham [:gerv]]

So we already expose the CPU, right? That's the armv7l bit. So you just want to expose the "hardware type". But what sort of values would that have? If we start putting brand names like "Nokia N900" in the user agent string, the logical end result of that is massive problems where websites have to update their sniffing every time a new mobile handset comes out with a new name! I mean, there's 30 different Android devices already, each of which would have a different equivalent value for "Nokia N900", and there will be hundreds more.

I'd never heard of UAProf before and you may not like it but, reading the wiki page, I can see why it exists. We can't put the burden of mapping hundreds of device names to sets of capabilities onto websites. That'll put us in a situation where sites just support the most common model names, then people start wanting to spoof user agents to make sites work, and it'll be like the old user agent creeping wars again (IE's UA acquires "Mozilla"; Safari's UA acquires "like Gecko") but much, much worse. 

You are going to end up with:

"Mozilla/5.0 (X11; U; Linux armv7l; en-US; rv:1.9.2a1pre) Gecko/20090322
Fennec/1.0b4pre (Nokia N920 (compatible; like N900 and N810 (# for YouTube support)); like iPhone/2.1 and Razr/Android (compatible))"

:-(

Gerv

Comment 27

[Doug Turner (:dougt)]

i don't not believe that this slipperly slope will happen in practice and I am not suggesting any "compatible" string in the UA.

Comment 28

[Doug Turner (:dougt)]

backing out the string change to the UA until gerv and I can agree.

Comment 29

[Doug Turner (:dougt)]

Attachment: no UA String change, but leave mDeviceInfo around.

Comment 30

[Doug Turner (:dougt)]

Attachment: no UA String change, but leave mDeviceInfo around.

Comment 31

[Doug Turner (:dougt)]

http://hg.mozilla.org/mozilla-central/rev/f12c8531e48d
http://hg.mozilla.org/releases/mozilla-1.9.2/rev/bb5e1bebe26f

Comment 32

[Gervase Markham [:gerv]]

Sorry, Doug, I thought I was CCed here. Apologies.

How do you want to continue this discussion? Here or elsewhere?

Gerv

Comment 33

[Doug Turner (:dougt)]

so, we had this on for a while.  many sites served up terribly dumbed-down pages because they sniffed the string "Nokia".  So, i am inclined to just mark this as WONTFIX.

Comment 34

[Dão Gottwald [:dao]]

Is the rest of the original patch going to be backed out as well?

Comment 35

[Dão Gottwald [:dao]]

(In reply to comment #34)
> Is the rest of the original patch going to be backed out as well?

ping?

Comment 36

[Doug Turner (:dougt)]

not the extra stuff in necko.

Comment 37

[Daniel Veditz [:dveditz]]

(In reply to comment #20)
> I am not sure parens add anything.
> 
> "Mozilla/5.0 (X11; U; Linux armv7l; en-US; rv:1.9.2a1pre) Gecko/20090322
> Fennec/1.0b4pre Nokia N900"

Sorry to comment in a long-dead bug, but the parens turn the added information into a comment rather than claiming to incorporate the versionless sub-product "Nokia" and a second versionless sub-product "N900".

http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.43