Closed Bug 804642 (baidu.com) Opened 12 years ago Closed 6 years ago

baidu.com UA sniffing serves non optimized content

Categories

(Web Compatibility :: Site Reports, defect, P1)

Product:
Web Compatibility
Component:
Site Reports
Platform:
ARM
All
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: lmandel, Assigned: angelc04, Mentored)

References

(
URL
)

Details

(Whiteboard: [country-cn] [serversniff] [tier1] [sitewait])

Attachments

(2 files)

246.png
202.54 KB, image/png
Details
baidu-devices.jpg
298.91 KB, image/jpeg
Details
baidu.com serves a static HTML site to Firefox for Android and a desktop site to Firefox OS. baidu has a much better mobile site that is served to the Android stock browser. Spoofing the UA to that of the Android stock browser results in the better mobile site being served to Firefox for Android. The site appears to be completely functional and does not appear to have layout issues.
Whiteboard: [country-cn]
Server side detection. To be contacted. → http -v GET http://baidu.com/ GET / HTTP/1.1 Accept: */* Accept-Encoding: gzip, deflate, compress Host: baidu.com User-Agent: HTTPie/0.7.2 HTTP/1.1 200 OK Accept-Ranges: bytes Cache-Control: max-age=86400 Connection: Keep-Alive Content-Length: 81 Content-Type: text/html Date: Fri, 25 Oct 2013 18:18:45 GMT ETag: "51-4b4c7d90" Expires: Sat, 26 Oct 2013 18:18:45 GMT Last-Modified: Tue, 12 Jan 2010 13:48:00 GMT Server: Apache <html> <meta http-equiv="refresh" content="0;url=http://www.baidu.com/"> </html> I tested with different User-Agent same response, at this level. The content varies with www.baidu.com and User-Agent. * Firefox Android - Server Apache with a content which seems to be for WAP. * Firefox OS - server BWS/1.0 with a content for desktop. * iPhone - server Apache - some content is tailored for Webkit only. * Opera Mobile (Presto) and Opera Mini - receive the WAP content # Firefox for Android → http -v GET http://www.baidu.com/ User-Agent:"$FANUA" GET / HTTP/1.1 Accept: */* Accept-Encoding: gzip, deflate, compress Host: www.baidu.com User-Agent: Mozilla/5.0 (Android; Mobile; rv:18.0) Gecko/18.0 Firefox/18.0 HTTP/1.1 200 OK Cache-Control: no-cache Connection: Keep-Alive Content-Encoding: gzip Content-type: text/html;charset=utf-8 Date: Fri, 25 Oct 2013 18:24:43 GMT P3P: CP=" OTI DSP COR IVA OUR IND COM " Server: apache Set-Cookie: BAIDUID=28A8E3E8ECA79AC521C5AEE5AE20107A:FG=1; max-age=31536000; expires=Sat, 25-Oct-14 18:24:43 GMT; domain=.baidu.com; path=/; version=1 Transfer-Encoding: chunked Vary: Accept-Encoding Version: HTTP/1.1 <!DOCTYPE html PUBLIC "-//WAPFORUM//DTD XHTML Mobile 1.0//EN" "http://www.wapforum.org/DTD/xhtml-mobile10.dtd"> […] # Firefox OS → http -v GET http://www.baidu.com/ User-Agent:"$FOSUA" GET / HTTP/1.1 Accept: */* Accept-Encoding: gzip, deflate, compress Host: www.baidu.com User-Agent: Mozilla/5.0 (Mobile; rv:18.0) Gecko/18.0 Firefox/18.0 HTTP/1.1 200 OK BDPAGETYPE: 1 BDQID: 0xf5b5a20500a75292 BDUSERID: 0 Cache-Control: private Connection: Keep-Alive Content-Encoding: gzip Content-Length: 4389 Content-Type: text/html;charset=utf-8 Date: Fri, 25 Oct 2013 20:20:12 GMT Expires: Fri, 25 Oct 2013 20:20:12 GMT P3P: CP=" OTI DSP COR IVA OUR IND COM " Server: BWS/1.0 Set-Cookie: BDSVRTM=21; path=/ Set-Cookie: H_PS_PSSID=3593_1448_2975_2981_3644_3545_3681_3661; path=/; domain=.baidu.com Set-Cookie: BAIDUID=262D6519775C847D529D823C90E5B278:FG=1; expires=Fri, 25-Oct-43 20:20:12 GMT; path=/; domain=.baidu.com <!DOCTYPE html><!--STATUS OK--><html><head><meta http-equiv="content-type" content="text/html;charset=utf-8"><title>百度一下，你就知道</title> # iPhone → http -v GET http://www.baidu.com/ User-Agent:"$APHUA" GET / HTTP/1.1 Accept: */* Accept-Encoding: gzip, deflate, compress Host: www.baidu.com User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 5_0 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9A334 Safari/7534.48.3 HTTP/1.1 200 OK Cache-Control: no-cache Connection: Keep-Alive Content-Encoding: gzip Content-type: text/html;charset=utf-8 Date: Fri, 25 Oct 2013 18:25:39 GMT P3P: CP=" OTI DSP COR IVA OUR IND COM " Server: apache Set-Cookie: BAIDUID=95ABF1DACA584E6BC684409502E0DCCC:FG=1; max-age=31536000; expires=Sat, 25-Oct-14 18:25:39 GMT; domain=.baidu.com; path=/; version=1 Transfer-Encoding: chunked Vary: Accept-Encoding Version: HTTP/1.1 <!DOCTYPE html> <html><!--STATUS OK--><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8" /><meta name="viewport" content="width=device-width,minimum-scale=1.0"/><link rel="apple-touch-icon-precomposed" href="http://m.baidu.com/static/index/screen_icon.png"/><meta name="format-detection" content="telephone=no"/>
URL: http://baidu.com → http://www.baidu.com/
Whiteboard: [country-cn] → [country-cn] [serversniff] [tier1] [contactready]
Attached image 246.png —
attaching screenshots for another example of Baidu sniffing - http://duokoo.baidu.com/novel/?from=1000969a (though I think the fancy site having a black background is a bug in the screenshot'ing process)
I nominate this as a sort of master bug for contacting Baidu, so I've marked the other baidu-subsite bugs as dependencies. (This isn't an empty META bug though - this bug is about the sniffing on the main baidu.com destination).
Alias: baidu.com
Priority: -- → P2
Whiteboard: [country-cn] [serversniff] [tier1] [contactready] → [country-cn] [serversniff] [tier1] [contactready][mentor=kdubost]
Assignee: nobody → yliu
Status: NEW → ASSIGNED
Mentor: kdubost
Whiteboard: [country-cn] [serversniff] [tier1] [contactready][mentor=kdubost] → [country-cn] [serversniff] [tier1] [contactready]
I have contacted the site, waiting for them to solve it.
Whiteboard: [country-cn] [serversniff] [tier1] [contactready] → [country-cn] [serversniff] [tier1] [sitewait]
See Also: → 1123197
Hi YiLiu, Just wondering have you have any response from Baidu? Thanks!
Flags: needinfo?(yliu)
Assignee: yliu → pcheng
Flags: needinfo?(yliu) → needinfo?(pcheng)
Hi josh, our BD team have contact baidu but they don't plan to fix in short time. because there are no firefox os device in China mainland yet so the user impact is very limited. I think this should not be a blocker for woodduck.
Flags: needinfo?(pcheng)
Hi Gary, Can you help to add Baidu.com to https://hg.mozilla.org/mozilla-central/file/tip/b2g/app/ua-update.json.in? Thanks!
Flags: needinfo?(gchen)
Hi Josh, Please do not push things like this. It's not the way we are trying to solve things. I proposed already to help you with UA override, asking someone else will not make things faster. Thanks for your understanding. First we need to test which UA will work. If the Comment #1 is still valid or not. Then read carefully the comment #9 by Peipei. I will test again on next Monday, but according to comment #1, we will not achieve a solution with *any* UA override. There is no point doing a UA override if we send the user to a broken mobile Web site. ^_^ The question before asking UA override is to discover if there is a UA override triggering 1. The right redirection 2. Without breaking the site. So I will do testing on Monday and I will report it my findings and we can decide *together* what we do.
Flags: needinfo?(gchen) → needinfo?(jocheng)
Attached image baidu-devices.jpg —
OK Let's see. What's the status with http://www.baidu.com/ * Firefox OS receives Desktop Content * Firefox Android receives a simplified Mobile Content * Chrome Mobile receives a more advanced Mobile Content Here Josh, I'm showing in this image what are the consequences of putting a Chrome Mobile UA override on Firefox OS. Compare the last two columns with the home page and a secondary page. Note that it is just a test on two pages. As you can see when overriding the Firefox OS UA with Chrome Mobile UA we get a broken version of the Web site. This is created by JavaScript scripts and CSS which are tailored for Blink. The Chrome UA override will **not solve** the issue, because receiving the mobile site is only half part of the issue. What we could do as a in between solution AND **if** Mozilla Beijing agrees is a UA override of Firefox OS to Firefox Android and receives the simplified Mobile Content. That would not solve all issues and probably would create secondary issues down the path. I haven't tested all the pages. Peipei has certainly tested more than me these issues already. Josh, does it help understand the issues? As I proposed in an email, I'm happy to give a full talk about all these issues to the rest of your team either in Whistler or before in Taipei or somewhere else. It's very important to understand what is happening when we change the UA strings (be on the device itself or through UA override). Thanks.
Flags: needinfo?(pcheng)
Hi Karl, Thank you very much for the clarify. I thought overriding will return page like "Chrome Android". The current "Firefox Android receives a simplified Mobile Content" seems just a WAP page. Peipei already updated me that she is now working with Baidu to solve the issue. Let's hope they can solve it soon. We are really looking forward to have your session about UA at Whistler. Thank you very very much for the help!
Flags: needinfo?(jocheng)
(In reply to Josh Cheng [:josh] from comment #13) > Thank you very much for the clarify. I thought overriding will return page > like "Chrome Android". Sometimes it is working, sometimes it is not, it really depends how the site has been made. Triggering with a different User Agent string the mobile domain name or mobile content is only the beginning. After this, once the content is sent there are many dependencies JavaScript, CSS, etc which have impact on the user experience. So each site have to be treated separately and sometimes we have simple solutions, and sometimes not so much. ^_^ Life.
Hi Josh, We got feedback from baidu today. It will took at least two weeks to fix this problem. Considering the amount of ffos device is very limited in China, they won't put this as high priority. We will try to look for other contact to see if we can push further.
Flags: needinfo?(pcheng)
Firefox Android receives a simplified version compared to Chrome on Android. Let's track this on https://webcompat.com/issues/3149
Closing as fixed.
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Priority: P2 → P1
Resolution: --- → FIXED
Product: Tech Evangelism → Web Compatibility
Component: Mobile → Site Reports
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: