726247 - Possible MiTM with releases.mozilla.org in China

Status: RESOLVED WORKSFORME

(mozilla.org Graveyard :: Server Operations, task)

Description

[Lu Wei]

Download files from releases.mozilla.org. The pages displayed as normal, download success as normal, but in fact the connection is not with Mozilla's server. A sample log (releases.mozilla.org is resoved to 218.6.25.199 at the time, but it's changing):

GET /pub/mozilla.org/thunderbird/releases/10.0-real/win32/en-US/Thunderbird%20Setup%2010.0.exe HTTP/1.1
Host: releases.mozilla.org
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:10.0) Gecko/20100101 Firefox/10.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en
Accept-Encoding: gzip, deflate
Referer: http://releases.mozilla.org/pub/mozilla.org/thunderbird/releases/10.0-real/win32/en-US/
Cookie: WT_FPC=id=2bb0cb0e84cc108c6511274053975974:lv=1328562797045:ss=1328562796719; dloadday=65.49.68.155.1325471858384717; wtspl=219427DNT: 1
Connection: keep-alive

HTTP/1.1 200 OK
X-Backend-Server: cn-web01
Last-Modified: Mon, 30 Jan 2012 01:43:57 GMT
Accept-Ranges: bytes
Content-Length: 16811832
Content-Type: application/octet-stream
Date: Fri, 10 Feb 2012 05:11:29 GMT
Server: Apache
Powered-By-ChinaCache: MISS from CHN-WZ-V-3CA
Powered-By-ChinaCache: HIT from CHN-PT-2-336
Connection: close

MZ......................@...............................................!..L.!This program cannot be run in DOS mode.
$........H...)u..)u..)u...~..)u.75{..)u......)u...q..)u..)t. )u.w&(..)u...~..)u.s/s..)u.Rich.)u.........PE..L...fJ.D.....
...

Comment 1

[Lu Wei]

This attack is raised in TB support group:
http://groups.google.com/group/mozilla.support.thunderbird/browse_thread/thread/a960a10ec01f1272/2d973d9ebe4e2f33

All chinese users and maybe other areas which under GFW's influence are affected. releases.mozilla.org is affected. Other release channels not checked, but may be attacked as well.

The fix, I think, is to enable HTTPS by default on all release channels.

Comment 2

[Lu Wei]

(In reply to LU Wei from comment #1)
> The fix, I think, is to enable HTTPS by default on all release channels.

And, to prevent GFW from even hijacking HTTPS, I think Bug 542689 should be re-opened too.

Comment 3

[Shyam Mani [:fox2mike]]

LU,

Can you run an md5sum on the downloaded binary? cn-web01 is a webserver in a Mozilla datacenter in China. I'm not a 100% sure, but I think in order to comply with local governmental regulations, we're supposed to send traffic from China to China for downloading firefox.

We use ChinaCache as a CDN to help with the traffic.

Also, releases.mozilla.org is a "pool" of servers, not a single machine. 

I don't think there is any cause for concern here at this point.

Comment 4

[David E. Ross]

Below is the result of a DNS lookup from a RoadRunner connection in the U.S. (Los Angeles area).  You will note that the IP address cited in the description -- 218.6.25.199 -- does not appear in this list.  

218.6.25.199 is the IP address of 199.25.6.218.broad.pt.fj.dynamic.163data.com.cn.  That IP address is owned by China Telecom.  The domain 163data.com.cn is listed by the CNNIC WhoIs server as "inactive".  

NSLookUp - Friday, February 10, 2012 22:09:38
Query: releases.mozilla.org
Type of Search: Use Winsock Function GetHostByX
Protocol: UDP
Verbose: Yes
Recursion Desired: Yes
Hostname: releases.geo.mozilla.com
IP Address: 128.61.111.9
IP Address: 64.50.236.214
IP Address: 131.188.12.212
IP Address: 216.165.129.141
IP Address: 149.20.36.135
IP Address: 202.177.202.154
IP Address: 129.101.198.59
IP Address: 204.246.0.136
IP Address: 155.98.64.83
IP Address: 156.56.247.196
IP Address: 204.152.184.113
Alias: releases.mozilla.org

Comment 5

[Shyam Mani [:fox2mike]]

David, 

That is correct. We still don't know how LU got that IP (that information isn't on this bug). It is quite likely there is an issue on his end vs all of our users in China (which is what this bug claims). Also, the resulting backend he's hitting (even after getting that IP) *seems* to be Mozilla owned, based on the headers he pasted.

Comment 6

[Dave Miller [:justdave]]

releases.mozilla.org is a cname to releases.geo.mozilla.com.  The nameserver which resolves *.geo.mozilla.com will look at your source IP and give you different results depending on where in the world you're doing the lookup from, by looking up your source IP in a database that maps IP addresses to a location.  The answer you get in the US will be different than the answer you get in China.

In China the correct result is a CNAME to releases.mozilla.chinacache.net, which is a CDN that we purchase bandwidth from to help distribute the files in China.  ChinaCache uses releases.cn.mozilla.com as its origin server, which is a load balancer in our data center in Beijing.  cn-web01, as listed in the headers of the above response, is one of the back-end servers behind that load balancer.

Comment 7

[Dave Miller [:justdave]]

And ChinaCache does lease IP space from ChinaNet (which is who the above-listed IP address belongs to) as well as about 4 other ISPs in China.

Comment 8

[Dave Miller [:justdave]]

(In reply to Shyam Mani [:fox2mike] from comment #3)
> I'm not a 100% sure, but I think in order to
> comply with local governmental regulations, we're supposed to send traffic
> from China to China for downloading firefox.

It wasn't anything to do with regulations but more for performance because getting traffic in and out of China is really slow.  If we host it in China people can actually download at a decent speed, and since we happened to have a data center there anyway...

Comment 9

[Dave Miller [:justdave]]

FWIW, if you would like to use https to download Firefox or Thunderbird, you can use https://ftp.mozilla.org/pub/mozilla.org/

Comment 10

[Lu Wei]

Thanks for all of you explained to me. Seems I am over panic under GFW. I have the habit to check file's signatures (if it has), and I have not encountered any fake files until now. The IP addresses I got is from Wireshark's log, besides 218.6.25.199 above, 113.107.107.176 and 59.57.12.141 have been recorded too. It is reasonable that they are all cache servers for load balance. 

Anyway, for an old user who knows signature checking, downloading from non-official place does not form any threat. It is user's responsibility to learn about security if he concerns about it.