694798 - Mozilla ftp site: connection refused

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Todd]

User Agent: Mozilla/5.0 (X11; Linux i686; rv:6.0.2) Gecko/20100101 Firefox/6.0.2
Build ID: 20110902133214

Steps to reproduce:

Would you guys please fix this for me?  Connection refused on ftp://releases.mozilla.org/pub/mozilla.org/firefox/releases/7.0.1/linux-i686/en-US/firefox-7.0.1.tar.bz2

64 bit is working.

Many thanks, -T

wget --output-document /home/CDs/Linux/Firefox/x32/firefox-7.0.1.tar.bz2 ftp://releases.mozilla.org/pub/mozilla.org/firefox/releases/7.0.1/linux-i686/en-US/firefox-7.0.1.tar.bz2
Downloading Firefox for Linux x32.  Please wait ...--2011-10-15 16:26:54--  ftp://releases.mozilla.org/pub/mozilla.org/firefox/releases/7.0.1/linux-i686/en-US/firefox-7.0.1.tar.bz2
           => `/home/CDs/Linux/Firefox/x32/firefox-7.0.1.tar.bz2'
Resolving releases.mozilla.org... 202.177.202.154, 204.152.184.113, 204.152.184.196, ...
Connecting to releases.mozilla.org|202.177.202.154|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done.    ==> PWD ... done.
==> TYPE I ... done.  ==> CWD /pub/mozilla.org/firefox/releases/7.0.1/linux-i686/en-US ... done.
==> SIZE firefox-7.0.1.tar.bz2 ... 15252909
==> PASV ... couldn't connect to 202.177.202.154 port 23716: Connection refused

Comment 1

[Justin Dow [:jabba]]

This might have been a transient issue. I just tried the exact command:

jabba@JabbaBookAir:~> wget --output-document sandbox/firefox-7.0.1.tar.bz2 ftp://releases.mozilla.org/pub/mozilla.org/firefox/releases/7.0.1/linux-i686/en-US/firefox-7.0.1.tar.bz2
--2011-11-14 10:00:30--  ftp://releases.mozilla.org/pub/mozilla.org/firefox/releases/7.0.1/linux-i686/en-US/firefox-7.0.1.tar.bz2
           => `sandbox/firefox-7.0.1.tar.bz2'
Resolving releases.mozilla.org... 131.188.12.212, 128.61.111.9, 129.101.198.59, ...
Connecting to releases.mozilla.org|131.188.12.212|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done.    ==> PWD ... done.
==> TYPE I ... done.  ==> CWD (1) /pub/mozilla.org/firefox/releases/7.0.1/linux-i686/en-US ... done.
==> SIZE firefox-7.0.1.tar.bz2 ... 15252909
==> PASV ... done.    ==> RETR firefox-7.0.1.tar.bz2 ... done.
Length: 15252909 (15M) (unauthoritative)

100%[========================================================================================================================================================================>] 15,252,909  1.64M/s   in 15s     

2011-11-14 10:00:47 (992 KB/s) - `sandbox/firefox-7.0.1.tar.bz2' saved [15252909]

jabba@JabbaBookAir:~>

Comment 2

[Todd]

(In reply to Justin Dow [:jabba] from comment #1)
> This might have been a transient issue.

Transient issue indeed.  It depends on which mirror you are using.  Reopening.

$ wget --output-document firefox-7.0.1.tar.bz2 ftp://releases.mozilla.org/pub/mozilla.org/firefox/releases/7.0.1/linux-i686/en-US/firefox-7.0.1.tar.bz2
--2011-11-14 09:08:26--  ftp://releases.mozilla.org/pub/mozilla.org/firefox/releases/7.0.1/linux-i686/en-US/firefox-7.0.1.tar.bz2
           => `firefox-7.0.1.tar.bz2'
Resolving releases.mozilla.org... 202.177.202.154, 204.152.184.113, 204.152.184.196, ...
Connecting to releases.mozilla.org|202.177.202.154|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done.    ==> PWD ... done.
==> TYPE I ... done.  ==> CWD /pub/mozilla.org/firefox/releases/7.0.1/linux-i686/en-US ... done.
==> SIZE firefox-7.0.1.tar.bz2 ... 15252909
==> PASV ... couldn't connect to 202.177.202.154 port 11071: Connection refused

Comment 3

[Reed Loden [:reed]]

If you want to use FTP, use ftp.mozilla.org. Not all the releases.mozilla.org servers support FTP.

Comment 4

[Todd]

(In reply to Reed Loden [:reed] (very busy) from comment #3)
> If you want to use FTP, use ftp.mozilla.org. Not all the
> releases.mozilla.org servers support FTP.

Not a good fix.  I was told by folks in these haunts not to use ftp.mozilla.org as they reserve the right to block traffic to control volume.  I was told specifically to use releases.

Now on the other hand, I was also told numerous times by the same powers that be that ftp is a requirement of the mirrors.

So, what I would like you to do is to find out which mirror are not supporting ftp and either upgrade them or drop them, as is your policy

Comment 5

[Justin Dow [:jabba]]

Normally when we block things on ftp.mozilla.org, we only block them from ftp. The same files are then still available via http://ftp.mozilla.org as it supports caching so it doesn't destroy our servers when it gets hammered during a release. I don't know all the ins and outs of it, but in my mind ftp is deprecated and I'm not sure why it would be preferred over http in this case.

Comment 7

[Corey Shields [:cshields]]

(In reply to Todd from comment #4)
> Not a good fix.  I was told by folks in these haunts not to use
> ftp.mozilla.org as they reserve the right to block traffic to control
> volume.  I was told specifically to use releases.

If you want to insure a download, you should be using our bouncer links.  This is our officially supported method of downloading (and as stated before, releases.mozilla.org is not for end user downloads.  sorry about that information, if you read it on one of our sites please let me know so we can get it fixed). We have a complex db of mirrors and mirror states that are checked every few minutes.  We know which mirrors are up and which mirrors can serve which files.  There should be no guess of what URL or hostname to use, just pick a link for what you need from this list:

http://www.mozilla.org/en-US/firefox/all.html

If you need a different version from what is latest (you mentioned 7.0.1) then copy the link and change the version number in the link.

Comment 8

[Todd]

(In reply to Corey Shields [:cshields] from comment #7)
> (In reply to Todd from comment #4)
> > Not a good fix.  I was told by folks in these haunts not to use
> > ftp.mozilla.org as they reserve the right to block traffic to control
> > volume.  I was told specifically to use releases.
> 
> If you want to insure a download, you should be using our bouncer links. 
> This is our officially supported method of downloading (and as stated
> before, releases.mozilla.org is not for end user downloads.  sorry about
> that information, if you read it on one of our sites please let me know so
> we can get it fixed). 

Here you go.  Dave Miller is the source:
https://bugzilla.mozilla.org/show_bug.cgi?id=597802#c4:

    Dave Miller [:justdave] bugzilla-reviewers 2010-09-19 19:34:20 PDT

    WONTFIX.  If you want to download releases, use the releases server.  The ftp server
    is only intended for nightlies and doesn't have the capacity to handle people direct-
    linking it on well-advertised releases (this is the whole reason we created the
    releases server pool).  If you try to download one of these via http from the ftp
    server you'll get redirected to the releases server.  Unfortunately, ftp has no
    redirect capability so our only option is to block them.

    The block is version-specific, and usually only applies to the latest version.
    Quite often, we forgot to move it along to the next version... until someone
    direct-links a release file via ftp and the server dies.  Then we block it so the
    server can function again.

    Every server in the releases.mozilla.org DNS pool should also support FTP (it's one
    of the requirements for being part of that pool).  The pathnames are exactly the
    same, except those servers are missing all of the nightly content and all but the most
    recent few releases.  So changing "ftp.mozilla.org" to "releases.mozilla.org" in your
    scripts will probably work just fine.

> We have a complex db of mirrors and mirror states that
> are checked every few minutes.  We know which mirrors are up and which
> mirrors can serve which files.  There should be no guess of what URL or
> hostname to use, just pick a link for what you need from this list:
> http://www.mozilla.org/en-US/firefox/all.html
> 
> If you need a different version from what is latest (you mentioned 7.0.1)
> then copy the link and change the version number in the link.

The above link is not practical for a script

Reopening.  Please follow your own requirements for being a part of the pool.  Or you could change the requirements.  Please get with Dave Miller and get this ironed out.  Please do not re-close this bug until you either fix the broken ftp mirror or changed your requirements.  Please make sure David Miller is on board with you before closing again.

Comment 9

[Corey Shields [:cshields]]

(In reply to Todd from comment #8)
> The above link is not practical for a script

Maybe you are misunderstanding the links..  If you can script an ftp link, then this is just as easy.

> Reopening.  Please follow your own requirements for being a part of the
> pool.  Or you could change the requirements.  Please get with Dave Miller
> and get this ironed out.  Please do not re-close this bug until you either
> fix the broken ftp mirror or changed your requirements.  Please make sure
> David Miller is on board with you before closing again.

Sure, I'll assign this to him.

Comment 10

[Dave Miller [:justdave]]

Looking at the comments it looks like 202.177.202.154 is the one that's broken...
Also looks like it's a firewall issue, because your command channel connects, and it's the data channel that's getting refused.  This tells me they don't have the ftp mapping set up correctly on their firewall.

154.202.177.202.in-addr.arpa domain name pointer pj-mirror01.mozilla.org.

That's actually one of ours.  I'll fix.

Comment 11

[Dave Miller [:justdave]]

Looking at the config, I suspect passive FTP probably worked, and it was only active FTP that was having issues.  This server is running its own iptables firewall as it's not behind a hardware firewall.  I've added the ip_conntrack_ftp module to the list of modules loaded when the firewall activates.  In theory this should fix it.

Confirmed using this command line from an outside IP:

wget --no-passive-ftp --output-document firefox-7.0.1.tar.bz2 ftp://pj-mirror01.mozilla.org/pub/mozilla.org/firefox/releases/7.0.1/linux-i686/en-US/firefox-7.0.1.tar.bz2

Comment 12

[Dave Miller [:justdave]]

For anyone who runs into this with another server in the future, the place in question to specify the module is in /etc/syconfig/iptables-config in the IPTABLES_MODULES variable.  Adding it there will affect the next restart.  You can load it at runtime by running "modprobe ip_conntrack_ftp".

Comment 13

[Nick Thomas [:nthomas] (UTC+12)]

In bug 701469 it was
  129.101.198.59 / mirror2.its.uidaho.edu
  155.98.64.83   / mozilla.cs.utah.edu
returning a 550 error, Failed to change directory.

Comment 14

[Todd]

(In reply to Dave Miller [:justdave] from comment #11)
> Looking at the config, I suspect passive FTP probably worked, and it was
> only active FTP that was having issues.  This server is running its own
> iptables firewall as it's not behind a hardware firewall.  I've added the
> ip_conntrack_ftp module to the list of modules loaded when the firewall
> activates.  In theory this should fix it.
> 
> Confirmed using this command line from an outside IP:
> 
> wget --no-passive-ftp --output-document firefox-7.0.1.tar.bz2
> ftp://pj-mirror01.mozilla.org/pub/mozilla.org/firefox/releases/7.0.1/linux-
> i686/en-US/firefox-7.0.1.tar.bz2

Thank you Dave! Very much appreciated!  This bug kept getting closed a lot until you took over.

-T  (the original poster)