Closed Bug 675163 Opened 13 years ago Closed 12 years ago

Get a FTP server setup on mozqa.com

Categories

(Mozilla QA Graveyard :: Infrastructure, defect)

Product:
Mozilla QA Graveyard
Component:
Infrastructure
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: vladmaniac, Assigned: jsmith)

Details

We need a FTP server setup on mozqa.com 
We don't want to use external servers as much as possible in our Mozmill tests
Blocks: 674874
You should assign these to me to do. I hadn't noticed this request.
Assignee: nobody → abillings
Thanks Al!
Status: NEW → ASSIGNED
Hi Al, do we have a status of some kind for this request? It will really really help me check in bug 674874 :) 

Lots of thanks!
(In reply to Maniac Vlad Florin (:vladmaniac) from comment #3)
> Hi Al, do we have a status of some kind for this request? It will really
> really help me check in bug 674874 :) 
> 
> Lots of thanks!

Al is on PTO until Sept 12.
Done. It works for non-ssl and ssl connections. Had to open up the Firewall too!

lftp -u abillings mozqa.com -e "debug;set ftp:ssl-protect-data true;ls;exit"
Password: 
---- Connecting to mozqa.com (67.23.44.24) port 21
<--- 220 (vsFTPd 2.0.5)
---> FEAT
<--- 211-Features:
<---  AUTH SSL
<---  AUTH TLS
<---  EPRT
<---  EPSV
<---  MDTM
<---  PASV
<---  PBSZ
<---  PROT
<---  REST STREAM
<---  SIZE
<---  TVFS
<--- 211 End
---> AUTH TLS
<--- 234 Proceed with negotiation.
---> USER abillings
Certificate: O=*.mozqa.com,OU=Domain Control Validated,CN=*.mozqa.com
 Issued by:        C=US,ST=Arizona,L=Scottsdale,O=GoDaddy.com\, Inc.,OU=http://certificates.godaddy.com/repository,CN=Go Daddy Secure Certification Authority,serialNumber=07969287
 Checking against: C=US,ST=Arizona,L=Scottsdale,O=GoDaddy.com\, Inc.,OU=http://certificates.godaddy.com/repository,CN=Go Daddy Secure Certification Authority,serialNumber=07969287
  Trusted
Certificate: C=US,ST=Arizona,L=Scottsdale,O=GoDaddy.com\, Inc.,OU=http://certificates.godaddy.com/repository,CN=Go Daddy Secure Certification Authority,serialNumber=07969287
 Issued by:        C=US,O=The Go Daddy Group\, Inc.,OU=Go Daddy Class 2 Certification Authority
 Checking against: C=US,O=The Go Daddy Group\, Inc.,OU=Go Daddy Class 2 Certification Authority
  Trusted
Certificate: C=US,O=The Go Daddy Group\, Inc.,OU=Go Daddy Class 2 Certification Authority
 Issued by:        L=ValiCert Validation Network,O=ValiCert\, Inc.,OU=ValiCert Class 2 Policy Validation Authority,CN=http://www.valicert.com/,EMAIL=info@valicert.com
 Checking against: L=ValiCert Validation Network,O=ValiCert\, Inc.,OU=ValiCert Class 2 Policy Validation Authority,CN=http://www.valicert.com/,EMAIL=info@valicert.com
  Trusted
Certificate: L=ValiCert Validation Network,O=ValiCert\, Inc.,OU=ValiCert Class 2 Policy Validation Authority,CN=http://www.valicert.com/,EMAIL=info@valicert.com
 Issued by: L=ValiCert Validation Network,O=ValiCert\, Inc.,OU=ValiCert Class 2 Policy Validation Authority,CN=http://www.valicert.com/,EMAIL=info@valicert.com
  Trusted
<--- 331 Please specify the password.
---> PASS XXXX
<--- 230 Login successful.
---> PWD
<--- 257 "/home/abillings"
---> PBSZ 0
<--- 200 PBSZ set to 0.
---> PROT P
<--- 200 PROT now Private.
---> PROT P
<--- 200 PROT now Private.
---> PASV
<--- 227 Entering Passive Mode (67,23,44,24,169,191)
---- Connecting data socket to (67.23.44.24) port 43455
---- Data connection established
---> LIST
<--- 150 Here comes the directory listing.
Certificate: O=*.mozqa.com,OU=Domain Control Validated,CN=*.mozqa.com
 Issued by:        C=US,ST=Arizona,L=Scottsdale,O=GoDaddy.com\, Inc.,OU=http://certificates.godaddy.com/repository,CN=Go Daddy Secure Certification Authority,serialNumber=07969287
 Checking against: C=US,ST=Arizona,L=Scottsdale,O=GoDaddy.com\, Inc.,OU=http://certificates.godaddy.com/repository,CN=Go Daddy Secure Certification Authority,serialNumber=07969287
  Trusted
Certificate: C=US,ST=Arizona,L=Scottsdale,O=GoDaddy.com\, Inc.,OU=http://certificates.godaddy.com/repository,CN=Go Daddy Secure Certification Authority,serialNumber=07969287
 Issued by:        C=US,O=The Go Daddy Group\, Inc.,OU=Go Daddy Class 2 Certification Authority
 Checking against: C=US,O=The Go Daddy Group\, Inc.,OU=Go Daddy Class 2 Certification Authority
  Trusted
Certificate: C=US,O=The Go Daddy Group\, Inc.,OU=Go Daddy Class 2 Certification Authority
 Issued by:        L=ValiCert Validation Network,O=ValiCert\, Inc.,OU=ValiCert Class 2 Policy Validation Authority,CN=http://www.valicert.com/,EMAIL=info@valicert.com
 Checking against: L=ValiCert Validation Network,O=ValiCert\, Inc.,OU=ValiCert Class 2 Policy Validation Authority,CN=http://www.valicert.com/,EMAIL=info@valicert.com
  Trusted
Certificate: L=ValiCert Validation Network,O=ValiCert\, Inc.,OU=ValiCert Class 2 Policy Validation Authority,CN=http://www.valicert.com/,EMAIL=info@valicert.com
 Issued by: L=ValiCert Validation Network,O=ValiCert\, Inc.,OU=ValiCert Class 2 Policy Validation Authority,CN=http://www.valicert.com/,EMAIL=info@valicert.com
  Trusted
<--- 226 Directory send OK.     
---- Got EOF on data connection
---- Closing data socket
drwxr-xr-x    2 500      500          4096 Mar 11  2011 Desktop
---> QUIT
---- Closing control socket
Status: ASSIGNED → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
Al, thanks for setting it up. Can you please point the initial directory for anonymous access to the litmus folder, so that the path to the files are equivalent to the HTTP(S) access? Right now it uses the default top-level folder.

ftp://ftp.mozqa.com/
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Can you give me the specific path you want, just for clarity?
We should end up at the same folder, independent of if we are using HTTP or FTP. Which means the root should be the appropriate folder on the file system as where http://mozqa.com/data/ points to.
Status: REOPENED → ASSIGNED
Done.

Anonymous FTP users go to /var/www/html/data
Status: ASSIGNED → RESOLVED
Closed: 13 years ago → 13 years ago
Resolution: --- → FIXED
Al, I'm sorry. Please blame me! But this was a wrong comment from me. I should have checked the HTTP configuration. Now we have different paths for the same content:

ftp://mozqa.com/
http://mozqa.com/data/ 

I think that the default location for anonymous access was ok, but that we should have added a symlink to '/var/www/html/data' as subfolder instead. Only that way we can ensure that path fragments for both protocols are identical.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
I just changed the top level FTP to the same as top level for http: /var/www/html/

We don't keep anything on the webservers where we're worried about security, AFAIK, so this fixes the issue.
Status: REOPENED → RESOLVED
Closed: 13 years ago → 13 years ago
Resolution: --- → FIXED
We will. At least under mozmill-env we will have PHP scripts. So we should not open up the HTTP root this way. The best way is still to reset the FTP root folder to the default value and create a symlink at this location.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
This turns out to be a pain in the ass.

vsftpd, which we use, disallows symlinks. I've made the symlink and you can see it but if you go to it, you get an error 550 and it refuses to change.

See http://radu.cotescu.com/vsftpd-and-symbolic-links/.

So, I have to explicitly mount the directory, etc. I'm not sure if that is really an ideal solution.
Status: REOPENED → ASSIGNED
Gah.

I had to make a dir and mount it pointing at /var/www/html/data

The commmand was:

mount --bind /var/www/html/data /var/ftp/data

Of course, the next time we reboot the server... :-)
Status: ASSIGNED → RESOLVED
Closed: 13 years ago → 13 years ago
Resolution: --- → FIXED
So are you saying that the mountpoint will not be re-set after a reboot currently? If that's the case we should create our own startup script.

Also I don't think we should play ping-pong on closing and reopening the bug. Closing would be appropriate when everything is done. I will not reopen for now, but I still think more work has to be done. At least when reading your last comment.
(In reply to Henrik Skupin (:whimboo) from comment #15)
> So are you saying that the mountpoint will not be re-set after a reboot
> currently? If that's the case we should create our own startup script.
> 
> Also I don't think we should play ping-pong on closing and reopening the
> bug. Closing would be appropriate when everything is done. I will not reopen
> for now, but I still think more work has to be done. At least when reading
> your last comment.

What is the status on this bug? Is it really fixed or not? 

Henrik - what can we do to de-block it?
Lets reopen for the final bits. Vlad, none of our tests are further blocked and you can go ahead. The only things are left on this bug are internals.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(In reply to Henrik Skupin (:whimboo) [away 11/25 - 12/03] from comment #17)
> Lets reopen for the final bits. Vlad, none of our tests are further blocked
> and you can go ahead. The only things are left on this bug are internals.

Bug 674874 has now its initial patch. Thanks! 

Should we remove it from 'Blocks' or just leave it there for sanity?
Al, can you please clarify your comment 14? Is that permanent or not?
It was not set up to be done on reboot at the time.
Jason, would you mind finishing up this bug?
Assignee: abillings → nobody
Status: REOPENED → ASSIGNED
(In reply to Henrik Skupin (:whimboo) from comment #21)
> Jason, would you mind finishing up this bug?

Sure I'll take a shot at it. What is the priority on this getting done? Must have? Good to have? Nice to have?
Assignee: nobody → jsmith
It's a must have. We have Litmus/Mozmill tests which rely on examples via this FTP connection. So we have to make sure that the FTP server can get started immediately after a reboot of the machine.
Jason, can you please have a look at this as soon as possible? The FTP server is down right now and it could be because of the power outage we had lately. So as mentioned earlier we are probably not able to restore the necessary settings.

This causes failures in our Mozmill tests so an immediate fix and start of the FTP server would be appreciated. Thanks.
Severity: normal → critical
(In reply to Henrik Skupin (:whimboo) from comment #24)
> Jason, can you please have a look at this as soon as possible? The FTP
> server is down right now and it could be because of the power outage we had
> lately. So as mentioned earlier we are probably not able to restore the
> necessary settings.
> 
> This causes failures in our Mozmill tests so an immediate fix and start of
> the FTP server would be appreciated. Thanks.

I'm a bit stuck right now trying to understand what Al did and where's he at right now with this FTP setup.
As far as I know, the FTP server should start on boot but you could check. The mount command in comment 14 doesn't run automatically and is necessary for this to work.

I don't recall a lot beyond this. It has been over six months.
(In reply to Al Billings [:abillings] from comment #26)
> As far as I know, the FTP server should start on boot but you could check.
> The mount command in comment 14 doesn't run automatically and is necessary
> for this to work.
> 
> I don't recall a lot beyond this. It has been over six months.

Tried rebooting the server and remounting. No luck.
(In reply to Jason Smith [:jsmith] from comment #27)
> (In reply to Al Billings [:abillings] from comment #26)
> > As far as I know, the FTP server should start on boot but you could check.
> > The mount command in comment 14 doesn't run automatically and is necessary
> > for this to work.
> > 
> > I don't recall a lot beyond this. It has been over six months.
> 
> Tried rebooting the server and remounting. No luck.

Wait, might not be right saying that. I was able to connect from one of my linux machines using sftp to mozqa.com.

Maybe I need to get clarification here - Exactly what is the problem here?
If you don't mount /var/www/html/data to /var/ftp/data you won't be able to get to the checked in mozilla test repo tests via FTP.
Can you go to a /data subdirectory when you FTP to the box now?
SFTP through a different server, I was able to get to /var/ftp/data. I double checked and confirmed that I did run the mount command as root (
mount --bind /var/www/html/data /var/ftp/data). 

Connected to mozqa.com.
sftp> cd /data
Couldn't canonicalise: No such file or directory
sftp> cd /var/www/html/data
sftp> ls
README.txt   firefox      thunderbird  webapps
sftp> cd /var/ftp/data
sftp> ls
README.txt   firefox      thunderbird  webapps
sftp>

If I access ftp://ftp.mozqa.com/data, nothing comes up.
Fixed it with the help of Kevin. ftp://ftp.mozqa.com/data/ is back online.
Severity: critical → normal
No longer blocks: 709932
Henrik - Trying to understand the comments here - What are the remaining bits summarized needed to be completed to close this bug out? I see a mention of a startup script, but what else?
When you restart the server no manual interaction should be necessary to get the FTP server setup and running. If that is done we can call this bug fixed.
(In reply to Henrik Skupin (:whimboo) from comment #34)
> When you restart the server no manual interaction should be necessary to get
> the FTP server setup and running. If that is done we can call this bug fixed.

Done using chkconfig. Requesting review. Should I reboot to test or is inspection enough? Here's what I did:

[root@mozqa sbin]# ./chkconfig vsftpd on
[root@mozqa sbin]# ./chkconfig --list
NetworkManager  0:off   1:off   2:off   3:off   4:off   5:off   6:off
avahi-daemon    0:off   1:off   2:off   3:on    4:on    5:on    6:off
avahi-dnsconfd  0:off   1:off   2:off   3:off   4:off   5:off   6:off
crond           0:off   1:off   2:on    3:on    4:on    5:on    6:off
cups            0:off   1:off   2:on    3:on    4:on    5:on    6:off
dc_client       0:off   1:off   2:off   3:off   4:off   5:off   6:off
dc_server       0:off   1:off   2:off   3:off   4:off   5:off   6:off
dnsmasq         0:off   1:off   2:off   3:off   4:off   5:off   6:off
firstboot       0:off   1:off   2:off   3:on    4:off   5:on    6:off
haldaemon       0:off   1:off   2:off   3:on    4:on    5:on    6:off
httpd           0:off   1:off   2:on    3:on    4:off   5:on    6:off
ip6tables       0:off   1:off   2:on    3:on    4:on    5:on    6:off
iptables        0:off   1:off   2:on    3:on    4:on    5:on    6:off
iscsi           0:off   1:off   2:off   3:on    4:on    5:on    6:off
iscsid          0:off   1:off   2:off   3:on    4:on    5:on    6:off
kdump           0:off   1:off   2:off   3:off   4:off   5:off   6:off
kudzu           0:off   1:off   2:off   3:on    4:on    5:on    6:off
lvm2-monitor    0:off   1:on    2:on    3:on    4:on    5:on    6:off
mcstrans        0:off   1:off   2:on    3:on    4:on    5:on    6:off
messagebus      0:off   1:off   2:off   3:on    4:on    5:on    6:off
multipathd      0:off   1:off   2:off   3:off   4:off   5:off   6:off
named           0:off   1:off   2:off   3:off   4:off   5:off   6:off
netconsole      0:off   1:off   2:off   3:off   4:off   5:off   6:off
netfs           0:off   1:off   2:off   3:on    4:on    5:on    6:off
netplugd        0:off   1:off   2:off   3:off   4:off   5:off   6:off
network         0:off   1:off   2:on    3:on    4:on    5:on    6:off
nfs             0:off   1:off   2:off   3:off   4:off   5:off   6:off
nfslock         0:off   1:off   2:off   3:on    4:on    5:on    6:off
ntpd            0:off   1:off   2:off   3:off   4:off   5:off   6:off
pcscd           0:off   1:off   2:on    3:on    4:on    5:on    6:off
portmap         0:off   1:off   2:off   3:on    4:on    5:on    6:off
rawdevices      0:off   1:off   2:off   3:on    4:on    5:on    6:off
rdisc           0:off   1:off   2:off   3:off   4:off   5:off   6:off
restorecond     0:off   1:off   2:on    3:on    4:on    5:on    6:off
rpcgssd         0:off   1:off   2:off   3:on    4:on    5:on    6:off
rpcidmapd       0:off   1:off   2:off   3:on    4:on    5:on    6:off
rpcsvcgssd      0:off   1:off   2:off   3:off   4:off   5:off   6:off
saslauthd       0:off   1:off   2:off   3:off   4:off   5:off   6:off
smb             0:off   1:off   2:off   3:off   4:off   5:off   6:off
sshd            0:off   1:off   2:on    3:on    4:on    5:on    6:off
syslog          0:off   1:off   2:on    3:on    4:on    5:on    6:off
vncserver       0:off   1:off   2:on    3:on    4:on    5:on    6:off
vsftpd          0:off   1:off   2:on    3:on    4:on    5:on    6:off
winbind         0:off   1:off   2:off   3:off   4:off   5:off   6:off
wpa_supplicant  0:off   1:off   2:off   3:off   4:off   5:off   6:off
xfs             0:off   1:off   2:on    3:on    4:on    5:on    6:off
xinetd          0:off   1:off   2:off   3:on    4:on    5:on    6:off

xinetd based services:
        chargen-dgram:  off
        chargen-stream: off
        daytime-dgram:  off
        daytime-stream: off
        discard-dgram:  off
        discard-stream: off
        echo-dgram:     off
        echo-stream:    off
        tcpmux-server:  off
        tftp:           off
        time-dgram:     off
        time-stream:    off
No longer blocks: 674874
Does it also solve the problem with automatically bind to the right data folder?
(In reply to Henrik Skupin (:whimboo) from comment #36)
> Does it also solve the problem with automatically bind to the right data
> folder?

Good catch (missed that one). I'll look into that.
(In reply to Jason Smith [:jsmith] from comment #37)
> (In reply to Henrik Skupin (:whimboo) from comment #36)
> > Does it also solve the problem with automatically bind to the right data
> > folder?
> 
> Good catch (missed that one). I'll look into that.

Looked into it. I edited /etc/fstab to include the following:

/var/www/html/data      /var/ftp/data           bind    defaults,bind   0 0

Rebooting the server, I saw the ftp server come up with the directory mounted correctly to show mozqa's data.
Status: ASSIGNED → RESOLVED
Closed: 13 years ago → 12 years ago
Resolution: --- → FIXED
Thanks Jason!
Product: Mozilla QA → Mozilla QA Graveyard
You need to log in before you can comment on or make changes to this bug.