Closed Bug 598259 Opened 14 years ago Closed 14 years ago

Set up access and routing to machine in Phoenix

Categories

(mozilla.org Graveyard :: Server Operations, task)

Product:
mozilla.org Graveyard
Component:
Server Operations
Platform:
x86
All
Type:
task
Priority:
Not set
Severity:
critical

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: nthomas, Assigned: aravind)

References

Details

RelEng needs to test pushing AUS snippets to a machine in the Phoenix data center, so please set up ssh access on a machine where we can dump up to a gigabyte of files. You could use the stage-ffxbld key in bug 455567. We'd like to be able to reach the PHX machine from dm-ausstage01.m.o, staging-stage.build.m.o, and machines in the build-vpn (eg mv-moz2-linux-ix-slave01.build.m.o). Once AUS in PHX is set up we can arrange access for production systems and revoke this testing access. This is for a joint IT & RelEng Q3 goal, which we'd like to work on Wednesday while RelEng are all in MV.
It'll be a dense tree of of small files, ~ 88K directories and nearly 140K of files for a base copy of the existing snippets, plus 25K dirs and 42K of files for each copy of new snippets we copy in. I know we've bumped the inode limit on dm-ausstage01:/opt so please double check that will be set large enough on the partition you're allocating, allowing for at least 1 base copy and 5 set of new snippets (210k dirs plus 350 files).
That should be s/of files/files/ in comment #1 - ie 140K and 42K individual files.
Easiest way to do this is probably through a VM in Phoenix that has an NFS mount to the filer there. Ravi - can you get this going? Copying Phong to help spin up a VM.
Assignee: server-ops → ravi
Severity: major → critical
dp-ausstage01 (to keep consistent naming) / 10.8.74.30 is online. Needs to be driven by puppet and needs stage-ffxbld account/keys.
ping, we'd like to use this machine today.
Flow has been opened between the two hosts. [root@dm-ausstage01 ~]# nc -vz 10.8.74.30 22 Connection to 10.8.74.30 22 port [tcp/ssh] succeeded!
Status: NEW → RESOLVED
Closed: 14 years ago
Resolution: --- → FIXED
(In reply to comment #0) > We'd like to be able to reach the PHX machine from dm-ausstage01.m.o, > staging-stage.build.m.o, and machines in the build-vpn (eg > mv-moz2-linux-ix-slave01.build.m.o). Once AUS in PHX is set up we can arrange > access for production systems and revoke this testing access. Could you please allow traffic between the staging slaves listed above and the dp-ausstage. Or you can set the access list like dm-ausstage01, in that all the build machines can access it (regardless of location).
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
dp-ausstage01 is also refusing the ssh key for stage-ffxbld. I've checked I'm using the private key that matches the public one in bug 455567.
Assignee: ravi → justdave
ravi had one part, punting over to justdave for the rest.
Not familiar with how this is set up. What user are you logging in with for the ssh? I see an ffxbld user, is it that one?
We'll be doing our testing with our staging machines so it makes sense to use our staging accounts right now, or at least the staging key. So .... ffxbld with the key in the bug 455567, thanks!
ok, account created, key set up, give it a shot.
I can log in by ssh from dm-ausstage01, but can't reach 10.8.74.30 from staging machines, ie comment 7. We can start with dm-ausstage in the meantime though.
Please enumerate all the sources and the destinations by FQDN or IP and the ports you will be connecting to. This will help me write the config only twice.
I am asking for destination: 10.8.74.30:22 sources: 10.2.71.82 staging-stage.build.mozilla.org 10.2.71.105 moz2-linux-slave03.build.mozilla.org 10.2.71.18 moz2-linux-slave04.build.mozilla.org 10.250.49.199 mv-moz2-linux-ix-slave01.build.mozilla.org This will be for testing purposes. If the 10.250 is a PITA then you drop that and we'll deal. Once we know what we'll be doing with production then we will want to change the list of sources.
I added the 4 additional hosts and am waiting for verification.
(In reply to comment #16) They all work except 10.250.49.199 mv-moz2-linux-ix-slave01.build.mozilla.org, but that'll be enough. Thanks.
(In reply to comment #3) > Easiest way to do this is probably through a VM in Phoenix that has an NFS > mount to the filer there. AIUI we use the netapp in MPT because it is HA, and mount via NFS to the webheads. I would prefer to test against this configuration rather than use the much faster storage on /. On dm-ausstage01 it takes us more than 30 minutes to create a nightly backup, and at least 5 mins to unpack it (it's still running as I write); vs ~ 30 seconds to unpack on dp-ausstage01. That's going to make a big difference to testing rsyncs from MPT to PHX.
Assignee: justdave → ravi
(In reply to comment #18) > (In reply to comment #3) > > Easiest way to do this is probably through a VM in Phoenix that has an NFS > > mount to the filer there. > > AIUI we use the netapp in MPT because it is HA, and mount via NFS to the > webheads. I would prefer to test against this configuration rather than use the > much faster storage on /. On dm-ausstage01 it takes us more than 30 minutes to > create a nightly backup, and at least 5 mins to unpack it (it's still running > as I write); vs ~ 30 seconds to unpack on dp-ausstage01. That's going to make a > big difference to testing rsyncs from MPT to PHX. Many things: 1. dp-ausstage01 is missing an NFS mount off the NetApp in Phoenix 2. dp-ausstage01 & the Phoenix NetApp are going to be faster than anything in SJC. 3. Ironically, dp-ausstage01:/ is on the NetApp in Phoenix. It's just fast. 4. For POC, you can keep writing to / until someone gets the NFS mount for you.
Assignee: ravi → aravind
Nick/Rob - do you guys have what you need?
mrz says we're OK with the current disk setup and we have access for the staging machines. That will be fine for testing.
Status: REOPENED → RESOLVED
Closed: 14 years ago → 14 years ago
Resolution: --- → FIXED
Still need the NAS setup - Aravind's point on that (but this doesn't block POC testing).
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
The mount is ready. Its commented out in fstab. 10.8.74.240:/vol/pio_aus2/ /opt nfs noatime,rsize=8192,wsize=8192,intr 0 0
Status: REOPENED → RESOLVED
Closed: 14 years ago → 14 years ago
Resolution: --- → FIXED
(In reply to comment #12) > ok, account created, key set up, give it a shot. FTR, the username is ffxbld, and the key is stage-ffxbld.
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.