change build slaves to use shared disk for builds, not local disk

RESOLVED WONTFIX

Status

Release Engineering
General
P2
normal
RESOLVED WONTFIX
9 years ago
5 years ago

People

(Reporter: joduinn, Assigned: joduinn)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Once shared disk is created, do experiment to see if we can:
- change slaves on each o.s. to use shared disk without significant performance hit
- change build framework and/or makefiles to use slightly different directory structure on shared disk to avoid collisions
- see if incremental builds get improved build times by having latest previous incremental build to work from

If this experiment works ok in staging, we can review whether its worth rolling out in production.
Assignee: nobody → joduinn
Priority: -- → P2
Summary of scribbled notes of my experiments on moz2-win32-slave03:

1) Reassign the original drive "E" to drive letter "Z". Mount this remote nfs samba drive, as a new drive "E" using the cltbld account done as part of bug#472185. 

2) From a cmd-dos command line, I could see, read (and I think write) to the new drive E just fine.

3) From an msys command line, I was not able to see, or mount, this drive in any way. Tried various combination of mount commands, restarting the win32 VM, auto-reconnecting to the drive, starting/stopping buildbot slave, etc, but was never able to see any files on the drive from a msys prompt. This was true even when, in different window, I started a cmd-dos-box, and was able to see the drive just fine. 

Some side discussions about bugs in msys connecting to network mounted disks, but no data either way. Investigations continue.

Added ted, bsmedberg as they had some ideas in today's dev meeting. Also noting ted tried something similar to this before and was never able to get write access - see details in bug#422176.
I have several network mapped drives on my home Windows machine and I can use them just fine in MSYS. I'm not aware of any issues there, but there may be some sensitivity to when they're mounted vs. when MSYS is started.

The bug you linked details me trying to get a writable network share on a VM on the build network. IT provided a NFS mount, but I was never able to make it writable on the Windows VM, IIRC. Could you double-check that you're able to write via the command prompt? As I recall, I was not able to write via explorer or MSYS.
I gave this is a try on moz2-win32-slave03. It worked after a bit of fuss. I couldn't get write access, though - I suspect we need to mount with a username and password for that to work. Or maybe IT needs to fix something on their side.

Here's what I did:
* Start -> Run -> \\bm-sun-xf01.build.mozilla.org
* Right click on 'build_tree', hit 'Map as Network Drive'
* My Computer -> Z: -> Right Click -> New Folder - unable to create - http://people.mozilla.org/~bhearsum/misc/explorer.png
* Open cmd.exe -> z: -> dir - shows directory listing - http://people.mozilla.org/~bhearsum/misc/dos.png
* mkdir foo - unable to create
* Open MSYS (start-l10n.bat) -> cd /z -> ls - shows directory listing - http://people.mozilla.org/~bhearsum/misc/msys2.png
* mkdir foo - unable to create

All of this worked right away as administrator, and worked as cltbld after I killed _all_ running MSYS processes and restarted it (some ssh-agent.exe's had to be killed in task manager).

I did the cltbld work in the VNC session.
I was able to mount shared disk as read-only, but never as read-write. Attempted with following configurations: username-password, username-no-password, and unrestricted access, each when logged into Win machine as cltbld and as root. In each case, I was able to see the disk, and read from it ok. I was also able to view the disk in MS Explorer, and confirmed read-write access, so I believe the disk is shared correctly. The only place where I could *not* get read-write access was within an MSYS shell, so I believe this is a bug in MSYS.

(Did find some discussions about MSYS-and-remote-disk-permission bugs on internet but nothing conclusive.)

Aki & jhford are working on an alternate approach in bug#498522 which looks promising, so I'm going to close this as WONTFIX.
Status: NEW → RESOLVED
Last Resolved: 9 years ago
Resolution: --- → WONTFIX
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.