Closed Bug 383063 Opened 17 years ago Closed 15 years ago

SeaMonkey Mac nightlies not copied to latest-trunk/

Categories

(mozilla.org :: FTP: Staging, task, P3)

Product:
mozilla.org
Component:
FTP: Staging
Platform:
PowerPC
macOS
Type:
task

Tracking

(Not tracked)

Status:
RESOLVED INVALID

People

(Reporter: kairo, Assigned: zach)

References

(
URL
)

Details

The SeaMonkey trunk Mac nightlies from cg-xserve02 are only uploaded to the dated directories, but not copied to latest-trunk.

Here's the relavent snippet from the latest nightly build log http://tinderbox.mozilla.org/showlog.cgi?log=SeaMonkey/1180857780.23131.gz&fulltext=1
-----------------------------------------------------------

./upload-packages.sh
ssh -2 -l seabld stage.mozilla.org mkdir -p /home/ftp/pub/seamonkey/nightly/2007-06-03-01-trunk /home/ftp/pub/seamonkey/nightly/latest-trunk

rsync -av -e ssh -2 /builds/tinderbox/SeaMonkey-Trunk/Darwin_8.8.1_Depend/2007060301/packages/ seabld@stage.mozilla.org:/home/ftp/pub/seamonkey/nightly/2007-06-03-01-trunk/
building file list ... done
rsync: failed to set times on "/home/ftp/pub/seamonkey/nightly/2007-06-03-01-trunk/.": Operation not permitted (1)
seamonkey-2.0a1pre.en-US.mac.dmg
rsync: failed to set times on "/home/ftp/pub/seamonkey/nightly/2007-06-03-01-trunk/.": Operation not permitted (1)

sent 23553013 bytes  received 40 bytes  9421221.20 bytes/sec
total size is 23549982  speedup is 1.00
rsync error: some files could not be transferred (code 23) at /SourceCache/rsync/rsync-24/rsync/main.c(717)
command failed!
OS: Mac System 9.x → Mac OS X
Seems like the failure is actually happening already when uploading to the dated dir, and then the command for copying to latest-trunk isn't even called.
The linux build finished first, and used the cltbld key to create the dated directory:

ssh -2 -l cltbld stage.mozilla.org mkdir -p /home/ftp/pub/seamonkey/nightly/2007-06-03-01-trunk /home/ftp/pub/seamonkey/nightly/latest-trunk

rsync -av -e ssh -2 /builds/tinderbox/SeaMonkey-Trunk/Linux_2.6.9-42.ELsmp_Depend/2007060301/packages/ cltbld@stage.mozilla.org:/home/ftp/pub/seamonkey/nightly/2007-06-03-01-trunk/
building file list ... done
./
seamonkey-2.0a1pre.en-US.linux-i686.tar.bz2

which leaves the directory as
drwxrwxr-x     2 cltbld seamonkey  4096 Jun  3 03:19 2007-06-03-01-trunk


With those permissions, I wonder why that failure happens, as seabld is a member of the seamonkey group, from what I see in /etc/group and that group has write permissions on the directory.

Interestingly, both sea-win32-tbox and cb-sea-linux-tbox, which upload as cltbld, have the same problem when copying over the files from the dated directory to latest-trunk which has
drwxrwsr-x  2     1038 seamonkey 4096 May 30 09:40 latest-trunk

In this case, exiting after the error is no problem, as copying to latest is the last thing supposed to happen there, and the coping itself succeeds. In the case of the Mac box, exiting after the error means that subsequent correction of permissions and copying to latest does not happen on those builds.

Apparently the group having write perm is not enough there to be able to set times if the directory is owned by a different user.
Zach, any ideas why this is happening ?

I've set the owner of latest-trunk to kairo (to match the other two latest dirs) but I don't expect that to help.
It's now after 1 am, FRI 15 JUN 2007 Mozilla Local Time (PT).
Do you know where your SeaMonkey trunk Mac builds are?

I just checked:
http://ftp.mozilla.org/pub/mozilla.org/seamonkey/nightly/latest-trunk/
[   ] seamonkey-2.0a1pre.en-US.mac.dmg             12-Jun-2007 19:27   22M

http://ftp.mozilla.org/pub/mozilla.org/seamonkey/nightly/2007-06-13-01-trunk/
[   ] seamonkey-2.0a1pre.en-US.mac.dmg            13-Jun-2007 03:16   22M

http://ftp.mozilla.org/pub/mozilla.org/seamonkey/nightly/2007-06-14-01-trunk/
[   ] seamonkey-2.0a1pre.en-US.mac.dmg             14-Jun-2007 03:24   23M


That's progress, but it snagged on something again for the 13th & 14th.


Thanks much for the hard work,
Eddie

The build from 12th is an unrelated build from a different machine. The failure on cg-xserve02 has not changed.
Kairo: what user account on stage does cg-xserve02 upload as? Is it kairo or seabld? 
(In reply to comment #6)
> The build from 12th is an unrelated build from a different machine. The failure
> on cg-xserve02 has not changed.

Yes, those other builds are my talkback experiments. My apologies, I should have shunted them to experimental/ or some such. 


Bug 383775 'accidentally' fixed this by using scp instead of rsync for the upload.

zach: cg-xserve02 is using seabld.
Severity: normal → trivial
Priority: -- → P3
I don't think this is fixed, nagios is still complaining and the dir listing looks like:

[   ] seamonkey-2.0a1pre.en-US.linux-i686.tar.bz2  25-Jun-2007 02:45   14M  
[   ] seamonkey-2.0a1pre.en-US.mac.dmg             20-Jun-2007 07:54   22M  
[   ] seamonkey-2.0a1pre.en-US.win32.installer.exe 25-Jun-2007 03:13  8.8M  
[   ] seamonkey-2.0a1pre.en-US.win32.zip           25-Jun-2007 03:14   12M 

Over to the FTP maestro's
Assignee: build → zach
Severity: trivial → normal
Component: Tinderbox Configuration → FTP: Staging
QA Contact: ccooper → preed
Very interesting... Do we have log output to know where it's snagging now?

Assuming that it's failing on the "setting times" stage again, can we try using scp without the -p switch so that it won't try to do this. 

I just looked at the most recent dated dir with a mac trunk nightly: 

-rw-rw-rw-    1 seabld seabld    23639988 Jun 25 03:20 seamonkey-2.0a1pre.en-US.mac.dmg

666 permissions are obviously unsafe, and everything else is 775. Maybe we should try that in the hope that it makes the problem go away? Not entirely sure why the execute bit would be needed, but it's probably worth a shot.
Severity: normal → major
So I'm pretty stumped here. The command that is allegedly failing is:

scp -r -p -oProtocol=2 /builds/tinderbox/SeaMonkey-Trunk/Darwin_8.8.1_Depend/2007070201/packages/. seabld@stage.mozilla.org:/home/ftp/pub/seamonkey/nightly/2007-07-02-01-trunk/

I don't have access to the build machine, but I can't see a reason why that's failing. The tinderbox client is unhelpful enough to not provide the actual error message (if there is one). Can someone with access to the build machine try manually running an scp to see if we can get a real error message?
The error happens on the server side in any case, from what I see.
scp actually does copy the file up to the server, but apparently fails when setting permissions, times, etc. on it and comes back with a non-zero return code which triggers the "command failed!" warning on tinderbox and stops the script, prohibiting execution of the remote rsync command to copy from the dated to the latest dir.
Even the Linux box fails, but only when doing this remote rsync, with practically the same error messages as in comment #0 ("failed to set times ...") - because of this happening at the last command it should execute, it doesn't do any harm, but still the same error is seen.
Invalid nowadays because we're not using those tinderboc client setups any more and we didn't see this problem with the buildbots ever.
Status: NEW → RESOLVED
Closed: 15 years ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.