769906 - Clear out upload/stage confusion with candidates/ directory

Status: RESOLVED WONTFIX

(SeaMonkey :: Release Engineering, defect)

Description

[Robert Kaiser]

IRC transcript, some things might be unrelated but it contains both the request and what I did to solve it for the moment:


<Callek> ewong|*: bah repacks will fail, need to figure that out
<Callek> :/
<Callek> KaiRo: if you have a few spare minutes can you peek at why http://stage.mozilla.org/pub/mozilla.org/seamonkey/nightly/2.11b4-candidates/ is perm denied? (I note that stuff changed a bit due to Bug 725839 and we failed at first on upload because of that, then I manually created our /home/.../candidates dir -- so builds uploaded fine, but now its not letting us grab builds)
* Callek NEEDS to head back to sleep
<Callek> if you find a solution, even if its one that won't last until next release, ewong can probably fix us up for rest of release
<Callek> KaiRo: but whatever the solution is, if you find one/have time, please tell me :-)
<KaiRo> Callek: we are uploading as the seabld user, right? if so, I have no clue why it would not work, that dir is all owned by seabld
<KaiRo> and the Linux-based builds uploaded fine
<KaiRo> Windows as well
<KaiRo> only Mac ones are not there - and repacks
<KaiRo> oh, we're looking at *downloading* builds from there
<KaiRo> [[e-mail]3 ~]$ ls -l /pub/mozilla.org/seamonkey/nightly/2.11b4-candidates/
<KaiRo> insgesamt 4
<KaiRo> drwxr-xr-x 7 seabld seamonkey 4096 30. Jun 03:22 build1
<KaiRo> [[e-mail]3 ~]$ ls -l /pub/mozilla.org/seamonkey/nightly/2.11b3-candidates/
<KaiRo> insgesamt 4
<KaiRo> drwxr-sr-x 11 seabld seamonkey 4096 22. Jun 18:23 build1
<KaiRo> the setguid seems to be the only difference here
<KaiRo> now I need to be seabld to change that, let's see if I can still find my way there
<KaiRo> Callek: any idea what the mac* directories in stage:/home/seabld are good for? seems to be a 2.7.1 build lying around there
<KaiRo> Callek: packages/ in there might be obsolete as well nowadays
<KaiRo> lrwxrwxrwx  1 seabld seamonkey      52 Jun 30 00:25 2.11b4-candidates -> /home/ftp/pub/seamonkey/candidates/2.11b4-candidates
<KaiRo> umm, wait, why is this a symlink?
<KaiRo> that's probably the reason, symlinks are probably not allowed there
<KaiRo> what did we change that we suddenly wrongly upload to seamonkey/candidates/ instead of seamonkey/nightly/ ??? That's probably the actual error
<KaiRo> ewong|afk: any idea why/how we changed in our config to upload to candidates instead of nightly?
<KaiRo> [[e-mail]3 ~]$ cd /home/ftp/pub/seamonkey/
<KaiRo> [[e-mail]3 seamonkey]$ ls -l candidates/
<KaiRo> total 4
<KaiRo> drwxr-sr-x 3 seabld seamonkey 4096 Jun 30 00:25 2.11b4-candidates
<KaiRo> [[e-mail]3 seamonkey]$ rm nightly/2.11b4-candidates
<KaiRo> [[e-mail]3 seamonkey]$ mv candidates/2.11b4-candidates nightly/
<KaiRo> [[e-mail]3 seamonkey]$ rmdir candidates/
<KaiRo> [[e-mail]3 seamonkey]$ ln -s nightly candidates
<KaiRo> that's what I did for the moment - this means that 1) 2.11b4-candidates is accessible the same way as the others, 2) if anything thinks it wants to upload to candidates/ via ssh, the files should go into nightly/ instead where they can be accessed correctly
* KaiRo thinks he should file a bug on that

Comment 1

[Robert Kaiser]

We still need to find a permanent solution so that things work correctly without that symlink in place, but I think we should basically be good for the moment.

Comment 2

[Robert Kaiser]

ewong pointed me to https://hg.mozilla.org/build/tools/rev/9473d51af6cb and this seems to be really the cause of most of the problems we have there, I commented in bug 725839 as well.

Comment 3

[Robert Kaiser]

<Callek> KaiRo: it is in prod for Firefox http://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/14.0b10-candidates/
<Callek> http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/14.0b10-candidates/
<KaiRo> Callek: interesting, no failures there?
<Callek> apparantly I *don't* have the puppet access I need, but I filed https://bugzilla.mozilla.org/show_bug.cgi?id=769928
<Callek> KaiRo: yea, loading the pages is fine on Firefox side
<KaiRo> now that's interesting
<KaiRo> lrwxrwxrwx  1 seabld seamonkey      52 Jun 30 00:25 2.11b4-candidates -> /home/ftp/pub/seamonkey/candidates/2.11b4-candidates
<KaiRo> vs.
<KaiRo> lrwxrwxrwx 1 ffxbld firefox 32 28. Jun 05:19 /pub/mozilla.org/firefox/nightly/14.0b10-candidates -> ../candidates/14.0b10-candidates
<KaiRo> so the symlink that was created for us is different
<Callek> KaiRo: hrm, I suspect this is script-bustage entirely then
<KaiRo> Callek: should I try if things work if I remodel the symlink to match the one of Firefox?
<Callek> since this script was deployed _after_ the Firefox side
<Callek> KaiRo: sure if you want to give it a shot, go ahead
<KaiRo> ok, will do
* KaiRo goes through the jump->master->stage indirection to get ontpo stage as seabld
<KaiRo> http://stage.mozilla.org/pub/mozilla.org/seamonkey/nightly/2.11b4-candidates/ looks good now
<KaiRo> lrwxrwxrwx  1 seabld seamonkey      31 Jun 30 10:13 2.11b4-candidates -> ../candidates/2.11b4-candidates

Comment 4

[Edmund Wong (:ewong)]

This is no longer applicable.

While I don't remember the conversation, it's still awesome to read that, particularly when I seemed to have a clue as opposed to now. ;/

Thanks KaiRo and Callek! Appreciate the help.