372836 - Tinderbox changes to work with MSYS/MozillaBuild

Status: RESOLVED FIXED

(Webtools Graveyard :: Tinderbox, defect, P2)

Description

[Benjamin Smedberg]

Attachment: Fix tinderbox + MozillaBuild, rev. 1

These minor changes allow tinderbox to build with MSYS/MozillaBuild.

Comment 1

[Robert Helmer [:rhelmer]]

Comment on attachment 257522 [details] [diff] [review]
Fix tinderbox + MozillaBuild, rev. 1

Looks find, I am curious why these are needed though (see below)

>Index: build-seamonkey-util.pl
>===================================================================
>RCS file: /cvsroot/mozilla/tools/tinderbox/build-seamonkey-util.pl,v
>retrieving revision 1.351
>diff -u -8 -r1.351 build-seamonkey-util.pl
>--- build-seamonkey-util.pl	9 Feb 2007 02:19:18 -0000	1.351
>+++ build-seamonkey-util.pl	6 Mar 2007 16:30:26 -0000
>@@ -287,16 +287,19 @@
>     #$Settings::ObjDir = '';
>     my $build_type = $Settings::BuildDepend ? 'Depend' : 'Clobber';
>     my $host = ShortHostname();
>     chomp($Settings::OS, $os_ver, $Settings::CPU, $host);
> 
>     # Redirecting stderr to stdout works on *nix, winnt, but not on win98.
>     $Settings::TieStderr = '2>&1';
> 
>+    # Mingw uname -r returns 1.0.10(0.46/3/2) - remove the parens section
>+    $os_ver =~ s/\(.*\)//;
>+


Why is this important? Do the parens cause problems elsewhere?


>     if ($Settings::OS eq 'AIX') {
>         my $osAltVer = `uname -v`;
>         chomp($osAltVer);
>         $os_ver = "$osAltVer.$os_ver";
>     }
> 
>     $Settings::OS = 'BSD_OS' if $Settings::OS eq 'BSD/OS';
>     $Settings::OS = 'IRIX'   if $Settings::OS eq 'IRIX64';
>@@ -413,22 +416,16 @@
> 
>     # Assume this file lives in the base dir, this will
>     # avoid human error from setting this manually.
>     $Settings::BaseDir = get_system_cwd();
> 
>     my $topsrcdir = "$Settings::BaseDir/$Settings::DirName/mozilla";
>     $objdir = "$topsrcdir/${Settings::ObjDir}";
> 
>-    if (not -e $objdir) {
>-        # Not checking errors here, because it's too early to set $status and the 
>-        # build will fail anyway; failing loudly is better than failing silently.
>-        run_shell_command("mkdir -p $objdir");
>-    }
>-


It's not really clear to me from the CVS log why this was added.. shouldn't the build system create the objdir if it doesn't exist? Why does this cause a problem for MSYS?

Comment 2

[Benjamin Smedberg]

> Why is this important? Do the parens cause problems elsewhere?

Yes. This becomes the directory name, and both parens and slashes in the directory name cause major hiccups. Besides which it is ugly and unwieldy.

> It's not really clear to me from the CVS log why this was added.. shouldn't the
> build system create the objdir if it doesn't exist? Why does this cause a
> problem for MSYS?

The build system does create the objdir in every incantation I know of. The problem is with stock cvs 1.11 if you check out into a directory that already exists but doesn't have a CVS/ directory:

$ mkdir mozilla
$ cvs co mozilla/client.mk
cvs checkout: in directory mozilla:
cvs checkout: cannot open CVS/Entries for reading: No such file or directory.

Comment 3

[Benjamin Smedberg]

Fixed on trunk.

Comment 4

[J. Paul Reed [:preed]]

I'm gonna reopen this and reassign it to me.

I guess MSYS doesn't support symlinks, and we rely extensively on symlinks for things like tinderbox autoupdate and cvs-based configs.

I can fix this, but I'll have to tweak the tinderbox code to do it.

Comment 5

[Benjamin Smedberg]

Can you use hardlinks?

Comment 6

[(not currently active) Ted Mielczarek]

Attachment: fix MSYS sysinfo [checked in]

This should fix the goofy OS version on the tinderbox.

Comment 7

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

(In reply to comment #1)
> It's not really clear to me from the CVS log why this was added.. shouldn't the
> build system create the objdir if it doesn't exist? Why does this cause a
> problem for MSYS?

Last I checked (pretty recently, I think), the build system doesn't use mkdir -p to create the objdir, so it fails if the objdir's parent directory doesn't exist.

Comment 8

[J. Paul Reed [:preed]]

(In reply to comment #5)
> Can you use hardlinks?

According to the test I did, no. Not sure if this is because MSYS doesn't support linking at all, or because we're using FAT32 on this filesystem, for build performance reasons.

Comment 9

[J. Paul Reed [:preed]]

Attachment: How I'm going to hell... let me count the ways...

This is a patch (untested, BTW; I'm gonna do that while people are pointing and laughing at this patch...) to manually copy files into place in all the locations where we expected (haha, joke's on us!) that symlinks would work.

In MSYS, symlinks don't exist (because they don't exist in Win32... except in Vista, but there are a lot of issues with... well... nevermind...), and hardlinks only work on NTFS. We use FAT32 for performance reasons in these directories.

Comment 10

[J. Paul Reed [:preed]]

Comment on attachment 276034 [details] [diff] [review]
How I'm going to hell... let me count the ways...

Don't bother reviewing this patch... it's close, but I had to jump through a couple of extra hoops to get things working (oh, how I hate thee, post-mozilla.pl!)

Expect another patch tomorrow when I see the one I currently have stays running all night.

Comment 11

[J. Paul Reed [:preed]]

Attachment: How I'm going to hell, let me count the ways II

Just a few changes that I had to make to get this reliably working...

Comment 12

[J. Paul Reed [:preed]]

Attachment: More sinning...

This fixes the problem where the builds wouldn't run on the testing tinderboxen.

Comment 13

[Robert Helmer [:rhelmer]]

Comment on attachment 276570 [details] [diff] [review]
More sinning...

ugh :P

Comment 14

[Chris Cooper [:coop] (he/him)]

Comment on attachment 276529 [details] [diff] [review]
How I'm going to hell, let me count the ways II

I'd much prefer a situation where we didn't rely on having an existing local checkout already on each machine and simply checked the harness code out fresh each time. It's not that much code, and it avoids much of this hackery.

I also wish we didn't rely on the presence of a file (post-mozilla-rel.pl) to determine the release status for the current build instance. Is it the future yet?

I'm r+ing this because it's tinderbox and you're simply adding more crap to the pile that is tinderbox, which is really the only short-term way to proceed. ;-)

Comment 15

[J. Paul Reed [:preed]]

Landed both of these patches, painful and ouchy though they are, and stabby though they make me.