136123 - Failed to compile 0.9.9 when using noatime

Status: RESOLVED FIXED

(SeaMonkey :: Build Config, defect)

Description

[Denis Vlasenko]

Was getting perl errors in config/make-jars.pl:
===============================================
        unlink $destPath;       # in case we had a symlink on unix
        copy($file, $destPath) || die "copy($file, $destPath) failed: $!";

        # fix the mod date so we don't jar everything (is this faster than 
just jarring everything?)
        #vda: my $atime = stat($file)->atime || die $!;
        #vda: my $mtime = stat($file)->mtime || die $!;
        #vda: utime($atime, $mtime, $destPath);

        return 1;
    }
    return 0;
}
=========
Commented offending lines (those with #vda: above).
This helped.

Comment 1

[hacker formerly known as [e-mail]]

This looks like a dupe of bug 118947, which a fix was never found for.  Either
the perl installation is busted and doesn't have a working stat() or the
filesystem doesn't have atime and/or mtime support.  Which linux distribution
are you using?  What type of filesystem?  Which version of perl?

Comment 2

[Denis Vlasenko]

My system was originally a Slackware 8 installation.
Although I replaced lots of stuff (new kernel, GCC, glibc,
Samba, Apache, KDE etc),
I didn't mess up with perl:

# perl -v
This is perl, v5.6.1 built for i386-linux

I'll download newer source of perl, install it and report back.
fs is a plain old ext2.

Comment 3

[hacker formerly known as [e-mail]]

I'm using perl 5.6.1 w/o a problem as are most RH7.x users.  What kernel version
are you using?  I faintly recall a kernel option to disable atime in some
filesystems awhile ago.  I don't know if it made it into the mainstream kernel
though.  

Comment 4

[Denis Vlasenko]

>I faintly recall a kernel option to disable atime

Yes! This is it. I mount all my filesystems -o noatime.
But how can this be related? Build process should never pay
attention to atime. Only mtime/ctime is important, right?

Nevertheless, I'll enable atime update and report back.
(this means tomorrow, I'm playing with 0.9.9 at home, not here)

Comment 5

[hacker formerly known as [e-mail]]

It looks like atime is used as part of an optimization when building jarfiles. 
nsinstall also uses atime when copying a file so I'm surprised that you aren't
seeing other bustage. 

Comment 6

[hacker formerly known as [e-mail]]

Attachment: make noatime non-fatal

Use mtime as atime if stat($file)->{atime} is undefined. (not sure if "undef"
is the proper return value in this case.)

Comment 7

[Denis Vlasenko]

Erm.. what's nsinstall? :-) I just copied built files. They worked.

Re noatime: mount -o noatime does not kill atime, filesystem just
stops _updating_ it AFAIK. It certainly reports atime for files.
If perl does not handle this, it's a perl bug.

IMHO build system *must not* rely on atime. It's an *access time*,
what if I read some files from another tty as they are building?
You say that can affect build?! If yes, that's a build system bug.

Comment 8

[hacker formerly known as [e-mail]]

nsinstall is the in-tree version of install that does some neat tricks like
symlinking instead of copying a file to save space.  Like I said, it appears to
be an optimization step.  And by looking at the nsinstall code, it doesn't seem
to be an uncommon practice.  (Note that 2 different groups wrote nsinstall &
make-jars.pl with at least a good 3 years between the projects.)  

If you accessed these files while the script was running, you would cause an
extra copy.  From reading the code, I believe that's the extent of atime's
affect on the build system.  I'm not sure I would count that as a bug as atime
is a standard filesystem attribute afaik and only linux provides a way to turn
it off (or the updating of it).  Perl should return a valid value regardless. 
Can you try the sample perl script that I posted at
http://bugzilla.mozilla.org/show_bug.cgi?id=118947#c4 and see what perl actually
returns for that stat?  

Comment 9

[Denis Vlasenko]

my $atime = stat($file)->atime || die $!;
my $mtime = stat($file)->mtime || die $!;
utime($atime, $mtime, $destPath);

You are trying to set destpath's [am]time to the file's ones.
Can we omit atime entirely as a workaround?

#weird: my $atime = stat($file)->atime || die $! does not work on some systems:
#"Died at ../../config/make-jars.pl line 216, <STDIN> line 2"
#TODO: find out wtf. 
my $mtime = stat($file)->mtime || die $!;
utime(0, $mtime, $destPath);

For real fix we need some perl guru to delve into stat(...) internals -
how come it can leave ->atime undefined?

Comment 10

[Denis Vlasenko]

>Can you try the sample perl script that I posted...
It works fine on -o noatime mounted device.
I even tested it with new file created on that device
while in noatime mode... although my kernel is 2.5.7,
bug was observed on 2.4.16...

Comment 11

[hacker formerly known as [e-mail]]

So, in noatime mode using kernel 2.5.7, you're saying that you had no trouble
whatsoever running the mozilla build or just the test perl script?  If so, that
means that there was a bug in what kernel 2.4.16 returned when something asked
for atime.  The patch that I attached should work around that problem, can you
verify that it does?

No, I don't plan to omit atime completely as I see nothing to indicate that the
usage of atime is wrong or non-standard.  

Comment 12

[hacker formerly known as [e-mail]]

FWIW, I mounted my build partition using noatime and I was not able to reproduce
the problem.  I'm running RH's 2.4.9-31.

Comment 13

[Denis Vlasenko]

>you're saying that you had no trouble whatsoever running
>the mozilla build or just the test perl script?

Only test script. Have to compile 2.5.7 at home to test mozilla.

>I see nothing to indicate that the
>usage of atime is wrong or non-standard.

atime is standard, thats ok, but using atime in build process
is _logically_ wrong. Remember make? It compares *mtime* of
source and target files in order to decide which target needs
rebuilding! atime is _irrelevant_.

>FWIW, I mounted my build partition using noatime and I was
>not able to reproduce the problem.  I'm running RH's 2.4.9-31.

I tried that at home:
noatime mount + fresh configure+make, got an error.
normal mount + fresh configure+make, everything is ok.
(2.4.16 kernel, had no opportunity to try 2.5.7 -
was compiling KDE 3.0 at the same time and having some sleep)

Tried to look into perl sources (there are a handful of stat*
files) but since I don't know perl, I was scared by ugly syntax
and ran away. YMMV. :-)

Comment 14

[John G. Myers]

I suspect that the reported atime is 0 (the unix epoch), not undefined.  An
atime of 0 should be legal, but would cause perl to evaluate the right hand side
of the "||" operator.

Comment 15

[Brian Ryner (not reading)]

Comment on attachment 78315 [details] [diff] [review]
make noatime non-fatal

r=bryner

Comment 16

[hacker formerly known as [e-mail]]

Patch has been checked into the trunk.

Comment 17

[hacker formerly known as [e-mail]]

*** Bug 151800 has been marked as a duplicate of this bug. ***