957955 - Build system loops forever around "Reticulating splines..." if browser/base/content/test/general/browser.ini has a timestamp in the future (e.g. after adjusting your system clock)

Status: RESOLVED DUPLICATE of bug 949319

(Firefox Build System :: General, defect)

Description

[Daniel Holbert [:dholbert]]

STR:
Run this in your sourcedir:
  ./mach clobber   # probably not necessary; just for good measure
  touch  -t 201501010000 browser/base/content/test/general/browser.ini
  ./mach build

EXPECTED RESULTS: I should be able to build.
ACTUAL RESULTS: Once my build gets past all the initial "checking whether XYZ..." steps, it cycles forever like so:
{
 0:50.53 config.status: executing src commands
 0:50.67 make[2]: warning:  Clock skew detected.  Your build may be incomplete.
 0:50.71 TEST-PASS | check-sync-dirs.py | /scratch/work/builds/mozilla-inbound/mozilla/js/src/config <= /scratch/work/builds/mozilla-inbound/mozilla/config
 0:51.08 make[2]: Warning: File `/scratch/work/builds/mozilla-inbound/mozilla/browser/base/content/test/general/browser.ini' has modification time 3.1e+07 s in the future
 0:51.08 Build configuration changed. Regenerating backend.
 0:51.35 Reticulating splines...
 0:58.78 Finished reading 1148 moz.build files in 0.59s
 0:58.78 Processed into 5639 build config descriptors in 2.71s
 0:58.78 Backend executed in 3.89s
 0:58.78 2458 total backend files; 0 created; 0 updated; 2458 unchanged; 0 deleted; 365 -> 1039 Makefile
 0:58.78 Total wall time: 7.43s; CPU time: 7.41s; Efficiency: 100%; Untracked: 0.24s
 0:59.14 make[2]: Warning: File `/scratch/work/builds/mozilla-inbound/mozilla/browser/base/content/test/general/browser.ini' has modification time 3.1e+07 s in the future
 0:59.14 Build configuration changed. Regenerating backend.
 0:59.35 Reticulating splines...
 1:06.65 Finished reading 1148 moz.build files in 0.61s
 1:06.65 Processed into 5639 build config descriptors in 2.79s
 1:06.65 Backend executed in 3.65s
 1:06.65 2458 total backend files; 0 created; 0 updated; 2458 unchanged; 0 deleted; 365 -> 1039 Makefile
 1:06.65 Total wall time: 7.29s; CPU time: 7.24s; Efficiency: 99%; Untracked: 0.24s
 1:07.04 make[2]: Warning: File `/scratch/work/builds/mozilla-inbound/mozilla/browser/base/content/test/general/browser.ini' has modification time 3.1e+07 s in the future
 1:07.05 Build configuration changed. Regenerating backend.
 1:07.32 Reticulating splines...
}
(etc)

EXPECTED RESULTS: The build should either succeed (despite this odd timestamp), OR it should fail. It should *not* just cycle forever.

(I happened to hit this issue because my computer's clock occasionally gets reset to 8 hours in the future (not sure why).  After I noticed that this had happened today, I adjusted the clock back 8 hours (after having pulled and updated timestamps to be 8 hours in the future). And then I was unable to build, due to this issue.)

Comment 1

[Daniel Holbert [:dholbert]]

[er, sorry for the duplicate EXPECTED RESULTS. You get the idea. :)]

Comment 2

[Daniel Holbert [:dholbert]]

This is on up-to-date mozilla-inbound, cset aeef43ce8bdb, FWIW.

Comment 3

[Gregory Szorc [:gps]]

config.status should only run once during the build. Therefore, this is a duplicate of bug 949319.

Future timestamps will wreck havoc on mtime-based build systems (like make). I can only dream that our build system will one day use a "strong validator" (to borrow language from the HTTP spec) like file content instead. But that introduces its own set of challenges.

Comment 4

[Michael Shal [:mshal]]

> Future timestamps will wreck havoc on mtime-based build systems (like make).
> I can only dream that our build system will one day use a "strong validator"
> (to borrow language from the HTTP spec) like file content instead. But that
> introduces its own set of challenges.

I find that checking a file's timestamp against it's last known timestamp works sufficiently well, and avoids issues that a full file content check would impose (being slower, and blowing out your disk cache, for example). I don't think make supports this mode, though, since it would need a datastore of last known timestamps.

Comment 5

[Gregory Szorc [:gps]]

... or you use modern filesystems (like ZFS) that store the file hash in the file metadata and verify file integrity on read. Obtaining the SHA-1 of a file is thus non-proportional to file size. Unfortunately, modern filesystems are not widely deployed. Sad panda.

Comment 6

[Mike Hommey [:glandium]]

Modern filesystems don't store file checksums, they store block checksums. Either way, those checksums are not available to userspace, so there's nothing to gain from them existing.