Open Bug 1315695 Opened 3 years ago Updated 2 years ago

build system doesn't cope with DST changes

Categories

(Firefox Build System :: General, defect)

Unspecified
Windows
defect
Not set

Tracking

(firefox52 wontfix)

ASSIGNED
Tracking Status
firefox52 --- wontfix

People

(Reporter: catlee, Assigned: gps)

Details

Attachments

(1 file)

Probably WONTFIX, but just in case this is a real bug...

We had an Aurora nightly build active during the DST change this weekend:
https://archive.mozilla.org/pub/firefox/nightly/2016/11/2016-11-06-00-40-04-mozilla-aurora/mozilla-aurora-win64-nightly-bm72-build1-build6.txt.gz

The change happened between these lines:
01:55:06     INFO -  mozmake.EXE[6]: Leaving directory 'c:/builds/moz2_slave/m-aurora-w64-ntly-000000000000/build/src/obj-firefox/js/src/shell'
01:12:42     INFO -  warning C4742: '`string'' has different alignment in 'c:\builds\moz2_slave\m-aurora-w64-ntly-000000000000\build\src\obj-firefox\media\webrtc\trunk\webrtc\base\base_rtc_base_approved\unified_cpp_trunk_webrtc_base0.cpp' and 'c:\builds\moz2_slave\m-aurora-w64-ntly-000000000000\build\src\xpcom\components\nscategorymanager.cpp': 1 and 2

Somehow this resulted in xul.dll being twice as large as normal (about 133MB vs 65MB), and busted updates (bug 1315530).

Is this a windows-ism where file system times are stored as timezone-less times, and so switching from DST to standard time would break something in the build system?
Some more context:

01:52:59     INFO -  Generating code
01:52:59     INFO -  Finished generating code
01:52:59     INFO -  touch -t `date +%Y%m%d%H%M.%S -d 'now+5seconds'` pgo.relink
01:52:59     INFO -  c:/builds/moz2_slave/m-aurora-w64-ntly-000000000000/build/src/obj-firefox/_virtualenv/Scripts/python.exe c:/builds/moz2_slave/m-aurora-w64-ntly-000000000000/build/src/config/nsinstall.py -t -m 755 'gdb-tests.exe' '../../../dist/bin'
01:52:59     INFO -  mozmake.EXE[6]: Leaving directory 'c:/builds/moz2_slave/m-aurora-w64-ntly-000000000000/build/src/obj-firefox/js/src/gdb'
01:53:19     INFO -  Generating code
01:53:19     INFO -  Finished generating code
01:53:19     INFO -  touch -t `date +%Y%m%d%H%M.%S -d 'now+5seconds'` pgo.relink
01:53:19     INFO -  c:/builds/moz2_slave/m-aurora-w64-ntly-000000000000/build/src/obj-firefox/_virtualenv/Scripts/python.exe c:/builds/moz2_slave/m-aurora-w64-ntly-000000000000/build/src/config/nsinstall.py -t -m 755 'jsapi-tests.exe' '../../../dist/bin'
01:53:19     INFO -  mozmake.EXE[6]: Leaving directory 'c:/builds/moz2_slave/m-aurora-w64-ntly-000000000000/build/src/obj-firefox/js/src/jsapi-tests'
01:55:05     INFO -  fallible.obj : MSIL .netmodule or module compiled with /GL found; restarting link with /LTCG; add /LTCG to the link command line to improve linker performance
01:55:05     INFO -  Generating code
01:55:05     INFO -  Finished generating code
01:55:05     INFO -  touch -t `date +%Y%m%d%H%M.%S -d 'now+5seconds'` pgo.relink
01:55:05     INFO -  c:/builds/moz2_slave/m-aurora-w64-ntly-000000000000/build/src/obj-firefox/_virtualenv/Scripts/python.exe c:/builds/moz2_slave/m-aurora-w64-ntly-000000000000/build/src/config/nsinstall.py -t -m 755 'js.exe' '../../../dist/bin'
01:55:06     INFO -  mozmake.EXE[6]: Leaving directory 'c:/builds/moz2_slave/m-aurora-w64-ntly-000000000000/build/src/obj-firefox/js/src/shell'
\moz2_slave\m-aurora-w64-ntly-000000000000\build\src\xpcom\components\nscategorymanager.cpp': 1 and 2
01:12:42     INFO -     Creating library xul.lib and object xul.exp
01:12:42     INFO -  Generating code
01:12:42     INFO -  Finished generating code
01:12:42     INFO -  test "$(dumpbin -exports xul.dll | grep _NSModule@@ | sort -k 3 | sed -n 's/^.*?\([^@]*\)@@.*$/\1/;1p;$p' | xargs echo)" != "start_kPStaticModules_NSModule end_kPStaticModules_NSModule" && echo "NSModules are not ordered appropriately" && exit 1 || exit 0
01:12:43     INFO -  touch -t `date +%Y%m%d%H%M.%S -d 'now+5seconds'` pgo.relink
01:12:43     INFO -  chmod +x xul.dll
01:12:43     INFO -  c:/builds/moz2_slave/m-aurora-w64-ntly-000000000000/build/src/obj-firefox/_virtualenv/Scripts/python.exe c:/builds/moz2_slave/m-aurora-w64-ntly-000000000000/build/src/config/nsinstall.py -t -m 644 'xul.dll' '../../dist/bin'
01:12:43     INFO -  c:/builds/moz2_slave/m-aurora-w64-ntly-000000000000/build/src/obj-firefox/_virtualenv/Scripts/python.exe c:/builds/moz2_slave/m-aurora-w64-ntly-000000000000/build/src/config/nsinstall.py -t -m 644 'xul.lib' '../../dist/sdk/lib'

DST change happened in the middle of PGO.

The "touch -t `date +%Y%m%d%H%M.%S -d 'now+5seconds'`" seem highly relevant, as it is using a timezone-less value. `man touch` says -t only accepts a "[[CC]YY]MMDDhhmm[.ss]" value, so it just plain doesn't handle timezones.

The fix here is likely to change that `touch` to some Python code that does math on UNIX time integers.

Since this can impact releases, I'll code up a fix.
Assignee: nobody → gps
Status: NEW → ASSIGNED
Well I did suggest once in the past, that in order to be less US centric and embrace our international contributors rather than using Mountain View time as Mozilla Standard time, we switch to using UTC for our servers and therefore become immune form all lame Daylight Saving (Summertime for you UK brethren) transitions.
(In reply to Bill Gianopoulos [:WG9s] from comment #3)
> Well I did suggest once in the past, that in order to be less US centric and
> embrace our international contributors rather than using Mountain View time
> as Mozilla Standard time, we switch to using UTC for our servers and
> therefore become immune form all lame Daylight Saving (Summertime for you UK
> brethren) transitions.

It is a best practice to set server timezones to UTC. Pretty much every new server I see spun up does this. These machines, however, have their configuration rooted from the "dark ages" when practically everything at Mozilla was set to California time. Since these machines are being phased out in favor of TaskCluster, it probably doesn't make sense to change them.
Comment on attachment 8808286 [details]
Bug 1315695 - Use Python to adjust pgo.relink mtime;

https://reviewboard.mozilla.org/r/91136/#review93500

This is terrible, but the whole pgo.relink thing is sorta terrible.
Attachment #8808286 - Flags: review?(ted) → review+
Pushed by gszorc@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/a77fb6a85fc8
Use Python to adjust pgo.relink mtime; r=ted
Backout by gszorc@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/530bbb41afc4
Backed out changeset a77fb6a85fc8 for breaking PGO
NSPR is a separate product, and Firefox should only use released snapshots of NSPR.

Please don't make local changes to the copy of NSPR in directory /nsprpub (except when you make a try build).

Instead, please suggest your changes to the NSPR component, and ask for a new NSPR snapshot to get uplifted.
Too late for firefox 52, mass-wontfix.
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.