953211 - startup cache precompilation broken on sparc64 again

Status: RESOLVED WONTFIX

(Core :: JavaScript Engine, defect)

Description

[Landry Breuil (:gaston)]

Seems we have a comeback of bug 912168 - since the last uplift, m-a builds fine but fails to package - xpcshell crashing.

Traceback (most recent call last):
  File "/home/buildslave/mozilla-aurora-sparc64/build/toolkit/mozapps/installer/packager.py", line 375, in <module>
    main()
  File "/home/buildslave/mozilla-aurora-sparc64/build/toolkit/mozapps/installer/packager.py", line 367, in main
    args.source, gre_path, base)
  File "/home/buildslave/mozilla-aurora-sparc64/build/toolkit/mozapps/installer/packager.py", line 148, in precompile_cache
    errors.fatal('Error while running startup cache precompilation')
  File "/home/buildslave/mozilla-aurora-sparc64/build/python/mozbuild/mozpack/errors.py", line 101, in fatal
    self._handle(self.FATAL, msg)
  File "/home/buildslave/mozilla-aurora-sparc64/build/python/mozbuild/mozpack/errors.py", line 96, in _handle
    raise ErrorMessage(msg)
mozpack.errors.ErrorMessage: Error: Error while running startup cache precompilation

See http://buildbot.rhaalovely.net/builders/mozilla-aurora-sparc64 - i resumed my aurora builder not so long ago, but m-c was fine iirc before the last uplift when 28 moved to aurora (looking a bit closer in that timeframe, m-c / 1426ffa9c was fine on the 5/12, after that it broke for various build issues)

Comment 1

[Steve Singer (:stevensn)]

This might be caused by the issue fixed in 950513 (or did you already have that patch applied)

Comment 2

[Landry Breuil (:gaston)]

Hmm, looks maybe related - trying https://hg.mozilla.org/releases/mozilla-aurora/rev/d56d13a32786 on my sparc64 aurora builder

Comment 3

[Landry Breuil (:gaston)]

Strangely, at first i thought https://hg.mozilla.org/releases/mozilla-aurora/rev/d56d13a32786 didnt fix things, but now that things have settled and both central and aurora build unpatched on sparc64, it seems packaging also succeeds on aurora! Cf http://buildbot.rhaalovely.net/builders/mozilla-aurora-sparc64/builds/292

So that bug might finally got fixed by 950513.. or something else. Will let it open for some days to ensure things dont subtly break again.

Comment 4

[John Paul Adrian Glaubitz]

Hello!

Has there been any news on this? This still seems to be broken.

I have fixed up Firefox on Debian/sparc64 (there are some obvious bugs - more on that later), but it will still fail at this particular point:

Executing /<<PKGBUILDDIR>>/build-browser/dist/bin/xpcshell -g /<<PKGBUILDDIR>>/build-browser/dist/bin/ -a /<<PKGBUILDDIR>>/build-browser/dist/bin/ -f /<<PKGBUILDDIR>>/toolkit/mozapps/installer/precompile_cache.js -e precompile_startupcache("resource://gre/");
Traceback (most recent call last):
  File "/<<PKGBUILDDIR>>/toolkit/mozapps/installer/packager.py", line 410, in <module>
    main()
  File "/<<PKGBUILDDIR>>/toolkit/mozapps/installer/packager.py", line 404, in main
    args.source, gre_path, base)
  File "/<<PKGBUILDDIR>>/toolkit/mozapps/installer/packager.py", line 161, in precompile_cache
    errors.fatal('Error while running startup cache precompilation')
  File "/<<PKGBUILDDIR>>/python/mozbuild/mozpack/errors.py", line 103, in fatal
    self._handle(self.FATAL, msg)
  File "/<<PKGBUILDDIR>>/python/mozbuild/mozpack/errors.py", line 98, in _handle
    raise ErrorMessage(msg)
mozpack.errors.ErrorMessage: Error: Error while running startup cache precompilation

The full build log can be found here:

> https://buildd.debian.org/status/fetch.php?pkg=firefox&arch=sparc64&ver=46.0.1-1&stamp=1463527916

If anyone is interested in fixing this bug but doesn't have access to a sparc64 porterbox, I can create accounts on a very fast SPARC-T5 server which has 192 GiB and 192 threads which allows building Firefox in a reasonable amount of time.

As for the bugs mentioned earlier, the Firefox source use the wrong compiler definitions when matching a sparc64 host. The code checks for __sparc64__ which is incorrect as gcc doesn't define that on sparc64 (and never did according to David Miller). The correct way to check for sparc64 is by checking that both __sparc__ and __arch64__ are defined. I recently fixed that in systemd and I'm working to patch other affected packages as well.

See this Debian bug report for more: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=824449

Thanks,
Adrian

Comment 5

[Martin Husemann]

All gcc versions on NetBSD always have defined __sparc64__:

gcc -dM -E - < /dev/null | egrep 'sparc|arch'
#define sparc 1
#define __sparc__ 1
#define __sparcv9 1
#define __sparc 1
#define __sparc64__ 1
#define __sparc_v9__ 1
#define __arch64__ 1

and I am typing this using Firefox 44.0.2 on a sparc64 machine, so I am sure it works (at least here, with tons of patches in pkgsrc of course)

Comment 6

[Martin Husemann]

A better way to phrase above builtins is: __saprc64__ is predefined on all BSDs, but not on SunOS or Linux. The canonical way (we use in pkgsrc) to test is __sparc__ && _LP64

Comment 7

[John Paul Adrian Glaubitz]

(In reply to Martin Husemann from comment #5)
> gcc -dM -E - < /dev/null | egrep 'sparc|arch'
> #define sparc 1
> #define __sparc__ 1
> #define __sparcv9 1
> #define __sparc 1
> #define __sparc64__ 1
> #define __sparc_v9__ 1
> #define __arch64__ 1

Yeah, this seems to be *BSD-specific [1], so I think upstream projects should not rely on __sparc64__ but on __sparc__ && __arch64__ which seems to be available on all operating systems.

> and I am typing this using Firefox 44.0.2 on a sparc64 machine, so I am sure
> it works (at least here, with tons of patches in pkgsrc of course)

This is awesome news! So, it seems someone fixed this particular bug in NetBSD but didn't share the patches upstream. I suspect the patches with *sparc* in the filename found here [2] should be enough. I will try this later today, after work. Might be a good idea to get these patches merged.

PS: Great to see that there are m68k patches, too. I'm working on Debian's m68k port, so I'm glad some work is done elsewhere, too.

Adrian

> [1] https://github.com/gcc-mirror/gcc/blob/master/gcc/config/sparc/freebsd.h
> [2] http://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/www/firefox/patches/