81087 - evaluate Intel's Linux C++ compiler [ICC]

Status: RESOLVED INCOMPLETE

(Core Graveyard :: Tracking, defect)

Description

[Cathleen]

Intel just released their Linux C compiler beta today.

http://www.intel.com/software/products/compilers/linuxbeta.htm

We need to try it out and see if this compiler can help us optimize our builds.


netscape developers: get your licence bank name and password here:
http://blues/users/cathleen/publish/performance/intelbeta.html
be sure you have STATIC IP for your linux box.

Comment 1

[jim_nance]

Some notes on my experience:

- Shaver needs to port xptcall :-)
- The compiler does NOT work with RH 7.1 headers.  RH 6.2 works.
- The resulting binary crashed during startup.  I hope this is due to xptcall.

Comment 2

[jim_nance]

Attachment: quick hack to configure scripts to get things working

Comment 3

[jim_nance]

Attachment: script I use to set up the environment

Comment 4

[jim_nance]

I have attached a diff that shows what I had to do to get configure working.  I
just hacked up the configure scripts as opposed to doing it the right way and
changing configure.in.  Also, when I was trying to remove -pipe, I removed every
one I could find.  I did not check to see which ones affected Linux and which
ones did not.

I also attached the script I used to run configure.  Its nothing fancy, but it
does show the configure options and what the name of the compiler is.

Finally you will also have to edit the Makefile for xptcall.  I dont remember
the directory path, but your build will stop in it so you can find it.  The
makefile has a generic target at the top, and its overridden a few lines down
once make determines that you are running on an x86 system.  Get rid of the line
that does the override and use the generic target.

Comment 5

[jim_nance]

I contacted intel tech support to inquire about help writting the xpt code.  
Lets see what happens.

Comment 6

[Cathleen]

posting from newsgroup...
=========================

Andi Kleen <[e-mail]> wrote:

  The Intel compiler supports Visual C++ style inline assembly with the
  right option. This would allow to compile the MMX JPEG code that is in 
  the mozilla jpeg library on Linux (currently it's only used on Windows I 
  think).  The MMX optimized JPEG functions are much faster, although they
  have lower quality.

  So even if it's not usable for most of mozilla it would be at least a nice
  option for mozilla/jpeg/*

  -Andi

Comment 7

[James Green]

Well either I'm missing something completely or this bug really shouldn't be
OS/Windows2K! Switching to Linux...

Comment 8

[Cormac F]

Adding meta keyword. Sorry for the spam!

Comment 9

[Olivier Cahagne]

Any progress with final release ?
http://developer.intel.com/software/products/compilers/c50/linux/

It would be interesting to see perf improvement with another compiler than gcc
on Linux/x86 platforms. See bench comparing g++ 3.0.1/Intel C++ 5.0.1 on Linux:
http://coyotegulch.com/hpc/intel_gcc_bench1.html

Has Kai's C++ compiler been also evaluated ?
http://www.kai.com/C_plus_plus/download/index.html#Linux

I can't find any bug report, but, anyway, Intel acquired Kai so they'll
eventually merge both compilers.

Comment 10

[Heikki Toivonen (remove -bugzilla when emailing directly)]

This compiler is actually available on Windows as well. There are evaluation
copies available at http://developer.intel.com/software/products/compilers/

Also, Openbench labs did some benchmarking (see full article at
http://www.open-mag.com/754088105111.htm ):

"KEY FINDINGS
? On Linux, numerically intense CPU performance improved by 47% compared to GNU
C v2.95
? On Windows, numerically intense CPU performance improved by 37% compared to MS
Visual Studio.
? CPU performance for Intel on Linux and MS Visual Studio.NET on Windows in a
dead heat.
? AMD Athon-based systems benefited equally with Pentium III-based systems."

Comment 11

[Jan Varga [:janv]]

I tried the latest icc 5.0.1 version on Linux and I think it is not ready for
prime time yet (or I should go RTFM ?). It uses different options (e.g. -fPIC =
-KPIC). I was not able to run mozilla configure even etc.
I tried to compile CVS version of PostgreSQL. This compiled just fine (they use
only plain C), but final linking died in a infinitive loop.

Comment 12

[Jan Varga [:janv]]

hmm, now it compiles, but it produces even more bloated code than gcc

Comment 13

[Jan Varga [:janv]]

I give up :(

Comment 14

[Andreas Lennartsson]

There is a 6.0 beta version of the Intel compiler for Linux.

I tried it and got the following error:

icc -shared -Wl,-soname -Wl,libnspr4.so -o libnspr4.so ./prvrsion.o
io/./prfdcach.o io/./prmwait.o io/./prmapopt.o io/./priometh.o io/./pripv6.o
io/./prlayer.o io/./prlog.o io/./prmmap.o io/./prpolevt.o io/./prprf.o
io/./prscanf.o io/./prstdio.o threads/./prcmon.o threads/./prrwlock.o
threads/./prtpd.o linking/./prlink.o malloc/./prmem.o md/./prosdep.o
memory/./prshm.o memory/./prshma.o memory/./prseg.o misc/./pralarm.o
misc/./pratom.o misc/./prcountr.o misc/./prdtoa.o misc/./prenv.o misc/./prerr.o
misc/./prerror.o misc/./prerrortable.o misc/./prinit.o misc/./prinrval.o
misc/./pripc.o misc/./prlog2.o misc/./prlong.o misc/./prnetdb.o misc/./prolock.o
misc/./prrng.o misc/./prsystem.o misc/./prthinfo.o misc/./prtpool.o
misc/./prtrace.o misc/./prtime.o malloc/./prmalloc.o pthreads/./ptsynch.o
pthreads/./ptio.o pthreads/./ptthread.o pthreads/./ptmisc.o md/unix/./unix.o
md/unix/./unix_errors.o md/unix/./uxproces.o md/unix/./uxrng.o md/unix/./uxshm.o
md/unix/./uxwrap.o md/unix/./linux.o md/unix/./os_Linux_x86.o -lpthread -ldl -lc
icc: NOTE: The evaluation period for this product ends on 15-may-2002 UTC.
ld: libnspr4.so: undefined versioned symbol name __register_frame_info@@GLIBC_2.0
ld: failed to set dynamic section sizes: Bad value
gmake[4]: *** [libnspr4.so] Error 1
gmake[4]: Leaving directory `/home/speede/dev/mozilla/nsprpub/pr/src'
gmake[3]: *** [export] Error 2
gmake[3]: Leaving directory `/home/speede/dev/mozilla/nsprpub/pr'
gmake[2]: *** [export] Error 2
gmake[2]: Leaving directory `/home/speede/dev/mozilla/nsprpub'
gmake[1]: *** [nspr] Error 2
gmake[1]: Leaving directory `/home/speede/dev/mozilla'
gmake: *** [default] Error 2

Not sure if this is any diffrent from 5.01.
Don't know enough about mozilla to fix it :-(

Comment 15

[Jesse Ruderman]

Have you tried contacting Intel for help?  I bet they'd love to be able to
advertise their compiler using such a well-known large project.  The numbers for
"numerically intense" calculations are nice, but "Mozilla used to be 5% slower
on Linux than Windows and is now 5% faster on Linux" would be better to
advertise.  Especially if they had an article describing how hard it was to get
Mozilla to compile with Intel, what kinds of re-optimizations were needed,
whether stability problems were introduced, etc.

Comment 16

[Mike Shaver (:shaver emeritus)]

Great idea, Jesse.  Let us know what you hear back. =)

Comment 17

[[not reading bugmail]]

There was a hugh keynote about compilers, making threading apps aware, vtune,
gcc, Visual Studio, with hyper-threading tools, threading debuggers, etc.. at
the latest IDF Spring '02..

Richard Wirt talks about hyper-threading, and all this stuff..

As taken from the intel keynote page:

Richard Wirt will explore exciting new trends in software development, providing
a preview of the newest applications for Web services and an inside look at what
developers need to know to design software for Hyper-Threading and the entire
threading spectrum. He will also outline the new features and capabilities of
the Intel Software Development Products for enabling Web services and threading
in application creation this year and into the future.

http://www.intel.com/idf/us/spr2002/conf_info/webcast.htm#richard
ftp://download.intel.com/pressroom/kits/events/idfspr_2002/wirt_presentation.pdf
http://www.intel.com/pressroom/archive/speeches/wirt20020225.htm


They got consultants all over the world doing this and that on various projects,
I think that they would help if the Mozilla community asked them..

-dennis 

Comment 18

[Noell Damron]

While trying to run the ./configure, the Intel 6.0 C++ compiler I received
the following error. 

./conftest: error while loading shared libraries: libcxa.so.1: cannot open
shared object file: No such file or directory.

Has anyone else experienced this problem?

Comment 19

[Steve Snyder]

Another data point.  Configuration of Moz v1.1 completes, but build stops with
this error:

ld: libnspr4.so: undefined versioned symbol name __register_frame_info@@GLIBC_2.0
ld: failed to set dynamic section sizes: Bad value
make[4]: *** [libnspr4.so] Error 1

Software in use:
  RedHat Linux v7.3 /w kernel 2.4.19
  Intel ICC v7.0, Build 20021021Z
  Mozilla v1.1 (from mozilla-1.1-0.src.rpm) 

Environment:
  export CC=icc
  export CXX=icpc

FYI.

Comment 20

[Olivier Cahagne]

interesting mail (Oct.2002) about Intel people submitting patches to Linus to
compile Linux kernel with their compiler:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/6450.html
Should we contact this person directly ?

Comment 21

[Sebastian Späth]

Today I tried compiling with INtels 7.0 and get a compile error. But it looks
more like my libs are not ok? Any clues?

/opt/intel/compiler70/ia32/bin/icpc -I/usr/X11R6/include -pthread  -DNDEBUG
-DTRIMMED -O2 -o libmozz.so  adler32.o crc32.o compress.o uncompr.o deflate.o
trees.o gzio.o zutil.o inflate.o infblock.o inftrees.o infcodes.o infutil.o
inffast.o        -ldl -lm    
/usr/lib/crt1.o: In function `_start':
/usr/lib/crt1.o(.text+0x18): undefined reference to `main'
make[3]: *** [libmozz.so] Error 1
make[3]: Leaving directory `/home/mozbuild/mozilla/modules/zlib/src'

Comment 22

[Olivier Cahagne]

using Intel compiler v.7.1, it's getting there, it now fails here:

[...]
vr_stubs.c
Building deps for vr_stubs.c
/opt/intel/compiler70/ia32/bin/icc -o vr_stubs.o
[...]
IG_H_ -DMOZILLA_CLIENT vr_stubs.c
/usr/include/bits/stat.h(70): error: incomplete type is not allowed
      struct timespec st_atim;          /* Time of last access.  */

and even if I mention the Intel headers before any other -I... in the Makefile,
it still doesn't compile:
/opt/intel/compiler70/ia32/substitute_headers/sys has a stat.h

It seems to be very close to compile, any idea how to tell Mozilla to look at
the substitute headers first ? Would ac_add_options --oldincludedir= work ?

Comment 23

[Michiel van Leeuwen (email: mvl+moz@)]

I got a working build with icc version 7.1 Build 20030307Z. (after some tweaks
to header files, and mozilla)
The results were quite bad:
Codesize increased from 21182845 to 51140467 bytes (yes, 2.5 times as big)
Txul and Ts both increased a few percent.
So, mozilla got bigger and slower.

If anyone wants the details of my changes, or more detailed test/codesighs
results, please let me know.

Comment 24

[Michiel van Leeuwen (email: mvl+moz@)]

Attachment: patch to mozilla

This are the changes i made to mozilla. Yes, they are ugly. Real ugly.
The added #includes are likely not needed with the changes I made to the
headers, or by not installing the substitute headers. I will test that soon.

Comment 25

[Olivier Cahagne]

it may be worth beta testing icc 8.0 beta as it brings more gcc compatibility
features:
http://www.ciemat.es/informatica/gsc/icpl8/C++ReleaseNotes.htm
The beta page mentions it's to be released soon:
http://www.intel.com/software/products/compilers/beta/register.htm

Comment 26

[Michiel van Leeuwen (email: mvl+moz@)]

Attachment: mozconfig

Attached the mozconfig file i used. I didn't use -g or debug, so that is not
where the size went.
The results of codesighs are at http://yellow.exedo.nl/~michiel/summary001.txt,
maybe it means anything to anyone :) (the file is about 15MB, be warned)

Comment 27

[Michiel van Leeuwen (email: mvl+moz@)]

(in all those comments, I forgot to mention I tested on debian linux, compared
speed tests with a gcc 3.3.2 build)

After uninstalling the substitute headers package, the build works without
tweaking any system/icc files. Also, the #include changes in the patch can be
removed. It is still slower and bigger, no change there. (would have suprised me
anyway :) )

Comment 28

[Olivier Cahagne]

icc 8.0.055 (first non-beta v.8) has been released on 8/12.

Comment 29

[Jack]

Intel C Compiler 8.0.055 came out four months ago?!  That's odd... I thought it 
was still in beta until this month.  In any case, that's good news for Mozilla, 
as it should make it easier to compile with ICC.

Comment 30

[Michiel van Leeuwen (email: mvl+moz@)]

oh, the joy of us vs european dates.

Comment 31

[Ali Ebrahim]

Updating summary to reflect that the Intel compiler is available on Windows and
Linux.

Also, how is this related to bug 222442?

Comment 32

[Michiel van Leeuwen (email: mvl+moz@)]

Reverting back, because the results on linux and windows can greatly differ. So
far, this bug is only linux, so lets keep it readable.

Comment 33

[Peter Kovář]

Catastrophic error: could not open source file "../../../../mozilla-config.h"

Excerpt from command line:
  -I/usr/X11R6.7.0/include -include ../../../../mozilla-config.h 

ICC Version 8.0   Build 20031016Z doesn't recognize -include in C Preprocessor
stage.

Comment 34

[cls]

*** Bug 294951 has been marked as a duplicate of this bug. ***

Comment 35

[Pu7o]

To note that it's currently possible to build mozilla with Intel's compiler, but it crashes on run when -O0 isn't used. (Optimization flags can be added after the -O0 though)

Comment 36

[Pu7o]

Attachment: mozconfig that works when building with icc

Comment 37

[Simon Strandman]

Hello

I'm trying to build Firefox with intel icc 10.0 on Mac OS X 10.4.10. (is this bug for linux only or icc in general btw?) Both latest trunkt and 2.0.0.4 fails with:

rm -f libnspr4.a
/usr/bin/ar cr libnspr4.a ./prvrsion.o io/./prfdcach.o io/./prmwait.o io/./prmapopt.o io/./priometh.o io/./pripv6.o io/./prlayer.o io/./prlog.o io/./prmmap.o io/./prpolevt.o io/./prprf.o io/./prscanf.o io/./prstdio.o threads/./prcmon.o threads/./prrwlock.o threads/./prtpd.o linking/./prlink.o malloc/./prmem.o md/./prosdep.o memory/./prshm.o memory/./prshma.o memory/./prseg.o misc/./pralarm.o misc/./pratom.o misc/./prcountr.o misc/./prdtoa.o misc/./prenv.o misc/./prerr.o misc/./prerror.o misc/./prerrortable.o misc/./prinit.o misc/./prinrval.o misc/./pripc.o misc/./prlog2.o misc/./prlong.o misc/./prnetdb.o misc/./prolock.o misc/./prrng.o misc/./prsystem.o misc/./prthinfo.o misc/./prtpool.o misc/./prtrace.o misc/./prtime.o malloc/./prmalloc.o pthreads/./ptsynch.o pthreads/./ptio.o pthreads/./ptthread.o pthreads/./ptmisc.o md/unix/./unix.o md/unix/./unix_errors.o md/unix/./uxproces.o md/unix/./uxrng.o md/unix/./uxshm.o md/unix/./uxwrap.o md/unix/./darwin.o md/unix/./os_Darwin_x86.o 
ranlib libnspr4.a
rm -f libnspr4.dylib
icc -dynamiclib -compatibility_version 1 -current_version 1 -all_load -install_name @executable_path/libnspr4.dylib -headerpad_max_install_names -o libnspr4.dylib ./prvrsion.o io/./prfdcach.o io/./prmwait.o io/./prmapopt.o io/./priometh.o io/./pripv6.o io/./prlayer.o io/./prlog.o io/./prmmap.o io/./prpolevt.o io/./prprf.o io/./prscanf.o io/./prstdio.o threads/./prcmon.o threads/./prrwlock.o threads/./prtpd.o linking/./prlink.o malloc/./prmem.o md/./prosdep.o memory/./prshm.o memory/./prshma.o memory/./prseg.o misc/./pralarm.o misc/./pratom.o misc/./prcountr.o misc/./prdtoa.o misc/./prenv.o misc/./prerr.o misc/./prerror.o misc/./prerrortable.o misc/./prinit.o misc/./prinrval.o misc/./pripc.o misc/./prlog2.o misc/./prlong.o misc/./prnetdb.o misc/./prolock.o misc/./prrng.o misc/./prsystem.o misc/./prthinfo.o misc/./prtpool.o misc/./prtrace.o misc/./prtime.o malloc/./prmalloc.o pthreads/./ptsynch.o pthreads/./ptio.o pthreads/./ptthread.o pthreads/./ptmisc.o md/unix/./unix.o md/unix/./unix_errors.o md/unix/./uxproces.o md/unix/./uxrng.o md/unix/./uxshm.o md/unix/./uxwrap.o md/unix/./darwin.o md/unix/./os_Darwin_x86.o  -framework CoreServices -framework CoreFoundation
xild: error unable_to_open_argument_file1: unable to open argument file 'executable_path/libnspr4.dylib'
make[5]: *** [libnspr4.dylib] Error 1
make[4]: *** [export] Error 2
make[3]: *** [export] Error 2
make[2]: *** [tier_nspr] Error 2
make[1]: *** [default] Error 2
make: *** [build] Error 2

My mozconfig only sets CC and CXX to icc and icpc and sources the firefox mozconfig.

Any ideas?

Comment 38

[Simon Strandman]

Oh, sorry, I didn´t read the title. I guess I should file a new bug about the OSX error.

Comment 39

[Jesse Ruderman]

Simon filed bug 386840, "Firefox doesn't build with Intel C++ Compiler (ICC) on Mac".

Comment 40

[Eric Thibodeau]

Has this build approach been checked out lately? I am currently attempting a build of Firefox 2.0.0.10 under Gentoo with ICC 10.1 and the following environment set:
export CC="icc"
export CXX="icpc"
export CFLAGS="-O2 -xT -parallel -fomit-frame-pointer -w"
export CXXFLAGS=" -O2 -xT -parallel -fomit-frame-pointer -w"
export LD="xild"
export AR="xiar"

Attempts to launch the browser segfaults as such:
/usr/libexec/mozilla-launcher: line 119: 27652 Segmentation fault      $(type -P aoss) "$mozbin" "$@"

Using basic compiler flags (-O2 for example, which is the default for ICC) renders the same result)

Comment 41

[Takanori MATSUURA]

On my Fedora 8 + ICC 10.1, "-frtti" is required to CXXFLAGS.

And the build is failed with the following error.

icpc -o xptcinvoke_x86_64_linux.o -c  -DMOZILLA_INTERNAL_API -DOSTYPE=\"Linux2.6.23.8-63\" -DOSARCH=Linux -DEXPORT_XPTC_API -D_IMPL_NS_COM -I./../.. -I./../../../../xptinfo/src  -I../../../../../../dist/include/string -I../../../../../../dist/include   -I../../../../../../dist/include/xpcom -I../../../../../../dist/include/nspr        -fPIC   -fno-rtti -fno-handle-exceptions -Wall -Wconversion -Wpointer-arith -Wcast-align -Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor -Wno-long-long -pedantic -O2 -parallel -fomit-frame-pointer -frtti -w -fshort-wchar -pthread -pipe  -DNDEBUG -DTRIMMED -O   -DMOZILLA_CLIENT -include ../../../../../../mozilla-config.h -Wp,-MD,.deps/xptcinvoke_x86_64_linux.pp xptcinvoke_x86_64_linux.cpp
xptcinvoke_x86_64_linux.cpp(152) (col. 21): error: register xmm0's type does not match variable type "double"

xptcinvoke_x86_64_linux.cpp(153) (col. 21): error: register xmm1's type does not match variable type "double"

xptcinvoke_x86_64_linux.cpp(154) (col. 21): error: register xmm2's type does not match variable type "double"

xptcinvoke_x86_64_linux.cpp(155) (col. 21): error: register xmm3's type does not match variable type "double"

xptcinvoke_x86_64_linux.cpp(156) (col. 21): error: register xmm4's type does not match variable type "double"

xptcinvoke_x86_64_linux.cpp(157) (col. 21): error: register xmm5's type does not match variable type "double"

xptcinvoke_x86_64_linux.cpp(158) (col. 21): error: register xmm6's type does not match variable type "double"

xptcinvoke_x86_64_linux.cpp(159) (col. 21): error: register xmm7's type does not match variable type "double"

compilation aborted for xptcinvoke_x86_64_linux.cpp (code 2)

Comment 42

[Dan Mosedale (:dmosedale, :dmose)]

Reading your command-line, it now appears to have both -frtti and -fno-rtti.  That's unlikely to work.

Comment 43

[Ludovic Hirlimann [:Usul]]

Is someone still pursuing these investigations ?

Comment 44

[Nicholas Nethercote [inactive]]

(In reply to Ludovic Hirlimann [:Usul] from comment #43)
> Is someone still pursuing these investigations ?

Almost four years have passed. I think the answer is "no", and having this bug open isn't useful.