Open Bug 1277578 Opened 8 years ago Updated 2 years ago

signaling unit tests produces large binaries

Categories

(Core :: WebRTC: Signaling, defect, P4)

Product:
Core
Component:
WebRTC: Signaling
Type:
defect

Tracking

()

People

(Reporter: jrmuizel, Unassigned)

References

Details

The tests in media/webrtc/signaling/test become 100MB binaries. This seems too large. Perhaps they are statically and dynamically linking against libxul?
They statically link in bits of Gecko:
https://dxr.mozilla.org/mozilla-central/rev/4d63dde701b47b8661ab7990f197b6b60e543839/media/webrtc/signaling/test/common.build#82

They're also not built on Windows (bug 862837). I think they should just be moved into xul-gtest nowadays.
I've complained about those tests a number of times, and said they should die or become gtests. I'm still waiting, but I'm not holding my breath...
Component: Build Config → WebRTC: Signaling
(In reply to Jeff Muizelaar [:jrmuizel] from comment #0)
> The tests in media/webrtc/signaling/test become 100MB binaries. This seems
> too large. Perhaps they are statically and dynamically linking against
> libxul?

On my local build on Mac I see them being ~45MB each. Which is still big. On which platform or where do you see them being over 100MB?
Also, what in-demand resource is this a problem for?
(In reply to Nils Ohlmeier [:drno] from comment #3)
> (In reply to Jeff Muizelaar [:jrmuizel] from comment #0)
> > The tests in media/webrtc/signaling/test become 100MB binaries. This seems
> > too large. Perhaps they are statically and dynamically linking against
> > libxul?
> 
> On my local build on Mac I see them being ~45MB each. Which is still big. On
> which platform or where do you see them being over 100MB?

Linux.

(In reply to Eric Rescorla (:ekr) from comment #4)
> Also, what in-demand resource is this a problem for?

My time. I noticed this when watching what was being built. These show up noticeably at the end.
Is this consuming build time, or are you just upset that you spent time noticing it?
build time
It would probably be helpful if you provided actual data here that they were consuming an undue amount of build time, given that, you know, they test stuff.
(In reply to Eric Rescorla (:ekr) from comment #8)
> It would probably be helpful if you provided actual data here that they were
> consuming an undue amount of build time, given that, you know, they test
> stuff.

It's not that much time. It's just they come at the end of the build so they are more noticeable. No one needs to drop anything to get this done, it would just be nice to lean things up.
There are other benefits than build time. That setup is fragile and error-prone.
(In reply to Mike Hommey [:glandium] from comment #10)
> There are other benefits than build time. That setup is fragile and
> error-prone.

Well, I'd certainly be willing to look at a patch that moved it into gtests and evaluate it in terms of convenience against the current setup.
FWIW, the reason they're huge on Linux and not elsewhere is that on Windows and OS X the linker doesn't link debug info into the final binary. They're huge because DWARF is huge. We could mitigate this problem by enabling -gsplit-dwarf when available (bug 905646). We haven't done that because it breaks some other tooling (like ccache).
Rank: 31
Priority: -- → P3
See Also: → 905646
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #12)
> FWIW, the reason they're huge on Linux and not elsewhere is that on Windows
> and OS X the linker doesn't link debug info into the final binary. They're
> huge because DWARF is huge. We could mitigate this problem by enabling
> -gsplit-dwarf when available (bug 905646). We haven't done that because it
> breaks some other tooling (like ccache).

ccache now handles -gsplit-dwarf since ccache 3.2.3

From release note:
https://ccache.samba.org/releasenotes.html#_ccache_3_2_5

Release date: 2015-08-16
New features and improvements

    Added support for compiler option -gsplit-dwarf.

I have been using it locally for sometime now with a great success.
Actually, I created the patch to ccache to use -gsplit-dwarf :-)

Why doesn't mozilla infrastructure update to the latest ccache?

But I don't know other toolchain thingy that may be affected by -gsplit-dwarf under linux. 
FYI, Latest binutils, especially, gnu gold linker works very happily and efficiently with object files generated by -gsplit-dwarf.

TIA
(In reply to ISHIKAWA, Chiaki from comment #13)
> Why doesn't mozilla infrastructure update to the latest ccache?

The Mozilla infrastructure primarily uses sccache:
https://github.com/glandium/sccache

We could add a version check for ccache in configure and fix bug 905646 by only enabling -gsplit-dwarf when either ccache is not in use or a new-enough ccache is present.
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #14)
> (In reply to ISHIKAWA, Chiaki from comment #13)
> > Why doesn't mozilla infrastructure update to the latest ccache?
> 
> The Mozilla infrastructure primarily uses sccache:
> https://github.com/glandium/sccache
> 

I did not know that.
Hmm. Maybe someone can pick up the patch to enable -gsplit-dwarf for ccache and
figure out how to modify sccache?

> We could add a version check for ccache in configure and fix bug 905646 by
> only enabling -gsplit-dwarf when either ccache is not in use or a new-enough
> ccache is present.

Right.

TIA
(In reply to Jeff Muizelaar [:jrmuizel] from comment #0)
> The tests in media/webrtc/signaling/test become 100MB binaries. This seems
> too large. Perhaps they are statically and dynamically linking against
> libxul?

BTW, I have noticed myself that the build has become tad slower in the last couple of months (not sure when it started, but early May for sure), and found, yes, under linux the binaries are each 60MB or more.
(This is for debug build, and even with -gsplit-dwarf, these are rather slow to link.
(This is for thunderbird build, but m-c tree part is also created)

I ran env LANG=C find . -size +50M -ls
in the MOZOBJ directory, and took out the initial part of each line.
So the first number in the following lines is the size of the file.

 215575692 Jun 14 01:25 ./js/src/libjs_static.a
  65570606 Jun 14 01:25 ./js/src/jsapi-tests/jsapi-tests
  62762778 Jun 14 01:25 ./js/src/gdb/gdb-tests
  63091549 Jun 14 01:29 ./media/webrtc/signaling/test/jsep_session_unittest
  61087354 Jun 14 01:29 ./media/webrtc/signaling/test/sdp_file_parser
  61246917 Jun 14 01:29 ./media/webrtc/signaling/test/mediaconduit_unittests
  62920527 Jun 14 01:29 ./media/webrtc/signaling/test/sdp_unittests
  61364914 Jun 14 01:29 ./media/webrtc/signaling/test/mediapipeline_unittest
  61342429 Jun 14 01:29 ./media/webrtc/signaling/test/jsep_track_unittest
  62911327 Jun 14 01:29 ./media/webrtc/signaling/test/signaling_unittests
 443142763 Jun 14 02:37 ./toolkit/library/libxul.so

It certainly will be nice if these are dynamically linked, etc.

TIA
(In reply to ISHIKAWA, Chiaki from comment #15)
> I did not know that.
> Hmm. Maybe someone can pick up the patch to enable -gsplit-dwarf for ccache
> and
> figure out how to modify sccache?

sccache already supports -gsplit-dwarf, but I don't think we want to enable it for CI anyway because I'm not sure Breakpad handles it.
Mass change P3->P4 to align with new Mozilla triage process.
Priority: P3 → P4
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.