1388023 - Field count exceeds 65k limitation

Status: NEW

(Firefox Build System :: Android Studio and Gradle Integration, enhancement)

Description

[Jing-wei Wu [:jwu]]

The field count in Fennec exceeds 65k limitation if we want to enable Photon and use constraint layout at the same time.

The error message showed in try server say now we have 65879 field count.

```
trouble writing output: Too many field references: 65879; max is 65536.
```

Maybe it's time to consider to support multi-dex in mach build?

https://treeherder.mozilla.org/#/jobs?repo=try&revision=0caf909637d9eeb067eb069e9bc391799545e122&selectedJob=121351942

Comment 1

[Sebastian Kaspari (:sebastian; :pocmo)]

According to the build output the constraint layout library added about 4500 fields. And we recently added Exoplayer2 with 1500+  fields.

So what are our options now?

1) Backout ConstraintLayout again. I've started using it in ActivityStream but it's still possible to rewrite those layouts - they might not be as simple and performant though.

2) Teach out build system to use MultiDex (in gradle it's a one-liner). Not sure how easy that is and whether it's worth it before switching to gradle builds.

3) Get rid of about 300 fields from somewhere else.

---

I looked through the field count list but there's nothing obvious that could be removed and gives us about 300 fields back. I'm also not confident that removing some of the "Australis only" code will give us more than 300 fields back. In any case this would be only temporarily helpful. So I think (3) is not really an option.

Using MultiDex seems to be the only long-term solution (2) - but it might take some time and introduces risk.

I think our best option is to go with (1) and remove ConstraintLayout for now. This will buy us some time and we can look into other solutions. And more importantly landing the Photon work is unblocked. ActivityStream will have to land in 57 too - but we should be able to implement it without ConstraintLayout. And if we can get (2) working then we can migrate later.

Comment 2

[Sebastian Kaspari (:sebastian; :pocmo)]

@nalexander: If you have a minute - have you ever though about Multidex support? Do you think it would be hard to add to our build system?

Comment 3

[Sebastian Kaspari (:sebastian; :pocmo)]

Tomcat helped backing out ConstraintLayout over in bug 1380639.

Comment 4

[Richard Newman [:rnewman]]

Breadcrumbs: Bug 1286677, Bug 1285511. AIUI we already multidex in some scenarios…

Comment 5

[Nick Alexander :nalexander [he/him]]

(In reply to Sebastian Kaspari (:sebastian) from comment #2)
> @nalexander: If you have a minute - have you ever though about Multidex
> support? Do you think it would be hard to add to our build system?

I have.  I did not dig too deeply -- just enough to understand what Gradle is doing -- but I concluded that it's essentially impossible to add to our build system in any generality.  I think we could do it "manually", if we pushed some isolated feature (say, Sync) into a separate DEX file, and then carefully audited entry-points, etc.  Although, Sync requires lots of system integration for the Sync Adapter, so perhaps that's a bad example.  Maybe ExoPlayer itself, which is very stand-alone?

Even if we did it, it would be a huge waste of time and effort to bake it into moz.build/Makefile.in.  I'd much rather put that effort into just using Gradle.

Anyway, I think the effort of just using Gradle would be significantly less than the effort to engineer even partial multidex support; and the _testing_ for hand-written multidex across devices and versions would be greater than the testing required for Gradle builds.

Comment 6

[Nick Alexander :nalexander [he/him]]

mcomella said something that made me think.  He said: "(not urgent) liuche: fyi that ConstraintLayout + photon changes has us exceed field limit (somehow before method limit!): https://bugzilla.mozilla.org/show_bug.cgi?id=1388023"

I wonder if what's happening is that we're getting tons of R.* fields, since we run aapt only once and use --extra-packages.  What that means is that _every_ R.java has _every_ resource field for every resource of each of the libraries we use, and that (should) result in each R.java having thousands of fields -- most of which will not be accessed.

I don't know how difficult it would be to run aapt multiple times, once for each package, with the minimal set of resources for the package.  I don't think that's trivial since the aapt invocations still have to "know" about each other, to allocate the resource table correctly.  But it might be an approach here.