Open Bug 1564677 Opened 4 months ago Updated 4 months ago

Wrench reftest border/no-aa.yaml is failing on Android emulator after rustc bump to 1.36

Categories

(Core :: Graphics: WebRender, defect, P3)

70 Branch
x86
Android
defect

Tracking

()

People

(Reporter: cbrewster, Unassigned)

References

(Regression)

Details

(Keywords: leave-open, regression)

Attachments

(1 file)

After landing changes from Bug 1555483, the border/no-aa.yaml test started to fail on the Android emulator:
https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=244b23c635c55f567960042f1aef1fbab3861b7b

Even after backing out those changes, the test is still failing:
https://treeherder.mozilla.org/#/jobs?repo=autoland&selectedJob=255637378&revision=7756a16551203ca7f9214ac5132f2a25f9812f85

The plan is to temporarily disable this test on the Android emulator so autoland can be reopened.

Pushed by dluca@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/5df186a7c7fe
Disable border/no-aa.yaml on Android emulator
Keywords: leave-open
Priority: -- → P3

I'm doing some backfill to see what caused this. The jobs are scheduled to only run if gfx/wr is modified, but maybe there was an emulator or config change that landed at some point that broke the test and it wasn't picked up because the job didn't get scheduled until the filter change landed.

Narrowing the range: https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=android%2Cem%2Cwrench&fromchange=91ec3db845b6175e08d91b9318d96504caef6ce6&tochange=244b23c635c55f567960042f1aef1fbab3861b7b

Most likely it's the rust version bump that caused this. Which seems to imply that the rust code in WR is somehow doing something undefined?

https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=android%2Cem%2Cwrench&fromchange=6363111ee8ce85cef922919baff5b511171789e1&tochange=2786509371657776451adaf95de7139962318102&group_state=expanded

Yup, seems like the rustc version bump caused this. Gankro, as resident gfx team member with most rustc knowledge, any thoughts on why a rustc compiler bump would have caused this kind of a wrench reftest failure? FWIW the wrench build in question is an Android APK that has both armv7-linux-androideabi and i686-linux-android targets, and is running on a x86-64 android emulator (so using the i686-linux-android version of the library).

Flags: needinfo?(a.beingessner)
Regressed by: 1563378
Summary: Wrench reftest border/no-aa.yaml is failing on Android emulator → Wrench reftest border/no-aa.yaml is failing on Android emulator after rustc bump to 1.36
Has Regression Range: --- → yes
Keywords: regression
OS: Unspecified → Android
Hardware: Unspecified → x86
Version: unspecified → 70 Branch

The most notable changes in 1.36 are:

  • Non-lexical-lifetimes (NLL) are now on by default for all Rust 2015 crates (thus changing the borrow checker implementation for those crates)
  • HashMap has a brand new implementation (hashbrown)

Both of these have pretty significant testing behind them, so it would be a bit surprising if they messed us up. Especially since I upgraded the main WR crates to use NLL months ago. That change should also be completely unobservable to code, modulo of course miscompilation. One way to check if it was the NLL change would be to try to build with a 1.35 nightly and see if passing -Zborrowck=migrate to every crate in our build produces the same issue. This should mostly emulate the change in 1.36 (1.36 changed the default from ast to migrate, and also removed ast mode).

The hashbrown change is technically observable, in that it changes the bucketing strategy, and so may affect how elements are ordered. So there's a chance we were incidentally relying on the old ordering, which with our hasher setup is deterministic but very chaotic. So we certainly wouldn't be intentionally relying on the ordering. There's no toggle to turn that on or off, but it was in tree for rustc 1.36.0-nightly (e305df184 2019-04-24). So you can compare that nightly against one that's a couple days older (not certain if it made it into 04-23).

If neither of those bear fruit, we'll probably just have to bisect rust nightlies to find when it broke.

Flags: needinfo?(a.beingessner)

I already upgraded my local emulator to r29.0.11 and that seems to cause wrench to panic in tex_image_3d on this test. So unless I downgrade my emulator (which I'm not in a hurry to do) I don't think I can repro this locally which means bisecting the rust changes is not going to be easy. If somebody else has the older emulator installed (r28.0.25) they might be better positioned to look into this.

You need to log in before you can comment on or make changes to this bug.