1531499 - windows/aarch64 - reftest suites report failure despite success

Status: RESOLVED FIXED

(Testing :: Reftest, defect, P5)

Description

[Treeherder Bug Filer]

#[markdown(off)]
Filed by: egao [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=230610810&repo=try

https://queue.taskcluster.net/v1/task/NtzwAeGnRFuXQsfvB2mgiw/runs/0/artifacts/public/logs/live_backing.log

https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/NtzwAeGnRFuXQsfvB2mgiw/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

Ru1 and R1 (and possibly others) have TBPL report failures despite having nothing go wrong in the test.

This is another workaround that builds on top of 1526777.

Comment 1

[Edwin Takahashi (:egao | infrequent contributor)]

Try: https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&revision=fa5874b5fb428c10606fc315c3e5443e14c82b33

Looks like windows10-aarch64/opt-reftest-no-accel-1 is doing fine with this workaround.

Flake8 failure has been addressed, should not affect the outcome of the try run.

For comparison, in this try run the windows10-aarch64/opt-reftest-no-accel-1 is reporting a failure despite nothing going wrong.

Comment 2

[Edwin Takahashi (:egao | infrequent contributor)]

Attachment: Bug 1531499 - additional workaround for Windows 32bit incorrectly reporting failure for reftest

• modified the criteria for using [0, 1] as the successful return code

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/bd930453714e
additional workaround for Windows 32bit incorrectly reporting failure for reftest r=jmaher,ahal

Comment 4

[Razvan Maries]

https://hg.mozilla.org/mozilla-central/rev/bd930453714e

Comment 5

[Intermittent Failures Robot]

1 failures in 2981 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1

Platform breakdown:

• windows10-aarch64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1531499&startday=2019-02-25&endday=2019-03-03&tree=all