1130926 - b2g reftest fails to run using a real device 'list index out of range' if you forget a reftest path

Status: RESOLVED INVALID

(Testing :: Reftest, defect)

Description

[Benoit Girard (:BenWa)]

Attachment: run log

I'm following the instructions here:
https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/Reftests#For_the_emulator

See my run log

Comment 1

[Andrew Halberstadt [:ahal]]

This is happening because you aren't passing in a tests path. You need to add e.g 'layout/reftests/reftest.list' to the end of the command line (yes crappy ux, but that's why we have mach :)).

That being said, I'm not sure when the last time someone got reftests working on devices was. Will be interesting to see how far you get. Also keep in mind that the tests assume a resolution of 800x1000, so if you do get it working, there may be false positives.

Comment 2

[Benoit Girard (:BenWa)]

Ahh ok, it seems to work better now except the device assume that their on the same network which I can't bridge easily ATM.

The error message is not acceptable IMO. We need either a better error message or do like other reftest drivers and run all the reftests.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

B2G is gone.