b2g reftest fails to run using a real device 'list index out of range' if you forget a reftest path

REOPENED
Unassigned

Status

Testing
Reftest
REOPENED
3 years ago
3 years ago

People

(Reporter: BenWa, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Reporter)

Description

3 years ago
Created attachment 8561196 [details]
run log

I'm following the instructions here:
https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/Reftests#For_the_emulator

See my run log
Flags: needinfo?(ahalberstadt)
This is happening because you aren't passing in a tests path. You need to add e.g 'layout/reftests/reftest.list' to the end of the command line (yes crappy ux, but that's why we have mach :)).

That being said, I'm not sure when the last time someone got reftests working on devices was. Will be interesting to see how far you get. Also keep in mind that the tests assume a resolution of 800x1000, so if you do get it working, there may be false positives.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Flags: needinfo?(ahalberstadt)
Resolution: --- → INVALID
(Reporter)

Comment 2

3 years ago
Ahh ok, it seems to work better now except the device assume that their on the same network which I can't bridge easily ATM.

The error message is not acceptable IMO. We need either a better error message or do like other reftest drivers and run all the reftests.
Status: RESOLVED → REOPENED
Resolution: INVALID → ---
Summary: b2g reftest fails to run using a real device 'list index out of range' → b2g reftest fails to run using a real device 'list index out of range' if you forget a reftest path
You need to log in before you can comment on or make changes to this bug.