1114474 - Cannot launch reftest on b2g desktop/emulator locally | ImportError: cannot import name expected

Status: RESOLVED FIXED

(Testing :: Reftest, defect)

Description

[Ting-Yu Lin [:TYLin] (PST, UTC-8)]

The reftests failed to start on b2g desktop with the following command. They failed to start on b2g emulator as well.

$ GAIA_PROFILE=~/Projects/gaia/profile/ ./mach reftest-b2g-desktop 
Error running mach:

    ['reftest-b2g-desktop']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

ImportError: cannot import name expected

  File "/home/tlin/Projects/gecko-dev/layout/tools/reftest/mach_commands.py", line 480, in run_reftest_b2g_desktop
    return self._run_reftest(test_file, suite='reftest', **kwargs)
  File "/home/tlin/Projects/gecko-dev/layout/tools/reftest/mach_commands.py", line 499, in _run_reftest
    test_file, suite=suite, **kwargs)
  File "/home/tlin/Projects/gecko-dev/layout/tools/reftest/mach_commands.py", line 166, in run_b2g_test
    imp.load_module('reftest', fh, path, ('.py', 'r', imp.PY_SOURCE))
  File "/home/tlin/Projects/gecko-dev/obj-b2g/_tests/reftest/runreftestb2g.py", line 16, in <module>
    from b2g_desktop import run_desktop_reftests
  File "/home/tlin/Projects/gecko-dev/obj-b2g/_tests/reftest/b2g_desktop.py", line 14, in <module>
    from marionette import Marionette, expected

Comment 1

[Ting-Yu Lin [:TYLin] (PST, UTC-8)]

The line failed to import was added in bug 1043699. https://hg.mozilla.org/mozilla-central/rev/dfd2eb97d840

Comment 2

[Ting-Yu Lin [:TYLin] (PST, UTC-8)]

Attachment: Fix reftest cannot launch on b2g desktop

Hi Armen,

With this patch, I can launch reftest on b2g desktop. Can it work for you on mulet?

Comment 3

[Armen [:armenzg]]

Comment on attachment 8539969 [details] [diff] [review]
Fix reftest cannot launch on b2g desktop

Review of attachment 8539969 [details] [diff] [review]:
-----------------------------------------------------------------

If this runs well on Try please send the review request to me.

We need to make sure this won't break b2g desktop jobs.
Mulet is not yet a tier-1 platform.

I get the feeling that this bug should fix how mach calls it.

Maybe jgriffin has more insightful information.

I will be back on the 5th if you want me to help you dig deeper.

Comment 4

[Ting-Yu Lin [:TYLin] (PST, UTC-8)]

My patch does not work on try. All reftests failed.
https://treeherder.mozilla.org/#/jobs?repo=try&revision=139bbcaeb459

Comment 5

[Ting-Yu Lin [:TYLin] (PST, UTC-8)]

'testing/marionette/client' was not found in sys.path. That's why the 'expected' module cannot import by saying "from marionette import Marionette, expected". 

However, that path had been removed by [1]. I guess it is not OK to add it back.

jgriffin, do you have any idea how to fix this?

[1] https://bugzilla.mozilla.org/page.cgi?id=splinter.html&bug=1050511&attachment=8502627

Comment 6

[Jonathan Griffin (:jgriffin)]

Ugh, the way that mach loads packages is a little inconsistent with how they're available to mozharness scripts.  Probably the easiest way to handle this is to try the new way and if that fails, try the old way.  This is a bit gross, but I think the alternatives are also a bit gross.

cc'ing Ahal and James in case either has a better idea.

Comment 7

[Andrew Halberstadt [:ahal]]

Yes, this is bug 1109183. Since that bug makes no mention of b2g desktop reftest being broken, I'll leave this open and add that as a dependency.

The workaround is to change this line [1] to:
'testing/marionette/client',

Unfortunately doing so breaks the marionette-test mach command, so the proper fix is a bit more complicated.

[1] http://dxr.mozilla.org/mozilla-central/source/build/mach_bootstrap.py#48

Comment 8

[Chris Manchester (limited bugmail, email directly)]

I'd be ok with backing out bug 1050511 until the packaging situation is sorted out if this is blocking people. The effect of backing that out is that some marionette tests will fail spuriously when run locally -- it sounds like the effect of the fix is worse than that.

Comment 9

[Ting-Yu Lin [:TYLin] (PST, UTC-8)]

I'll unassigned this bug to myself since the solution doesn't seem trivial. My patch or ahal's workaround in comment 7 should work locally, so this might not block people.

Comment 11

[Mason Chang [Inactive] [:mchang]]

Attachment: Temp fix for b2g reftests

From comment 7, updated non-bitrot version.

Comment 12

[Chris Manchester (limited bugmail, email directly)]

Attachment: Unbreak some mach consumers of marionette

This is a version of the workaround that doesn't break marionette tests. If this works for people it's probably worth taking in the short term.

Comment 13

[Andrew Halberstadt [:ahal]]

Comment on attachment 8561854 [details] [diff] [review]
Unbreak some mach consumers of marionette

Review of attachment 8561854 [details] [diff] [review]:
-----------------------------------------------------------------

Horribly hacky and gets the job done. Perfect.

Comment 14

[Chris Manchester (limited bugmail, email directly)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/935c00a15ced

Comment 15

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/935c00a15ced