Closed Bug 1345109 Opened 3 years ago Closed 3 years ago

No tests being collected in Marionette harness tests


(Testing :: Marionette, enhancement, P1)



(firefox54 fixed, firefox55 fixed)

Tracking Status
firefox54 --- fixed
firefox55 --- fixed


(Reporter: ahal, Assigned: ahal)




(2 files)

It looks like a I broke the marionette-harness tests in bug 1335873.. sorry! It was a bit of a perfect storm.

For some reason, they only fail if you are using the default mozconfig (i.e no 'mozconfig' files in the srcdir and no $MOZCONFIG env set), which is why I didn't catch it locally.

Plus, there is a second bug in the python-test mach command, which fails to error out if no tests were detected.. which is why I didn't catch it in automation.

I'll try to get this fixed asap.
Priority: -- → P1
Mike Shal has been helping me diagnose this. We understand what's going wrong now, but not how to fix it. Essentially, 'JS_STANDALONE' gets set to 1, which causes the marionette python.ini to get bypassed:

I had originally added that check to fix the busted 'sm-pkg' task which was the cause for bug 1335873 getting backed out. So removing that condition fixes these harness tests, but then I still need to figure out how to not break 'sm-pkg'. I have some try runs going, I'll update here again in a bit.
See Also: → 1345209
Comment on attachment 8844574 [details]
Bug 1345109 - Ensure |mach python-test| errors out if no tests are found,

Glad this is fixed!
Attachment #8844574 - Flags: review?(mshal) → review+
Comment on attachment 8844575 [details]
Bug 1345109 - Delete 'JS_STANDALONE' from EmptyConfig when generating test metadata,

::: build/
(Diff revision 1)
>      except BuildEnvironmentNotFoundException:
>          print("No build detected, test metadata may be incomplete.")
> -        config = EmptyConfig(build_obj.topsrcdir)
> +
> +        # If 'JS_STANDALONE' is set, tests that don't require an objdir won't
> +        # be picked up due to bug 1345209.
> +        config = EmptyConfig.default_config

Can you use a different variable name here? Otherwise 'config' is a dict now and then becomes an EmptyConfig object later. Maybe it would be simpler if EmptyConfig.default_config was called default_substs. Then you would have:

substs = EmptyConfig.default_substs
config = EmptyConfig(build_obj.topsrcdir, substs)
Attachment #8844575 - Flags: review?(mshal) → review+
Pushed by
Ensure |mach python-test| errors out if no tests are found, r=mshal
Delete 'JS_STANDALONE' from EmptyConfig when generating test metadata, r=mshal
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla55
Please uplift to 54. Test-only uplift, needed to uplift Bug 1347483.
Whiteboard: [checkin-needed-aurora]
You need to log in before you can comment on or make changes to this bug.