No tests being collected in Marionette harness tests

RESOLVED FIXED in Firefox 54

Status

Testing
Marionette
P1
normal
RESOLVED FIXED
a year ago
a year ago

People

(Reporter: ahal, Assigned: ahal)

Tracking

unspecified
mozilla55
Points:
---

Firefox Tracking Flags

(firefox54 fixed, firefox55 fixed)

Details

MozReview Requests

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(2 attachments)

(Assignee)

Description

a year ago
It looks like a I broke the marionette-harness tests in bug 1335873.. sorry! It was a bit of a perfect storm.

For some reason, they only fail if you are using the default mozconfig (i.e no 'mozconfig' files in the srcdir and no $MOZCONFIG env set), which is why I didn't catch it locally.

Plus, there is a second bug in the python-test mach command, which fails to error out if no tests were detected.. which is why I didn't catch it in automation.

I'll try to get this fixed asap.
Priority: -- → P1
(Assignee)

Comment 1

a year ago
Mike Shal has been helping me diagnose this. We understand what's going wrong now, but not how to fix it. Essentially, 'JS_STANDALONE' gets set to 1, which causes the marionette python.ini to get bypassed:
https://dxr.mozilla.org/mozilla-central/source/moz.build#88

I had originally added that check to fix the busted 'sm-pkg' task which was the cause for bug 1335873 getting backed out. So removing that condition fixes these harness tests, but then I still need to figure out how to not break 'sm-pkg'. I have some try runs going, I'll update here again in a bit.
(Assignee)

Updated

a year ago
See Also: → bug 1345209
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)

Comment 4

a year ago
mozreview-review
Comment on attachment 8844574 [details]
Bug 1345109 - Ensure |mach python-test| errors out if no tests are found,

https://reviewboard.mozilla.org/r/117964/#review119716

Glad this is fixed!
Attachment #8844574 - Flags: review?(mshal) → review+

Comment 5

a year ago
mozreview-review
Comment on attachment 8844575 [details]
Bug 1345109 - Delete 'JS_STANDALONE' from EmptyConfig when generating test metadata,

https://reviewboard.mozilla.org/r/117966/#review119724

::: build/gen_test_backend.py:22
(Diff revision 1)
>      except BuildEnvironmentNotFoundException:
>          print("No build detected, test metadata may be incomplete.")
> -        config = EmptyConfig(build_obj.topsrcdir)
> +
> +        # If 'JS_STANDALONE' is set, tests that don't require an objdir won't
> +        # be picked up due to bug 1345209.
> +        config = EmptyConfig.default_config

Can you use a different variable name here? Otherwise 'config' is a dict now and then becomes an EmptyConfig object later. Maybe it would be simpler if EmptyConfig.default_config was called default_substs. Then you would have:

substs = EmptyConfig.default_substs
...
config = EmptyConfig(build_obj.topsrcdir, substs)
Attachment #8844575 - Flags: review?(mshal) → review+
Comment hidden (mozreview-request)

Comment 7

a year ago
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/233d22402d7a
Ensure |mach python-test| errors out if no tests are found, r=mshal
https://hg.mozilla.org/integration/autoland/rev/2324b5d27792
Delete 'JS_STANDALONE' from EmptyConfig when generating test metadata, r=mshal

Comment 8

a year ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/233d22402d7a
https://hg.mozilla.org/mozilla-central/rev/2324b5d27792
Status: ASSIGNED → RESOLVED
Last Resolved: a year ago
status-firefox55: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla55
Please uplift to 54. Test-only uplift, needed to uplift Bug 1347483.
Whiteboard: [checkin-needed-aurora]
You need to log in before you can comment on or make changes to this bug.