Closed Bug 1462489 Opened 6 years ago Closed 6 years ago

Run mochitests from the tup build

Categories

(Firefox Build System :: General, enhancement)

Product:
Firefox Build System
Component:
General
Version:
3 Branch
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(firefox62 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla62
Tracking Flags:
Tracking Status
firefox62 --- fixed

People

(Reporter: chmanchester, Assigned: chmanchester)

References

Details

Attachments

(4 files)

Bug 1462489 - Process some chrome manifest entries from dist/xpi-stage from the tup backend.
59 bytes, text/x-review-board-request
Details
Bug 1462489 - Use test addons from dist/xpi-stage when running mochitests locally.
59 bytes, text/x-review-board-request
ahal
: review+
Details
Bug 1462489 - Put packaged test addons in the test package with the test packager rather than Makefile.in
59 bytes, text/x-review-board-request
Details
Bug 1462489 - Re-generate the build backend when a build backend output is missing.
59 bytes, text/x-review-board-request
Details 
I've been able to run xpcshell tests from a tup build just fine, but the mochitests fail due to: "invalid setup: missing mochikit extension"
Aha, there's a stray libs rule that copies mochijar from dist/xpi-stage to _tests/testing/mochitest: https://searchfox.org/mozilla-central/rev/da499aac682d0bbda5829327b60a865cbc491611/testing/mochitest/Makefile.in#11
I wonder if we could just fix the default path in runtests.py to look in dist/xpi-stage, and fix the test packaging bits to copy it out of there into the current place in the test package?
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #2)
> I wonder if we could just fix the default path in runtests.py to look in
> dist/xpi-stage, and fix the test packaging bits to copy it out of there into
> the current place in the test package?

+1. I think the main reason we put everything in those staging test dirs is because it was previously a PITA to create the test package in testsuite-targets.mk. But now with test_archive.py, it's trivial to pull in things from all over the sourcedir/objdir.
Assignee: nobody → cmanchester
Comment on attachment 8980455 [details]
Bug 1462489 - Use test addons from dist/xpi-stage when running mochitests locally.

https://reviewboard.mozilla.org/r/246618/#review252926

Lgtm!
Attachment #8980455 - Flags: review?(ahal) → review+
Comment on attachment 8980454 [details]
Bug 1462489 - Process some chrome manifest entries from dist/xpi-stage from the tup backend.

https://reviewboard.mozilla.org/r/246616/#review254188
Attachment #8980454 - Flags: review+
Attachment #8980454 - Flags: review?(core-build-config-reviews)
Comment on attachment 8980456 [details]
Bug 1462489 - Put packaged test addons in the test package with the test packager rather than Makefile.in

https://reviewboard.mozilla.org/r/246620/#review254190

::: python/mozbuild/mozbuild/action/test_archive.py:372
(Diff revision 1)
> +            'source': buildconfig.topobjdir,
> +            'base': 'dist/xpi-stage',
> +            'pattern': 'mochijar/**',
> +            'dest': 'mochitest'
> +        },
> +        {

Is this entry intended to be something else? It's a duplicate of the one above.
Attachment #8980456 - Flags: review+
Attachment #8980456 - Flags: review?(core-build-config-reviews)
Comment on attachment 8980457 [details]
Bug 1462489 - Re-generate the build backend when a build backend output is missing.

https://reviewboard.mozilla.org/r/246622/#review254194
Attachment #8980457 - Flags: review+
Attachment #8980457 - Flags: review?(core-build-config-reviews)
Comment on attachment 8980456 [details]
Bug 1462489 - Put packaged test addons in the test package with the test packager rather than Makefile.in

https://reviewboard.mozilla.org/r/246620/#review254190

> Is this entry intended to be something else? It's a duplicate of the one above.

Nope, it's just duplicated for no reason. Thanks for noticing that!
Pushed by cmanchester@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/dfed23857cea
Process some chrome manifest entries from dist/xpi-stage from the tup backend. r=mshal
https://hg.mozilla.org/integration/autoland/rev/3cd6a8ee8f66
Use test addons from dist/xpi-stage when running mochitests locally. r=ahal
https://hg.mozilla.org/integration/autoland/rev/daa33716b140
Put packaged test addons in the test package with the test packager rather than Makefile.in r=mshal
https://hg.mozilla.org/integration/autoland/rev/41cf0d53a95c
Re-generate the build backend when a build backend output is missing. r=mshal
Backed out 4 changesets (bug 1462489) for breaking mochitest on testing/mochitest/tests/python/test_basic_mochitest_plain.py

Log:
https://treeherder.mozilla.org/logviewer.html#?job_id=181011268&repo=autoland&lineNumber=355

 0:11.62 ===================== 1 passed, 1 warnings in 0.28 seconds =====================
[task 2018-05-30T23:11:04.028Z]  0:11.72 /builds/worker/checkouts/gecko/testing/mochitest/tests/python/test_basic_mochitest_plain.py
[task 2018-05-30T23:11:04.639Z]  0:12.33 ============================= test session starts ==============================
[task 2018-05-30T23:11:04.639Z]  0:12.33 platform linux2 -- Python 2.7.12, pytest-3.2.5, py-1.5.3, pluggy-0.4.0 -- /builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/_virtualenvs/init/bin/python
[task 2018-05-30T23:11:04.639Z]  0:12.33 rootdir: /builds/worker/checkouts/gecko, inifile:
[task 2018-05-30T23:11:04.667Z]  0:12.36 collecting ... collected 6 items
[task 2018-05-30T23:11:04.668Z]  0:12.36 
[task 2018-05-30T23:11:13.553Z]  0:21.24 testing/mochitest/tests/python/test_basic_mochitest_plain.py::test_output_pass TEST-UNEXPECTED-FAIL
[task 2018-05-30T23:11:21.572Z]  0:29.26 testing/mochitest/tests/python/test_basic_mochitest_plain.py::test_output_fail TEST-UNEXPECTED-FAIL
[task 2018-05-30T23:11:21.573Z]  0:29.26 testing/mochitest/tests/python/test_basic_mochitest_plain.py::test_output_crash SKIPPED
[task 2018-05-30T23:11:31.213Z]  0:38.90 testing/mochitest/tests/python/test_basic_mochitest_plain.py::test_output_asan TEST-UNEXPECTED-FAIL
[task 2018-05-30T23:11:31.213Z]  0:38.90 testing/mochitest/tests/python/test_basic_mochitest_plain.py::test_output_assertion SKIPPED
[task 2018-05-30T23:11:31.216Z]  0:38.91 testing/mochitest/tests/python/test_basic_mochitest_plain.py::test_output_leak SKIPPED
[task 2018-05-30T23:11:31.217Z]  0:38.91 
[task 2018-05-30T23:11:31.217Z]  0:38.91 =================================== FAILURES ===================================
[task 2018-05-30T23:11:31.217Z]  0:38.91 _______________________________ test_output_pass _______________________________
[task 2018-05-30T23:11:31.218Z]  0:38.91 
[task 2018-05-30T23:11:31.218Z]  0:38.91 runtests = <function inner at 0x7f7e2e05c500>

Push with failures:
https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=41cf0d53a95cc8966659a5eafb93d3826b97ea62&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-classifiedState=unclassified

Backout:
https://hg.mozilla.org/integration/autoland/rev/aca4b42750423341b74ba8cfbd77d27069f70fb2
Flags: needinfo?(cmanchester)
I would have fixed it this anyway, so it's not really much of an inconvenience, but these tests are marked as tier-2, so as far as I know this failure should not have resulted in a backout.
Flags: needinfo?(cmanchester)
Attachment #8980455 - Flags: review+ → review?(ahal)
Comment on attachment 8980455 [details]
Bug 1462489 - Use test addons from dist/xpi-stage when running mochitests locally.

https://reviewboard.mozilla.org/r/246618/#review254392
Attachment #8980455 - Flags: review?(ahal) → review+
Pushed by cmanchester@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/7f3e1d87baa3
Process some chrome manifest entries from dist/xpi-stage from the tup backend. r=mshal
https://hg.mozilla.org/integration/autoland/rev/5133f89842f6
Use test addons from dist/xpi-stage when running mochitests locally. r=ahal
https://hg.mozilla.org/integration/autoland/rev/ce2105ffae17
Put packaged test addons in the test package with the test packager rather than Makefile.in r=mshal
https://hg.mozilla.org/integration/autoland/rev/e0b837323b48
Re-generate the build backend when a build backend output is missing. r=mshal
Depends on: 1466958
Version: Version 3 → 3 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: