Closed
Bug 1295826
Opened 8 years ago
Closed 6 years ago
[Tracking] Fix SeaMonkey's Testing infrastructure.
Categories
(SeaMonkey :: Testing Infrastructure, defect)
SeaMonkey
Testing Infrastructure
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: ewong, Assigned: ewong)
References
(Blocks 1 open bug)
Details
Attachments
(14 files, 8 obsolete files)
6.66 KB,
patch
|
frg
:
review+
|
Details | Diff | Splinter Review |
9.73 KB,
patch
|
frg
:
review+
|
Details | Diff | Splinter Review |
29.21 KB,
patch
|
frg
:
review+
|
Details | Diff | Splinter Review |
882 bytes,
patch
|
Details | Diff | Splinter Review | |
1.54 MB,
patch
|
Details | Diff | Splinter Review | |
2.35 MB,
patch
|
Details | Diff | Splinter Review | |
1.00 MB,
patch
|
Details | Diff | Splinter Review | |
1.16 MB,
patch
|
Details | Diff | Splinter Review | |
925.38 KB,
text/plain
|
Details | |
4.50 KB,
patch
|
Details | Diff | Splinter Review | |
3.21 KB,
patch
|
Details | Diff | Splinter Review | |
848 bytes,
patch
|
Details | Diff | Splinter Review | |
2.46 MB,
text/plain
|
Details | |
62.29 KB,
patch
|
frg
:
review+
|
Details | Diff | Splinter Review |
Right now, our tests are in a mess (we haven't caught up with the changes in how the tests are packaged and what's downloaded) and instead of getting the slaves to build broken tests (when their processing can be used in building regular builds), the unittests should be disabled.
Assignee | ||
Comment 1•8 years ago
|
||
Attachment #8781803 -
Flags: review?(bugspam.Callek)
Assignee | ||
Updated•8 years ago
|
Summary: Disable all tests until bug 1209378 is fixed. → [Tracking] Fix SeaMonkey's Testing infrastructure.
Assignee | ||
Comment 2•8 years ago
|
||
Bug 1209378 was morphed to a TB bug since it fixed their packageURL issue. So morphing this bug to track our SeaMonkey testing infra.
Assignee | ||
Comment 3•8 years ago
|
||
Assignee: nobody → ewong
Attachment #8781803 -
Attachment is obsolete: true
Status: NEW → ASSIGNED
Attachment #8781803 -
Flags: review?(bugspam.Callek)
Attachment #8782008 -
Flags: review?(bugspam.Callek)
Assignee | ||
Comment 4•8 years ago
|
||
Comment on attachment 8782008 [details] [diff] [review] [configs] disable unittests and l10n dep nightlies going for a post-land-review
Assignee | ||
Comment 5•8 years ago
|
||
https://hg.mozilla.org/build/buildbot-configs/rev/3bcc86136b6b6e030d785be4ae6b93c4483783f1 Bug 1295826 - Disable unittests and l10n dep builds. r=post-land-review
Assignee | ||
Comment 6•8 years ago
|
||
(In reply to Edmund Wong (:ewong) from comment #5) > https://hg.mozilla.org/build/buildbot-configs/rev/ > 3bcc86136b6b6e030d785be4ae6b93c4483783f1 > Bug 1295826 - Disable unittests and l10n dep builds. r=post-land-review Backed out http://hg.mozilla.org/build/buildbot-configs/rev/a1eb00a0b64b for possibly disabling the nightlies.
Assignee | ||
Updated•7 years ago
|
Attachment #8782008 -
Attachment is obsolete: true
Attachment #8782008 -
Flags: review?(bugspam.Callek)
Assignee | ||
Comment 7•7 years ago
|
||
Attachment #8895683 -
Flags: review?(frgrahl)
Assignee | ||
Comment 8•7 years ago
|
||
Attachment #8895684 -
Flags: review?(frgrahl)
Assignee | ||
Comment 9•7 years ago
|
||
Attachment #8895686 -
Flags: review?(frgrahl)
Assignee | ||
Comment 10•7 years ago
|
||
To note: The way tests are run has changed *very* dramatically since our last successful test run and as such, I'm just gleaming what I can from the tests run from firefox and translating their steps to buildbot. Back then, I wasn't even familiar with the tests so let it be known that this new way of testing is also unfamiliar. The steps as given 'worked' for mochitest-1 for linux32 (well, the tests ran... just not successfully and I'm not sure if it's due to how I'm running the tests or our binaries are not compatible with the tests or something else). These three patches are a starting point to get our tests running. Once these patches are pushed and a successful debug build is done, we should know whether or not more is needed. (I am betting on "more is needed").
Assignee | ||
Comment 11•7 years ago
|
||
further rationale for the patches: tooltool: - we need the tooltool manifests in tools because 1) we don't pull the source so we can't put the testing manifests in suite/config, so putting them in tools/testing is the 'best' place since we do pull tools - for linux*, we need both gtk3 and gcc tooltool packages (firefox doesn't use the gtk3/gcc tooltool packages .. why? because I'm guessing their Taskcluster images have proper gtk3/gcc built in..) to run xpcshell buildbot-config: - the new steps require pip and the usage of pypi.pub.build.mozilla.org/pub. (I'm assuming since it's public, I don't need permission to get the pypi packages). buildbotcustom: - a new set of steps required: a) install a virtualenv b) install some requirements (requirements.txt) c) install mozbase requirements d) install marionette requirements e) install websocket requirements [ requires being run in a mock env] f) set up the right tooltool packages g) mozinstall the application h) run test That is the extent of the new steps. I'm hoping to have them running so I can adapt future patches to fix whatever needs fixing.
Assignee | ||
Comment 12•7 years ago
|
||
marionette_reqs are to be installed after the unpack of tests as that's where the configs are from.
Attachment #8895686 -
Attachment is obsolete: true
Attachment #8895686 -
Flags: review?(frgrahl)
Attachment #8895699 -
Flags: review?(frgrahl)
Assignee | ||
Comment 13•7 years ago
|
||
to note: # of chunks have changed. In the past, there were 5 chunks. Now, Firefox has 10. Since I'm not familiar with the testing part of things (serge, where are you?), I'll have to play it by ear and will go for 10. (pending patches)
Comment 14•7 years ago
|
||
Comment on attachment 8895683 [details] [diff] [review] [tools] add test_tooltool_manifests to testing/tooltool_manifest For Fx the manifests seem to be located in testing\config\tooltool-manifests. Samy for our c-c products. If nothing whould oppose it should imho be also done this way in tools to have an identical structure for all manifest files so should be testing\config\tooltool-manifests. > testing/tooltool-manifests/linux32/releng.manifest digests have blanks in them. Overall: indent is not consistent between them. Some start always on column 1 and other are indented. Might be the same for the packages where you copied them already. Can live with it if you think not worth fixing. r+ with digest errors above fixed and first question answered/fixed. Thanks for tackling it.
Attachment #8895683 -
Flags: review?(frgrahl) → review+
Comment 15•7 years ago
|
||
Comment on attachment 8895684 [details] [diff] [review] [configs] add pip_server and test_tooltool_manifest_src entries r+ with eventually the path for the manifests changed.
Attachment #8895684 -
Flags: review?(frgrahl) → review+
Comment 16•7 years ago
|
||
Comment on attachment 8895699 [details] [diff] [review] [custom] add steps to run tests Just looked only more or less formally over it but looks good. Just lets try it and see what needs fixing afterwards.
Attachment #8895699 -
Flags: review?(frgrahl) → review+
Assignee | ||
Comment 17•7 years ago
|
||
(In reply to Frank-Rainer Grahl (:frg) from comment #14) > Comment on attachment 8895683 [details] [diff] [review] > [tools] add test_tooltool_manifests to testing/tooltool_manifest > > For Fx the manifests seem to be located in > testing\config\tooltool-manifests. Samy for our c-c products. > > If nothing whould oppose it should imho be also done this way in tools to > have an identical structure for all manifest files so should be > testing\config\tooltool-manifests. > ah. sorry should be manifests > > testing/tooltool-manifests/linux32/releng.manifest > > digests have blanks in them. Will fix this. > > Overall: indent is not consistent between them. Some start always on column > 1 and other are indented. Might be the same for the packages where you > copied them already. Can live with it if you think not worth fixing. > > r+ with digest errors above fixed and first question answered/fixed. > > Thanks for tackling it. I'm not going to indent any of the manifests. As you mentioned, the indentation is inconsistent so either we indent the manifests, or we just have them left justified. Edmund
Assignee | ||
Updated•7 years ago
|
Severity: normal → critical
Assignee | ||
Comment 18•7 years ago
|
||
over irc, the decision was to do the indentation on the manifests.
Attachment #8895683 -
Attachment is obsolete: true
Attachment #8896996 -
Flags: review+
Assignee | ||
Comment 19•7 years ago
|
||
Attachment #8896996 -
Attachment is obsolete: true
Attachment #8896998 -
Flags: review?(frgrahl)
Assignee | ||
Comment 20•7 years ago
|
||
Attachment #8896998 -
Attachment is obsolete: true
Attachment #8896998 -
Flags: review?(frgrahl)
Attachment #8897003 -
Flags: review?(frgrahl)
Assignee | ||
Comment 21•7 years ago
|
||
Attachment #8895684 -
Attachment is obsolete: true
Attachment #8897005 -
Flags: review?(frgrahl)
Comment 22•7 years ago
|
||
Comment on attachment 8897003 [details] [diff] [review] [tools] add tooltool manifests for tests (w/o space in digests and move to under testing/config) r+++
Attachment #8897003 -
Flags: review?(frgrahl) → review+
Comment 23•7 years ago
|
||
Comment on attachment 8897005 [details] [diff] [review] [configs] add tooltool manifest src paths for testing infra, and add pip server url And another r+
Attachment #8897005 -
Flags: review?(frgrahl) → review+
Assignee | ||
Comment 24•7 years ago
|
||
found major issues in the previous patch after I had pushed it. (backed out)
Attachment #8895699 -
Attachment is obsolete: true
Attachment #8897712 -
Flags: review?(frgrahl)
Comment 25•7 years ago
|
||
Comment on attachment 8897712 [details] [diff] [review] [custom] add steps to run tests Looks good but so did the last one :) r+
Attachment #8897712 -
Flags: review?(frgrahl) → review+
Assignee | ||
Comment 26•7 years ago
|
||
Assignee | ||
Comment 27•7 years ago
|
||
Assignee | ||
Comment 28•7 years ago
|
||
Assignee | ||
Updated•7 years ago
|
Attachment #8924810 -
Attachment description: [tools] additional package required → [tools] additional package required (virtualenv) [checked-in]
Assignee | ||
Comment 29•7 years ago
|
||
Assignee | ||
Comment 30•7 years ago
|
||
Assignee | ||
Comment 31•7 years ago
|
||
This is the very *FIRST* mochitest-1 test run in over three years or more. A lot of stuff have happened since then and the test run shows how far we need to fix as after a bit, it crashes and produces a crapton of tracebacks... *sigh*
Assignee | ||
Comment 32•7 years ago
|
||
Assignee | ||
Comment 33•6 years ago
|
||
Assignee | ||
Comment 34•6 years ago
|
||
Assignee | ||
Comment 36•6 years ago
|
||
currently our tests are timing out.
Assignee | ||
Comment 37•6 years ago
|
||
(In reply to Edmund Wong (:ewong) from comment #36) > Created attachment 8931180 [details] > timeout log > > currently our tests are timing out. attached is the log.. a lot of assertions and not quite good looking results :(
Assignee | ||
Updated•6 years ago
|
Blocks: operation_greentest
Assignee | ||
Comment 38•6 years ago
|
||
With this patch, the tests "run" [the infrastructure is in place, just need to fix the actual tests, which will require new bugs]. Since this bug is getting long in the tooth, once I've applied the reviewed patch, I'm closing this bug. [just a heads up] (unless someone disagrees)
Attachment #8939357 -
Flags: review?(frgrahl)
Comment 39•6 years ago
|
||
Comment on attachment 8939357 [details] [diff] [review] [custom] proposed patch [checked-in] Thanks
Attachment #8939357 -
Flags: review?(frgrahl) → review+
Assignee | ||
Comment 40•6 years ago
|
||
Comment on attachment 8939357 [details] [diff] [review] [custom] proposed patch [checked-in] Pushed to buildbotcustom: https://hg.mozilla.org/build/buildbotcustom/rev/7e5d72dc0fd2
Attachment #8939357 -
Attachment description: [custom] proposed patch → [custom] proposed patch [checked-in]
Assignee | ||
Comment 41•6 years ago
|
||
Closing bug as 'fixed' only because the infrastructure fix was the purpose. The actual fixing of the tests will be done in subsequent bugs.
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•