Run unit tests on pre-existing nightly builds

RESOLVED FIXED

Status

Release Engineering
General
P2
normal
RESOLVED FIXED
8 years ago
4 years ago

People

(Reporter: joduinn, Assigned: catlee)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(4 attachments)

Once we are able to run unittests on incremental builds, we should have the automation run unittests on nightly and release builds also. 

Its separate work, and blocked, so filing separate bug in Future to track this.
(Assignee)

Updated

8 years ago
Depends on: 463605
(Assignee)

Updated

8 years ago
Depends on: 460282
Tweak summary to distinguish from bug#457753 and bug#372581.
Summary: run unittests on nightly and release builds → Run unit tests on pre-existing nightly builds
Ted said this will enable unit-testing of PGO builds. Is this the case?

We want to enable PGO for a few places where it's disabled (bug 419893), but I'm loathe to do that without knowing that it'll pass the tests.
bug 457753 is a Q3 goal, I assume this (being very similar) will be fixed at the same time? Should it be un-futured?
Blocks: 511159
Blocks: 419893
(In reply to comment #3)
> bug 457753 is a Q3 goal, I assume this (being very similar) will be fixed at
> the same time? Should it be un-futured?

Yep, not a goal, but happening about the same time, as its similar work.

Lukas said she would take this on last week, as is similar to the work she is
doing for running unittests on nightly builds and on debug builds, but neither
of us updated the bug with that info.

Moved from Future and assigned to Lukas, to match reality.
Assignee: nobody → lsblakk
Component: Release Engineering: Future → Release Engineering
Thanks for the update John, and thanks Lukas for working on this. PGO'd unit testing is blocking the enabling of PGO for several areas of code, which could improve application performance significantly.
(Assignee)

Updated

8 years ago
Assignee: lsblakk → catlee
(Assignee)

Comment 6

8 years ago
Created attachment 401012 [details] [diff] [review]
buildbotcustom changes for running unittests on nightly/depend (optimized) builds
Attachment #401012 - Flags: review?(lsblakk)
Attachment #401012 - Flags: review?(bhearsum)
Attachment #401012 - Flags: review?(bhearsum) → review+
Attachment #401012 - Flags: review?(lsblakk) → review+
(Assignee)

Comment 7

8 years ago
Created attachment 401020 [details] [diff] [review]
buildbot-configs for unittests on optimized builds
Attachment #401020 - Flags: review?(lsblakk)
Attachment #401020 - Flags: review?(bhearsum)
Comment on attachment 401020 [details] [diff] [review]
buildbot-configs for unittests on optimized builds

After our chat on IRC, this is fine.
Attachment #401020 - Flags: review?(bhearsum) → review+
(Assignee)

Updated

8 years ago
Priority: -- → P2
(Assignee)

Comment 9

8 years ago
Comment on attachment 401012 [details] [diff] [review]
buildbotcustom changes for running unittests on nightly/depend (optimized) builds

changeset:   418:47b1cd1c81dc
Attachment #401012 - Flags: checked-in+
(Assignee)

Comment 10

8 years ago
Comment on attachment 401020 [details] [diff] [review]
buildbot-configs for unittests on optimized builds

changeset:   1544:83db6bbf7abd
Attachment #401020 - Flags: review?(lsblakk) → checked-in+
(Assignee)

Comment 11

8 years ago
These look like they're running properly on m-c
Please drop --enable-tests from the mozconfigs. It's the default, so having it there is redundant.
Created attachment 403150 [details] [diff] [review]
Fix mac nightly bustage

Today's mac m-c nightly ran out of space packaging the tests. It was asking for 5G and got 5.66G; tests seem to be about 91M unpacked so increasing it to 6G should hopefully be enough.

http://hg.mozilla.org/build/buildbot-configs/rev/3e61bd4a8509
p-m & p-m02 reconfig'd.
Attachment #403150 - Flags: checked-in+
(Assignee)

Updated

8 years ago
Depends on: 519194
is this ready to be marked fixed? is this running on all branches w/ tinderboxes?
(In reply to comment #14)
> is this ready to be marked fixed? is this running on all branches w/
> tinderboxes?

(if it needs to. maybe it doesn't?)
(Assignee)

Comment 16

8 years ago
It's currently only running on mozilla-central.  I'd like to see it on the mozilla-1.9.2 branch, and project branches as well at some point.  Not sure if it's worth doing on the 1.9.1 branch at this point.
OS X 10.5.2 mozilla-central test opt everythingelse went green after I landed bug 518641. Yay!

OS X 10.5.2 mozilla-central test opt everythingelse is still orange for two reasons:
1) It's failing to run mochitest-a11y. Our nightly builds don't have accessibility enabled on mac, so we'll have to turn these tests off on opt builds currently.
2) One failing xpcshell test: bug 519727
Depends on: 519727
(Assignee)

Comment 18

8 years ago
Created attachment 404698 [details] [diff] [review]
Disable mochitest-a11y on non-unittest mac builds

Neither opt nor debug builds are compiled with --enable-accessibility on mac, so running the mochitest-a11y tests will always fail.

This patch creates a new UnittestPackagedBuildFactory for mac opt and debug tests that makes sure that they don't try and run mochitest-a11y.
Attachment #404698 - Flags: review?(bhearsum)
Attachment #404698 - Flags: review?(bhearsum) → review+
(Assignee)

Comment 19

8 years ago
Comment on attachment 404698 [details] [diff] [review]
Disable mochitest-a11y on non-unittest mac builds

changeset:   445:b6e8ad13700c
Attachment #404698 - Flags: checked-in+
(Assignee)

Comment 20

8 years ago
Looks good.  Future work (e.g. running on m-1.9.2, bug 523388) will be tracked in other bugs.
Status: NEW → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → FIXED
Depends on: 518641
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.