Closed
Bug 624851
Opened 14 years ago
Closed 11 years ago
Enable {crash,ref}test-ipc for Mac machines on Try
Categories
(Release Engineering :: General, defect, P3)
Tracking
(fennec-)
RESOLVED
FIXED
Tracking | Status | |
---|---|---|
fennec | - | --- |
People
(Reporter: cjones, Assigned: mozilla)
References
Details
(Whiteboard: [unittest][new suite])
Attachments
(5 files, 2 obsolete files)
8.92 KB,
patch
|
armenzg
:
review+
mozilla
:
checked-in+
|
Details | Diff | Splinter Review |
2.65 KB,
patch
|
armenzg
:
review+
mozilla
:
checked-in+
|
Details | Diff | Splinter Review |
6.86 KB,
patch
|
armenzg
:
review+
mozilla
:
checked-in+
|
Details | Diff | Splinter Review |
11.67 KB,
patch
|
Details | Diff | Splinter Review | |
8.61 KB,
patch
|
Details | Diff | Splinter Review |
See also bug 623613. AFAIK we have all the infra support we need for this already, just need to flip the switch to enable these test suites when the platform deps are ready.
Updated•14 years ago
|
Whiteboard: [unittest][new suite]
Reporter | ||
Comment 1•14 years ago
|
||
Forgot to note that for the time being, we'll need to run these tests with GPU rendering *disabled*, because our multi-process gfx code doesn't understand D3D yet. (This caused lots of problems on tryserver! ;) )
Reporter | ||
Comment 2•14 years ago
|
||
For my records, tests initially failing in my VM
REFTEST TEST-UNEXPECTED-FAIL | file:///c:/Users/cjones/mozilla/mozilla-central/modules/libpr0n/test/reftest/pngsuite-ancillary/ccwn2c08.png | image comparison (==)
REFTEST TEST-UNEXPECTED-FAIL | file:///c:/Users/cjones/mozilla/mozilla-central/modules/libpr0n/test/reftest/pngsuite-ancillary/ccwn3p08.png | image comparison (==)
REFTEST TEST-UNEXPECTED-FAIL | file:///c:/Users/cjones/mozilla/mozilla-central/modules/libpr0n/test/reftest/apng/delaytest.html?bug546272.png | image comparison (==)
REFTEST TEST-UNEXPECTED-FAIL | file:///c:/Users/cjones/mozilla/mozilla-central/modules/libpr0n/test/reftest/generic/accept-image-catchall.html | image comparison (==)
REFTEST TEST-UNEXPECTED-FAIL | file:///c:/Users/cjones/mozilla/mozilla-central/layout/reftests/bugs/289480.html#top | image comparison (==)
REFTEST TEST-UNEXPECTED-FAIL | file:///c:/Users/cjones/mozilla/mozilla-central/layout/reftests/bugs/564991-1.html | image comparison (==)
REFTEST TEST-UNEXPECTED-FAIL | file:///c:/Users/cjones/mozilla/mozilla-central/layout/reftests/bugs/579985-1.html | image comparison (==)
REFTEST TEST-UNEXPECTED-FAIL | file:///c:/Users/cjones/mozilla/mozilla-central/layout/reftests/pixel-rounding/background-image-tiling.html | image comparison (==)
REFTEST TEST-UNEXPECTED-FAIL | file:///c:/Users/cjones/mozilla/mozilla-central/layout/reftests/svg/as-image/background-image-rect-1png.html | image comparison (==)
REFTEST TEST-UNEXPECTED-FAIL | file:///c:/Users/cjones/mozilla/mozilla-central/layout/reftests/svg/as-image/list-simple-1.html | image comparison (==)
REFTEST TEST-UNEXPECTED-FAIL | file:///c:/Users/cjones/mozilla/mozilla-central/layout/reftests/svg/image/image-fill-01.svg | image comparison (==)
REFTEST TEST-UNEXPECTED-FAIL | file:///c:/Users/cjones/mozilla/mozilla-central/layout/reftests/svg/image/image-svg-inline-01.html | image comparison (==)
REFTEST TEST-UNEXPECTED-FAIL | file:///c:/Users/cjones/mozilla/mozilla-central/layout/reftests/svg/smil/anim-image-href-01.svg | image comparison (==)
Reporter | ||
Comment 3•14 years ago
|
||
(Also for my reference, tests failing in-process in my VM
REFTEST TEST-UNEXPECTED-FAIL | file:///c:/Users/cjones/mozilla/mozilla-central/layout/reftests/bugs/289480.html#top | image comparison (==)
REFTEST TEST-UNEXPECTED-FAIL | file:///c:/Users/cjones/mozilla/mozilla-central/layout/reftests/bugs/579985-1.html | image comparison (==)
REFTEST TEST-UNEXPECTED-FAIL | file:///c:/Users/cjones/mozilla/mozilla-central/gfx/tests/reftest/611498-1.html | image comparison (==)
)
Reporter | ||
Comment 4•14 years ago
|
||
Good news:
http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTry/1294962549.1294965381.21061.gz
reftest: 5568/20/304 L-FAIL
Of the failures, the vast majority are differences I can't distinguish even on one pixel visually, even zoomed in fully. Not sure what's up, wouldn't block on those.
One is a legitimate failure probably caused by bug 605618. Should be easy to fix.
The rest are apparently failures to load the test plugin from the content process. Not sure what's up with that. Very much willing to disable those tests, since they're a sideshow of a sideshow.
Not sure what's up with the alleged leak.
Reporter | ||
Comment 5•14 years ago
|
||
Oh, the "leak" is probably
TEST-UNEXPECTED-FAIL | tab process 3284 | automationutils.processLeakLog() | missing output line for total leaks!
We _exit() in content processes when the chrome process shuts down. Probably need a LogTerm() before doing that, for unit tests.
Reporter | ||
Comment 6•14 years ago
|
||
The reftests are mainly failing for niggly pixel differences due to the duct-tape-and-rubber-band rendering setup. We're never going to ship out-of-process content with this code, so I don't think it's worth investing time sorting out these failures.
I *do* think it's worth running the crashtests so that we know of big problems caused by platform changes.
Summary: Enable reftests+crashtests for out-of-process content on Windows machines → Enable crashtests for out-of-process content on Windows machines
Reporter | ||
Comment 7•14 years ago
|
||
I think this should block because it's going to be our only automated way to catch broken windows nightlies of fennec, which extension developers are expected to use.
tracking-fennec: --- → ?
Comment 8•14 years ago
|
||
Same question as in bug 623613:
Do you want this pref set for the existing reftest / crashtest runs, or do you
want an additional set of runs with this pref set?
Which branches should this be applied to?
Reporter | ||
Comment 9•14 years ago
|
||
This needs to be an additional run for now, say CIPC (crashtest IPC) for short.
Re: branches, let's say
- m-c
- tracemonkey
- places
- project twigs
- electrolysis
in that order of importance.
Updated•14 years ago
|
tracking-fennec: ? → 2.0-
Updated•14 years ago
|
Priority: -- → P3
Hardware: x86_64 → x86
Reporter | ||
Comment 10•14 years ago
|
||
On my last try push (263d877928ac)
- opt crashtest-ipc passed on both win7 and winXP
- debug failed for the same reasons as in bug 623613 comment 32:
s: talos-r3-w7-004
REFTEST TEST-UNEXPECTED-PASS | file:///c:/talos-slave/test/build/reftest/tests/editor/libeditor/html/crashtests/615015-1.html | assertion count 0 is less than expected 6 assertions
REFTEST TEST-UNEXPECTED-PASS | file:///c:/talos-slave/test/build/reftest/tests/layout/generic/crashtests/586806-1.html | assertion count 0 is less than expected 11 assertions
Bug 590426 - Intermittent 586806-1.html | timed out waiting for reftest-wait to be removed (after onload fired) TEST-UNEXPECTED-FAIL | tab process 1096 | automationutils.processLeakLog() | leaked 1648 bytes during test execution
TEST-UNEXPECTED-FAIL | tab process 1096 | automationutils.processLeakLog() | leaked 1 instance of AtomImpl with size 20 bytes
TEST-UNEXPECTED-FAIL | tab process 1096 | automationutils.processLeakLog() | leaked 5 instances of FontTableBlobData with size 12 bytes each (60 bytes total)
TEST-UNEXPECTED-FAIL | tab process 1096 | automationutils.processLeakLog() | leaked 2 instances of PuppetWidget with size 216 bytes each (432 bytes total)
TEST-UNEXPECTED-FAIL | tab process 1096 | automationutils.processLeakLog() | leaked 1 instance of gfxASurface with size 24 bytes
TEST-UNEXPECTED-FAIL | tab process 1096 | automationutils.processLeakLog() | leaked 1 instance of gfxFont with size 112 bytes
So the opt windows crashtest-ipc are ready to be enabled! :)
Comment 11•14 years ago
|
||
cjones what is your expected schedule? and your expected priority?
Is this something still being actively worked on? For which version of Firefox do you want this?
I have my hands full so I would like to get things out in the right order depending on urgency.
OS: Linux → Windows 7
Priority: P3 → P4
Reporter | ||
Comment 12•14 years ago
|
||
I think this leak has been fixed so we could probably turn these tests on. I'm not 100% sure though. Fixing them up is somewhere near the middle of my priority queue.
For FF6 on desktop, this is bug very-good-to-have but not blocking priority. This is a hard blocker for the FF7 timeframe.
This bug would be useful for fennec, to test the fennec-windows-desktop builds shipped for extension developers, but I can't comment on the priority of that.
Comment 13•14 years ago
|
||
The current load on Windows machines is insane.
What I could do is look into enabling it but with jobs of low priority until we do buy the new testing infrastructure pool that would have more capacity.
Priority: P4 → P3
Whiteboard: [unittest][new suite] → [unittest][new suite][win7] ffx6 good to have; ffx7 hardblocker
Reporter | ||
Updated•12 years ago
|
Summary: Enable crashtests for out-of-process content on Windows machines → Enable crashtest-ipc for Windows machines
Assignee | ||
Comment 14•12 years ago
|
||
Attachment #719313 -
Flags: review?(armenzg)
Assignee | ||
Comment 15•12 years ago
|
||
Assignee | ||
Comment 16•12 years ago
|
||
linux has no changes.
Updated•12 years ago
|
Attachment #719313 -
Flags: review?(armenzg) → review+
Assignee | ||
Comment 17•12 years ago
|
||
Comment on attachment 719313 [details] [diff] [review]
enable reftest-ipc and crashtest-ipc for windows and mac
http://hg.mozilla.org/build/buildbot-configs/rev/977a79f18af3
Attachment #719313 -
Flags: checked-in+
Assignee | ||
Comment 18•12 years ago
|
||
I see Ripc and Cipc for osx and windows in inbound.
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Assignee | ||
Comment 19•12 years ago
|
||
Copying from the linux unittest configs.
Attachment #721447 -
Flags: review?(armenzg)
Updated•12 years ago
|
Attachment #721447 -
Flags: review?(armenzg) → review+
Assignee | ||
Comment 20•12 years ago
|
||
Comment on attachment 721447 [details] [diff] [review]
actually have ipc test configs for win+mac in mozharness
http://hg.mozilla.org/build/mozharness/rev/b12acff27f2f
Ran unit.sh, merged to production.
Attachment #721447 -
Flags: checked-in+
Assignee | ||
Comment 21•12 years ago
|
||
Per Philor, these are largely broken.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Summary: Enable crashtest-ipc for Windows machines → Enable {crash,ref}test-ipc for Windows and Mac machines on Try
Whiteboard: [unittest][new suite][win7] ffx6 good to have; ffx7 hardblocker → [unittest][new suite]
Assignee | ||
Comment 22•12 years ago
|
||
I'll attach the builderlist diffs shortly.
Attachment #721542 -
Flags: review?(armenzg)
Assignee | ||
Comment 23•12 years ago
|
||
Just removing ipc mac tests
Attachment #719314 -
Attachment is obsolete: true
Assignee | ||
Comment 25•12 years ago
|
||
These tests are currently hidden. Who is going to get them green?
Updated•12 years ago
|
Attachment #721542 -
Flags: review?(armenzg) → review+
Assignee | ||
Comment 26•12 years ago
|
||
Comment on attachment 721542 [details] [diff] [review]
only schedule win+mac ipc tests on try
http://hg.mozilla.org/build/buildbot-configs/rev/e8077c9ee77d
Attachment #721542 -
Flags: checked-in+
Comment 27•12 years ago
|
||
Merged and reconfiguration completed.
Assignee | ||
Comment 28•12 years ago
|
||
These are now enabled on Try only.
How should we track fixing these tests on Try?
Assignee: nobody → cjones.bugs
Comment 29•12 years ago
|
||
And to make that question even more questionable, between the time you pushed and the time of the reconfig the Mac ones (which were only rather bent, unlike the Windows ones that are totally broken) got greened up, so they're running green and hidden on try.
Comment 30•11 years ago
|
||
Found in triage.
Not clear if this is still needed, or what current state is. Anyone?
Component: Release Engineering → Release Engineering: Automation (General)
QA Contact: catlee
Updated•11 years ago
|
Product: mozilla.org → Release Engineering
Comment 31•11 years ago
|
||
Status:
Somebody needed to have filed bug 906382 to run Mac crashtest-ipc everywhere, and somebody needs to fix it.
Somebody needed to have filed bug 906383 to green up Mac reftest-ipc, and somebody needs to fix it, so somebody can file a bug to turn on Mac reftests everywhere.
Windows was shut off again on try, so it needs somebody new to care who will file a new turn-it-on-on-try bug, and will green it up.
Assignee: cjones.bugs → aki
Status: REOPENED → RESOLVED
Closed: 12 years ago → 11 years ago
OS: Windows 7 → Mac OS X
Resolution: --- → FIXED
Summary: Enable {crash,ref}test-ipc for Windows and Mac machines on Try → Enable {crash,ref}test-ipc for Mac machines on Try
Updated•7 years ago
|
Component: General Automation → General
You need to log in
before you can comment on or make changes to this bug.
Description
•