513926 - Tracking bug for Reftest Failures on Windows CE

Status: RESOLVED WONTFIX

(Core :: General, defect, P3)

Description

[cmtalbert]

Attachment: First pass of results on Tegra

This is a tracking bug to serve help track Reftest failures on Windows CE tegra based devices.  These tests all failed using a reftest over http approach which was proven to work on a standard firefox build.

However, many of these tests failed on windows CE.  As we discover the root causes of these failures, we will make new bugs and have them block this bug so that we can keep track of all of them.

I'm attaching the first full pass of logs from a Windows CE tegra build from 20090824.  The zip file contains the raw logs and grepped failures.  The files had to be run in three large batches due to Firefox crashing, hence the three files.

Comment 1

[cmtalbert]

The only failures we observed when running these reftests over http with a standard firefox build are detailed in bug 513377.

Comment 2

[Jonathan Griffin (:jgriffin)]

For instructions on how to run reftests remotely on the tegra, see:

http://wiki.github.com/jonallengriffin/moz-remote-reftest

Comment 3

[Justin Dolske [:Dolske]]

One thing I immediately notice is that since the device is running in 16 bit graphics mode, modules/libpr0n/test/reftest/colordepth.html fails (because it explicitly checks for 24/32bpp), and other libpr0n tests fail (because they require 24/32bpp).

These should just be turned off for WinCE.

It would be interesting to run reftests on desktop Firefox in 16bpp mode, to see if other reftests fail because of this. [It shouldn't, as when we had problems with the main Win32 test VMs reverting to 16bpp mode, these seemed to be the only tests affected. But maybe some have since slipped in.]

Comment 4

[Justin Dolske [:Dolske]]

Also, have you run the tests twice, to see if any of these are intermittent-failures on WinCE?

Comment 5

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

We probably don't need a separate bug for each failed reftest here.

> The REFTEST IMAGE data which is normally available for failed reftests, for use
in the reftest analyzer, is not available for tests run on the tegra device.

Why are these lines not available?  They're printed just as dump()s, in the same way that the success/failure is printed.

Comment 6

[Jonathan Griffin (:jgriffin)]

> Why are these lines not available?  They're printed just as dump()s, 
> in the same way that the success/failure is printed.

These dump()'s had to be removed in order to run the reftests, as otherwise they cause Firefox on the tegra to crash.

Comment 7

[cmtalbert]

(In reply to comment #3)
> One thing I immediately notice is that since the device is running in 16 bit
> graphics mode, modules/libpr0n/test/reftest/colordepth.html fails (because it
> explicitly checks for 24/32bpp), and other libpr0n tests fail (because they
> require 24/32bpp).
> 
> These should just be turned off for WinCE.
> 
> It would be interesting to run reftests on desktop Firefox in 16bpp mode, to
> see if other reftests fail because of this. [It shouldn't, as when we had
> problems with the main Win32 test VMs reverting to 16bpp mode, these seemed to
> be the only tests affected. But maybe some have since slipped in.]
I did this yesterday and have a whole slew of tests that failed when run in 16 bit color mode on a windows system.  Unfortunately they were not identical to the failures on tegra (there were some that only failed on desktop), but I think all the libpr0n and color depth tests can probably be discarded for now in favor of fixing the other issues.

Comment 8

[cmtalbert]

Comment on attachment 397875 [details]
First pass of results on Tegra


** New Logs: http://people.mozilla.org/~ctalbert/wince/20090901 **

I went back to investigate the crash on writing dataURLs out and found that it no longer occurs.  These logs have the dataURLs written out so they will be far, far more useful.  We'll be sure to file bugs sooner next time.  I was too consumed with just trying to get the testing to stand up last time.  I apologize.  

I now have new logs, so I'm marking this one obsolete.

These new logs were generated using a 1.9.2 pull from yesterday's nightly build.  And the tegra run was performed using the standard Tegra nightly: Mozilla/5.0 (Windows; U; WindowsCE 6.0; en-US; rv:1.9.2a2pre) Gecko/20090901 Namoroka/3.6a2pre

Also, it crashed only once on test 2548.  However, re-running the run from test 2548 did not recreate the crash.

The full logs and the failure-only log will be at http://people.mozilla.org/~ctalbert/wince/20090901 because the logs are larger than the maximum file size that bugzilla allows.

Comment 9

[Jonathan Griffin (:jgriffin)]

> Also, have you run the tests twice, to see if any of these are
> intermittent-failures on WinCE?

Yes, I just attached logs for a second run to all of the dependent bugs.  All failures previously reported still occur.

Comment 10

[Ed Morley [:emorley]]

WinCE/Windows Mobile support has been removed from the main build system, Spidermonkey, mobile installer, in-app updater and so on (see bug 614720, bug 554087 and all their dependants). Until such point where MS decide to release a Windows Phone 7 NDK and the decision is made to port to that platform, this is WONTFIX.

Filter bugmail on WinCEMassWONTFIX.