850472 - indexedDB tests are pretty close to perma-orange on Win8

Status: RESOLVED DUPLICATE of bug 783913

(Core :: IPC, defect)

Description

[Jim Mathies [:jimm]]

starting with:

17106 ERROR TEST-UNEXPECTED-FAIL | /tests/dom/indexedDB/test/test_add_put.html | Test timed out.

Bug 783913 covers intermittent failures for these on other platforms.

I've run these locally and see no issues. Maybe we should just disable on Win8.

Comment 1

[Phil Ringnalda (:philor)]

Just do the world a favor and completely remove those tests - after 7 months as a topfail, we finally learned that they are completely unowned, and there's zero chance of anyone fixing them.

Comment 2

[Jim Mathies [:jimm]]

(In reply to Phil Ringnalda (:philor) from comment #1)
> Just do the world a favor and completely remove those tests - after 7 months
> as a topfail, we finally learned that they are completely unowned, and
> there's zero chance of anyone fixing them.

Looks like test_ipc.html is the root cause so maybe we could disable that only and keep the indexedDB tests that run in process.

Comment 3

[Ben Turner (not reading bugmail, use the needinfo flag!)]

Honestly, comment 1 baffles me. As philor and others know well, both khuey and I have spent a lot of time looking into this issue. The tests always pass locally and we've been unable to catch the failure in our record+replay VMs despite lots of attempts. This does not in any way mean that the tests are unowned or that they will never be fixed.

The reason they're still enabled is that they're the closest thing we have to exercising the IPC code paths that are vital for B2G. Disabling them on a per-platform basis (as was done for Android, for instance) means we could break IPC and never really know about it. It's a big risk to take. On the other hand leaving them enabled means that we have to deal with a steady (like, 5- to 10-ish per day?) stream of pseudo-random oranges.

Comment 4

[Jim Mathies [:jimm]]

(In reply to ben turner [:bent] from comment #3)
> Honestly, comment 1 baffles me. As philor and others know well, both khuey
> and I have spent a lot of time looking into this issue. The tests always
> pass locally and we've been unable to catch the failure in our record+replay
> VMs despite lots of attempts. This does not in any way mean that the tests
> are unowned or that they will never be fixed.
> 
> The reason they're still enabled is that they're the closest thing we have
> to exercising the IPC code paths that are vital for B2G. Disabling them on a
> per-platform basis (as was done for Android, for instance) means we could
> break IPC and never really know about it. It's a big risk to take. On the
> other hand leaving them enabled means that we have to deal with a steady
> (like, 5- to 10-ish per day?) stream of pseudo-random oranges.

The situation is going to get more complicated pretty soon. We're migrating all of our tests to new hardware, which is what Win8 tests are currently running on. So that steady stream is going to become a river soonish on Windows, which means we either need to find a fix or disable temporarily until we have the cycles to do so.

Comment 5

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

(In reply to Jim Mathies [:jimm] from comment #4)
> The situation is going to get more complicated pretty soon. We're migrating
> all of our tests to new hardware, which is what Win8 tests are currently
> running on. So that steady stream is going to become a river soonish on
> Windows, which means we either need to find a fix or disable temporarily
> until we have the cycles to do so.

Why do you think it's hardware related instead of OS related?

Comment 6

[Jim Mathies [:jimm]]

(In reply to Kyle Huey [:khuey] ([e-mail]) from comment #5)
> (In reply to Jim Mathies [:jimm] from comment #4)
> > The situation is going to get more complicated pretty soon. We're migrating
> > all of our tests to new hardware, which is what Win8 tests are currently
> > running on. So that steady stream is going to become a river soonish on
> > Windows, which means we either need to find a fix or disable temporarily
> > until we have the cycles to do so.
> 
> Why do you think it's hardware related instead of OS related?

Per Ben's comments, we haven't been able to reproduce locally. I can't reproduce locally on Win8 or Win7. We have sporadic failures on Win7 on mac minis, and perma-orange on win8 using the new ix servers. That to me seems to point to a timing / hardware dep issue.

Comment 7

[Jim Mathies [:jimm]]

(In reply to Jim Mathies [:jimm] from comment #6)
> (In reply to Kyle Huey [:khuey] ([e-mail]) from comment #5)
> > (In reply to Jim Mathies [:jimm] from comment #4)
> > > The situation is going to get more complicated pretty soon. We're migrating
> > > all of our tests to new hardware, which is what Win8 tests are currently
> > > running on. So that steady stream is going to become a river soonish on
> > > Windows, which means we either need to find a fix or disable temporarily
> > > until we have the cycles to do so.
> > 
> > Why do you think it's hardware related instead of OS related?
> 
> Per Ben's comments, we haven't been able to reproduce locally. I can't
> reproduce locally on Win8 or Win7. We have sporadic failures on Win7 on mac
> minis, and perma-orange on win8 using the new ix servers. That to me seems
> to point to a timing / hardware dep issue.

(Or at least that the new hardware is going to make the problem worse. It might not be hardware related.)

Comment 8

[Jim Mathies [:jimm]]

Also they seem to fail more often on debug builds - 

https://tbpl.mozilla.org/?tree=Cedar&showall=0&rev=6a01997ec08c

Comment 9

[Jim Mathies [:jimm]]

Have we considered changing the way these tests run? We have one mochitest 'test_ipc.html' with an incredibly long timeout that runs another set of mochitests in iframes. Maybe we should change this so that these ipc tests run individually as normal tests?

Comment 10

[Ben Turner (not reading bugmail, use the needinfo flag!)]

(In reply to Jim Mathies [:jimm] from comment #9)
> Have we considered changing the way these tests run? We have one mochitest
> 'test_ipc.html' with an incredibly long timeout that runs another set of
> mochitests in iframes. Maybe we should change this so that these ipc tests
> run individually as normal tests?

The timeout for test_ipc.html is irrelevant, I think, as it's not that test_ipc.html is timing out. The issue is somewhere between the point at which test_ipc.html finishes and when the first non-ipc test starts.

Comment 11

[Jim Mathies [:jimm]]

armenzg is going to enable the win8 tests on mi, mc, and try as hidden so we'll be able to get a feel for how bad things are on the new hardware in a day or so.