1060716 - No longer possible to edit/rerun a mochitest reliably without restarting the harness

Status: NEW

(Testing :: Mochitest, defect)

Description

[Boris Zbarsky [:bzbarsky]]

STEPS TO REPRODUCE:

1)  mach mochitest-plain dom/tests/mochitest/general/test_interfaces.html and
    don't quit it.
2)  Edit the test by commenting out the line that says "AnalyzerNode"
3)  Focus URL bar in the test browser instance and hit enter to rerun the test.

About half the time, the test reruns and passes instead of rerunning and failing.  This caused me quite a bit of confusion earlier today when I was debugging a test, because it failed when it shouldn't have and passed when it shouldn't have...

This is a regression from bug 1014062: enter in the URL bar forces revalidation for the toplevel page, but that doesn't apply to subframes, so the subframe may or may not actually get reloaded from not-cache, depending on black magic in the HTTP implementation.

Using reload/shift-reload is not an option, unfortunately, because that will really confuse tests that involve history in any way.  :(

Comment 1

[Boris Zbarsky [:bzbarsky]]

> Using reload/shift-reload is not an option, unfortunately

After using this some more, it's not an option because it's not reliable.  I haven't debugged the harness, but I assume the issue is that it's racy in terms of when the load in the subframe starts, and depending on when that happens it may or may not inherit the reload/cache-bypass flags from the toplevel document.

Case in point, I just tried shift-reloading and that loaded the cached version of the test.  Right-clicking and doing "show only this frame" followed by reloading worked, though...

Comment 2

[:Gijs (he/him)]

(taking this discussion here because it seems to make more sense here than in the fixed bug)

(In reply to Boris Zbarsky [:bz] from bug 1014062 comment #25)
> (In reply to Gijs from bug 1014062 comment #24)
> > and have that just remove and re-set src= to the input's contents on the iframe
> 
> We could try that, but I'm not sure it would trigger the "revalidate or
> bypass cache" behavior that's needed here, sadly.  :(  I seem to recall us
> doing something special for enter in url bar in terms of revalidation....
> but maybe we're luck and I'm wrong.

Hrm... alternatively, we could pushState the current test path as a query string in the URL, and if the test framework is loaded with a query string present, load that test (passing a cache-invalidation query string if necessary, ie ?_=timestamp or something). That way, you get both the URL manipulation back and we should hopefully deal with caching adequately...

Comment 3

[Boris Zbarsky [:bzbarsky]]

> passing a cache-invalidation query string if necessary

I'd worry a bit about that affecting some tests that examine location.href, but we could try...

Comment 4

[:Gijs (he/him)]

(In reply to Boris Zbarsky [:bz] from comment #3)
> > passing a cache-invalidation query string if necessary
> 
> I'd worry a bit about that affecting some tests that examine location.href,
> but we could try...

Mm, I think I could live with this not working for those tests if we fix the general case. I mean, I'd prefer something less icky... but am not sure we have a better plan at this point, short of reverting the bug which isn't really desirable...

Comment 5

[(not currently active) Ted Mielczarek]

Can we just write some SpecialPowers-using code to manually blow out the cache when we load the top-level document?

Comment 6

[:Gijs (he/him)]

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #5)
> Can we just write some SpecialPowers-using code to manually blow out the
> cache when we load the top-level document?

We could, although that's likely to not improve the reload times (bug 1066158)...