Closed Bug 1268177 Opened 5 years ago Closed 4 years ago

Intermittent test_session_zombification.html | application timed out after 330 seconds with no output

Categories

(Firefox for Android :: Testing, defect)

defect
Not set
normal

Tracking

()

RESOLVED FIXED
Firefox 49
Tracking Status
firefox49 --- fixed

People

(Reporter: KWierso, Assigned: JanH)

References

Details

(Keywords: intermittent-failure)

Attachments

(1 file)

Assignee: nobody → jh+bugzilla
That log indicates that the test is hanging here (https://hg.mozilla.org/mozilla-central/file/fc15477ce628599519cb0055f52cc195d640dc94/mobile/android/tests/browser/chrome/test_session_zombification.html#l77), which is somewhat mystifying. Looking at the log, we do receive a DOMContentLoaded event, so promiseBrowserEvent should resolve and return control back to the test, but in this case that never happens.
However at precisely that point in time, we've also received a memory-pressure event (https://treeherder.mozilla.org/logviewer.html#?job_id=26648365&repo=mozilla-inbound#L7772).
I can't test my theory right now because I've manged to bork my development VM while trying to upgrade Ubuntu, but I strongly suspect that the mochitest main page - which was in a background tab at that time - was zombified into oblivion and with it the test script as well.
If so, a possible solution could be to just generally disallow automatic background tab zombifications during mochitests.
Component: General → Testing
Blocks: 1268981
During tests that open additional tabs, the main test tab controlling the test is at risk of being zombified if a memory pressure event arrives, which breaks the test and leads to the test timing out. Therefore, we now disable background tab zombifications during tests.

This means that there's a slightly increased risk of being OOM killed instead, however
- the tabs opened by the tests themselves are normally relatively short-lived anyway
- we're no worse off than if the tab containing the test harness code had been zombified.

Review commit: https://reviewboard.mozilla.org/r/49797/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/49797/
Attachment #8747280 - Flags: review?(margaret.leibovic)
Comment on attachment 8747280 [details]
MozReview Request: Bug 1268177 - Disable background tab zombifications during tests. r=margaret

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/49797/diff/1-2/
See Also: → 1267481
See Also: 1267481
Comment on attachment 8747280 [details]
MozReview Request: Bug 1268177 - Disable background tab zombifications during tests. r=margaret

https://reviewboard.mozilla.org/r/49797/#review47535

Sounds like a reasonable approach to me. Have you run this on try to verify this reduces the occurence of this failure?
Attachment #8747280 - Flags: review?(margaret.leibovic) → review+
See Also: → 1270334
No, but I can reproduce it locally by forcing a memory pressure event (adb shell am broadcast -a org.mozilla.gecko.FORCE_MEMORY_PRESSURE) while the test harness tab is in background. With this patch enabled, background tabs are no longer zombified and the test just keeps going, so I'm fairly confident it'll work on try, too.

I don't know if it's just a coincidence or not, but while looking through some logcats from try a common theme seems to be that often the backup service is active on the emulator around the time the memory pressure event is triggered - I'm wondering how easy it would be to disable the backup service on the emulator images used for try.
(In reply to Jan Henning [:JanH] from comment #5)

> I'm wondering how easy it would be to disable the backup
> service on the emulator images used for try.

That sounds like a good idea. If you can tell me how to disable the backup service, I can update the emulator images.
Good question - maybe Sebastian knows better?
Flags: needinfo?(s.kaspari)
Looking at an emulator, it might be enough to turn backups off under "Settings -> Backup & reset", but I'm not sure whether something might be hidden elsewhere - my normal phone is somewhat older, so I'm not really familiar with it
https://hg.mozilla.org/mozilla-central/rev/509fe4d4251e
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 49
See Also: 1270334
Duplicate of this bug: 1270334
(In reply to Jan Henning [:JanH] from comment #7)
> Good question - maybe Sebastian knows better?

No idea. Backup for the app is disabled[1] and I assume we do not have a Google account assigned with the emulator images? So what is creating the backup and uploading to where? Is it actually doing anything? Maybe worth investigating in a follow-up?

[1] https://dxr.mozilla.org/mozilla-central/source/mobile/android/base/AndroidManifest.xml.in#30
Flags: needinfo?(s.kaspari)
(In reply to Sebastian Kaspari (:sebastian) from comment #13)
> (In reply to Jan Henning [:JanH] from comment #7)
> > Good question - maybe Sebastian knows better?
> 
> No idea. Backup for the app is disabled[1] and I assume we do not have a
> Google account assigned with the emulator images? 

That's right - no account.

> So what is creating the
> backup and uploading to where? Is it actually doing anything? Maybe worth
> investigating in a follow-up?

I don't know what it is doing. Maybe I'll just look into disabling backup for future images, but leave this one as-is for now.
(In reply to Sebastian Kaspari (:sebastian) from comment #13)
> (In reply to Jan Henning [:JanH] from comment #7)
> > Good question - maybe Sebastian knows better?
> 
> So what is creating the backup and uploading to where? Is it actually doing anything?
> Maybe worth investigating in a follow-up?

If you look at a random logcat (e.g. http://mozilla-releng-blobs.s3.amazonaws.com/blobs/mozilla-central/sha512/f8e92a1fc1bb7ea61779be9a2c8c1d9b477711787962bbbdf8a32e63991a56392ce0924854ee030e09dca1bed97ab0ba6e73d7059d48f88218a1db328b43deee) and search for "backup" you'll find a number of entries from the BackupManagerService.
See Also: → 1271836
You need to log in before you can comment on or make changes to this bug.