Closed Bug 1638183 Opened 4 years ago Closed 4 years ago

Intermittent TEST-UNEXPECTED-CRASH | /IndexedDB/<random> | expected OK

Categories

(Core :: Storage: IndexedDB, defect, P5)

Product:
Core
Component:
Storage: IndexedDB
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla78
Tracking Flags:
Tracking Status
firefox78 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: egao)

References

Details

(Keywords: intermittent-failure)

Attachments

(1 file, 1 obsolete file)

Bug 1638183 - increase xorg client count to 2048 for ubuntu1804-test docker image
47 bytes, text/x-phabricator-request
Details | Review

Filed by: dluca [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=302349842&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/d9bhwLNOQDmwBKB6c-YnVA/runs/0/artifacts/public/logs/live_backing.log

 INFO - TEST-START | /css/selectors/focus-display-none-001.html
[task 2020-05-14T22:47:52.955Z] 22:47:52     INFO - Closing window 27
[task 2020-05-14T22:47:53.066Z] 22:47:53     INFO - PID 1054 | -----------------------------------------------------
[task 2020-05-14T22:47:53.067Z] 22:47:53     INFO - PID 1054 | Suppressions used:
[task 2020-05-14T22:47:53.067Z] 22:47:53     INFO - PID 1054 |   count      bytes template
[task 2020-05-14T22:47:53.069Z] 22:47:53     INFO - PID 1054 |      24        744 nsComponentManagerImpl
[task 2020-05-14T22:47:53.069Z] 22:47:53     INFO - PID 1054 |     633      18287 libfontconfig.so
[task 2020-05-14T22:47:53.069Z] 22:47:53     INFO - PID 1054 | -----------------------------------------------------
[task 2020-05-14T22:47:53.213Z] 22:47:53     INFO - PID 1054 | ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
[task 2020-05-14T22:47:53.310Z] 22:47:53     INFO - IOError on command, setting status to CRASH
[task 2020-05-14T22:47:53.344Z] 22:47:53     INFO - TEST-UNEXPECTED-CRASH | /IndexedDB/idbobjectstore_get6.htm | expected OK
[task 2020-05-14T22:47:53.345Z] 22:47:53     INFO - TEST-INFO took 1831ms
[task 2020-05-14T22:47:55.048Z] 22:47:55     INFO - PID 13893 | Maximum number of clients reached-----------------------------------------------------
[task 2020-05-14T22:47:55.049Z] 22:47:55     INFO - PID 13893 | Suppressions used:
[task 2020-05-14T22:47:55.049Z] 22:47:55     INFO - PID 13893 |   count      bytes template
[task 2020-05-14T22:47:55.049Z] 22:47:55     INFO - PID 13893 |      24        744 nsComponentManagerImpl
[task 2020-05-14T22:47:55.049Z] 22:47:55     INFO - PID 13893 |     633      18287 libfontconfig.so
[task 2020-05-14T22:47:55.049Z] 22:47:55     INFO - PID 13893 | -----------------------------------------------------
[task 2020-05-14T22:47:55.139Z] 22:47:55     INFO - PID 1054 | -----------------------------------------------------
[task 2020-05-14T22:47:55.139Z] 22:47:55     INFO - PID 1054 | Suppressions used:
[task 2020-05-14T22:47:55.139Z] 22:47:55     INFO - PID 1054 |   count      bytes template
[task 2020-05-14T22:47:55.139Z] 22:47:55     INFO - PID 1054 |      24        744 nsComponentManagerImpl
[task 2020-05-14T22:47:55.139Z] 22:47:55     INFO - PID 1054 |     633      18287 libfontconfig.so
[task 2020-05-14T22:47:55.139Z] 22:47:55     INFO - PID 1054 | -----------------------------------------------------
[task 2020-05-14T22:47:55.444Z] 22:47:55     INFO - PID 1054 | -----------------------------------------------------
[task 2020-05-14T22:47:55.445Z] 22:47:55     INFO - PID 1054 | Suppressions used:
[task 2020-05-14T22:47:55.445Z] 22:47:55     INFO - PID 1054 |   count      bytes template
[task 2020-05-14T22:47:55.445Z] 22:47:55     INFO - PID 1054 |      24        744 nsComponentManagerImpl
[task 2020-05-14T22:47:55.445Z] 22:47:55     INFO - PID 1054 |     633      18287 libfontconfig.so
[task 2020-05-14T22:47:55.445Z] 22:47:55     INFO - PID 1054 | -----------------------------------------------------
[task 2020-05-14T22:47:55.494Z] 22:47:55     INFO - PID 1054 | 1589496475484	Marionette	INFO	Stopped listening on port 46836
[task 2020-05-14T22:47:55.936Z] 22:47:55     INFO - PID 1054 | -----------------------------------------------------
[task 2020-05-14T22:47:55.936Z] 22:47:55     INFO - PID 1054 | Suppressions used:
[task 2020-05-14T22:47:55.937Z] 22:47:55     INFO - PID 1054 |   count      bytes template
[task 2020-05-14T22:47:55.937Z] 22:47:55     INFO - PID 1054 |      24        744 nsComponentManagerImpl
[task 2020-05-14T22:47:55.937Z] 22:47:55     INFO - PID 1054 |     633      18287 libfontconfig.so
[task 2020-05-14T22:47:55.937Z] 22:47:55     INFO - PID 1054 | -----------------------------------------------------
[task 2020-05-14T22:47:56.324Z] 22:47:56     INFO - ..
[task 2020-05-14T22:47:56.324Z] 22:47:56     INFO - TEST-OK | /css/selectors/focus-display-none-001.html | took 3381ms
[task 2020-05-14T22:47:56.325Z] 22:47:56     INFO - TEST-START | /css/selectors/focus-visible-005.html
Summary: Intermittent TEST-UNEXPECTED-CRASH | /IndexedDB/idbobjectstore_get6.htm | expected OK → Intermittent TEST-UNEXPECTED-CRASH | /IndexedDB/<random> | expected OK

I see

[task 2020-05-14T22:47:52.391Z] 22:47:52     INFO - PID 1054 | Maximum number of clients reachedUnable to init server: Could not connect: Connection refused
[task 2020-05-14T22:47:52.393Z] 22:47:52     INFO - PID 1054 | (/builds/worker/workspace/build/application/firefox/firefox:14566): Gtk-WARNING **: 22:47:52.379: cannot open display: :0

and then

[task 2020-05-14T22:47:53.344Z] 22:47:53     INFO - TEST-UNEXPECTED-CRASH | /IndexedDB/idbobjectstore_get6.htm | expected OK
[task 2020-05-14T22:47:53.345Z] 22:47:53     INFO - TEST-INFO took 1831ms
[task 2020-05-14T22:47:55.048Z] 22:47:55     INFO - PID 13893 | Maximum number of clients reached-----------------------------------------------------
[task 2020-05-14T22:47:55.049Z] 22:47:55     INFO - PID 13893 | Suppressions used:
[task 2020-05-14T22:47:55.049Z] 22:47:55     INFO - PID 13893 |   count      bytes template
[task 2020-05-14T22:47:55.049Z] 22:47:55     INFO - PID 13893 |      24        744 nsComponentManagerImpl
[task 2020-05-14T22:47:55.049Z] 22:47:55     INFO - PID 13893 |     633      18287 libfontconfig.so
[task 2020-05-14T22:47:55.049Z] 22:47:55     INFO - PID 13893 | ----------------------------------------------------

but no stack trace. I am pretty sure to have seen this kind of message on another intermittent, though I cannot find it right now. As also old thunderbird bug 607654 suggests, this seems to be caused by a lack of (window) handles on OS level.

Not sure, if the "Suppressions used" block gives some hint on the culprit? In any case it does not look like an IndexedDB problem at all.

Flags: needinfo?(dluca)
Flags: needinfo?(dluca)

Hi Geoff, given your involvement in bug 1590604, did we ever do something to tackle those Gtk display errors in the test harness? I assume, there is just some coincidence that cause some tests to hit this OS limit more frequently than others? Or might this indicate a leak of window handles?

Reading this post we might want to adjust the Option "MaxClients" "512" of the X-Server to something higher than the default ?

Flags: needinfo?(gbrown)

No, as far as I know, nothing was done to address Gtk display errors. I'm afraid I don't have much insight into the issue either.

:egao has recent experience with the linux docker configuration; maybe he can look at the MaxClients setting?

Flags: needinfo?(gbrown) → needinfo?(egao)

This is the first time I've seen this particular failure myself. I can try building a revised docker image with that option set.

Doe this test consistently produce this failure?

Flags: needinfo?(egao)

I have a try push where I increased the option mentioned in Comment 3 to 2048. Only one instance of the chunk has completed as of this writing, but the test seems to be passing as expected:

[task 2020-05-26T05:58:17.416Z] 05:58:17     INFO - TEST-START | /IndexedDB/idbobjectstore_get6.htm
[task 2020-05-26T05:58:17.422Z] 05:58:17     INFO - Closing window 599
[task 2020-05-26T05:58:17.605Z] 05:58:17     INFO - PID 1070 | ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
[task 2020-05-26T05:58:17.696Z] 05:58:17     INFO - PID 1070 | -----------------------------------------------------
[task 2020-05-26T05:58:17.696Z] 05:58:17     INFO - PID 1070 | Suppressions used:
[task 2020-05-26T05:58:17.696Z] 05:58:17     INFO - PID 1070 |   count      bytes template
[task 2020-05-26T05:58:17.696Z] 05:58:17     INFO - PID 1070 |      24        744 nsComponentManagerImpl
[task 2020-05-26T05:58:17.696Z] 05:58:17     INFO - PID 1070 |     633      18287 libfontconfig.so
[task 2020-05-26T05:58:17.696Z] 05:58:17     INFO - PID 1070 | -----------------------------------------------------
[task 2020-05-26T05:58:19.186Z] 05:58:19     INFO - .
[task 2020-05-26T05:58:19.187Z] 05:58:19     INFO - TEST-OK | /IndexedDB/idbobjectstore_get6.htm | took 1772ms

I've retriggered the chunk some more times. Assuming that this was a permafail, hopefully this solution can stick.

Assignee: nobody → egao
Attachment #9151831 - Attachment is obsolete: true
Pushed by egao@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/7bd39fb73c08
increase xorg client count to 2048 for ubuntu1804-test docker image r=jmaher

https://hg.mozilla.org/mozilla-central/rev/7bd39fb73c08

Status: NEW → RESOLVED
Closed: 4 years ago
status-firefox78: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla78
See Also: → 1629093
See Also: → semi-headless
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: