Closed
Bug 1121205
Opened 10 years ago
Closed 9 years ago
Bugscache population deletes all rows before re-populating, can lead to races/problems
Categories
(Tree Management :: Treeherder: Data Ingestion, defect, P3)
Tree Management
Treeherder: Data Ingestion
Tracking
(Not tracked)
RESOLVED
DUPLICATE
of bug 1231361
People
(Reporter: KWierso, Unassigned)
References
Details
>[14:16] RyanVM bug 1113038 isn't being suggested anymore
>[14:16] firebot https://bugzil.la/1113038 — ASSIGNED, bmcbride@mozilla.com — Intermittent browser_UITour3.js | Popup was opened -
>[14:17] RyanVM nor is bug 1109932
>[14:17] firebot https://bugzil.la/1109932 — NEW, nobody@mozilla.org — Intermittent We have not been able to establish a telnet connection with the emulator
>[14:17] RyanVM which is #1 on orangefactor
I also see bug 1115781 not being suggested for https://treeherder.mozilla.org/logviewer.html#?job_id=1157876&repo=b2g-inbound
Bug 1111137 is not being suggested for https://treeherder.mozilla.org/logviewer.html#?job_id=5422543&repo=mozilla-inbound but this might be because the version number has bumped from 37 to 38 on trunk.
It's not all failing, though. Several bugs are still being correctly suggested for some failures.
Reporter | ||
Comment 1•10 years ago
|
||
Comment 2•10 years ago
|
||
(In reply to Wes Kocher (:KWierso) from comment #0)
> I also see bug 1115781 not being suggested for
> https://treeherder.mozilla.org/logviewer.html#?job_id=1157876&repo=b2g-
> inbound
-> https://treeherder.mozilla.org/#/jobs?repo=b2g-inbound&revision=f9625445803a&filter-searchStr=b2g_emulator_vm%20b2g-inbound%20opt%20test%20reftest-8
Grabbing the bug suggestion artefact URL from the web console:
https://treeherder.mozilla.org/api/project/b2g-inbound/artifact/?job_id=1157876&name=Bug+suggestions&type=json
{
"search": "TEST-UNEXPECTED-FAIL | http://10.0.2.2:8888/tests/dom/canvas/test/reftest/webgl-orientation-test.html?__&alpha&preserve | application timed out after 330 seconds with no output",
"bugs": {
"open_recent": [],
"all_others": []
}
},
The search term used _should_ have been:
"webgl-orientation-test.html?__&alpha&preserve"
Which should have been queried using:
https://treeherder.mozilla.org/api/bugscache/?search=webgl-orientation-test.html?__&alpha&preserve
-> Which at least at the moment, returns bug suggestions. Strange.
{
"open_recent": [{
"crash_signature": "",
"resolution": "",
"summary": "Intermittent webgl-orientation-test.html?__&alpha&________ | application timed out after 330 seconds with no output",
"relevance": 1.00,
"keywords": "intermittent-failure",
"os": "Gonk (Firefox OS)",
"id": 1108506
},
{
"crash_signature": "",
"resolution": "",
"summary": "Intermittent webgl-orientation-test.html?__&alpha&preserve | application timed out after 330 seconds with no output",
"relevance": 1.00,
"keywords": "intermittent-failure",
"os": "Gonk (Firefox OS)",
"id": 1105683
}],
"all_others": []
}
> Bug 1111137 is not being suggested for
> https://treeherder.mozilla.org/logviewer.html#?job_id=5422543&repo=mozilla-
> inbound
-> https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=e7da7bc52c94&filter-searchStr=Android%204.0%20armv7%20API%2010%2B%20mozilla-inbound%20debug%20test%20mochitest-8
The bug suggestions artefact:
https://treeherder.mozilla.org/api/project/mozilla-inbound/artifact/?job_id=5422543&name=Bug+suggestions&type=json
{
"search": "520 INFO TEST-UNEXPECTED-FAIL | netwerk/test/mochitests/test_user_agent_overrides.html | Navigator UA not overridden at step 1 - got Mozilla/5.0 (Android; Tablet; rv:38.0) Gecko/38.0 Firefox/38.0, expected DummyUserAgent",
"bugs": {
"open_recent": [{
"crash_signature": "",
"resolution": "",
"summary": "Intermittent test_user_agent_overrides.html | uncaught exception - NS_ERROR_FAILURE: at http://mochi.test:8888/tests/netwerk/test/mochitests/test_user_agent_overrides.html:38",
"relevance": 1.00,
"keywords": "intermittent-failure",
"os": "Android",
"id": 1065132
}],
"all_others": [{
"crash_signature": "",
"resolution": "WORKSFORME",
"summary": "Intermittent netwerk/test/mochitests/test_user_agent_overrides.html | application timed out after 330 seconds with no output",
"relevance": 1.00,
"keywords": "intermittent-failure",
"os": "Mac OS X",
"id": 920514
}]
}
},
The presumed search term:
"test_user_agent_overrides.html"
Bugs lookup:
https://treeherder.mozilla.org/api/bugscache/?search=test_user_agent_overrides.html
Result:
{
"open_recent": [{
"crash_signature": "",
"resolution": "",
"summary": "Intermittent test_user_agent_overrides.html | uncaught exception - NS_ERROR_FAILURE: at http://mochi.test:8888/tests/netwerk/test/mochitests/test_user_agent_overrides.html:38",
"relevance": 1.00,
"keywords": "intermittent-failure",
"os": "Android",
"id": 1065132
},
{
"crash_signature": "",
"resolution": "",
"summary": "Intermittent test_user_agent_overrides.html | Navigator UA not overridden at step 1 - got Mozilla/5.0 (Android; Mobile; rv:37.0) Gecko/37.0 Firefox/37.0, expected DummyUserAgent",
"relevance": 1.00,
"keywords": "intermittent-failure,leave-open",
"os": "Android",
"id": 1111137
}],
"all_others": [{
"crash_signature": "",
"resolution": "WORKSFORME",
"summary": "Intermittent netwerk/test/mochitests/test_user_agent_overrides.html | application timed out after 330 seconds with no output",
"relevance": 1.00,
"keywords": "intermittent-failure",
"os": "Mac OS X",
"id": 920514
}]
}
-> Which strangely includes the missing bug, at least now.
Comment 3•10 years ago
|
||
Random thought: I've just noticed that when we update the bugscache, it first purges *all* of the old bugs, and then repopulates it:
https://github.com/mozilla/treeherder-service/blob/master/treeherder/model/derived/refdata.py#L1308
If the insert of the updated jobs fails, or if a lookup races with the insert, then I guess we'd miss bugs, right? Ideally we should diff the old and new bugs list, and just prune the ones that are missing, then update the others.
Mauro, can you think of any other explanation?
Flags: needinfo?(mdoglio)
Updated•10 years ago
|
Component: Treeherder → Treeherder: Data Ingestion
OS: Windows 8.1 → All
Priority: -- → P1
Hardware: x86_64 → All
Reporter | ||
Comment 4•10 years ago
|
||
And now I see a suggestion for 1113038.
Updated•10 years ago
|
Assignee: nobody → emorley
Comment 5•10 years ago
|
||
I can't think of any other reason, your idea sounds correct to me.
Flags: needinfo?(mdoglio)
Updated•10 years ago
|
Status: NEW → ASSIGNED
Updated•10 years ago
|
Assignee: emorley → nobody
Status: ASSIGNED → NEW
Priority: P1 → P2
Updated•10 years ago
|
Summary: Some intermittent bug suggestions are no longer being suggested. → Bugscache population deletes all rows before re-populating, can lead to races/problems
Comment 6•9 years ago
|
||
I can't recall the last time I've seen something like this play out. I'm inclined to WFM at this point.
Flags: needinfo?(emorley)
Comment 7•9 years ago
|
||
I think this is more luck than anything else, since the code as written isn't ideal. I'll leave this open to improve it, though the priority is lower if it's not occurring often :-)
Flags: needinfo?(emorley)
Priority: P2 → P3
Comment 8•9 years ago
|
||
This was magically fixed when I landed bug 1231361. \o/
(thanks to :emorley for pointing out what I did)
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → DUPLICATE
You need to log in
before you can comment on or make changes to this bug.
Description
•