Closed Bug 689209 Opened 14 years ago Closed 14 years ago

Frequent Android "Graph server unreachable" with "No machine_name called 'tegra-X' can be found"

Categories

(Release Engineering :: General, defect)

Product:
Release Engineering
Component:
General
Platform:
ARM
Android
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: mbrubeck, Assigned: bear)

References

Details

(Keywords: intermittent-failure, Whiteboard: [android_tier_1][android][tegra])

Attachments

(1 file, 1 obsolete file)

patch to add new tegras to graph server
4.90 KB, patch
jmaher
: review+
Details | Diff | Splinter Review
Since around this weekend, I've been seeing a lot of Android Tegra Talos jobs that finish successfully but then fail with a message like this: https://tbpl.mozilla.org/php/getParsedLog.php?id=6553909&tree=Firefox RETURN: twinopen: 334.84 FAIL: Graph server unreachable (5 attempts) RETURN:send failed, graph server says: RETURN:No machine_name called 'tegra-113' can be found RETURN: File "/data/www/graphs.mozilla.org/server/pyfomatic/collect.py", line 264, in handleRequest RETURN: metadata = MetaDataFromTalos(databaseCursor, databaseModule, inputStream) RETURN: File "/data/www/graphs.mozilla.org/server/pyfomatic/collect.py", line 59, in __init__ RETURN: self.doDatabaseThings(databaseCursor) RETURN: File "/data/www/graphs.mozilla.org/server/pyfomatic/collect.py", line 88, in doDatabaseThings RETURN: raise DatabaseException("No machine_name called '%s' can be found" % self.machine_name) RETURN: RETURN: Other examples, earliest first: https://tbpl.mozilla.org/php/getParsedLog.php?id=6532908&tree=Firefox https://tbpl.mozilla.org/php/getParsedLog.php?id=6531106&tree=Firefox https://tbpl.mozilla.org/php/getParsedLog.php?id=6536954&tree=Firefox https://tbpl.mozilla.org/php/getParsedLog.php?id=6537219&tree=Firefox https://tbpl.mozilla.org/php/getParsedLog.php?id=6536949&tree=Firefox https://tbpl.mozilla.org/php/getParsedLog.php?id=6533128&tree=Firefox https://tbpl.mozilla.org/php/getParsedLog.php?id=6537340&tree=Firefox https://tbpl.mozilla.org/php/getParsedLog.php?id=6537343&tree=Firefox https://tbpl.mozilla.org/php/getParsedLog.php?id=6537513&tree=Firefox
Component: Talos → Release Engineering
Product: Testing → mozilla.org
QA Contact: talos → release
Version: Trunk → other
Whiteboard: [orange] → [orange][android_tier_1]
Assignee: nobody → bear
Whiteboard: [orange][android_tier_1] → [orange][android_tier_1][android][tegra]
Attachment #562493 - Flags: review?(jmaher)
Comment on attachment 562493 [details] [diff] [review] patch to add new tegras to graph server Review of attachment 562493 [details] [diff] [review]: ----------------------------------------------------------------- thanks!
Attachment #562493 - Flags: review?(jmaher) → review+
Depends on: 689232
bug 689232 filed for ServerOps to add new sql statements to graphs and graphs-stage
(In reply to Phil Ringnalda (:philor) from comment #4) > Don't forget to disable them until they're ready for production... > > https://tbpl.mozilla.org/php/getParsedLog.php?id=6562191&tree=Mozilla-Inbound > https://tbpl.mozilla.org/php/getParsedLog.php?id=6561838&tree=Mozilla-Inbound > https://tbpl.mozilla.org/php/getParsedLog.php?id=6560504&tree=Mozilla-Inbound > https://tbpl.mozilla.org/php/getParsedLog.php?id=6561115&tree=Mozilla-Inbound > https://tbpl.mozilla.org/php/getParsedLog.php?id=6560941&tree=Mozilla-Inbound > https://tbpl.mozilla.org/php/getParsedLog.php?id=6560158&tree=Mozilla-Inbound > https://tbpl.mozilla.org/php/getParsedLog.php?id=6560303&tree=Mozilla-Inbound > https://tbpl.mozilla.org/php/getParsedLog.php?id=6559996&tree=Mozilla-Inbound > https://tbpl.mozilla.org/php/getParsedLog.php?id=6559421&tree=Mozilla-Inbound > https://tbpl.mozilla.org/php/getParsedLog.php?id=6558298&tree=Mozilla-Inbound > https://tbpl.mozilla.org/php/getParsedLog.php?id=6558125&tree=Mozilla-Inbound > https://tbpl.mozilla.org/php/getParsedLog.php?id=6558460&tree=Mozilla-Inbound > https://tbpl.mozilla.org/php/getParsedLog.php?id=6558464&tree=Mozilla-Inbound > https://tbpl.mozilla.org/php/getParsedLog.php?id=6556351&tree=Mozilla-Inbound > https://tbpl.mozilla.org/php/getParsedLog.php?id=6556765&tree=Mozilla-Inbound The machines themselves are fine. The problem is missing entries in graphserver. We dropped the ball here, sorry about the disruption. Per irc, these tegras were disabled in production ~19:20 and will remain disabled until the sql changes are landed in bug#689232. Leaving this bug open to track.
bumped to include all new 160 tegras
Attachment #562493 - Attachment is obsolete: true
Attachment #562835 - Flags: review?(jmaher)
Comment on attachment 562835 [details] [diff] [review] patch to add new tegras to graph server I don't see anything out of the ordinary here
Attachment #562835 - Flags: review?(jmaher) → review+
the graph server sql table has been adjusted to have the new tegras
Status: NEW → RESOLVED
Closed: 14 years ago
Resolution: --- → FIXED
Whiteboard: [orange][android_tier_1][android][tegra] → [android_tier_1][android][tegra]
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: