Closed Bug 766267 Opened 12 years ago Closed 12 years ago

socorro staging access to hbase is down

Categories

(Mozilla Metrics :: Metrics Operations, task)

Product:
Mozilla Metrics
Component:
Metrics Operations
Platform:
x86_64
Linux
Type:
task
Priority:
Not set
Severity:
major

Tracking

(Not tracked)

Status:
RESOLVED FIXED
Milestone:
Unreviewed

People

(Reporter: lars, Assigned: tmary)

Details

the processors in Socorro staging are unable to talk to HBase.  Every thread is getting a socket read error:

2012-06-19 12:36:38,262 CRITICAL - Thread-2 - File "/data/socorro/thirdparty/thrift/transport/TSocket.py", line 108, in read
    raise TTransportException(type=TTransportException.END_OF_FILE, message='TSocket read 0 bytes')
2012-06-19 12:36:38,262 CRITICAL - Thread-2 - FatalException: the connection is not viable.  retries fail:
2012-06-19 12:36:38,263 CRITICAL - Thread-2 - major failure in crash storage - retry in 300 seconds
hbaseHost=10.8.100.209
hbasePort=9090
Assignee: server-ops → nobody
Group: metrics-private
Severity: normal → major
Component: Server Operations: Web Operations → Metrics Operations
Product: mozilla.org → Mozilla Metrics
QA Contact: cshields → metrics-operations
Version: other → unspecified
Group: metrics-private
Please check/verify - should be up now

--
Assignee: nobody → tmeyarivan
the monitor running on socorroadm.stage.private.phx1.mozilla.com are having no trouble with HBase.

the processors running on socorro-processor1.stage.metrics.phx1.mozilla.com still _cannot_ connect to HBase.
I was about to write that the problem is resolved.  Suddenly, the situation is reversed, the processor is talking with HBase, but the monitor is giving connection errors.
and just as suddenly, everything is fine...
and it's all broken again moments later...
As per last IRC msg, its working as expected

--
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.