Investigate I/O error on try1-linux-slave

RESOLVED FIXED

Status

Release Engineering
General
P3
normal
RESOLVED FIXED
11 years ago
5 years ago

People

(Reporter: bhearsum, Assigned: bhearsum)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Assignee)

Description

11 years ago
A build failed on the Linux try slave because the filesystem was mounted read-only. Found this in the logs:
Aug  6 23:34:21 try1-linux-slave kernel: sd 0:0:0:0: SCSI error: return code = 0
x00020008
Aug  6 23:34:21 try1-linux-slave kernel: end_request: I/O error, dev sda, sector
 17758975
Aug  6 23:34:21 try1-linux-slave kernel: Buffer I/O error on device sda1, logica
l block 2219864
Aug  6 23:34:21 try1-linux-slave kernel: lost page write due to I/O error on sda
1

Going to look at this tomorrow (Thursday)
(Assignee)

Comment 1

11 years ago
'mount -o remount,rw /' did not fix it. 'mount' reports it as being rw but buildbot still cannot write to the filesystem. I rebooted it and it's OK for now. Will look for the source of the problem tonight/tomorrow.
Assignee: build → bhearsum
(Assignee)

Updated

11 years ago
Status: NEW → ASSIGNED
(Assignee)

Comment 2

11 years ago
The VI Client shows an error on August 6th, "Insufficient disk space on datastore". It seems unlikely that this would affect things 2 days later, but this may be it.

I ran a full fsck on the / filesystem which turned up no errors. I'm doing a few test builds now while watching the logs.
Priority: P3 → P2
Whiteboard: i think i found the problem
(Assignee)

Updated

11 years ago
Priority: P2 → P3
(Assignee)

Updated

11 years ago
Blocks: 387958
(Assignee)

Updated

11 years ago
Whiteboard: i think i found the problem → needs further investigation
(Assignee)

Updated

11 years ago
Blocks: 393092
(Assignee)

Comment 3

11 years ago
Apparently this machine ran out of disk space last night and started erroring out. I don't see anything in the logs about an I/O error though.

Coop tells me that the hard drive on this machine is too small. I've added a 30G drive as /builds.
(Assignee)

Comment 4

11 years ago
This has been fine since the upgrade and many builds have been done in that time. Marking this FIXED. It will be re-opened if the problem recurs.
Status: ASSIGNED → RESOLVED
Last Resolved: 11 years ago
Resolution: --- → FIXED
Whiteboard: needs further investigation
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.