SQL errors causing jobs to not be added to builds-4hr.js.gz

RESOLVED FIXED

Status

Data & BI Services Team
DB: MySQL
--
blocker
RESOLVED FIXED
6 years ago
3 years ago

People

(Reporter: philor, Assigned: mpressman)

Tracking

Details

(Reporter)

Description

6 years ago
http://build.mozilla.org/builds/builds-4hr.js.gz is completely empty, and as a result no completed jobs are shown on tbpl.

Condition for reopening the tree (after the last premature reopening):

The tip push on https://tbpl.mozilla.org/?tree=Mozilla-Inbound must show every single job which runs and is visible for both Android XUL and Android in a completed state (orange, fine, but not purple or red from not actually running).

No "I think it probably will" or "that might have fixed it." Every single job, in a completed state.
(Reporter)

Comment 1

6 years ago
Typed this so many times lately, I forgot to type it again:

mozilla-inbound, mozilla-central, mozilla-aurora and mozilla-beta are closed.
(Reporter)

Updated

6 years ago
Blocks: 712877
(Reporter)

Updated

6 years ago
Blocks: 712988
I'm seeing this in some logs:
sqlalchemy.exc.InternalError: (InternalError) (144, "Table './buildbot/properties' is marked as crashed and last (automatic?) repair failed") 'DESCRIBE `properties`' ()

Can we repair that table?
per irc w/catlee, this is fallout from the DB outage earlier.
Assignee: nobody → server-ops-database
Component: Release Engineering → Server Operations: Database
QA Contact: release → cshields
Summary: Jobs are not being added to builds-4hr.js.gz → SQL errors causing jobs to not be added to builds-4hr.js.gz
txt'd sheeri as she requested in irc earlier this evening.
(Assignee)

Comment 5

6 years ago
Running table repair on properties now

Updated

6 years ago
Assignee: server-ops-database → mpressman
(Assignee)

Comment 6

6 years ago
buildbot.properties table successfully repaired, select queries against each against ran without error.

The table was heavily fragmented. Prior to repair the index file on disk was 1018M, after repair it is 706M.
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED

Comment 7

6 years ago
confirmed that the updater log is no longer showing an error
Status: RESOLVED → UNCONFIRMED
Ever confirmed: false
Resolution: FIXED → ---

Updated

6 years ago
Status: UNCONFIRMED → RESOLVED
Last Resolved: 6 years ago6 years ago
Resolution: --- → FIXED

Comment 8

6 years ago
see bug 712988 as this is still happening.
(In reply to Matt Pressman [:mpressman] from comment #6)
> buildbot.properties table successfully repaired, select queries against each
> against ran without error.
> 
> The table was heavily fragmented. Prior to repair the index file on disk was
> 1018M, after repair it is 706M.

Thanks Matt!
Blocks: 926246
Product: mozilla.org → Data & BI Services Team
You need to log in before you can comment on or make changes to this bug.