Grab some logs from public BZ ETL

RESOLVED FIXED

Status

()

bugzilla.mozilla.org
Infrastructure
RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: ekyle, Assigned: fubar)

Tracking

Details

(Reporter)

Description

4 years ago
The BZ ETL is failing.  I need someone to pull the logs, which will help me diagnose the problem:

Specifically, there should be mention of "Ensure the following private bugs are deleted" along with "968159".  Please confirm

There are two other errors, which I need stack traces for:
1) "modified_ts can not be Null"
2) "must have created_ts"

For new time, what is the URL for BZ ETL?

Thanks!
(Assignee)

Comment 1

4 years ago
Re: 968159, it is NOT being deleted. Still shows up under "Updating N bugs:".

modified_ts can not be Null:

15:31:17 - WARNING: Problem processing row: {
    "_merge_order": 2,
    "attach_id": null,
    "bug_id": 970329,
    "field_name": "cc",
    "modified_by": null,
    "modified_ts": null,
    "new_value": "geoffrey@mozillafoundation.org",
    "old_value": null
}
        at File "/data/www/Bugzilla-ETL/bzETL/parse_bug_history.py", line 163, i
n processRow
        at File "/data/www/Bugzilla-ETL/bzETL/bz_etl.py", line 104, in etl
        at File "/data/www/Bugzilla-ETL/bzETL/util/threads.py", line 225, in _ru
n

caused by
        ERROR: modified_ts can not be Null
        at File "/data/www/Bugzilla-ETL/bzETL/parse_bug_history.py", line 176, i
n uid
        at File "/data/www/Bugzilla-ETL/bzETL/parse_bug_history.py", line 187, i
n startNewBug
        at File "/data/www/Bugzilla-ETL/bzETL/parse_bug_history.py", line 103, i
n processRow
        at File "/data/www/Bugzilla-ETL/bzETL/bz_etl.py", line 104, in etl
        at File "/data/www/Bugzilla-ETL/bzETL/util/threads.py", line 225, in _ru
n

and created_ts error:

08:41:19 - PROBLEM expecting a created_ts (did you install the timezone database
 into your MySQL instance?)
08:41:19 - WARNING: Problem processing row: {
    "_merge_order": 9,
    "attach_id": null,
    "bug_id": 969274,
    "field_name": "dependson",
    "modified_by": "jmcf@tid.es",
    "modified_ts": 1391762381000,
    "new_value": "967272",
    "old_value": null
}
        at File "/data/www/Bugzilla-ETL/bzETL/parse_bug_history.py", line 163, i
n processRow
        at File "/data/www/Bugzilla-ETL/bzETL/bz_etl.py", line 104, in etl
        at File "/data/www/Bugzilla-ETL/bzETL/util/threads.py", line 225, in _ru
n

caused by
        ERROR: must have created_ts
        at File "/data/www/Bugzilla-ETL/bzETL/parse_bug_history.py", line 279, i
n processBugsActivitiesTableItem
        at File "/data/www/Bugzilla-ETL/bzETL/parse_bug_history.py", line 158, i
n processRow
        at File "/data/www/Bugzilla-ETL/bzETL/bz_etl.py", line 104, in etl
        at File "/data/www/Bugzilla-ETL/bzETL/util/threads.py", line 225, in _ru
n

Lastly, I'm not sure what you mean by "what is the URL for BZ ETL?"
Assignee: server-ops-webops → klibby
(Reporter)

Comment 2

4 years ago
> Re: 968159, it is NOT being deleted. Still shows up under "Updating N bugs:".

Please look at the logs at the list following "Ensure the following private bugs are deleted".  I am curious if 968159 is was detected by the private logic and not deleting properly, or completely missed.

Since the upgrade to 8G, this problem disappeared.  I highly suspect 968159 to be in that list a few times.


> Lastly, I'm not sure what you mean by "what is the URL for BZ ETL?"
>

In the deep future, the "BZ ETL" will not be enough to adequately describe the machine(s) running the ETL process.  What unique-sequence-of-characters (URL, if you will) can I use so that future IT people can locate these machine(s) easily? 


I can not isolate what the other errors are about, yet.  But while I think about that, I have another issue (https://bugzilla.mozilla.org/show_bug.cgi?id=971421)
Flags: needinfo?(klibby)
(Assignee)

Comment 3

4 years ago
968159 only shows up in the (current) log file three times, none of which are in the private bugs block. 

The ETL server is etl1.bugs.scl3; the ESFrontline server is esfrontline1.bugs.scl3.
Flags: needinfo?(klibby)
(Reporter)

Updated

4 years ago
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
Component: WebOps: Bugzilla → Infrastructure
Product: Infrastructure & Operations → bugzilla.mozilla.org
You need to log in before you can comment on or make changes to this bug.