970309 - Grab some logs from public BZ ETL

Status: RESOLVED FIXED

(bugzilla.mozilla.org :: Infrastructure, defect)

Description

[Kyle Lahnakoski [:ekyle]]

The BZ ETL is failing.  I need someone to pull the logs, which will help me diagnose the problem:

Specifically, there should be mention of "Ensure the following private bugs are deleted" along with "968159".  Please confirm

There are two other errors, which I need stack traces for:
1) "modified_ts can not be Null"
2) "must have created_ts"

For new time, what is the URL for BZ ETL?

Thanks!

Comment 1

[Kendall Libby [:fubar] (he/him)]

Re: 968159, it is NOT being deleted. Still shows up under "Updating N bugs:".

modified_ts can not be Null:

15:31:17 - WARNING: Problem processing row: {
    "_merge_order": 2,
    "attach_id": null,
    "bug_id": 970329,
    "field_name": "cc",
    "modified_by": null,
    "modified_ts": null,
    "new_value": "[e-mail]",
    "old_value": null
}
        at File "/data/www/Bugzilla-ETL/bzETL/parse_bug_history.py", line 163, i
n processRow
        at File "/data/www/Bugzilla-ETL/bzETL/bz_etl.py", line 104, in etl
        at File "/data/www/Bugzilla-ETL/bzETL/util/threads.py", line 225, in _ru
n

caused by
        ERROR: modified_ts can not be Null
        at File "/data/www/Bugzilla-ETL/bzETL/parse_bug_history.py", line 176, i
n uid
        at File "/data/www/Bugzilla-ETL/bzETL/parse_bug_history.py", line 187, i
n startNewBug
        at File "/data/www/Bugzilla-ETL/bzETL/parse_bug_history.py", line 103, i
n processRow
        at File "/data/www/Bugzilla-ETL/bzETL/bz_etl.py", line 104, in etl
        at File "/data/www/Bugzilla-ETL/bzETL/util/threads.py", line 225, in _ru
n

and created_ts error:

08:41:19 - PROBLEM expecting a created_ts (did you install the timezone database
 into your MySQL instance?)
08:41:19 - WARNING: Problem processing row: {
    "_merge_order": 9,
    "attach_id": null,
    "bug_id": 969274,
    "field_name": "dependson",
    "modified_by": "[e-mail]",
    "modified_ts": 1391762381000,
    "new_value": "967272",
    "old_value": null
}
        at File "/data/www/Bugzilla-ETL/bzETL/parse_bug_history.py", line 163, i
n processRow
        at File "/data/www/Bugzilla-ETL/bzETL/bz_etl.py", line 104, in etl
        at File "/data/www/Bugzilla-ETL/bzETL/util/threads.py", line 225, in _ru
n

caused by
        ERROR: must have created_ts
        at File "/data/www/Bugzilla-ETL/bzETL/parse_bug_history.py", line 279, i
n processBugsActivitiesTableItem
        at File "/data/www/Bugzilla-ETL/bzETL/parse_bug_history.py", line 158, i
n processRow
        at File "/data/www/Bugzilla-ETL/bzETL/bz_etl.py", line 104, in etl
        at File "/data/www/Bugzilla-ETL/bzETL/util/threads.py", line 225, in _ru
n

Lastly, I'm not sure what you mean by "what is the URL for BZ ETL?"

Comment 2

[Kyle Lahnakoski [:ekyle]]

> Re: 968159, it is NOT being deleted. Still shows up under "Updating N bugs:".

Please look at the logs at the list following "Ensure the following private bugs are deleted".  I am curious if 968159 is was detected by the private logic and not deleting properly, or completely missed.

Since the upgrade to 8G, this problem disappeared.  I highly suspect 968159 to be in that list a few times.


> Lastly, I'm not sure what you mean by "what is the URL for BZ ETL?"
>

In the deep future, the "BZ ETL" will not be enough to adequately describe the machine(s) running the ETL process.  What unique-sequence-of-characters (URL, if you will) can I use so that future IT people can locate these machine(s) easily? 


I can not isolate what the other errors are about, yet.  But while I think about that, I have another issue (https://bugzilla.mozilla.org/show_bug.cgi?id=971421)

Comment 3

[Kendall Libby [:fubar] (he/him)]

968159 only shows up in the (current) log file three times, none of which are in the private bugs block. 

The ETL server is etl1.bugs.scl3; the ESFrontline server is esfrontline1.bugs.scl3.