Note: There are a few cases of duplicates in user autocompletion which are being worked on.

enable change notifier for balrog admin app

RESOLVED FIXED

Status

Cloud Services
Operations: Deployment Requests
RESOLVED FIXED
10 months ago
9 months ago

People

(Reporter: bhearsum, Assigned: bhearsum)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Assignee)

Description

10 months ago
This was requested way back when we originally talked about the migration. I ended up implementing support for it in https://bugzilla.mozilla.org/show_bug.cgi?id=1251338, but we didn't enable it. It should work if you set the following environment variables:
SMTP_HOST
SMTP_PORT
NOTIFY_TO_ADDR
NOTIFY_FROM_ADDR
SMTP_USERNAME (optional)
SMTP_PASSWORD (optional)

NOTIFY_TO_ADDR should be balrog-db-changes@mozilla.com. We should probably test it out in stage first, but it would be best if it were disabled there once we know it works (too spammy).
(Assignee)

Updated

10 months ago
No longer blocks: 1304054
(Assignee)

Updated

10 months ago
Blocks: 1246675
(Assignee)

Comment 1

10 months ago
We decided that we should have a separate list for stage, to make sure we don't bust it in the future. I created junk-balrog-db-changes@mozilla.com for this.
(Assignee)

Updated

10 months ago
Depends on: 1305703
(Assignee)

Comment 2

10 months ago
We tried enabling this in stage today, and got an exception:
{"EnvVersion": "2.0", "Type": "Rules", "Severity": 3, "Fields": {"message": "Failed to connect to SMTP server:", "traceback": "Uncaught exception:\n  File \"./auslib/db.py\", line 1790, in send_email\n    conn.starttls()\n  File \"/usr/local/lib/python2.7/smtplib.py\", line 644, in starttls\n    raise SMTPException(\"STARTTLS extension not supported by server.\")\n<class 'smtplib.SMTPException'>\nSMTPException('STARTTLS extension not supported by server.',)\n", "requestid": 140010317872208, "error": "SMTPException('STARTTLS extension not supported by server.',)"}, "Timestamp": 1475080251884000000, "Logger": "Balrog", "Hostname": "ip-172-31-48-255", "Pid": 8}

Looks like I'll need to tweak the code to make it possible to disable starttls.
(Assignee)

Comment 3

10 months ago
We also hit another error, which happens prior to the smtp connection in certain cases:
ion=transaction)\n  File \"./auslib/db.py\", line 1530, in addLocaleToRelease\n    transaction=transaction)\n  File \"./auslib/db.py\", line 498, in update\n    return self._prepareUpdate(transaction, where, what, changed_by, old_data_version)\n  File \"./auslib/db.py\", line 456, in _prepareUpdate\n    self.onUpdate(self, \"UPDATE\", changed_by, query)\n  File \"./auslib/db.py\", line 1838, in bleet\n    row = table.sel$

I'll steal this bug for now while I work out fixes for both issues.
Assignee: bwong → bhearsum
(Assignee)

Comment 4

10 months ago
(In reply to Ben Hearsum (:bhearsum) from comment #3)
> We also hit another error, which happens prior to the smtp connection in
> certain cases:
> ion=transaction)\n  File \"./auslib/db.py\", line 1530, in
> addLocaleToRelease\n    transaction=transaction)\n  File \"./auslib/db.py\",
> line 498, in update\n    return self._prepareUpdate(transaction, where,
> what, changed_by, old_data_version)\n  File \"./auslib/db.py\", line 456, in
> _prepareUpdate\n    self.onUpdate(self, \"UPDATE\", changed_by, query)\n 
> File \"./auslib/db.py\", line 1838, in bleet\n    row = table.sel$
> 
> I'll steal this bug for now while I work out fixes for both issues.

Ben - any chance you can dig up the full traceback for this one? It looks like the pastebin this came from has expired. It should be from around 4pm eastern on the 28th.
Flags: needinfo?(bwong)
This is the long line you are looking for...

{
    "EnvVersion": "2.0",
    "Fields": {
        "error": "SMTPException('STARTTLS extension not supported by server.',)",
        "message": "Failed to connect to SMTP server:",
        "requestid": 140372048409168,
        "traceback": "Uncaught exception:\n  File \"./auslib/db.py\", line 1790, in send_email\n    conn.starttls()\n  File \"/usr/local/lib/python2.7/smtplib.py\", line 644, in starttls\n    raise SMTPException(\"STARTTLS extension not supported by server.\")\n<class 'smtplib.SMTPException'>\nSMTPException('STARTTLS extension not supported by server.',)\n"
    },
    "Hostname": "xx",
    "Logger": "Balrog",
    "Pid": 8,
    "Severity": 3,
    "Timestamp": 1475092806206509056,
    "Type": "Rules"
}
Flags: needinfo?(bwong)
(Assignee)

Comment 6

10 months ago
(In reply to Benson Wong [:mostlygeek] from comment #5)
> This is the long line you are looking for...
> 
> {
>     "EnvVersion": "2.0",
>     "Fields": {
>         "error": "SMTPException('STARTTLS extension not supported by
> server.',)",
>         "message": "Failed to connect to SMTP server:",
>         "requestid": 140372048409168,
>         "traceback": "Uncaught exception:\n  File \"./auslib/db.py\", line
> 1790, in send_email\n    conn.starttls()\n  File
> \"/usr/local/lib/python2.7/smtplib.py\", line 644, in starttls\n    raise
> SMTPException(\"STARTTLS extension not supported by server.\")\n<class
> 'smtplib.SMTPException'>\nSMTPException('STARTTLS extension not supported by
> server.',)\n"
>     },
>     "Hostname": "xx",
>     "Logger": "Balrog",
>     "Pid": 8,
>     "Severity": 3,
>     "Timestamp": 1475092806206509056,
>     "Type": "Rules"
> }

Actually, I've got this full traceback in comment #3. The one I'm looking or should have "addLocaleToRelease" in the traceback.
[pid: 8|app: 0|req: 350/359] 127.0.0.1 () {42 vars in 628 bytes} [Wed Sep 28 19:40:44 2016] HEAD /api/csrf_token => generated 0 bytes in 1 msecs (HTTP/1.1 200) 6 headers in 384 bytes (0 switches on core 0)
{"EnvVersion": "2.0", "Type": "auslib.admin.base", "Severity": 3, "Fields": {"message": "Exception on /releases/Thunderbird-45.4.0-build1/builds/Linux_x86-gcc3/en-US [PUT]", "traceback": "Uncaught exception:\n  F
ion=transaction)\n  File \"./auslib/db.py\", line 1530, in addLocaleToRelease\n    transaction=transaction)\n  File \"./auslib/db.py\", line 498, in update\n    return self._prepareUpdate(transaction, where, what
{"EnvVersion": "2.0", "Type": "auslib.admin.base", "Severity": 3, "Fields": {"message": "Caught ISE 500 error.", "requestid": 140372046519120}, "Timestamp": 1475091644615927808, "Logger": "Balrog", "Hostname": "i
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1836, in __call__
return self.wsgi_app(environ, start_response)
File "./auslib/admin/base.py", line 37, in __call__
return self.app(environ, start_response)
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1820, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1577, in make_response
rv = self.response_class.force_type(rv, request.environ)
File "/usr/local/lib/python2.7/site-packages/werkzeug/wrappers.py", line 847, in force_type
response = BaseResponse(*_run_wsgi_app(response, environ))
File "/usr/local/lib/python2.7/site-packages/werkzeug/test.py", line 871, in run_wsgi_app
app_rv = app(environ, start_response)
TypeError: 'exceptions.IndexError' object is not callable
(Assignee)

Comment 8

9 months ago
Created attachment 8799754 [details] [review]
fix issues with change notifier

Rail, I'm hoping to get this in stage today. Can you have a look if you have time?
Attachment #8799754 - Flags: review?(rail)
Attachment #8799754 - Flags: review?(rail) → review+

Comment 9

9 months ago
Commit pushed to master at https://github.com/mozilla/balrog

https://github.com/mozilla/balrog/commit/aa85ffe8361514818e41211c5a283c21641e9ee4
bug 1304082: fix issues with change notifier in production (#142). r=rail
(Assignee)

Updated

9 months ago
Depends on: 1311036
(Assignee)

Comment 10

9 months ago
This is in production and working. Prod sends notifications to https://groups.google.com/a/mozilla.com/forum/#!forum/balrog-db-changes.

Thanks to Ben & others for helping resolve the issues found in stage.
Status: NEW → RESOLVED
Last Resolved: 9 months ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.