1087500 - upgrade balrog's Flask stack

Status: RESOLVED FIXED

(Release Engineering Graveyard :: Applications: Balrog (backend), defect)

Description

[[e-mail] (:bhearsum)]

While Peter was working on the new Balrog UI he discovered that our current Flask stack doesn't support form submissions as JSON blobs (which Angular depends on). The latest Flask-wtforms does, and this is probably a good opportunity to get up to date with other things as well.

I've been working on this in https://github.com/bhearsum/balrog/tree/upgrade-flask-wtf, and I got tests passing with minimal effort. The biggest changes seem to be some config variables changing names. I still need to do a bunch of testing by hand.

Comment 1

[[e-mail] (:bhearsum)]

Attachment: upgrade our flask stack

This patch contains everything except the changes to vendor/. Surprisingly little needed to change here, but the highlights are:
* Fix some add_url_rule calls not to duplicateview names -- this became an error at some point in Flask.
* Fix a bunch of imports that moved.
* s/TextField/StringField/, because the former is deprecated and spews a billion warnings when running tests.
* Fix Rules.addRule to return just the rule id rather than a single item list containing the rule id (and fix a test that relied on this wacky behaviour).
* s/CSRF_ENABLED/WTF_CSRF_ENABLED/g

The vendor changes include new versions of Flask, Werkzeug, WTForms, Flask-WTF and itsdangerous (a library that newer versions of Flask depend on). In order to ensure clean installs, I deleted all old files from these before installing the new ones. You can see all of this in https://github.com/bhearsum/balrog/compare/upgrade-flask-wtf - though I'm not sure how useful it is.

All tests pass, and I've tried all of the functionality I could think of locally - it all seems to work! Nonetheless, I want to point a staging master at it for some jobs for additional verification before going to prod.

Comment 2

[Peter Bengtsson [:peterbe]]

Comment on attachment 8509748 [details] [diff] [review]
upgrade our flask stack

Review of attachment 8509748 [details] [diff] [review]:
-----------------------------------------------------------------

There's not much to review here. It it works it works. Much of the changes aren't really business logic. It's just the new ways of doing things. 

I haven't tested it locally yet but I trust you have. By the way, we really should have travis/jenkins tests on things like this automated.

Comment 3

[[e-mail] (:bhearsum)]

(In reply to Peter Bengtsson [:peterbe] from comment #2)
> Comment on attachment 8509748 [details] [diff] [review]
> upgrade our flask stack
> 
> Review of attachment 8509748 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> There's not much to review here. It it works it works. Much of the changes
> aren't really business logic. It's just the new ways of doing things. 
> 
> I haven't tested it locally yet but I trust you have. By the way, we really
> should have travis/jenkins tests on things like this automated.

We do have Balrog tests running, unfortunately on our private Jenkins instance. Balrog issues can cause tree closures and issues with release updates though, so I always like to do more verification around them.

Comment 4

[[e-mail] (:bhearsum)]

Comment on attachment 8509748 [details] [diff] [review]
upgrade our flask stack

Massimo, are you comfortable reviewing this? See comment #1 for more details + how I plan to do additional testing.

Comment 5

[Massimo Gervasini [:massimo]]

Comment on attachment 8509748 [details] [diff] [review]
upgrade our flask stack

Hi Ben,

The code looks good to me, no issues on my local tests.

Not a blocker, I have a warning from pyflakes:
auslib/db.py:7: 'urlparse' imported but unused.
Could you remove the urlparse import?

Comment 6

[[e-mail] (:bhearsum)]

Comment on attachment 8509748 [details] [diff] [review]
upgrade our flask stack

I pushed this. I'll be doing some more testing in dev today before asking for a push to prod.

Comment 7

[[github robot]]

Commit pushed to master at https://github.com/mozilla/balrog

https://github.com/mozilla/balrog/commit/72d815b9d80cf6fc09e1f4c5c50b8fd74e1c3588
bug 1087500: upgrade balrog's Flask stack. r=mgerva

Comment 8

[[e-mail] (:bhearsum)]

OK, I did some resubmissions to balrog dev. They all worked, and I was able to get updates back after changing the rules. I filed bug 1090954 to get this pushed.

Comment 9

[[e-mail] (:bhearsum)]

Working fine in production AFAICT. I saw some nightly repacks submit successfully, and I got updates for them!