1294451 - use compression in balrog client tools

Status: RESOLVED WONTFIX

(Release Engineering :: General, defect)

Description

[[e-mail] (:bhearsum)]

The Balrog server supports gzip compression in POST/PUT bodies, so there's no reason not to use compression in the clients...

Comment 1

[Vedant Chakravadhanula(:vedantc98)]

I'd like to take up this bug, but I'll need a little more guidance as to how to go about it.
Thanks!

Comment 2

[[e-mail] (:bhearsum)]

(In reply to Vedant Chakravadhanula(:vedantc98) from comment #1)
> I'd like to take up this bug, but I'll need a little more guidance as to how
> to go about it.
> Thanks!

Sorry, somebody else is already working on it actually!

Comment 3

[nikshepsvn]

would love to work on this if someone can tell me how to start off

Comment 4

[[e-mail] (:bhearsum)]

(In reply to nikshepsvn from comment #3)
> would love to work on this if someone can tell me how to start off

The goal here is to change the Balrog client library (located at https://github.com/mozilla/balrog/blob/b97d1c70ba4489976333ab9b96bcaacbfc41c944/client/balrogclient/api.py) to compress requests sent to the server. To do this, you'll need to compress the body of the request and set the Content-Encoding header correctly (per https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Encoding).

To make sure things work correctly, I'd suggest running the development environment (http://mozilla-balrog.readthedocs.io/en/latest/contribute.html#contributing-to-balrog), and making a test request like this both before and after your changes:
$ PYTHONPATH=client python
>>> from balrogclient.api import Release
>>> release = Release("Firefox-mozilla-central-nightly-latest", api_root="http://localhost:8080/api")
>>> release_data = release.get_data()[0]
>>> release.update_release("Firefox", "sha512", json.dumps(release_data))
<Response [200]>

Comment 5

[rajalakshmi.v15]

Hi,
May I know if the issue is still open. If yes, I would like to take it up.
Thanks.

Comment 6

[nikshepsvn]

hey there, I forgot about this bug and will continue working on it!

Comment 7

[nikshepsvn]

should be done in the next few days.

Comment 8

[nikshepsvn]

https://github.com/mozilla/balrog/pull/652 - PR for fix

Comment 9

[nikshepsvn]

Attachment: PR link

Comment 10

[Nick Thomas [:nthomas] (UTC+12)]

Note to myself - this code is used in balrog scriptworkers via balrogscript [1], which have a clone of the hg tools repo [2] so they can import [3] balrog.submitter.cli [4], which in turn imports [5] balrogclient from lib/python/vendor/ [6]. This is all done with python 2.7.

A deploy is a tools landing like https://hg.mozilla.org/build/tools/rev/d2da1f514d5e to update balrog client, then manually pull the tools repo on the N workers (probably doing a single one as a canary first).

[1] https://github.com/mozilla-releng/balrogscript
[2] https://github.com/mozilla-releng/build-puppet/blob/master/modules/balrog_scriptworker/manifests/init.pp#L80
[3] https://github.com/mozilla-releng/balrogscript/blob/master/balrogscript/script.py#L24
[4] https://hg.mozilla.org/build/tools/file/default/lib/python/balrog/submitter/cli.py
[5] https://hg.mozilla.org/build/tools/file/default/lib/python/balrog/submitter/api.py#l6
[6] https://hg.mozilla.org/build/tools/file/default/lib/python/vendor/balrogclient-0.0.4

Comment 11

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to rajalakshmi.v15 from comment #5)
> May I know if the issue is still open. If yes, I would like to take it up.

Sorry, it's already being worked on.

Comment 12

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 8988730 [details] [review]
PR link

Please see my comments on Github.

Comment 13

[[e-mail] (:bhearsum)]

I'm closing this because:
a) It turns out the Balrog server doesn't support compression already
but mostly because
b) The requests are pretty small already, and adding the overhead of decompression to the server handling them is not worth it.