1120114 - Fix some ship-it errors 2

Status: RESOLVED FIXED

(Release Engineering :: Release Automation, defect)

Comment 1

[Rail Aliiev [:rail]]

Attachment: puppet.diff

* Python 2.7 comes with argparse
* no need to install buildtools (ship-it notifier is a part of tools)
* use older requests shipped in tools

Comment 2

[Rail Aliiev [:rail]]

Attachment: custom.diff

To prevent the hack in http://hg.mozilla.org/build/tools/file/9c708b59dd9c/buildfarm/release/shipit-notifier.py#l66

Comment 3

[Rail Aliiev [:rail]]

Comment on attachment 8547091 [details] [diff] [review]
puppet.diff

remote:   https://hg.mozilla.org/build/puppet/rev/a095b2b25e07
remote:   https://hg.mozilla.org/build/puppet/rev/47176a2ae0b4

Comment 4

[Rail Aliiev [:rail]]

Comment on attachment 8547093 [details] [diff] [review]
custom.diff

https://hg.mozilla.org/build/buildbotcustom/rev/f86f604385a6

Comment 5

[Chris Cooper [:coop] (he/him)]

In production: https://hg.mozilla.org/build/buildbotcustom/rev/f86f604385a6

Comment 6

[Rail Aliiev [:rail]]

Attachment: fix_shipit_notifier-puppet.diff

* add certify to handle certs properly
* sort the list

Comment 7

[Rail Aliiev [:rail]]

Attachment: fix_ship_it_notifier-tools.diff

To make the client work. Server side is still b0rken :(

Comment 8

[Rail Aliiev [:rail]]

I've seen 2 major errors in ship-it:

1) HTTP 400, Client error:

2015-01-13 05:30:30,980 - kickoff.views.status.post#39: User Input Failed - [['Chunk Num is required.'], ['Chunk Total is required.']] - (Fennec-36.0b1-build1, release-mozilla-beta-android_build)

Neither chunkNum and chunkTotal are defined in most of the request. Requires some logical fixes in the code, I think.


2) HTTP 500.

015-01-13 05:55:31,037 - kickoff.log_exception#1306: Exception on /releases/Fennec-36.0b1-build1/status [POST]
Traceback (most recent call last):
File "/data/www/ship-it.mozilla.org/release-kickoff/vendor/lib/python/flask/app.py", line 1687, in wsgi_app
response = self.full_dispatch_request()
File "/data/www/ship-it.mozilla.org/release-kickoff/vendor/lib/python/flask/app.py", line 1360, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/data/www/ship-it.mozilla.org/release-kickoff/vendor/lib/python/flask/app.py", line 1358, in full_dispatch_request
rv = self.dispatch_request()
File "/data/www/ship-it.mozilla.org/release-kickoff/vendor/lib/python/flask/app.py", line 1344, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/data/www/ship-it.mozilla.org/release-kickoff/vendor/lib/python/flask/views.py", line 83, in view
return self.dispatch_request(*args, **kwargs)
File "/data/www/ship-it.mozilla.org/release-kickoff/vendor/lib/python/flask/views.py", line 150, in dispatch_request
return meth(*args, **kwargs)
File "/data/www/ship-it.mozilla.org/release-kickoff/kickoff/views/status.py", line 56, in post
ReleaseEvents.event_name == releaseEventsUpdate.event_name
TypeError: <lambda>() takes exactly 2 arguments (3 given)

Comment 9

[[e-mail] (:bhearsum)]

Comment on attachment 8548206 [details] [diff] [review]
fix_ship_it_notifier-tools.diff

Review of attachment 8548206 [details] [diff] [review]:
-----------------------------------------------------------------

::: buildfarm/release/shipit-notifier.py
@@ +23,5 @@
> +NAME_KEYS = ('product', 'version', 'build_number')
> +PROPERTIES_KEYS = ('platform', 'chunkTotal', 'chunkNum', 'event_group')
> +
> +
> +def get_propperties_dict(props):

typo: propperties vs properties. r=me with that fixed.

Comment 10

[Rail Aliiev [:rail]]

Comment on attachment 8548205 [details] [diff] [review]
fix_shipit_notifier-puppet.diff

remote:   https://hg.mozilla.org/build/puppet/rev/d509971a2588
remote:   https://hg.mozilla.org/build/puppet/rev/b3d40d4b2940

Comment 11

[Rail Aliiev [:rail]]

Comment on attachment 8548206 [details] [diff] [review]
fix_ship_it_notifier-tools.diff

https://hg.mozilla.org/build/tools/rev/36fe58203498

Comment 12

[Rail Aliiev [:rail]]

Attachment: fix-400-500.diff

This should address the 500 and 400 errors. I completely removed release name validation. I can revert the change if there is a reason to run that. It passed my local curl-based tests.

Comment 13

[Rail Aliiev [:rail]]

Comment on attachment 8549014 [details] [diff] [review]
fix-400-500.diff

https://git.mozilla.org/?p=build/release-kickoff.git;a=commitdiff;h=5c8967241d6fdef7e8333cfd63f9d17a35ed8220

Comment 14

[Rail Aliiev [:rail]]

woooooo!

https://ship-it.mozilla.org/releases/Firefox-36.0b1-build2/status.html
http://people.mozilla.org/~raliiev/sattap/d48bdb05.png

1) The tagging was finished before I deployed the change
2) No sure why Repacks had animated circle, probably related to 
2015-01-14 18:53:09,968 Processing release-mozilla-beta-win32_repack_complete
2015-01-14 18:53:09,968 Ignoring message without event_group set

or broken logic handling the chunks.

Comment 15

[Rail Aliiev [:rail]]

Attachment: repack_event_group.diff

to please https://git.mozilla.org/?p=build/release-kickoff.git;a=blob;f=kickoff/model.py;h=2bdafbbe60f8d49094b81b268ad9e3a8e23ebbe0;hb=HEAD#l353

Comment 16

[Rail Aliiev [:rail]]

FTR, https://git.mozilla.org/?p=build/release-kickoff.git;a=blob;f=kickoff/model.py;h=2bdafbbe60f8d49094b81b268ad9e3a8e23ebbe0;hb=HEAD#l354 may cause the sum be < 1 due to floating point math.

Comment 17

[Rail Aliiev [:rail]]

Comment on attachment 8549340 [details] [diff] [review]
repack_event_group.diff

https://hg.mozilla.org/build/buildbotcustom/rev/2b02c9c12a9c

Comment 18

[Chris Cooper [:coop] (he/him)]

In production: https://hg.mozilla.org/build/buildbotcustom/rev/2b02c9c12a9c