1283853 - Uptake monitoring broke in 48 beta 5

Status: RESOLVED FIXED

(Release Engineering :: Release Automation: Other, defect)

Description

[Justin Wood (:Callek)]

22:40:13     INFO - #####
22:40:13     INFO - ##### Running monitor-uptake step.
22:40:13     INFO - #####
22:40:13     INFO - Running main action method: monitor_uptake
22:40:13     INFO - Starting the loop to determine the uptake monitoring ...
22:40:13    FATAL - Uncaught exception: Traceback (most recent call last):
22:40:13    FATAL -   File "/builds/slave/rel-m-beta-fx_uptk_mntr-000000/scripts/mozharness/base/script.py", line 1765, in run
22:40:13    FATAL -     self.run_action(action)
22:40:13    FATAL -   File "/builds/slave/rel-m-beta-fx_uptk_mntr-000000/scripts/mozharness/base/script.py", line 1707, in run_action
22:40:13    FATAL -     self._possibly_run_method(method_name, error_if_missing=True)
22:40:13    FATAL -   File "/builds/slave/rel-m-beta-fx_uptk_mntr-000000/scripts/mozharness/base/script.py", line 1647, in _possibly_run_method
22:40:13    FATAL -     return getattr(self, method_name)()
22:40:13    FATAL -   File "scripts/scripts/release/uptake_monitoring.py", line 175, in monitor_uptake
22:40:13    FATAL -     uptake = self._get_release_uptake(auth)
22:40:13    FATAL -   File "scripts/scripts/release/uptake_monitoring.py", line 158, in _get_release_uptake
22:40:13    FATAL -     return min(dl)
22:40:13    FATAL - ValueError: min() arg is an empty sequence
22:40:13    FATAL - Running post_fatal callback...
22:40:13    FATAL - Exiting -1
22:40:13     INFO - Running post-run listener: copy_logs_to_upload_dir
22:40:13     INFO - Copying logs to upload dir...
22:40:13     INFO - mkdir: /builds/slave/rel-m-beta-fx_uptk_mntr-000000/build/upload/logs
program finished with exit code 255

http://buildbot-master72.build.mozilla.org:8001/builders/release-mozilla-beta-firefox_uptake_monitoring/builds/3
https://tools.taskcluster.net/task-group-inspector/#YXKVDa6mQg-BRxl82Os7Hw/vjN9LTZ4TD2Fw1pVDjVwbQ/0

Had a bunch of retries:
http://buildbot-master74.build.mozilla.org:8001/builders/release-mozilla-beta-firefox_uptake_monitoring/builds/1
http://buildbot-master72.build.mozilla.org:8001/builders/release-mozilla-beta-firefox_uptake_monitoring/builds/4
http://buildbot-master72.build.mozilla.org:8001/builders/release-mozilla-beta-firefox_uptake_monitoring/builds/5
http://buildbot-master72.build.mozilla.org:8001/builders/release-mozilla-beta-firefox_uptake_monitoring/builds/6


Also one JUST ran is:

http://buildbot-master72.build.mozilla.org:8001/builders/release-mozilla-beta-firefox_uptake_monitoring/builds/7/steps/run_script/logs/stdio

Comment 1

[Mihai Tabara [:mtabara]⌚️GMT]

Mea culpa, this is my fault.

"ValueError: min() arg is an empty sequence" suggests that no uptake monitoring value could be determined. The issue is unrelated to the actual uptake monitoring or the partial versions but instead is related to the platforms parameter passed down via buildbot. Based on the intersection between the buildprops platforms and the local platforms from the mozharness configs, certain uptake checks are performed (http://hg.mozilla.org/mozilla-central/file/tip/testing/mozharness/scripts/release/uptake_monitoring.py#l132)

In bug 1278312 we fixed the uptake monitoring in graph2 and turned it on starting 48.0b5.

So, in 48.0b4 the buildprops platforms looked like this (via http://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-beta-noarch/release-mozilla-beta-firefox_uptake_monitoring-bm70-build1-build0.txt.gz):

---
02:22:03     INFO -  'platforms': (u'linux', u'linux64', u'macosx64', u'win32', u'win64'),
---

while in 48.0b5 the buildprops platforms looks like this (via http://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-beta-noarch/release-mozilla-beta-firefox_uptake_monitoring-bm72-build1-build3.txt.gz):
---
22:40:10     INFO -  'platforms': (u"('linux'",
22:40:10     INFO -                u"'linux64'",
22:40:10     INFO -                u"'win32'",
22:40:10     INFO -                u"'win64'",
22:40:10     INFO -                u"'macosx64')"),
---

It got messed up by this line from my commit https://github.com/mozilla/releasetasks/pull/181/files#diff-70163ba31e82fd96586fb6765ed90914L38

Because of it, no platform intersection could be determined in the uptake monitoring reckon, hence the empty sequence.
Will push a fix in a bit.

Comment 2

[Mihai Tabara [:mtabara]⌚️GMT]

Attachment: Bug 1283853 - Fix platforms parameter in uptake monitoring buildprops. r=rail

Comment 3

[Mihai Tabara [:mtabara]⌚️GMT]

Merged https://github.com/mozilla/releasetasks/commit/f3dc5cac0ddb930941358abfb150ddde07415644.
Will leave this open till next beta, 48.0b6, confirms it's working just fine.

Comment 4

[Mihai Tabara [:mtabara]⌚️GMT]

48.0b6 worked as expected \o/

See:
* http://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-beta-noarch/release-mozilla-beta-firefox_uptake_monitoring-bm72-build1-build9.txt.gz
* https://tools.taskcluster.net/task-graph-inspector/#Z2BNOjUqTC-P6du___P5oA/ERZN22ZMQxaNfvQsXluIZw/0