1417568 - beetmoverscript push-to-cdns should not silently return success if nothing has been copied over

Status: RESOLVED FIXED

(Release Engineering :: Release Automation: Other, enhancement)

Description

[Mihai Tabara [:mtabara]⌚️GMT]

Fennec 58.0b3 had some additional tasks and we needed to fire up another graph. We slipped a wrong buildnumber (buildno 1) instead of buildno 4 but push-to-cdns silently return successful even though it copied nothing. 

We need to amend beetmovercript behavior to fail if nothing is to be copied over.

Comment 1

[Mihai Tabara [:mtabara]⌚️GMT]

Log of that file[1]

[1]: https://public-artifacts.taskcluster.net/BEN7wSATRvWrpiWGtjJMfQ/0/public/logs/live_backing.log

Comment 2

[Mihai Tabara [:mtabara]⌚️GMT]

Steps:
* start a Fennec staging release on maple
* follow steps[1] to trigger a follow-up second graph that contains a push-to-cdnstest
* the job won't run unless beetmover-checksums from previous graph will have been successful
* if we wait for all the tasks to run, we will have artifacts under candidates so we won't be able to catch the corner case
* for that reason we force the running of the task

Note:
* editing an older task + removing deps and resubmitting no longer works as it breaks CoT.

This way we have a fully generated graph, that is CoT able.
Following up we correctly failed the task[2] with:

2017-11-15 21:05:15,042 - asyncio - DEBUG - Using selector: EpollSelector
2017-11-15 21:05:15,043 - beetmoverscript.task - INFO - Action types: ['push-to-releases']
2017-11-15 21:05:15,046 - beetmoverscript.task - INFO - Buckets: ['dep']
2017-11-15 21:05:15,046 - beetmoverscript.task - INFO - Action types: ['push-to-releases']
2017-11-15 21:05:15,097 - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): s3.amazonaws.com
Traceback (most recent call last):
  File "/builds/scriptworker/lib/python3.5/site-packages/beetmoverscript/script.py", line 429, in main
    loop.run_until_complete(async_main(context))
  File "/tools/python35/lib/python3.5/asyncio/base_events.py", line 387, in run_until_complete
    return future.result()
  File "/tools/python35/lib/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
  File "/tools/python35/lib/python3.5/asyncio/tasks.py", line 239, in _step
    result = coro.send(None)
  File "/builds/scriptworker/lib/python3.5/site-packages/beetmoverscript/script.py", line 215, in async_main
    await action_map[context.action](context)
  File "/builds/scriptworker/lib/python3.5/site-packages/beetmoverscript/script.py", line 122, in push_to_releases
    candidates_prefix)
scriptworker.exceptions.ScriptWorkerTaskException: No artifacts to copy from pub/mobile/candidates/58.0b6-candidates/build1/ so there is no reason to continue.
exit code: 1


[1]: https://github.com/mozilla/releasewarrior/blob/master/how-tos/fennec-temp-relpro.md
[2]: https://tools.taskcluster.net/groups/PfEI-MwGR-m73rR3W5bQlw/tasks/buxwXdDjQye2lwS98V4K3g/details

Comment 3

[Mihai Tabara [:mtabara]⌚️GMT]

Attachment: Fail fast if no artifacts are to be copied in beetmoverscript.

I am thinking of merging this to master branch but not bumping beetmoverscript. It's not urgent and can be picked up by the next set of changes. I'm just trying to minimize the amount of change freeze emails I'm sending :)

Therefore, please ignore the version.txt changes.

Comment 4

[Mihai Tabara [:mtabara]⌚️GMT]

Comment on attachment 8928689 [details] [review]
Fail fast if no artifacts are to be copied in beetmoverscript.

https://github.com/mozilla-releng/beetmoverscript/commit/071df154b33d44dd12298012437243c7f2becd66

Comment 5

[Mihai Tabara [:mtabara]⌚️GMT]

These changes will get picked up by the next beetmoverscript bump.
Because we're still in a change freeze state, I want to hold on from deploying too many things.
This is not urgent so can wait until next week or so.

Work is done, closing this.