schedule publishing in balrog fails: Changes may not be scheduled in the past

RESOLVED DUPLICATE of bug 1389312

Status

Release Engineering
Release Automation
P1
normal
RESOLVED DUPLICATE of bug 1389312
6 months ago
3 months ago

People

(Reporter: jlorenzo, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [releaseduty])

(Reporter)

Description

6 months ago
Similar to bug 1389312 and bug 1388330, "schedule publishing in balrog" failed in 56.0b7[1]. The technical reason is the following: 

> 02:28:45     INFO -  Balrog request to https://aus4-admin.mozilla.org/api/scheduled_changes/rules
> 02:28:45     INFO -  Data sent: {u'comment': None, 'change_type': 'update', u'fallbackMapping': u'Firefox-56.0b6-build2', u'buildID': None, u'backgroundRate': 100, u'product': u'Firefox', u'instructionSet': None, u'mapping': 'Firefox-56.0b7-build2', u'alias': u'firefox-beta', u'headerArchitecture': None, u'buildTarget': None, 'data_verison': 336, u'osVersion': None, u'distVersion': None, u'mig64': None, u'data_version': 336, 'when': 1504022400000, u'priority': 90, u'locale': None, u'version': None, u'update_type': u'minor', u'memory': None, u'distribution': None, u'rule_id': '32', u'channel': u'beta'}
> 02:28:45     INFO -  "POST /api/scheduled_changes/rules HTTP/1.1" 400 116
> 02:28:45     INFO -  Caught HTTPError: {"status": 400, "type": "about:blank", "detail": "Changes may not be scheduled in the past", "title": "Bad Request"}

The human reason is that we got delayed in signing off builds. 

I worked it around by submitting a new task manually[2]. For the record, I had to change the timestamp at 2 places in the task definition.

I wonder if we shouldn't just change the worker to take the current timestamp (on beta), instead of the what was defined in ship-it. However, I don't know if this value is used elsewhere, though

[1] https://tools.taskcluster.net/groups/xzIcLIHqSL2rEcHsUf9j9Q/tasks/hXXWT9JxRAuENUXvkL-pCQ/runs/1/logs/public%2Flogs%2Flive_backing.log.gz
[2] https://tools.taskcluster.net/groups/xzIcLIHqSL2rEcHsUf9j9Q/tasks/G1L8zR45TGGbhz4QOe4g5A/details
Priority: -- → P2
Whiteboard: [releaseduty]
I don't think hardcoding special behaviour for Beta is the best fix here -- this is something that can happen on any channel if the Task runs after the release eta has passed.

My suggestion is to change the balrog submitter to take the more recent of "release eta" and "5 seconds from now". This should fix the issue for all channels, and will ensure that release eta is respected whenever possible on Beta.
Priority: P2 → P1
Status: NEW → RESOLVED
Last Resolved: 3 months ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1389312
You need to log in before you can comment on or make changes to this bug.