Closed
Bug 1231318
Opened 9 years ago
Closed 8 years ago
Taskcluster builds don't use the same buildid as buildbot builds
Categories
(Taskcluster :: General, defect)
Taskcluster
General
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: glandium, Assigned: dustin)
References
Details
Attachments
(1 file)
2.27 KB,
patch
|
jlund
:
review+
|
Details | Diff | Splinter Review |
All builds for a given push, in buildbot, use the same buildid, passed down to the build through MOZ_BUILD_DATE. I don't remember when this was done, but I do remember it was done on purpose. So it would be good for taskcluster builds to do the same.
Assignee | ||
Comment 2•8 years ago
|
||
Rail, can you give me some pointers on how I should set this and how it's used?
Assignee: nobody → dustin
Flags: needinfo?(rail)
Comment 3•8 years ago
|
||
IIRC it's in http://hg.mozilla.org/build/buildbotcustom/file/tip/misc_scheduler.py#l82
Flags: needinfo?(rail)
Assignee | ||
Comment 4•8 years ago
|
||
From a look at that code, it's generated via time.strftime("%Y%m%d%H%M%S", time.localtime(now)) and uses the database to ensure uniqueness (noting that these DB calls are made synchronously, and that there is only one scheduler master, there is no race condition here). Replicating that in a mach command is pretty tricky, since mach doesn't have access to any common storage. One method I can see is for mach to use the TaskCluster proxy to talk to taskcluster and get access to an Azure table dedicated to this purpose, then use Azure Table Storage's consistency guarantees to select a unique buildid. That would mean that `mach taskcluster-graph` wouldn't work for devs anymore (at least, not without TC credentials). It would also mean that `mach` has to include an Azure client. Other options might include a buildid generator, perhaps as a relengapi component The more "modern" way to do this sort of thing is to generate a UUID randomly - this is how taskids are created, for example. So if we could change the format of the buildid to something like 201601211103901-HdpAswIdR-eVmlwqcv4DBg then we could be (probabalistically) sure of uniqueness without any need for a centralized arbiter of uniqueness. Which way should we go?
Reporter | ||
Comment 5•8 years ago
|
||
The point is that all builds for a given changeset take the same buildid. Randomly unique is not helping that, and you need a centralized arbitrer.
Assignee | ||
Comment 6•8 years ago
|
||
That part's easy -- the decision task can pick the buildid and stick it into the task description for each build task. The issue is that if two decision tasks run at the same time (within 1s), they will pick the same buildid. From my chat with rail, and from your comment, it sounds like this is not an important concern?
Comment 7•8 years ago
|
||
All builds for the same push sharing the same UUID would be useful too, I think. buildids as they're implemented aren't guaranteed to be unique across branches.
Assignee | ||
Comment 8•8 years ago
|
||
How is the UUID specified? And it is it just a regular v4 UUID?
Flags: needinfo?(catlee)
Assignee | ||
Comment 9•8 years ago
|
||
Ah, I'm going to guess catlee meant BuildUID. But I wouldn't put it past us to have separate BUILDID, BUILDUID, BUILDUUID, and even BUILDUUUIDs :)
Flags: needinfo?(catlee)
Assignee | ||
Comment 10•8 years ago
|
||
It looks like builduid is only used for sendchange, which we don't use in taskcluster. The buildid, on the other hand, goes into to the build environment as MOZ_BUILD_DATE. Mozharness not-so-helpfully tries to read that from a buildbot property, but I think we can hack around that.
Assignee | ||
Comment 11•8 years ago
|
||
https://treeherder.mozilla.org/#/jobs?repo=try&revision=084ba2c69330
Assignee | ||
Comment 12•8 years ago
|
||
" The review mercurial server is down - please use Bugzilla for any new code reviews at this time " ..it's like living in the dark ages
Assignee | ||
Comment 13•8 years ago
|
||
The try job in TC is looking good (I see MOZ_BUILD_DATE set to the same value as specified in the task description).
Attachment #8713313 -
Flags: review?(jlund)
Comment 14•8 years ago
|
||
BTW, build IDs are checked as a part of the update procedure and the updater won't let you update if current build ID > new build ID.
Comment 15•8 years ago
|
||
Comment on attachment 8713313 [details] [diff] [review] patch.patch Review of attachment 8713313 [details] [diff] [review]: ----------------------------------------------------------------- ::: testing/taskcluster/tasks/build.yml @@ +47,5 @@ > GECKO_HEAD_REV: '{{head_rev}}' > GECKO_HEAD_REF: '{{head_ref}}' > TOOLTOOL_REPO: 'https://git.mozilla.org/build/tooltool.git' > TOOLTOOL_REV: 'master' > + MOZ_BUILD_DATE: '{{pushdate}}' given rail's comment: will this pushdate clash with buildbot based buildids? as in, will it ever produce buildids that are < a buildid in the update server?
Attachment #8713313 -
Flags: review?(jlund) → review+
Reporter | ||
Comment 16•8 years ago
|
||
It should use the same as buildbot...
Assignee | ||
Comment 17•8 years ago
|
||
That's the same format as Buildbot. I'm not sure what event Buildbot uses to determine the time represented in the build date, but all of the available times (push, schedule, job start, mozharness run) are within a few hours at most so I don't see how this could be problematic over the timescale of updates. Unless I'm missing something you're all speaking about obliquely? Is there actually a problem?
Comment 18•8 years ago
|
||
> Unless I'm missing something you're all speaking about obliquely? Is there
> actually a problem?
sounds like you thought it through. makes sense and wfm :)
Assignee | ||
Comment 19•8 years ago
|
||
https://hg.mozilla.org/integration/mozilla-inbound/rev/c337204b80ad4f3da051a3037808891c5a3c2269 Bug 1231318: pass MOZ_BUILD_DATE to builds based on pushdate; r=jlund
Comment 20•8 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/c337204b80ad
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Assignee | ||
Comment 21•8 years ago
|
||
What's the best way to verify that this has worked? Does this show up in about:<something>?
Status: RESOLVED → REOPENED
Flags: needinfo?(mh+mozilla)
Resolution: FIXED → ---
Reporter | ||
Comment 22•8 years ago
|
||
The first line of target.txt contains it.
Flags: needinfo?(mh+mozilla)
Assignee | ||
Comment 23•8 years ago
|
||
Confirmed: https://public-artifacts.taskcluster.net/BWCV686SRtKwqQ9MpFFUcg/0/public/build/target.txt 20160204142342 https://hg.mozilla.org/integration/mozilla-inbound/rev/65e246baede44b6c75f0603d4cc3901e0d7a45a9
Status: REOPENED → RESOLVED
Closed: 8 years ago → 8 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•