1231318 - Taskcluster builds don't use the same buildid as buildbot builds

Status: RESOLVED FIXED

(Taskcluster :: General, defect)

Description

[Mike Hommey [:glandium]]

All builds for a given push, in buildbot, use the same buildid, passed down to the build through MOZ_BUILD_DATE. I don't remember when this was done, but I do remember it was done on purpose. So it would be good for taskcluster builds to do the same.

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

Rail, can you give me some pointers on how I should set this and how it's used?

Comment 3

[Rail Aliiev [:rail]]

IIRC it's in http://hg.mozilla.org/build/buildbotcustom/file/tip/misc_scheduler.py#l82

Comment 4

[Dustin J. Mitchell [:dustin] (he/him)]

From a look at that code, it's generated via time.strftime("%Y%m%d%H%M%S", time.localtime(now)) and uses the database to ensure uniqueness (noting that these DB calls are made synchronously, and that there is only one scheduler master, there is no race condition  here).

Replicating that in a mach command is pretty tricky, since mach doesn't have access to any common storage.  One method I can see is for mach to use the TaskCluster proxy to talk to taskcluster and get access to an Azure table dedicated to this purpose, then use Azure Table Storage's consistency guarantees to select a unique buildid.  That would mean that `mach taskcluster-graph` wouldn't work for devs anymore (at least, not without TC credentials).  It would also mean that `mach` has to include an Azure client.  Other options might include a buildid generator, perhaps as a relengapi component

The more "modern" way to do this sort of thing is to generate a UUID randomly - this is how taskids are created, for example.  So if we could change the format of the buildid to something like 201601211103901-HdpAswIdR-eVmlwqcv4DBg then we could be (probabalistically) sure of uniqueness without any need for a centralized arbiter of uniqueness.

Which way should we go?

Comment 5

[Mike Hommey [:glandium]]

The point is that all builds for a given changeset take the same buildid. Randomly unique is not helping that, and you need a centralized arbitrer.

Comment 6

[Dustin J. Mitchell [:dustin] (he/him)]

That part's easy -- the decision task can pick the buildid and stick it into the task description for each build task.  The issue is that if two decision tasks run at the same time (within 1s), they will pick the same buildid.  From my chat with rail, and from your comment, it sounds like this is not an important concern?

Comment 7

[Chris AtLee [:catlee]]

All builds for the same push sharing the same UUID would be useful too, I think. 

buildids as they're implemented aren't guaranteed to be unique across branches.

Comment 8

[Dustin J. Mitchell [:dustin] (he/him)]

How is the UUID specified?  And it is it just a regular v4 UUID?

Comment 9

[Dustin J. Mitchell [:dustin] (he/him)]

Ah, I'm going to guess catlee meant BuildUID.  But I wouldn't put it past us to have separate BUILDID, BUILDUID, BUILDUUID, and even BUILDUUUIDs :)

Comment 10

[Dustin J. Mitchell [:dustin] (he/him)]

It looks like builduid is only used for sendchange, which we don't use in taskcluster.

The buildid, on the other hand, goes into to the build environment as MOZ_BUILD_DATE.  Mozharness not-so-helpfully tries to read that from a buildbot property, but I think we can hack around that.

Comment 11

[Dustin J. Mitchell [:dustin] (he/him)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=084ba2c69330

Comment 12

[Dustin J. Mitchell [:dustin] (he/him)]

" The review mercurial server is down - please use Bugzilla for any new code reviews at this time "

..it's like living in the dark ages

Comment 13

[Dustin J. Mitchell [:dustin] (he/him)]

Attachment: patch.patch

The try job in TC is looking good (I see MOZ_BUILD_DATE set to the same value as specified in the task description).

Comment 14

[Rail Aliiev [:rail]]

BTW, build IDs are checked as a part of the update procedure and the updater won't let you update if current build ID > new build ID.

Comment 15

[Jordan Lund (:jlund)]

Comment on attachment 8713313 [details] [diff] [review]
patch.patch

Review of attachment 8713313 [details] [diff] [review]:
-----------------------------------------------------------------

::: testing/taskcluster/tasks/build.yml
@@ +47,5 @@
>        GECKO_HEAD_REV: '{{head_rev}}'
>        GECKO_HEAD_REF: '{{head_ref}}'
>        TOOLTOOL_REPO: 'https://git.mozilla.org/build/tooltool.git'
>        TOOLTOOL_REV: 'master'
> +      MOZ_BUILD_DATE: '{{pushdate}}'

given rail's comment: will this pushdate clash with buildbot based buildids? as in, will it ever produce buildids that are < a buildid in the update server?

Comment 16

[Mike Hommey [:glandium]]

It should use the same as buildbot...

Comment 17

[Dustin J. Mitchell [:dustin] (he/him)]

That's the same format as Buildbot.

I'm not sure what event Buildbot uses to determine the time represented in the build date, but all of the available times (push, schedule, job start, mozharness run) are within a few hours at most so I don't see how this could be problematic over the timescale of updates.

Unless I'm missing something you're all speaking about obliquely?  Is there actually a problem?

Comment 18

[Jordan Lund (:jlund)]

> Unless I'm missing something you're all speaking about obliquely?  Is there
> actually a problem?

sounds like you thought it through. makes sense and wfm :)

Comment 19

[Dustin J. Mitchell [:dustin] (he/him)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/c337204b80ad4f3da051a3037808891c5a3c2269
Bug 1231318: pass MOZ_BUILD_DATE to builds based on pushdate; r=jlund

Comment 20

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/c337204b80ad

Comment 21

[Dustin J. Mitchell [:dustin] (he/him)]

What's the best way to verify that this has worked?  Does this show up in about:<something>?

Comment 22

[Mike Hommey [:glandium]]

The first line of target.txt contains it.

Comment 23

[Dustin J. Mitchell [:dustin] (he/him)]

Confirmed:

https://public-artifacts.taskcluster.net/BWCV686SRtKwqQ9MpFFUcg/0/public/build/target.txt
20160204142342
https://hg.mozilla.org/integration/mozilla-inbound/rev/65e246baede44b6c75f0603d4cc3901e0d7a45a9