829145 - Reveal gaia changesets for b2g builds on tbpl

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P2)

Description

[Armen [:armenzg]]

This will make things easier when sheriffing.

Comment 1

[Ed Morley [:emorley]]

Good idea, thank you :-)

Comment 2

[Armen [:armenzg]]

We can probably download the sources.xml file and TinderboxPrint the gaia repo and changeset.

<!--Mercurial-Information: <project name="integration/gaia-central" path="gaia" remote="hgmozillaorg" revision="6e784c6fee0f"/> -->

We can probably add it to "run_test" or add another action:
http://hg.mozilla.org/build/mozharness/file/default/scripts/b2g_panda.py#l117

This can be tested on Ash by pushing to this mozharness repo as if it was try:
http://hg.mozilla.org/users/asasaki_mozilla.com/ash-mozharness

Comment 3

[Nick Thomas [:nthomas] (UTC+12)]

Compile jobs already have a gaia_revision property which can be TinderboxPrint'd. We could pass that to tests as a property on the sendchange.

Comment 4

[Armen [:armenzg]]

Attachment: print URL to gaia change

Comment 5

[Armen [:armenzg]]

Attachment: TinderboxPrint the URL pointing to the scripts repo/revision used

mozharness is an external factor to the job and it might be good to keep track of it.

Comment 6

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 704593 [details] [diff] [review]
print URL to gaia change

I'll switch this to feedback+ for a couple of reasons
* you'll get urls like this:
    http://hg.mozilla.org/integration/gaia-nightly/e84cc9f3e1e5
  which doesn't work, instead of 
    http://hg.mozilla.org/integration/gaia-nightly/rev/e84cc9f3e1e5
* we should ask edmorley what tbpl needs here (maybe you did this already and didn't mention it on the bug)

Comment 7

[Armen [:armenzg]]

Attachment: TinderboxPrint the URL of the gaia revision

Meh :( that was the old patch. updated.

jgriffin, edmorley, would this work for you?

Comment 8

[Jonathan Griffin (:jgriffin)]

Comment on attachment 704670 [details] [diff] [review]
TinderboxPrint the URL of the gaia revision

Could we linkify this?  Some people looking at these results may not realize the gaia checkout refers to our hg clone, and not the original github repo.

Comment 9

[Armen [:armenzg]]

(In reply to Jonathan Griffin (:jgriffin) from comment #8)
> Comment on attachment 704670 [details] [diff] [review]
> TinderboxPrint the URL of the gaia revision
> 
> Could we linkify this?  Some people looking at these results may not realize
> the gaia checkout refers to our hg clone, and not the original github repo.

Would you like me to skip the hg URL at all?
We have a mapper application to match hg to git. I will look to see how it works.

edmorley, can I TinderboxPrint an HTML hyperlink? would tbpl be OK with it?

Comment 10

[Ed Morley [:emorley]]

Comment on attachment 704670 [details] [diff] [review]
TinderboxPrint the URL of the gaia revision

This is fine, but as your later comments mentions, using markup would be preferred.

Eg printbuildrev.py does:
https://hg.mozilla.org/build/tools/file/6b11bdd34fcb/buildfarm/utils/printbuildrev.py#l30

Comment 11

[Ed Morley [:emorley]]

Note we'll also need to remove:
https://hg.mozilla.org/webtools/tbpl/file/ac35be9cb467/js/MachineResult.js#l55

(Believe it was added ages ago due to having multiple revisions pasted in the logs; they've since been cleaned up so should be fine)

Comment 12

[Jonathan Griffin (:jgriffin)]

(In reply to Armen Zambrano G. [:armenzg] from comment #9)
> (In reply to Jonathan Griffin (:jgriffin) from comment #8)
> > Comment on attachment 704670 [details] [diff] [review]
> > TinderboxPrint the URL of the gaia revision
> > 
> > Could we linkify this?  Some people looking at these results may not realize
> > the gaia checkout refers to our hg clone, and not the original github repo.
> 
> Would you like me to skip the hg URL at all?
> We have a mapper application to match hg to git. I will look to see how it
> works.
> 
> edmorley, can I TinderboxPrint an HTML hyperlink? would tbpl be OK with it?

If we could get the link to the gaia commit, that would be best, since that's what most developers work against.  If not, even the hg link would be better than nothing.

Comment 13

[Armen [:armenzg]]

Let's go with the hg URLs for now.

13:20 armenzg: should I tell the A-team we can only offer them the hg URLs to the gaia repos?
13:21 catlee: yeah, it's possible that the mapfile won't have the needed data in time for dep builds
13:21 catlee: unless we rework how they're published and consumed

Comment 14

[Chris AtLee [:catlee]]

Comment on attachment 704650 [details] [diff] [review]
TinderboxPrint the URL pointing to the scripts repo/revision used

Review of attachment 704650 [details] [diff] [review]:
-----------------------------------------------------------------

::: process/factory.py
@@ +6310,5 @@
> +            property='script_repo',
> +            command=['echo', scriptRepo],
> +            workdir='scripts',
> +            haltOnFailure=True,
> +        ))

Please use SetBuildProperty instead of SetProperty here. No need to have a slave echo out a value just so we can put it into a property.

@@ +6320,5 @@
> +            haltOnFailure=True,
> +        ))
> +        self.addStep(ShellCommand(
> +            name='print_url_to_script_revision_used',
> +            command=['echo', "TinderboxPrint:", WithProperties("%(script_repo)s/rev/%(script_repo_revision)s")],

If all you're using the script_repo property for is this, then it's cleaner to do something like

WithProperties("%s/rev/%%(script_repo_revision)s" % scriptRepo)

and then you don't need the SetBuildProperty step at all.

Do we need to add something to the TinderboxPrint to indicate this is the script repo as opposed to some other repo?

Comment 15

[Chris AtLee [:catlee]]

Comment on attachment 704670 [details] [diff] [review]
TinderboxPrint the URL of the gaia revision

Review of attachment 704670 [details] [diff] [review]:
-----------------------------------------------------------------

::: scripts/b2g_build.py
@@ +400,5 @@
>              branch = gaia_config.get('branch')
>              vcs = gaia_config['vcs']
>              rev = self.vcs_checkout(repo=repo, dest=dest, vcs=vcs, branch=branch)
>              self.set_buildbot_property('gaia_revision', rev, write_to_file=True)
> +            self.info("TinderboxPrint: gaia_checkout: %s/rev/%s" % (repo, rev))

r+ with gaia_revlink instead of gaia_checkout

Comment 16

[Armen [:armenzg]]

Attachment: TinderboxPrint the URL pointing to the scripts repo/revision used

The running repo is:
http://hg.mozilla.org/users/asasaki_mozilla.com/ash-mozharness
the result is this:
TinderboxPrint: ash-mozharness_rev_link http://hg.mozilla.org/users/asasaki_mozilla.com/ash-mozharness/rev/7ad6473689d4

Comment 17

[Armen [:armenzg]]

Attachment: hyperlink to gaia changeset

r=bhearsum
I nevertheless want to give it a spin first.

Comment 18

[Ed Morley [:emorley]]

Comment on attachment 705964 [details] [diff] [review]
hyperlink to gaia changeset

Please can the href text be trimmed to just the changeset hash, rather than the full URL? (similar to the example I gave in comment 10) :-)

Comment 19

[Ed Morley [:emorley]]

eg:
gaia_revlink: 616244739680

Comment 20

[Chris AtLee [:catlee]]

(In reply to Ed Morley [:edmorley UTC+0] from comment #19)
> eg:
> gaia_revlink: 616244739680

If that's what you want, then let's call it 'gaia_revision'.

Comment 21

[Armen [:armenzg]]

Comment on attachment 705964 [details] [diff] [review]
hyperlink to gaia changeset

Everyone happy! Rejoice! :)

gaia_revision: <a href='%(repo)s/rev/%(rev)s'>%(rev)s</a>" % dict(repo=repo, rev=rev)
http://hg.mozilla.org/build/mozharness/rev/139b0a1ff7b1

Comment 22

[Armen [:armenzg]]

Comment on attachment 705964 [details] [diff] [review]
hyperlink to gaia changeset

Until we find out what's wrong in bug 834456.

Comment 23

[Chris AtLee [:catlee]]

Comment on attachment 705502 [details] [diff] [review]
TinderboxPrint the URL pointing to the scripts repo/revision used

Review of attachment 705502 [details] [diff] [review]:
-----------------------------------------------------------------

::: process/factory.py
@@ +6560,5 @@
> +        ))
> +        self.addStep(ShellCommand(
> +            name='print_url_to_script_revision_used',
> +            command=['echo', 'TinderboxPrint:', 
> +                "%s_rev_link" % scriptRepo.split('/')[-1],

change rev_link to revlink to be consistent with your other patch
does self.getRepoName(scriptRepo) get what you want, rather than scriptRepo.split('/')[-1]?

@@ +6561,5 @@
> +        self.addStep(ShellCommand(
> +            name='print_url_to_script_revision_used',
> +            command=['echo', 'TinderboxPrint:', 
> +                "%s_rev_link" % scriptRepo.split('/')[-1],
> +                WithProperties("%s/rev/%%(script_repo_revision)s" % scriptRepo)], 

nit: trailing whitespace

Comment 24

[Armen [:armenzg]]

Comment on attachment 705502 [details] [diff] [review]
TinderboxPrint the URL pointing to the scripts repo/revision used

http://hg.mozilla.org/build/buildbotcustom/rev/ce2310555604

Review notes added.

Comment 25

[Armen [:armenzg]]

I will deal with attachment 705964 [details] [diff] [review] in bug 834456.

Comment 26

[Armen [:armenzg]]

I will take care of attachment 705502 [details] [diff] [review] for the mozharness revision on bug 835688.

bug 834456 is nice to have but we have a working link on the job summary.

Comment 27

[Ed Morley [:emorley]]

The TinderboxPrint format that landed isn't actually that's listed in comment 21, which is why it's not linkified.

However, I've filed bug 845388 to fix the general mess that is the idea of markup in TinderboxPrint :-)