834354 - gaia-central B2G jobs should report to buildbot on their own tree, similar to Addon-SDK/jetpack

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P1)

Description

[Jonathan Griffin (:jgriffin)]

We need a new view in TBPL that will show Gaia UI tests ("G" in TBPL) per commit to gaia-central (http: //hg.mozilla.org/integration/gaia-central), similar to the current Addon-SDK view:  https://tbpl.mozilla.org/?tree=Jetpack

This view would aggregate test results across the various gecko branches on which the tests are run.

Comment 1

[Ed Morley [:emorley]]

This needs to be done primarily in buildbot (/mozharness/...), similar to how the jetpack tree works.

The only jetpack specific code in TBPL is the tree entry (the same as any other tree):
https://hg.mozilla.org/webtools/tbpl/file/183241a96fe5/js/Config.js#l94
and the tooltips/letters shown for each:
https://hg.mozilla.org/webtools/tbpl/file/183241a96fe5/js/Data.js#l569
https://hg.mozilla.org/webtools/tbpl/file/183241a96fe5/js/Config.js#l382

The gaiu UI test jobs need to report themselves to buildbot as being on a "gaia-central" tree. We'll then make the necessary Config.js and Data.js changes to add a new tree/job name.

Comment 2

[Phil Ringnalda (:philor)]

Well, that would be our share of bug 834356; this seems to actually be asking that buildbot include the g-c rev in every gaia-ui test job on every tree, and that we have a tree which looks at g-c_revision instead of revision, and displays:

e8c582246a18  Gm-c Gm-c Gm-c Gm-c Gm-i Gm-i Gm-i Gm-i Gm-i Gm-i Gm-i Gm-i Gm-i Gm-i Gm-i Gm-i Gm-i Gm-i Gm-i Gm-i Gm-i 

until there's another push to g-c so that m-c/m-i runs run against a new g-c rev, so that we show the jobs which run against inbound both on inbound's tree and on a separate tree-that's-no-particular-tree.

Comment 3

[Phil Ringnalda (:philor)]

FWIW, I think g-c *should* follow the actual Jetpack model, and run on-push on other trees hidden, where people concerned with it can use &noignore=1 to chase down bustages, with a g-c tree that's triggered by gaia pushes to see the results of their own pushes, until the annoyance of that wears them down and they finally land their code in m-c so they can become unhidden.

Comment 4

[Armen [:armenzg]]

The ideal solution (and the easiest is) if gaia was part of mozilla-central or if we controlled it with a manifest.

I want to be explicit about this.
I can take 2 approaches to this.

OPTION A
- Create a project branch per "gaia" branch (mozilla/project_branches.py)
-- which checkouts one of the associated repos
-- but it polls the repo branches
-- simple patches
-- we build a gecko repo ("repo_path") but we checkout a gaia repo ("poll_repo")(different than all other project branches)

OPTION B
- Create a b2g project per "gaia" branch (mozilla/b2g_config.py)
-- same as above
-- more complicated patches

Unlike Jetpack, we have to generate a build for gaia to be tested.
Hence, the approach of triggering 4 Jetpack jobs (for 4 different branches) for every Jetpack change does not translate well to Gaia (in fact, I don't think that model was correct even for Jetpack).

I'm leaning towards option A since we really only want extra granularity for an external project.

Comment 5

[Armen [:armenzg]]

Attachment: build config changes

Comment 6

[Armen [:armenzg]]

Attachment: test build configs

Comment 7

[Armen [:armenzg]]

Attachment: custom changes

Comment 8

[Aki Sasaki (not active)]

Comment on attachment 712562 [details] [diff] [review]
build config changes

I think this works.

Comment 9

[Aki Sasaki (not active)]

Comment on attachment 712564 [details] [diff] [review]
custom changes

A comment that actually applies to the first two patches: we may want to call this branch gaia-central if we plan on having to do something similar with other branches.

Comment 10

[Armen [:armenzg]]

I would like to be more explicit; I would like to associate them to the branch from where we build (rather than the gaia branch):
* gaia-mozilla-central (for master)
* gaia-mozilla-b2g18   (for v1-train)
* gaia-mozilla-b2g18_v1_0_0 (for v1.0.0)
* gaia-mozilla-b2g18_v1_0_1 (for v1.0.0)

Would this work for you?

The patches are ready to be reviewed and landed (I will adjust the tree names upon landing).

Comment 11

[Armen [:armenzg]]

I was waiting for success on staging before requesting reviews.

Comment 12

[Aki Sasaki (not active)]

(In reply to Armen Zambrano G. [:armenzg] from comment #10)
> I would like to be more explicit; I would like to associate them to the
> branch from where we build (rather than the gaia branch):
> * gaia-mozilla-central (for master)
> * gaia-mozilla-b2g18   (for v1-train)
> * gaia-mozilla-b2g18_v1_0_0 (for v1.0.0)
> * gaia-mozilla-b2g18_v1_0_1 (for v1.0.0)
> 
> Would this work for you?
> 
> The patches are ready to be reviewed and landed (I will adjust the tree
> names upon landing).

This might be more confusing, since that means people need to switch pages, say, mid-1.0.1 when that moves from mozilla-b2g18 to mozilla-b2g18_v1_0_1.  A gaia-v1_0_1 page would follow the gecko branch switch and people could keep their bookmarks/tabs the same.

I guess it depends who this page is oriented towards.

Comment 13

[Armen [:armenzg]]

Comment on attachment 712564 [details] [diff] [review]
custom changes

b800f14c86f6

Comment 14

[Armen [:armenzg]]

http://hg.mozilla.org/build/buildbot-configs/rev/b72d58a259ce

We discussed on IRC that the naming would be something like this (since the pages will be oriented to Gaia devs):
* gaia-mozilla-master
* gaia-mozilla-v1-train
* gaia-mozilla-v1.0.0 
* gaia-mozilla-v1.0.1

Are periods bad?

Waiting for reconfiguration.

Comment 15

[Aki Sasaki (not active)]

(In reply to Armen Zambrano G. [:armenzg] from comment #14)
> http://hg.mozilla.org/build/buildbot-configs/rev/b72d58a259ce
> 
> We discussed on IRC that the naming would be something like this (since the
> pages will be oriented to Gaia devs):
> * gaia-mozilla-master
> * gaia-mozilla-v1-train
> * gaia-mozilla-v1.0.0 
> * gaia-mozilla-v1.0.1

I think the email names were better; not sure what the "mozilla-" is for.

> Are periods bad?

I think this is a question for tbpl.  I don't care strongly if it's periods or underscores.

Comment 16

[Armen [:armenzg]]

(In reply to Aki Sasaki [:aki] from comment #15)
> (In reply to Armen Zambrano G. [:armenzg] from comment #14)
> > http://hg.mozilla.org/build/buildbot-configs/rev/b72d58a259ce
> > 
> > We discussed on IRC that the naming would be something like this (since the
> > pages will be oriented to Gaia devs):
> > * gaia-mozilla-master
> > * gaia-mozilla-v1-train
> > * gaia-mozilla-v1.0.0 
> > * gaia-mozilla-v1.0.1
> 
> I think the email names were better; not sure what the "mozilla-" is for.
> 
Bah.
It should have been:
* gaia-master
* gaia-v1-train
* gaia-v1.0.0 
* gaia-v1.0.1

> > Are periods bad?
> 
> I think this is a question for tbpl.  I don't care strongly if it's periods
> or underscores.

edmorley, would periods on the name give us any trouble?

Comment 17

[Armen [:armenzg]]

Heads-up and discussion about naming:
https://groups.google.com/d/topic/mozilla.dev.gaia/NabKrlkf1ow/discussion

Comment 18

[Armen [:armenzg]]

Attachment: add Gaia trees

The last three branches will not be added to buildbot on this pass.
I want to make sure that I iron out everything first on Gaia-Master.

Comment 19

[Aki Sasaki (not active)]

The currently landed patches are in production.

Comment 20

[Armen [:armenzg]]

I see it running on buildbot:
http://buildbot-master49.srv.releng.scl3.mozilla.com:8001/builders/b2g_gaia-master_panda_dep
http://buildbot-master29.build.scl1.mozilla.com:8201/builders/b2g_panda%20gaia-master%20opt%20test%20gaia-ui-test

Comment 21

[Ed Morley [:emorley]]

Comment on attachment 713071 [details] [diff] [review]
add Gaia trees

Review of attachment 713071 [details] [diff] [review]:
-----------------------------------------------------------------

Regarding the periods in the name - I don't think it should be a problem - though only one way to tell for sure :-) I do prefer them to underscores however (particularly in the TBPL UI), so would like to give it a shot.

prettierName is only required if a name other than the key name is desired in the UI, so these are redundant.

r=me without the prettierName entries.

Comment 22

[Armen [:armenzg]]

Comment on attachment 713071 [details] [diff] [review]
add Gaia trees

Removed prettierNames:
https://hg.mozilla.org/webtools/tbpl/rev/b8a5c7b21be9

Comment 23

[Ed Morley [:emorley]]

Comment on attachment 713071 [details] [diff] [review]
add Gaia trees

I'd like to test this with the new Chief setup (bug 827473), so if possible, hold off filing a push to production bug for now please :-)

Comment 24

[Armen [:armenzg]]

I can wait.

I see it running on https://tbpl-dev.allizom.org/?tree=Gaia-Master

I have to fix an issue I did not notice before.
The changesets of gaia-central do not exist on mozilla-central - duh!

On staging I probably used force builds and I thought the build due to a poll where pointing to my users/armenzg_mozilla.com/mozilla-central and not having the changesets.

Comment 25

[Armen [:armenzg]]

Attachment: --checkout-revision default

What do you think of this?

I want to call the script with --checkout-revision default (it has to be added) which would update the build to default rather than the gaia revision which shows up as a buildbot property from the poling.

Comment 26

[Aki Sasaki (not active)]

Comment on attachment 713520 [details] [diff] [review]
--checkout-revision default

This will probably work.
What happens in terms of the buildbot properties?
We need to make sure that the actual revision (not default) shows up for tbpl.

Comment 27

[Armen [:armenzg]]

Attachment: add --checkout-revision to b2g_build.py

Comment 28

[Armen [:armenzg]]

Attachment: add --checkout-revision to b2g_build.py (v2)

Comment 29

[Armen [:armenzg]]

Comment on attachment 713520 [details] [diff] [review]
--checkout-revision default

This cannot land before the other change makes it to mozharness/production.

I got these properties:
gaia_revision 	40c3b26ea181
gecko_revision 	081cf5b0121e
which matches:
http://hg.mozilla.org/integration/gaia-central/rev/40c3b26ea181
http://hg.mozilla.org/mozilla-central/rev/081cf5b0121e

Comment 30

[Aki Sasaki (not active)]

Comment on attachment 713545 [details] [diff] [review]
add --checkout-revision to b2g_build.py (v2)

I'm not sure I like this because:

1) behavior that doesn't intuitively belong to a --checkout-revision gets attached to --checkout-revision (namely, the buildbot_property('revision') gets set differently if it's set or not), and
2) we base other information (sources xml, try upload_path) on the 'revision' buildbot property.

Could you:

1) add a comment about why checkout_revision works this way in checkout_gecko (right above your additions),
2) set 'gecko_revision' to rev in the else block as well,
3) change the update_source_manifest() self.buildbot_properties['revision']'s to self.buildbot_properties['gecko_revision'] ?

I can clarify if the above doesn't make sense.

Comment 31

[Armen [:armenzg]]

Attachment: add --checkout-revision to b2g_build.py (v3)

Like this?

Comment 32

[Aki Sasaki (not active)]

Comment on attachment 714023 [details] [diff] [review]
add --checkout-revision to b2g_build.py (v3)

>+        if self.config.has_key("checkout_revision"):
>+            rev = self.vcs_checkout(repo=repo, dest=dirs['src'], revision=self.config["checkout_revision"])
>+            self.set_buildbot_property('revision', self.config["checkout_revision"], write_to_file=True)
>+            self.set_buildbot_property('gecko_revision', rev, write_to_file=True)
>+        else:
>+            rev = self.vcs_checkout(repo=repo, dest=dirs['src'], revision=self.query_revision())
>+            self.set_buildbot_property('revision', rev, write_to_file=True)
>+            self.set_buildbot_property('gecko_revision', rev, write_to_file=True)

You could move the gecko_revision setting outside the if/else block.  I thought of this after my previous comment.

>             self.warning("Couldn't find git_base_url in manifest; using %s" % git_base_url)
>         new_sources = []
> 
>         for line in sources.splitlines():
>             new_sources.append(line)
>             if 'Gonk specific things' in line:
>                 new_sources.append('  <!-- Mercurial-Information: <remote fetch="http://hg.mozilla.org/" name="hgmozillaorg"> -->')
>                 new_sources.append('  <!-- Mercurial-Information: <project name="%s" path="gecko" remote="hgmozillaorg" revision="%s"/> -->' %
>-                                   (self.buildbot_config['properties']['repo_path'], self.buildbot_properties['revision']))
>+                                   (self.buildbot_config['properties']['repo_path'], self.buildbot_properties['gecko_revision']))
>                 new_sources.append('  <!-- Mercurial-Information: <project name="%s" path="gaia" remote="hgmozillaorg" revision="%s"/> -->' %
>                                    (gaia_config['repo'].replace('http://hg.mozilla.org/', ''), self.buildbot_properties['gaia_revision']))
> 
>                 if self.query_do_translate_hg_to_git():
>                     url = manifest_config['translate_base_url']
>                     gecko_git = self.query_translated_revision(url, 'gecko', self.buildbot_properties['revision'])

Don't forget this line ^^ (s,revision,gecko_revision,)

r=me with those fixes.

Comment 33

[Armen [:armenzg]]

Attachment: add --checkout-revision to b2g_build.py (v4)

upload calls query_revision() and we would have been using the gaia revision.
Even though that is for try. I don't think I need this patch.

Would you want me to use the previous patch?

Comment 34

[Aki Sasaki (not active)]

Comment on attachment 714076 [details] [diff] [review]
add --checkout-revision to b2g_build.py (v4)

Yeah, I think in this case

a) try isn't using gaia revisions
b) if it were, we might want to upload using the gaia revision

So I saw this before I commented earlier, and decided to keep try uploads with query_revision().

Previous patch is fine.  We can use this if we do turn on gaia revisions on a try-like branch, and we want to upload using the gecko revision.

Comment 35

[Armen [:armenzg]]

Comment on attachment 714023 [details] [diff] [review]
add --checkout-revision to b2g_build.py (v3)

If everything goes well I will land the buildbotcustom change and merge mozharness (default->production).

Landed:
http://hg.mozilla.org/build/mozharness/rev/48e022f5de22

Also testing on Ash one more time to be sure:
https://tbpl.mozilla.org/?tree=Ash&rev=a373379d43cf
http://hg.mozilla.org/users/asasaki_mozilla.com/ash-mozharness/rev/48e022f5de22

Comment 36

[Armen [:armenzg]]

Comment on attachment 713520 [details] [diff] [review]
--checkout-revision default

http://hg.mozilla.org/build/buildbot-configs/rev/6f1d4fc40a79

Comment 37

[Armen [:armenzg]]

Comment on attachment 714023 [details] [diff] [review]
add --checkout-revision to b2g_build.py (v3)

Merged to production:
http://hg.mozilla.org/build/mozharness/rev/428c474cc43e

Reconfig will come soon.

Comment 38

[Armen [:armenzg]]

This is done:
https://tbpl-dev.allizom.org/?tree=Gaia-Master