Closed
Bug 834354
Opened 11 years ago
Closed 11 years ago
gaia-central B2G jobs should report to buildbot on their own tree, similar to Addon-SDK/jetpack
Categories
(Release Engineering :: General, defect, P1)
Release Engineering
General
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: jgriffin, Assigned: armenzg)
References
()
Details
(Keywords: sheriffing-P1)
Attachments
(6 files, 3 obsolete files)
1.50 KB,
patch
|
mozilla
:
review+
mozilla
:
feedback+
armenzg
:
checked-in+
|
Details | Diff | Splinter Review |
1.42 KB,
patch
|
mozilla
:
review+
mozilla
:
feedback+
armenzg
:
checked-in+
|
Details | Diff | Splinter Review |
1.55 KB,
patch
|
mozilla
:
review+
mozilla
:
feedback+
armenzg
:
checked-in+
|
Details | Diff | Splinter Review |
1.68 KB,
patch
|
emorley
:
review+
armenzg
:
checked-in+
|
Details | Diff | Splinter Review |
1.29 KB,
patch
|
mozilla
:
review+
mozilla
:
feedback+
armenzg
:
checked-in+
|
Details | Diff | Splinter Review |
4.32 KB,
patch
|
mozilla
:
review+
armenzg
:
checked-in+
|
Details | Diff | Splinter Review |
We need a new view in TBPL that will show Gaia UI tests ("G" in TBPL) per commit to gaia-central (http: //hg.mozilla.org/integration/gaia-central), similar to the current Addon-SDK view: https://tbpl.mozilla.org/?tree=Jetpack This view would aggregate test results across the various gecko branches on which the tests are run.
Comment 1•11 years ago
|
||
This needs to be done primarily in buildbot (/mozharness/...), similar to how the jetpack tree works. The only jetpack specific code in TBPL is the tree entry (the same as any other tree): https://hg.mozilla.org/webtools/tbpl/file/183241a96fe5/js/Config.js#l94 and the tooltips/letters shown for each: https://hg.mozilla.org/webtools/tbpl/file/183241a96fe5/js/Data.js#l569 https://hg.mozilla.org/webtools/tbpl/file/183241a96fe5/js/Config.js#l382 The gaiu UI test jobs need to report themselves to buildbot as being on a "gaia-central" tree. We'll then make the necessary Config.js and Data.js changes to add a new tree/job name.
Updated•11 years ago
|
Component: Tinderboxpushlog → Release Engineering
Keywords: sheriffing-P1
Product: Webtools → mozilla.org
Summary: Need gaia-central-centric view in TBPL, ala the Addon-SDK view → gaia-central B2G jobs should report to buildbot on their own tree, similar to Addon-SDK/jetpack
Version: Trunk → other
Comment 2•11 years ago
|
||
Well, that would be our share of bug 834356; this seems to actually be asking that buildbot include the g-c rev in every gaia-ui test job on every tree, and that we have a tree which looks at g-c_revision instead of revision, and displays: e8c582246a18 Gm-c Gm-c Gm-c Gm-c Gm-i Gm-i Gm-i Gm-i Gm-i Gm-i Gm-i Gm-i Gm-i Gm-i Gm-i Gm-i Gm-i Gm-i Gm-i Gm-i Gm-i until there's another push to g-c so that m-c/m-i runs run against a new g-c rev, so that we show the jobs which run against inbound both on inbound's tree and on a separate tree-that's-no-particular-tree.
Comment 3•11 years ago
|
||
FWIW, I think g-c *should* follow the actual Jetpack model, and run on-push on other trees hidden, where people concerned with it can use &noignore=1 to chase down bustages, with a g-c tree that's triggered by gaia pushes to see the results of their own pushes, until the annoyance of that wears them down and they finally land their code in m-c so they can become unhidden.
Assignee | ||
Updated•11 years ago
|
Assignee: nobody → armenzg
Status: NEW → ASSIGNED
Priority: -- → P1
Assignee | ||
Comment 4•11 years ago
|
||
The ideal solution (and the easiest is) if gaia was part of mozilla-central or if we controlled it with a manifest. I want to be explicit about this. I can take 2 approaches to this. OPTION A - Create a project branch per "gaia" branch (mozilla/project_branches.py) -- which checkouts one of the associated repos -- but it polls the repo branches -- simple patches -- we build a gecko repo ("repo_path") but we checkout a gaia repo ("poll_repo")(different than all other project branches) OPTION B - Create a b2g project per "gaia" branch (mozilla/b2g_config.py) -- same as above -- more complicated patches Unlike Jetpack, we have to generate a build for gaia to be tested. Hence, the approach of triggering 4 Jetpack jobs (for 4 different branches) for every Jetpack change does not translate well to Gaia (in fact, I don't think that model was correct even for Jetpack). I'm leaning towards option A since we really only want extra granularity for an external project.
Assignee | ||
Comment 5•11 years ago
|
||
Attachment #712562 -
Flags: feedback?(aki)
Assignee | ||
Comment 6•11 years ago
|
||
Attachment #712563 -
Flags: feedback?(aki)
Assignee | ||
Comment 7•11 years ago
|
||
Attachment #712564 -
Flags: feedback?(aki)
Comment 8•11 years ago
|
||
Comment on attachment 712562 [details] [diff] [review] build config changes I think this works.
Attachment #712562 -
Flags: feedback?(aki) → feedback+
Updated•11 years ago
|
Attachment #712563 -
Flags: feedback?(aki) → feedback+
Comment 9•11 years ago
|
||
Comment on attachment 712564 [details] [diff] [review] custom changes A comment that actually applies to the first two patches: we may want to call this branch gaia-central if we plan on having to do something similar with other branches.
Attachment #712564 -
Flags: feedback?(aki) → feedback+
Assignee | ||
Comment 10•11 years ago
|
||
I would like to be more explicit; I would like to associate them to the branch from where we build (rather than the gaia branch): * gaia-mozilla-central (for master) * gaia-mozilla-b2g18 (for v1-train) * gaia-mozilla-b2g18_v1_0_0 (for v1.0.0) * gaia-mozilla-b2g18_v1_0_1 (for v1.0.0) Would this work for you? The patches are ready to be reviewed and landed (I will adjust the tree names upon landing).
Assignee | ||
Updated•11 years ago
|
Attachment #712562 -
Flags: review?(aki)
Assignee | ||
Updated•11 years ago
|
Attachment #712563 -
Flags: review?(aki)
Assignee | ||
Updated•11 years ago
|
Attachment #712564 -
Flags: review?(aki)
Assignee | ||
Comment 11•11 years ago
|
||
I was waiting for success on staging before requesting reviews.
Comment 12•11 years ago
|
||
(In reply to Armen Zambrano G. [:armenzg] from comment #10) > I would like to be more explicit; I would like to associate them to the > branch from where we build (rather than the gaia branch): > * gaia-mozilla-central (for master) > * gaia-mozilla-b2g18 (for v1-train) > * gaia-mozilla-b2g18_v1_0_0 (for v1.0.0) > * gaia-mozilla-b2g18_v1_0_1 (for v1.0.0) > > Would this work for you? > > The patches are ready to be reviewed and landed (I will adjust the tree > names upon landing). This might be more confusing, since that means people need to switch pages, say, mid-1.0.1 when that moves from mozilla-b2g18 to mozilla-b2g18_v1_0_1. A gaia-v1_0_1 page would follow the gecko branch switch and people could keep their bookmarks/tabs the same. I guess it depends who this page is oriented towards.
Updated•11 years ago
|
Attachment #712562 -
Flags: review?(aki) → review+
Updated•11 years ago
|
Attachment #712563 -
Flags: review?(aki) → review+
Updated•11 years ago
|
Attachment #712564 -
Flags: review?(aki) → review+
Assignee | ||
Comment 13•11 years ago
|
||
Comment on attachment 712564 [details] [diff] [review] custom changes b800f14c86f6
Attachment #712564 -
Flags: checked-in+
Assignee | ||
Updated•11 years ago
|
Attachment #712563 -
Flags: checked-in+
Assignee | ||
Updated•11 years ago
|
Attachment #712562 -
Flags: checked-in+
Assignee | ||
Comment 14•11 years ago
|
||
http://hg.mozilla.org/build/buildbot-configs/rev/b72d58a259ce We discussed on IRC that the naming would be something like this (since the pages will be oriented to Gaia devs): * gaia-mozilla-master * gaia-mozilla-v1-train * gaia-mozilla-v1.0.0 * gaia-mozilla-v1.0.1 Are periods bad? Waiting for reconfiguration.
Comment 15•11 years ago
|
||
(In reply to Armen Zambrano G. [:armenzg] from comment #14) > http://hg.mozilla.org/build/buildbot-configs/rev/b72d58a259ce > > We discussed on IRC that the naming would be something like this (since the > pages will be oriented to Gaia devs): > * gaia-mozilla-master > * gaia-mozilla-v1-train > * gaia-mozilla-v1.0.0 > * gaia-mozilla-v1.0.1 I think the email names were better; not sure what the "mozilla-" is for. > Are periods bad? I think this is a question for tbpl. I don't care strongly if it's periods or underscores.
Assignee | ||
Comment 16•11 years ago
|
||
(In reply to Aki Sasaki [:aki] from comment #15) > (In reply to Armen Zambrano G. [:armenzg] from comment #14) > > http://hg.mozilla.org/build/buildbot-configs/rev/b72d58a259ce > > > > We discussed on IRC that the naming would be something like this (since the > > pages will be oriented to Gaia devs): > > * gaia-mozilla-master > > * gaia-mozilla-v1-train > > * gaia-mozilla-v1.0.0 > > * gaia-mozilla-v1.0.1 > > I think the email names were better; not sure what the "mozilla-" is for. > Bah. It should have been: * gaia-master * gaia-v1-train * gaia-v1.0.0 * gaia-v1.0.1 > > Are periods bad? > > I think this is a question for tbpl. I don't care strongly if it's periods > or underscores. edmorley, would periods on the name give us any trouble?
Assignee | ||
Comment 17•11 years ago
|
||
Heads-up and discussion about naming: https://groups.google.com/d/topic/mozilla.dev.gaia/NabKrlkf1ow/discussion
Assignee | ||
Comment 18•11 years ago
|
||
The last three branches will not be added to buildbot on this pass. I want to make sure that I iron out everything first on Gaia-Master.
Attachment #713071 -
Flags: review?(emorley)
Comment 19•11 years ago
|
||
The currently landed patches are in production.
Assignee | ||
Comment 20•11 years ago
|
||
I see it running on buildbot: http://buildbot-master49.srv.releng.scl3.mozilla.com:8001/builders/b2g_gaia-master_panda_dep http://buildbot-master29.build.scl1.mozilla.com:8201/builders/b2g_panda%20gaia-master%20opt%20test%20gaia-ui-test
Comment 21•11 years ago
|
||
Comment on attachment 713071 [details] [diff] [review] add Gaia trees Review of attachment 713071 [details] [diff] [review]: ----------------------------------------------------------------- Regarding the periods in the name - I don't think it should be a problem - though only one way to tell for sure :-) I do prefer them to underscores however (particularly in the TBPL UI), so would like to give it a shot. prettierName is only required if a name other than the key name is desired in the UI, so these are redundant. r=me without the prettierName entries.
Attachment #713071 -
Flags: review?(emorley) → review+
Assignee | ||
Comment 22•11 years ago
|
||
Comment on attachment 713071 [details] [diff] [review] add Gaia trees Removed prettierNames: https://hg.mozilla.org/webtools/tbpl/rev/b8a5c7b21be9
Attachment #713071 -
Flags: checked-in+
Comment 23•11 years ago
|
||
Comment on attachment 713071 [details] [diff] [review] add Gaia trees I'd like to test this with the new Chief setup (bug 827473), so if possible, hold off filing a push to production bug for now please :-)
Assignee | ||
Comment 24•11 years ago
|
||
I can wait. I see it running on https://tbpl-dev.allizom.org/?tree=Gaia-Master I have to fix an issue I did not notice before. The changesets of gaia-central do not exist on mozilla-central - duh! On staging I probably used force builds and I thought the build due to a poll where pointing to my users/armenzg_mozilla.com/mozilla-central and not having the changesets.
Assignee | ||
Comment 25•11 years ago
|
||
What do you think of this? I want to call the script with --checkout-revision default (it has to be added) which would update the build to default rather than the gaia revision which shows up as a buildbot property from the poling.
Attachment #713520 -
Flags: feedback?(aki)
Comment 26•11 years ago
|
||
Comment on attachment 713520 [details] [diff] [review] --checkout-revision default This will probably work. What happens in terms of the buildbot properties? We need to make sure that the actual revision (not default) shows up for tbpl.
Attachment #713520 -
Flags: feedback?(aki) → feedback+
Assignee | ||
Comment 27•11 years ago
|
||
Assignee | ||
Comment 28•11 years ago
|
||
Assignee | ||
Updated•11 years ago
|
Attachment #713545 -
Flags: review?(aki)
Assignee | ||
Comment 29•11 years ago
|
||
Comment on attachment 713520 [details] [diff] [review] --checkout-revision default This cannot land before the other change makes it to mozharness/production. I got these properties: gaia_revision 40c3b26ea181 gecko_revision 081cf5b0121e which matches: http://hg.mozilla.org/integration/gaia-central/rev/40c3b26ea181 http://hg.mozilla.org/mozilla-central/rev/081cf5b0121e
Attachment #713520 -
Flags: review?(aki)
Assignee | ||
Updated•11 years ago
|
Attachment #713536 -
Attachment is obsolete: true
Comment 30•11 years ago
|
||
Comment on attachment 713545 [details] [diff] [review] add --checkout-revision to b2g_build.py (v2) I'm not sure I like this because: 1) behavior that doesn't intuitively belong to a --checkout-revision gets attached to --checkout-revision (namely, the buildbot_property('revision') gets set differently if it's set or not), and 2) we base other information (sources xml, try upload_path) on the 'revision' buildbot property. Could you: 1) add a comment about why checkout_revision works this way in checkout_gecko (right above your additions), 2) set 'gecko_revision' to rev in the else block as well, 3) change the update_source_manifest() self.buildbot_properties['revision']'s to self.buildbot_properties['gecko_revision'] ? I can clarify if the above doesn't make sense.
Attachment #713545 -
Flags: review?(aki) → review-
Updated•11 years ago
|
Attachment #713520 -
Flags: review?(aki) → review+
Assignee | ||
Comment 31•11 years ago
|
||
Like this?
Attachment #713545 -
Attachment is obsolete: true
Attachment #714023 -
Flags: review?(aki)
Comment 32•11 years ago
|
||
Comment on attachment 714023 [details] [diff] [review] add --checkout-revision to b2g_build.py (v3) >+ if self.config.has_key("checkout_revision"): >+ rev = self.vcs_checkout(repo=repo, dest=dirs['src'], revision=self.config["checkout_revision"]) >+ self.set_buildbot_property('revision', self.config["checkout_revision"], write_to_file=True) >+ self.set_buildbot_property('gecko_revision', rev, write_to_file=True) >+ else: >+ rev = self.vcs_checkout(repo=repo, dest=dirs['src'], revision=self.query_revision()) >+ self.set_buildbot_property('revision', rev, write_to_file=True) >+ self.set_buildbot_property('gecko_revision', rev, write_to_file=True) You could move the gecko_revision setting outside the if/else block. I thought of this after my previous comment. > self.warning("Couldn't find git_base_url in manifest; using %s" % git_base_url) > new_sources = [] > > for line in sources.splitlines(): > new_sources.append(line) > if 'Gonk specific things' in line: > new_sources.append(' <!-- Mercurial-Information: <remote fetch="http://hg.mozilla.org/" name="hgmozillaorg"> -->') > new_sources.append(' <!-- Mercurial-Information: <project name="%s" path="gecko" remote="hgmozillaorg" revision="%s"/> -->' % >- (self.buildbot_config['properties']['repo_path'], self.buildbot_properties['revision'])) >+ (self.buildbot_config['properties']['repo_path'], self.buildbot_properties['gecko_revision'])) > new_sources.append(' <!-- Mercurial-Information: <project name="%s" path="gaia" remote="hgmozillaorg" revision="%s"/> -->' % > (gaia_config['repo'].replace('http://hg.mozilla.org/', ''), self.buildbot_properties['gaia_revision'])) > > if self.query_do_translate_hg_to_git(): > url = manifest_config['translate_base_url'] > gecko_git = self.query_translated_revision(url, 'gecko', self.buildbot_properties['revision']) Don't forget this line ^^ (s,revision,gecko_revision,) r=me with those fixes.
Attachment #714023 -
Flags: review?(aki) → review+
Assignee | ||
Comment 33•11 years ago
|
||
upload calls query_revision() and we would have been using the gaia revision. Even though that is for try. I don't think I need this patch. Would you want me to use the previous patch?
Attachment #714076 -
Flags: review?(aki)
Comment 34•11 years ago
|
||
Comment on attachment 714076 [details] [diff] [review] add --checkout-revision to b2g_build.py (v4) Yeah, I think in this case a) try isn't using gaia revisions b) if it were, we might want to upload using the gaia revision So I saw this before I commented earlier, and decided to keep try uploads with query_revision(). Previous patch is fine. We can use this if we do turn on gaia revisions on a try-like branch, and we want to upload using the gecko revision.
Attachment #714076 -
Flags: review?(aki)
Assignee | ||
Comment 35•11 years ago
|
||
Comment on attachment 714023 [details] [diff] [review] add --checkout-revision to b2g_build.py (v3) If everything goes well I will land the buildbotcustom change and merge mozharness (default->production). Landed: http://hg.mozilla.org/build/mozharness/rev/48e022f5de22 Also testing on Ash one more time to be sure: https://tbpl.mozilla.org/?tree=Ash&rev=a373379d43cf http://hg.mozilla.org/users/asasaki_mozilla.com/ash-mozharness/rev/48e022f5de22
Attachment #714023 -
Flags: checked-in+
Assignee | ||
Updated•11 years ago
|
Attachment #714076 -
Attachment is obsolete: true
Assignee | ||
Comment 36•11 years ago
|
||
Comment on attachment 713520 [details] [diff] [review] --checkout-revision default http://hg.mozilla.org/build/buildbot-configs/rev/6f1d4fc40a79
Attachment #713520 -
Flags: checked-in+
Assignee | ||
Comment 37•11 years ago
|
||
Comment on attachment 714023 [details] [diff] [review] add --checkout-revision to b2g_build.py (v3) Merged to production: http://hg.mozilla.org/build/mozharness/rev/428c474cc43e Reconfig will come soon.
Assignee | ||
Comment 38•11 years ago
|
||
This is done: https://tbpl-dev.allizom.org/?tree=Gaia-Master
Status: ASSIGNED → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Updated•11 years ago
|
Product: mozilla.org → Release Engineering
You need to log in
before you can comment on or make changes to this bug.
Description
•