Closed Bug 894980 Opened 7 years ago Closed 7 years ago

Return the try server the ability to test custom talos

Categories

(Release Engineering :: General, defect)

defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: armenzg, Assigned: jyeo)

References

Details

Attachments

(2 files, 1 obsolete file)

Before, when the a-team wanted to test a new talos version, they would upload a new talos.zip to their people account, make a talos.json change to point to it and push it to try.

We have now lost that ability because we checkout the repo rather than download a talos.zip.
https://tbpl.mozilla.org/?tree=Try&jobname=talos&rev=1af89a37b3a5
https://tbpl.mozilla.org/php/getParsedLog.php?id=25372036&tree=Try&full=1#error0

I assume that talos.zip-url will be deprecated.
Why do we have a talos.tar.gz to download for? Is it so we can grab python packages?

As a workflow I think it would be ideal if we could grab new talos versions through a revision and a user repo.
Open to suggestions.

     "talos.zip": {
         "url": "http://build.mozilla.org/talos/zips/talos.a11542b55a70.zip",
         "path": ""
     },
     "global": {
         "talos_url": "http://puppetagain.pub.build.mozilla.org/data/python/packages/talos-7d4ae29b7d1f.tar.gz",
        "talos_revision": "7d4ae29b7d1f"
     },
talos.zip-url will be deprecated.
global-talos_url was used before you made your change.

Maybe we can specify the talos_repo in addition to talos_revision?  Then people could create a user hg repo and push their changes to it, and point at that talos_repo and talos_revision.
OS: Mac OS X → All
Hardware: x86 → All
a custom talos user repo + revision would be just fine.
I created another function to query_talos_revision for completeness sake. I hope that's okay.
Attachment #777275 - Flags: review?(aki)
Attached patch Talos json with repo string (obsolete) — Splinter Review
Attachment #777289 - Flags: review?(aki)
Comment on attachment 777289 [details] [diff] [review]
Talos json with repo string

We should keep talos.zip until we're fully rolled out; buildbot talos uses it.

So once mozharness talos is rolled out to all mozilla-central level branches, we can land a "remove talos.zip from talos.json" patch to mozilla-inbound, but we can't roll that out to aurora, beta, release, etc. until those branches are also using mozharness talos.

r=me for the talos_url removal and talos_repo addition.
Attachment #777289 - Flags: review?(aki) → review+
Comment on attachment 777275 [details] [diff] [review]
Use talos repo from talos json

I would guess that self.has_cloned_talos isn't useful anymore, since that's the default (and only?) workflow.  I'm ok leaving that alone for now, though, unless you want to do cleanup.

This looks good, thanks!
Attachment #777275 - Flags: review?(aki) → review+
Attachment #777289 - Attachment is obsolete: true
Attachment #777275 - Flags: checked-in+
Comment on attachment 777361 [details] [diff] [review]
s/talos_url/talos_repo/ pointing to hg.m.o/build/talos

Carrying forward the r+ from obsolete patch.
I will land the patch once m-i is open.

I will merge mozharness tomorrow morning.
Attachment #777361 - Flags: review+
Merged to production.
Testing on try one last time to be sure:
https://tbpl.mozilla.org/?tree=Try&rev=b0b2e2e2cff8

-        "talos_url": "http://puppetagain.pub.build.mozilla.org/data/python/pack
-        "talos_revision": "7d4ae29b7d1f"
+        "talos_repo": " ssh://hg.mozilla.org/users/armenzg_mozilla.com/talos",
+        "talos_revision": "c41ec0e321c6"
Trying now with http:// instead of ssh://
https://tbpl.mozilla.org/?tree=Try&rev=7e4be7297875
(In reply to Armen Zambrano G. [:armenzg] (Release Enginerring) from comment #10)
> ssh://hg.mozilla.org/users/armenzg_mozilla.com/talos",

I'm not sure if ssh works here.

I pushed another try commit here: https://tbpl.mozilla.org/?tree=Try&rev=6b5220dc4064

-        "talos_url": "http://puppetagain.pub.build.mozilla.org/data/python/packages/talos-7d4ae29b7d1f.tar.gz",
-        "talos_revision": "7d4ae29b7d1f"
+        "talos_repo": "http://hg.mozilla.org/users/armenzg_mozilla.com/talos",
+        "talos_revision": "c41ec0e321c6"
(In reply to Jason Yeo [:jyeo] from comment #12)
> (In reply to Armen Zambrano G. [:armenzg] (Release Enginerring) from comment
> #10)
> > ssh://hg.mozilla.org/users/armenzg_mozilla.com/talos",
> 
> I'm not sure if ssh works here.
> 
> I pushed another try commit here:
> https://tbpl.mozilla.org/?tree=Try&rev=6b5220dc4064
> 
> -        "talos_url":
> "http://puppetagain.pub.build.mozilla.org/data/python/packages/talos-
> 7d4ae29b7d1f.tar.gz",
> -        "talos_revision": "7d4ae29b7d1f"
> +        "talos_repo":
> "http://hg.mozilla.org/users/armenzg_mozilla.com/talos",
> +        "talos_revision": "c41ec0e321c6"

Comment 11 is a push without ssh:// (that is what I was meaning).

You can probably cancel your push from comment 12 if you believe the push from comment 11 is sufficient (please cancel if so to save resources).
(In reply to Armen Zambrano G. [:armenzg] (Release Enginerring) from comment #10)
> Testing on try one last time to be sure:
> https://tbpl.mozilla.org/?tree=Try&rev=b0b2e2e2cff8
> 
> -        "talos_url":
> "http://puppetagain.pub.build.mozilla.org/data/python/pack
> -        "talos_revision": "7d4ae29b7d1f"
> +        "talos_repo": "
> ssh://hg.mozilla.org/users/armenzg_mozilla.com/talos",

Noting that you have a space before the ssh:// here, so it's probably not a valid test.
jmaher, I believe you have a syntax error on your talos repo
http://hg.mozilla.org/users/jmaher_mozilla.com/tpain/rev/443f419ec164

09:44:04     INFO -    File "C:\slave\test\build\venv\lib\site-packages\talos\run_tests.py", line 289
09:44:04 CRITICAL -      except talosError, e:
09:44:04     INFO -           ^
09:44:04     INFO -  IndentationError: expected an indented block
(In reply to Aki Sasaki [:aki] from comment #14)
> you have a space before the ssh:// here, so it's probably not a
> valid test.

I tried removing the space and executing it on the slave. Here's what I got:

[cltbld@talos-linux64-ix-053.test.releng.scl3.mozilla.com ~]$ hg --config ui.merge=internal:merge clone "ssh://hg.mozilla.org/users/armenzg_mozilla.com/talos" /builds/slave/talos-slave/test/build/talos_repo
remote: Permission denied (publickey).
abort: no suitable response from remote hg!
[cltbld@talos-linux64-ix-053.test.releng.scl3.mozilla.com ~]$

Let's just stick to read-only http addresses for the repo.
(In reply to Jason Yeo [:jyeo] from comment #12)
> I pushed another try commit here:
> https://tbpl.mozilla.org/?tree=Try&rev=6b5220dc4064

10:00:56     INFO - Cloning http://hg.mozilla.org/users/armenzg_mozilla.com/talos to /builds/slave/talos-slave/test/build/talos_repo.

We're good to go:
https://hg.mozilla.org/integration/mozilla-inbound/rev/bb0b437e60c7
https://hg.mozilla.org/mozilla-central/rev/bb0b437e60c7
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Wee!
No longer depends on: 895721
We have the ability back.
We will have to fix bug 895721 before we deploy bug 713055 across the board.
Product: mozilla.org → Release Engineering
Component: General Automation → General
You need to log in before you can comment on or make changes to this bug.