Last Comment Bug 1250904 - When using one-click loaner do not run the tests; only set them up
: When using one-click loaner do not run the tests; only set them up
Status: RESOLVED FIXED
[frontend]
:
Product: Taskcluster
Classification: Other
Component: Task Configuration (show other bugs)
: unspecified
: Unspecified Unspecified
-- normal
: ---
Assigned To: Andrew Halberstadt [:ahal]
:
:
Mentors:
Depends on: 1253421 1272110 1273926 1274645 1275308 1276216 1279040
Blocks: 1262260 1262605
  Show dependency treegraph
 
Reported: 2016-02-24 07:24 PST by Armen Zambrano - Back on March 27th [:armenzg] (EDT/UTC-4)
Modified: 2016-06-08 13:28 PDT (History)
6 users (show)
See Also:
QA Whiteboard:
Iteration: ---
Points: ---

MozReview Requests
Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:
Show discarded requests

Attachments
one_click_loaner.diff (6.21 KB, patch)
2016-02-26 14:36 PST, Armen Zambrano - Back on March 27th [:armenzg] (EDT/UTC-4)
jopsen: feedback+
Details | Diff | Splinter Review
improve the SSH interactive session (7.61 KB, patch)
2016-03-03 12:33 PST, Armen Zambrano - Back on March 27th [:armenzg] (EDT/UTC-4)
no flags Details | Diff | Splinter Review
MozReview Request: Bug 1250904 - Re-organize dot-files in testing/docker/desktop-test, r?armenzg (58 bytes, text/x-review-board-request)
2016-05-20 14:41 PDT, Andrew Halberstadt [:ahal]
armenzg: review+
Details | Review
MozReview Request: Bug 1250904 - Add an hgrc for 'worker' user in testing/docker/desktop-test, r?armenzg (58 bytes, text/x-review-board-request)
2016-05-31 07:04 PDT, Andrew Halberstadt [:ahal]
armenzg: review+
Details | Review
MozReview Request: Bug 1250904 - Don't run mozharness if TASKCLUSTER_INTERACTIVE is set, r?armenzg (58 bytes, text/x-review-board-request)
2016-05-31 07:04 PDT, Andrew Halberstadt [:ahal]
armenzg: review+
Details | Review
MozReview Request: Bug 1250904 - Start a wizard upon initiating an interactive shell, r?armenzg (58 bytes, text/x-review-board-request)
2016-05-31 07:04 PDT, Andrew Halberstadt [:ahal]
armenzg: review+
Details | Review

Description User image Armen Zambrano - Back on March 27th [:armenzg] (EDT/UTC-4) 2016-02-24 07:24:59 PST
We can just append --no-run-tests which gets passed down to Mozharness.
We can do this inside of bin/test.sh when TASKCLUSTER_INTERACTIVE=true is set.

jonasfj: how can I add messages when a developer connects to the shell like interface?
Comment 1 User image Jonas Finnemann Jensen (:jonasfj) 2016-02-26 12:37:08 PST
Create a file:
  /etc/taskcluster-motd
On the docker image.


It's a hack for now:
https://github.com/taskcluster/taskcluster-tools/blob/7ae2f24954e3cd58f945bdd0b8956d3b137c20e1/shell/app.jsx#L27

Maybe we'll formalize it later :)
Comment 2 User image Armen Zambrano - Back on March 27th [:armenzg] (EDT/UTC-4) 2016-02-26 13:51:41 PST
Thanks jonasfj. I will look into it.
Comment 3 User image Armen Zambrano - Back on March 27th [:armenzg] (EDT/UTC-4) 2016-02-26 13:54:30 PST
jonasfj: where can I change what command a one-click-loaner task executes?
Comment 4 User image Armen Zambrano - Back on March 27th [:armenzg] (EDT/UTC-4) 2016-02-26 13:56:21 PST
I forgot. nvm.
Comment 5 User image Armen Zambrano - Back on March 27th [:armenzg] (EDT/UTC-4) 2016-02-26 14:22:59 PST
https://treeherder.mozilla.org/#/jobs?repo=try&revision=9234f9082b75
Comment 6 User image Armen Zambrano - Back on March 27th [:armenzg] (EDT/UTC-4) 2016-02-26 14:36:19 PST
Created attachment 8724259 [details] [diff] [review]
one_click_loaner.diff

What do you think about this?

Perhaps I can wait for a file to be created insted of telling the developer to go and look at the output of the task being executed.

It would reduce the developer having know much about it.
I could also tell them where the Mozharness log is if they want to inspect what was accomplished during the setup time.
Comment 7 User image Armen Zambrano - Back on March 27th [:armenzg] (EDT/UTC-4) 2016-02-26 14:37:53 PST
FYI this still needs to be tested (still building) but I want your initial feedback.
Comment 8 User image Jonas Finnemann Jensen (:jonasfj) 2016-02-29 13:03:29 PST
Comment on attachment 8724259 [details] [diff] [review]
one_click_loaner.diff

Review of attachment 8724259 [details] [diff] [review]:
-----------------------------------------------------------------

This looks like a really good start.

Question: Will this not download the binary that the task would otherwise download?
I think it should. Then there is no need to specify --binary-path, right?

---
Also with respect to the waiting thing I propose:
We add /etc/taskcluster-interactive-shell.sh as something like:
    #!/bin/bash
    while [ ! -f /bin/run-tests.sh ]; do echo "Waiting..."; sleep 1; done;
    echo "Ready, you can now run tests with: run-tests.sh"; 
    exec bash -li

Then not only is the "run-tests.sh" binary a simple command to run the tests.
But it is also the file we wait for in the interactive session.

Note: In the /etc/taskcluster-interactive-shell.sh script I intentionally
didn't use "#!/bin/bash -e" as a user pressing ctrl+c to interrupt the loop
should still get the bash shell from "exec bash -li"

---
Background: https://github.com/taskcluster/taskcluster-tools/blob/7ae2f24954e3cd58f945bdd0b8956d3b137c20e1/shell/app.jsx#L37
Basically, if there is a file: "/etc/taskcluster-interactive-shell.sh" in the container,
then the interactive shell will launch that script instead of bash.
So you can customize the shell the user is given, or even as done above wait for a resource
before letting the user play with the shell.

::: testing/taskcluster/scripts/tester/test-linux.sh
@@ +122,5 @@
>  for cfg in $MOZHARNESS_CONFIG; do
>    config_cmds="${config_cmds} --config-file ${cfg}"
>  done
>  
> +if [ ${RUN_LOCALLY} == true ]; then

I have very limited bash skills. But isn't the norm to use defined vs undefined ?
Or empty string == false, and non-empty string == true.
When doing flags with env vars.

It's just I suspect someone will do RUN_LOCALLY=TRUE,
and similarly for TASKCLUSTER_INTERACTIVE=TRUE :)

Anyways, like I said I have limited bash skills.

@@ +144,5 @@
> +  echo "You're reading this because you were waiting for your interactive session to be ready to be used."
> +  echo "Congratulations! We have set up everything for you (downloaded binaries, tests and set up the Python virtualenv)."
> +  echo ""
> +  echo "You can now run the tests by typing this command on your interactive session:"
> +  echo " sudo -E -u worker  python2.7 $WORKSPACE/${MOZHARNESS_SCRIPT} ${config_cmds} '${@}' --run-tests"

I propose:
echo -e '#!/bin/bash\nsudo -E -u worker python 2.7 $WOR....' > /bin/run-tests.sh
chmod +x /bin/run-tests.sh

Then tell people to just run: "run-tests.sh" :)

@@ +147,5 @@
> +  echo "You can now run the tests by typing this command on your interactive session:"
> +  echo " sudo -E -u worker  python2.7 $WORKSPACE/${MOZHARNESS_SCRIPT} ${config_cmds} '${@}' --run-tests"
> +  echo "You will probably need to append --binary-path to make it work."
> +  echo "If there are any issues please file a bug or ping us on #ateam"
> +  echo "https://bugzilla.mozilla.org/enter_bug.cgi?product=Testing&component=TaskCluster"

You could append this message to "/etc/taskcluster-motd" with: echo "..." >> /etc/taskcluster-motd
Possibly in addition to printing it here :)

@@ +149,5 @@
> +  echo "You will probably need to append --binary-path to make it work."
> +  echo "If there are any issues please file a bug or ping us on #ateam"
> +  echo "https://bugzilla.mozilla.org/enter_bug.cgi?product=Testing&component=TaskCluster"
> +
> +  sleep 3600 # This will keep this shell script from exiting and ending their interactive session

Decent hack for now.. We it's probably going to waste a lot of CPU cycles on nothing :)
But let's improve this later. I suspect we might have to do something smart worker-side.
Comment 9 User image Armen Zambrano - Back on March 27th [:armenzg] (EDT/UTC-4) 2016-02-29 13:27:04 PST
(In reply to Jonas Finnemann Jensen (:jonasfj) from comment #8)
> Question: Will this not download the binary that the task would otherwise
> download?
> I think it should. Then there is no need to specify --binary-path, right?
> 
When running Mozharness with the action --run-tests, it forgets that in the previous action it extracted the package.
This could be improved in the MH side.

I will look into the rest of the comments tomorrow.

Thanks Jonas!
Comment 10 User image Armen Zambrano - Back on March 27th [:armenzg] (EDT/UTC-4) 2016-03-03 12:32:07 PST
https://treeherder.mozilla.org/#/jobs?repo=try&revision=e8f614e9ae41
Comment 11 User image Armen Zambrano - Back on March 27th [:armenzg] (EDT/UTC-4) 2016-03-03 12:33:57 PST
Created attachment 8726375 [details] [diff] [review]
improve the SSH interactive session

I've taken most of your suggestions into account.
I'm testing this on try.
Comment 12 User image Armen Zambrano - Back on March 27th [:armenzg] (EDT/UTC-4) 2016-03-04 12:46:28 PST
https://treeherder.mozilla.org/#/jobs?repo=try&revision=d01b694129f1
Comment 13 User image Armen Zambrano - Back on March 27th [:armenzg] (EDT/UTC-4) 2016-03-04 13:07:59 PST
https://treeherder.mozilla.org/#/jobs?repo=try&revision=1798165017c1
Comment 14 User image Armen Zambrano - Back on March 27th [:armenzg] (EDT/UTC-4) 2016-04-25 08:07:48 PDT
Not working on this at the moment.
Comment 15 User image Andrew Halberstadt [:ahal] 2016-05-10 07:24:59 PDT
Going to take a shot at finishing this. I'm hitting a couple problems when connecting with one-click loaner (haven't tried setting interactive in the task yet). Unfortunately the turn around time to test changes is pretty slow on this (as it depends on try jobs finishing and then connecting interactively).
Comment 16 User image Andrew Halberstadt [:ahal] 2016-05-20 14:41:15 PDT
Created attachment 8755029 [details]
MozReview Request: Bug 1250904 - Re-organize dot-files in testing/docker/desktop-test, r?armenzg

Review commit: https://reviewboard.mozilla.org/r/54342/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/54342/
Comment 17 User image Andrew Halberstadt [:ahal] 2016-05-31 07:04:03 PDT
Created attachment 8758277 [details]
MozReview Request: Bug 1250904 - Add an hgrc for 'worker' user in testing/docker/desktop-test, r?armenzg

This creates some defaults, enables common built-in extensions and sets up
Ubuntu's CA file for checking host certificates.

Review commit: https://reviewboard.mozilla.org/r/56564/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/56564/
Comment 18 User image Andrew Halberstadt [:ahal] 2016-05-31 07:04:03 PDT
Created attachment 8758278 [details]
MozReview Request: Bug 1250904 - Don't run mozharness if TASKCLUSTER_INTERACTIVE is set, r?armenzg

Rather than guess what the developer means to do when they checkout an interactive tester,
we'll prompt them from the interactive shell. This means we need to bypass the mozharness
script initially. To make it easy to run again later, we save the exact command to a
'run-mozharness' binary.

Review commit: https://reviewboard.mozilla.org/r/56566/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/56566/
Comment 19 User image Andrew Halberstadt [:ahal] 2016-05-31 07:04:03 PDT
Created attachment 8758279 [details]
MozReview Request: Bug 1250904 - Start a wizard upon initiating an interactive shell, r?armenzg

To make things as easy as possible, run a little wizard to help developers
choose what to do. This way they don't need to memorize any commands or read
any wiki pages.

Review commit: https://reviewboard.mozilla.org/r/56568/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/56568/
Comment 20 User image Andrew Halberstadt [:ahal] 2016-05-31 07:04:03 PDT
Comment on attachment 8755029 [details]
MozReview Request: Bug 1250904 - Re-organize dot-files in testing/docker/desktop-test, r?armenzg

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/54342/diff/1-2/
Comment 21 User image Andrew Halberstadt [:ahal] 2016-05-31 07:05:59 PDT
Here's a try run:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=0074fe7857d99c615430982474da7a7c02c7aa5b

You can test this out by doing the one-click loaner for a job. Mochitests hit a pulseaudio permissions issue that I'll tackle in a separate bug, but other suites like xpcshell work well.
Comment 22 User image Armen Zambrano - Back on March 27th [:armenzg] (EDT/UTC-4) 2016-06-02 05:44:36 PDT
https://reviewboard.mozilla.org/r/54340/#review53942

This looks great.

How long does it take to clone the repo?
If too long, are there any optimizations that the TaskCluster team or gps have?

There might be on the image a tool the TaskCluster team created called tvcs. I don't know if that adds any value but thought of mentioning it.
Comment 23 User image Armen Zambrano - Back on March 27th [:armenzg] (EDT/UTC-4) 2016-06-02 05:49:08 PDT
Comment on attachment 8755029 [details]
MozReview Request: Bug 1250904 - Re-organize dot-files in testing/docker/desktop-test, r?armenzg

https://reviewboard.mozilla.org/r/54342/#review53946
Comment 24 User image Armen Zambrano - Back on March 27th [:armenzg] (EDT/UTC-4) 2016-06-02 05:50:09 PDT
Comment on attachment 8758277 [details]
MozReview Request: Bug 1250904 - Add an hgrc for 'worker' user in testing/docker/desktop-test, r?armenzg

https://reviewboard.mozilla.org/r/56564/#review53948
Comment 25 User image Armen Zambrano - Back on March 27th [:armenzg] (EDT/UTC-4) 2016-06-02 05:50:22 PDT
Comment on attachment 8758278 [details]
MozReview Request: Bug 1250904 - Don't run mozharness if TASKCLUSTER_INTERACTIVE is set, r?armenzg

https://reviewboard.mozilla.org/r/56566/#review53950
Comment 26 User image Armen Zambrano - Back on March 27th [:armenzg] (EDT/UTC-4) 2016-06-02 05:50:34 PDT
Comment on attachment 8758279 [details]
MozReview Request: Bug 1250904 - Start a wizard upon initiating an interactive shell, r?armenzg

https://reviewboard.mozilla.org/r/56568/#review53952
Comment 27 User image Andrew Halberstadt [:ahal] 2016-06-02 08:49:32 PDT
https://reviewboard.mozilla.org/r/54340/#review53942

That's a good idea, I think it actually does do some optimizations. If you don't mind I'd like to tackle that in a follow-up though, it's a bit lower priority than things like getting mach working.
Comment 28 User image Andrew Halberstadt [:ahal] 2016-06-02 08:52:27 PDT
Here's a previous up-to-date try run I did to prove it doesn't break existing non-interactive jobs:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=0074fe7857d99c615430982474da7a7c02c7aa5b
Comment 29 User image Andrew Halberstadt [:ahal] 2016-06-02 08:54:33 PDT
Actually, I should probably wait for bug 1276216 to get fixed, otherwise all taskcluster jobs will turn orange until the next push (which is harmless in terms of tree health, but will confuse the sheriffs).
Comment 30 User image Pulsebot 2016-06-07 12:05:40 PDT
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/aa3e7ff72452
Re-organize dot-files in testing/docker/desktop-test, r=armenzg
https://hg.mozilla.org/integration/mozilla-inbound/rev/7b3ad5d93fbd
Add an hgrc for 'worker' user in testing/docker/desktop-test, r=armenzg
https://hg.mozilla.org/integration/mozilla-inbound/rev/12b3b927435f
Don't run mozharness if TASKCLUSTER_INTERACTIVE is set, r=armenzg
https://hg.mozilla.org/integration/mozilla-inbound/rev/fb0507f42e74
Start a wizard upon initiating an interactive shell, r=armenzg

Note You need to log in before you can comment on or make changes to this bug.