1250904 - When using one-click loaner do not run the tests; only set them up

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[Armen [:armenzg]]

We can just append --no-run-tests which gets passed down to Mozharness.
We can do this inside of bin/test.sh when TASKCLUSTER_INTERACTIVE=true is set.

jonasfj: how can I add messages when a developer connects to the shell like interface?

Comment 1

[Jonas Finnemann Jensen (:jonasfj)]

Create a file:
  /etc/taskcluster-motd
On the docker image.


It's a hack for now:
https://github.com/taskcluster/taskcluster-tools/blob/7ae2f24954e3cd58f945bdd0b8956d3b137c20e1/shell/app.jsx#L27

Maybe we'll formalize it later :)

Comment 2

[Armen [:armenzg]]

Thanks jonasfj. I will look into it.

Comment 3

[Armen [:armenzg]]

jonasfj: where can I change what command a one-click-loaner task executes?

Comment 4

[Armen [:armenzg]]

I forgot. nvm.

Comment 5

[Armen [:armenzg]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=9234f9082b75

Comment 6

[Armen [:armenzg]]

Attachment: one_click_loaner.diff

What do you think about this?

Perhaps I can wait for a file to be created insted of telling the developer to go and look at the output of the task being executed.

It would reduce the developer having know much about it.
I could also tell them where the Mozharness log is if they want to inspect what was accomplished during the setup time.

Comment 7

[Armen [:armenzg]]

FYI this still needs to be tested (still building) but I want your initial feedback.

Comment 8

[Jonas Finnemann Jensen (:jonasfj)]

Comment on attachment 8724259 [details] [diff] [review]
one_click_loaner.diff

Review of attachment 8724259 [details] [diff] [review]:
-----------------------------------------------------------------

This looks like a really good start.

Question: Will this not download the binary that the task would otherwise download?
I think it should. Then there is no need to specify --binary-path, right?

---
Also with respect to the waiting thing I propose:
We add /etc/taskcluster-interactive-shell.sh as something like:
    #!/bin/bash
    while [ ! -f /bin/run-tests.sh ]; do echo "Waiting..."; sleep 1; done;
    echo "Ready, you can now run tests with: run-tests.sh"; 
    exec bash -li

Then not only is the "run-tests.sh" binary a simple command to run the tests.
But it is also the file we wait for in the interactive session.

Note: In the /etc/taskcluster-interactive-shell.sh script I intentionally
didn't use "#!/bin/bash -e" as a user pressing ctrl+c to interrupt the loop
should still get the bash shell from "exec bash -li"

---
Background: https://github.com/taskcluster/taskcluster-tools/blob/7ae2f24954e3cd58f945bdd0b8956d3b137c20e1/shell/app.jsx#L37
Basically, if there is a file: "/etc/taskcluster-interactive-shell.sh" in the container,
then the interactive shell will launch that script instead of bash.
So you can customize the shell the user is given, or even as done above wait for a resource
before letting the user play with the shell.

::: testing/taskcluster/scripts/tester/test-linux.sh
@@ +122,5 @@
>  for cfg in $MOZHARNESS_CONFIG; do
>    config_cmds="${config_cmds} --config-file ${cfg}"
>  done
>  
> +if [ ${RUN_LOCALLY} == true ]; then

I have very limited bash skills. But isn't the norm to use defined vs undefined ?
Or empty string == false, and non-empty string == true.
When doing flags with env vars.

It's just I suspect someone will do RUN_LOCALLY=TRUE,
and similarly for TASKCLUSTER_INTERACTIVE=TRUE :)

Anyways, like I said I have limited bash skills.

@@ +144,5 @@
> +  echo "You're reading this because you were waiting for your interactive session to be ready to be used."
> +  echo "Congratulations! We have set up everything for you (downloaded binaries, tests and set up the Python virtualenv)."
> +  echo ""
> +  echo "You can now run the tests by typing this command on your interactive session:"
> +  echo " sudo -E -u worker  python2.7 $WORKSPACE/${MOZHARNESS_SCRIPT} ${config_cmds} '${@}' --run-tests"

I propose:
echo -e '#!/bin/bash\nsudo -E -u worker python 2.7 $WOR....' > /bin/run-tests.sh
chmod +x /bin/run-tests.sh

Then tell people to just run: "run-tests.sh" :)

@@ +147,5 @@
> +  echo "You can now run the tests by typing this command on your interactive session:"
> +  echo " sudo -E -u worker  python2.7 $WORKSPACE/${MOZHARNESS_SCRIPT} ${config_cmds} '${@}' --run-tests"
> +  echo "You will probably need to append --binary-path to make it work."
> +  echo "If there are any issues please file a bug or ping us on #ateam"
> +  echo "https://bugzilla.mozilla.org/enter_bug.cgi?product=Testing&component=TaskCluster"

You could append this message to "/etc/taskcluster-motd" with: echo "..." >> /etc/taskcluster-motd
Possibly in addition to printing it here :)

@@ +149,5 @@
> +  echo "You will probably need to append --binary-path to make it work."
> +  echo "If there are any issues please file a bug or ping us on #ateam"
> +  echo "https://bugzilla.mozilla.org/enter_bug.cgi?product=Testing&component=TaskCluster"
> +
> +  sleep 3600 # This will keep this shell script from exiting and ending their interactive session

Decent hack for now.. We it's probably going to waste a lot of CPU cycles on nothing :)
But let's improve this later. I suspect we might have to do something smart worker-side.

Comment 9

[Armen [:armenzg]]

(In reply to Jonas Finnemann Jensen (:jonasfj) from comment #8)
> Question: Will this not download the binary that the task would otherwise
> download?
> I think it should. Then there is no need to specify --binary-path, right?
> 
When running Mozharness with the action --run-tests, it forgets that in the previous action it extracted the package.
This could be improved in the MH side.

I will look into the rest of the comments tomorrow.

Thanks Jonas!

Comment 10

[Armen [:armenzg]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=e8f614e9ae41

Comment 11

[Armen [:armenzg]]

Attachment: improve the SSH interactive session

I've taken most of your suggestions into account.
I'm testing this on try.

Comment 12

[Armen [:armenzg]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=d01b694129f1

Comment 13

[Armen [:armenzg]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=1798165017c1

Comment 14

[Armen [:armenzg]]

Not working on this at the moment.

Comment 15

[Andrew Halberstadt [:ahal]]

Going to take a shot at finishing this. I'm hitting a couple problems when connecting with one-click loaner (haven't tried setting interactive in the task yet). Unfortunately the turn around time to test changes is pretty slow on this (as it depends on try jobs finishing and then connecting interactively).

Comment 16

[Andrew Halberstadt [:ahal]]

Attachment: MozReview Request: Bug 1250904 - Re-organize dot-files in testing/docker/desktop-test, r?armenzg

Review commit: https://reviewboard.mozilla.org/r/54342/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/54342/

Comment 17

[Andrew Halberstadt [:ahal]]

Attachment: MozReview Request: Bug 1250904 - Add an hgrc for 'worker' user in testing/docker/desktop-test, r?armenzg

This creates some defaults, enables common built-in extensions and sets up
Ubuntu's CA file for checking host certificates.

Review commit: https://reviewboard.mozilla.org/r/56564/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/56564/

Comment 18

[Andrew Halberstadt [:ahal]]

Attachment: MozReview Request: Bug 1250904 - Don't run mozharness if TASKCLUSTER_INTERACTIVE is set, r?armenzg

Rather than guess what the developer means to do when they checkout an interactive tester,
we'll prompt them from the interactive shell. This means we need to bypass the mozharness
script initially. To make it easy to run again later, we save the exact command to a
'run-mozharness' binary.

Review commit: https://reviewboard.mozilla.org/r/56566/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/56566/

Comment 19

[Andrew Halberstadt [:ahal]]

Attachment: MozReview Request: Bug 1250904 - Start a wizard upon initiating an interactive shell, r?armenzg

To make things as easy as possible, run a little wizard to help developers
choose what to do. This way they don't need to memorize any commands or read
any wiki pages.

Review commit: https://reviewboard.mozilla.org/r/56568/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/56568/

Comment 20

[Andrew Halberstadt [:ahal]]

Comment on attachment 8755029 [details]
MozReview Request: Bug 1250904 - Re-organize dot-files in testing/docker/desktop-test, r?armenzg

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/54342/diff/1-2/

Comment 21

[Andrew Halberstadt [:ahal]]

Here's a try run:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=0074fe7857d99c615430982474da7a7c02c7aa5b

You can test this out by doing the one-click loaner for a job. Mochitests hit a pulseaudio permissions issue that I'll tackle in a separate bug, but other suites like xpcshell work well.

Comment 22

[Armen [:armenzg]]

https://reviewboard.mozilla.org/r/54340/#review53942

This looks great.

How long does it take to clone the repo?
If too long, are there any optimizations that the TaskCluster team or gps have?

There might be on the image a tool the TaskCluster team created called tvcs. I don't know if that adds any value but thought of mentioning it.

Comment 23

[Armen [:armenzg]]

Comment on attachment 8755029 [details]
MozReview Request: Bug 1250904 - Re-organize dot-files in testing/docker/desktop-test, r?armenzg

https://reviewboard.mozilla.org/r/54342/#review53946

Comment 24

[Armen [:armenzg]]

Comment on attachment 8758277 [details]
MozReview Request: Bug 1250904 - Add an hgrc for 'worker' user in testing/docker/desktop-test, r?armenzg

https://reviewboard.mozilla.org/r/56564/#review53948

Comment 25

[Armen [:armenzg]]

Comment on attachment 8758278 [details]
MozReview Request: Bug 1250904 - Don't run mozharness if TASKCLUSTER_INTERACTIVE is set, r?armenzg

https://reviewboard.mozilla.org/r/56566/#review53950

Comment 26

[Armen [:armenzg]]

Comment on attachment 8758279 [details]
MozReview Request: Bug 1250904 - Start a wizard upon initiating an interactive shell, r?armenzg

https://reviewboard.mozilla.org/r/56568/#review53952

Comment 27

[Andrew Halberstadt [:ahal]]

https://reviewboard.mozilla.org/r/54340/#review53942

That's a good idea, I think it actually does do some optimizations. If you don't mind I'd like to tackle that in a follow-up though, it's a bit lower priority than things like getting mach working.

Comment 28

[Andrew Halberstadt [:ahal]]

Here's a previous up-to-date try run I did to prove it doesn't break existing non-interactive jobs:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=0074fe7857d99c615430982474da7a7c02c7aa5b

Comment 29

[Andrew Halberstadt [:ahal]]

Actually, I should probably wait for bug 1276216 to get fixed, otherwise all taskcluster jobs will turn orange until the next push (which is harmless in terms of tree health, but will confuse the sheriffs).

Comment 30

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/aa3e7ff72452
Re-organize dot-files in testing/docker/desktop-test, r=armenzg
https://hg.mozilla.org/integration/mozilla-inbound/rev/7b3ad5d93fbd
Add an hgrc for 'worker' user in testing/docker/desktop-test, r=armenzg
https://hg.mozilla.org/integration/mozilla-inbound/rev/12b3b927435f
Don't run mozharness if TASKCLUSTER_INTERACTIVE is set, r=armenzg
https://hg.mozilla.org/integration/mozilla-inbound/rev/fb0507f42e74
Start a wizard upon initiating an interactive shell, r=armenzg

Comment 31

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/aa3e7ff72452
https://hg.mozilla.org/mozilla-central/rev/7b3ad5d93fbd
https://hg.mozilla.org/mozilla-central/rev/12b3b927435f
https://hg.mozilla.org/mozilla-central/rev/fb0507f42e74