1381770 - Miscellaneous tooltool-download.sh changes

Status: RESOLVED FIXED

(Firefox Build System :: General, enhancement)

Comment 1

[Mike Hommey [:glandium]]

Attachment: Bug 1381770 - Use $WORKSPACE instead of $HOME/workspace in tooltool-download.sh.

All the current users of tooltool-download.sh set $WORKSPACE. This will
allow to reuse the script on different types of workers, that don't have
$WORKSPACE set to $HOME/workspace, but still have the source in
$WORKSPACE/build/src.

Review commit: https://reviewboard.mozilla.org/r/158232/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/158232/

Comment 2

[Mike Hommey [:glandium]]

Attachment: Bug 1381770 - Enable multiple download retries in tooltool-download.sh.

The scripts that use tooltool-download.sh don't run regularly, but when
they do, they might hit some download problems (the relengapi proxy
tends to be rather unreliable for some reason), and in that case, it
would be better to retry a few times, like other job types, rather than
fail directly.

Review commit: https://reviewboard.mozilla.org/r/158234/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/158234/

Comment 3

[Mike Hommey [:glandium]]

Attachment: Bug 1381770 - In tooltool-download.sh, only use the relengapi proxy when it's configured.

Review commit: https://reviewboard.mozilla.org/r/158236/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/158236/

Comment 4

[Mike Hommey [:glandium]]

Attachment: Bug 1381770 - In tooltool-download.sh, automatically setup the relengapi authentication file.

Review commit: https://reviewboard.mozilla.org/r/158238/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/158238/

Comment 5

[Mike Hommey [:glandium]]

Attachment: Bug 1381770 - Don't use the relengapi proxy for Linux toolchain jobs.

As the relengapi proxy doesn't appear to be very reliable, let's just
use the public releng API, since we only need public artifacts anyways
(no need for the relengapi token, which Linux TC workers don't have)

Review commit: https://reviewboard.mozilla.org/r/158240/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/158240/

Comment 6

[Michael Shal [:mshal]]

Comment on attachment 8887387 [details]
Bug 1381770 - Use $WORKSPACE instead of $HOME/workspace in tooltool-download.sh.

https://reviewboard.mozilla.org/r/158232/#review164252

Comment 7

[Michael Shal [:mshal]]

Comment on attachment 8887388 [details]
Bug 1381770 - Enable multiple download retries in tooltool-download.sh.

https://reviewboard.mozilla.org/r/158234/#review164254

Comment 8

[Michael Shal [:mshal]]

Comment on attachment 8887389 [details]
Bug 1381770 - In tooltool-download.sh, only use the relengapi proxy when it's configured.

https://reviewboard.mozilla.org/r/158236/#review164258

::: taskcluster/scripts/misc/tooltool-download.sh:7
(Diff revision 1)
>  
>  cd $WORKSPACE/build/src
>  
> +TOOLTOOL_DL_FLAGS=
> +
> +if [ -n "$RELENGAPI_PORT" ]; then

Where does RELENGAPI_PORT get set? Would it make sense to pass in a TOOLTOOL_URL instead and set the --tooltool-url flag based on that?

Comment 9

[Michael Shal [:mshal]]

Comment on attachment 8887390 [details]
Bug 1381770 - In tooltool-download.sh, automatically setup the relengapi authentication file.

https://reviewboard.mozilla.org/r/158238/#review164260

Comment 10

[Michael Shal [:mshal]]

Comment on attachment 8887391 [details]
Bug 1381770 - Don't use the relengapi proxy for Linux toolchain jobs.

https://reviewboard.mozilla.org/r/158240/#review164262

Comment 11

[Mike Hommey [:glandium]]

(In reply to Michael Shal [:mshal] from comment #8)
> Comment on attachment 8887389 [details]
> Bug 1381770 - In tooltool-download.sh, only use the relengapi proxy when
> it's configured.
> 
> https://reviewboard.mozilla.org/r/158236/#review164258
> 
> ::: taskcluster/scripts/misc/tooltool-download.sh:7
> (Diff revision 1)
> >  
> >  cd $WORKSPACE/build/src
> >  
> > +TOOLTOOL_DL_FLAGS=
> > +
> > +if [ -n "$RELENGAPI_PORT" ]; then
> 
> Where does RELENGAPI_PORT get set? Would it make sense to pass in a
> TOOLTOOL_URL instead and set the --tooltool-url flag based on that?

It's set $somewhere when relengapi-proxy is enabled.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5dbf26e7d92d
Use $WORKSPACE instead of $HOME/workspace in tooltool-download.sh. r=mshal
https://hg.mozilla.org/integration/autoland/rev/688c2808b32c
Enable multiple download retries in tooltool-download.sh. r=mshal
https://hg.mozilla.org/integration/autoland/rev/ba2a0e45421b
In tooltool-download.sh, only use the relengapi proxy when it's configured. r=mshal
https://hg.mozilla.org/integration/autoland/rev/66bb1f3a35f9
In tooltool-download.sh, automatically setup the relengapi authentication file. r=mshal
https://hg.mozilla.org/integration/autoland/rev/a2ae5410cfd3
Don't use the relengapi proxy for Linux toolchain jobs. r=mshal

Comment 13

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/5dbf26e7d92d
https://hg.mozilla.org/mozilla-central/rev/688c2808b32c
https://hg.mozilla.org/mozilla-central/rev/ba2a0e45421b
https://hg.mozilla.org/mozilla-central/rev/66bb1f3a35f9
https://hg.mozilla.org/mozilla-central/rev/a2ae5410cfd3