1003927 - Allow developers to download files from tooltool

Status: RESOLVED FIXED

(Infrastructure & Operations :: RelOps: General, task)

Description

[Armen [:armenzg]]

We currently have artifacts in tooltool that are not easily recreated.
Developers in those cases would ask us to share it with them.
To do so, we copy the file from pvt/build/sha512 into pub/ to allow them to download it and then delete it.

Is there a way we could allow them to access the files with LDAP credentials?

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

Do those files really need to be private?  If not, they should probably be on the pub version (in which case talk to jlund).

Otherwise, yes, in principle we could add LDAP auth for private files, although I'll still need some help from Jordan since we'd need to change hostnames and use HTTPS.

Which LDAP gropus would be allowed?

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

Sorry, s/jlund/simone/g.

Comment 3

[Armen [:armenzg]]

They cannot be public.

I don't know which LDAP group. Can we choose the people that have a high commit access? (not just try access).

Comment 4

[Dustin J. Mitchell [:dustin] (he/him)]

We could potentially set up a new LDAP group for non-moco people who have access, and then additionally allow access for all moco.

Can you lay out exactly what you'd like this to look at?  Basically, a list of tooltool repos (URLs), who or what has access to them, and what files are located there?

Comment 5

[Armen [:armenzg]]

For now I'm only interested on developers being to download any file from here:
http://tooltool.pvt.build.mozilla.org/build/sha512

We might want to consider other locations from where we download private bits like the talos pagesets.

For now let's focus on tooltool since most proprietary contents that we use in our build and test jobs are located.

BTW, it is great that we have the pub location if I need to put something there temporarily for a developer to download. That was great! (aka we can work around it; this is not urgent but good to have)

All Moco employees is a good start.

Thank you Dustin.

Comment 6

[Armen [:armenzg]]

Hi Dustin,
What is needed for this to happen?
I assume sometime next month I will loose my Build VPN access as a Release Engineer and I would like to figure out how to make this work.

I want to make it easier for developers and the a-team to mimic locally what we run in production.

Comment 7

[Dustin J. Mitchell [:dustin] (he/him)]

I think that what we would do is add URLs like
  https://secure.pub.build.mozilla.org/tooltool/pvt/build/sha512
that would have the access restrictions described above, and behind the scenes access the same files.

I'm not sure how to make that discoverable, though..

Comment 8

[Dustin J. Mitchell [:dustin] (he/him)]

Et voilà:
  https://secure.pub.build.mozilla.org/tooltool/pvt/build/

basically all of tooltool.pub is mirrored at secure/tooltool/pub, and similarly for pvt.  Right now, it's available to releng, relops, and sheriffs, but we can change that on request.