1512644 - need gcp proxy for minidump-stackwalker

Status: RESOLVED FIXED

(Socorro :: Infra, task, P1)

Description

[Will Kahn-Greene [:willkg] ET needinfo? me]

minidump_stackwalker (aka mdsw) accesses the private symbols bucket for private symbols. This requires permissions and auth headers to do, but mdsw can't do that. So Miles wrote a proxy that mdsw talks to that adds the appropriate headers to access the private symbols bucket.

When Tecken moves to GCP, we're going to also need a proxy for accessing the private symbols in GCP.

This bug covers that work.

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

At some point, me or someone will rewrite mdsw in rust and theoretically we can adjust it to add auth headers then. However, I don't know when that's going to happen and it's likely it'll happen after Tecken moves to GCP. I think we're going to need to write the proxy as an interim measure.

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

This is the one we currently have written in Go for AWS: https://github.com/mozilla-services/aws-signing-proxy

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

I'm grabbing this for now and going to use it as a reason to get set up with GCP. If I can't get it done, I'll pass it off.

Also, I'm making this a P1 since it blocks Tecken.

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

Last week I spent some time looking at aws-signing-proxy. I was thinking I'd write this in Python but aws-signing-proxy is in Go and there are some interesting things about it. After talking with Miles and Brian a bit, I decided to learn Go and write an equivalent.

That sort of worked out.

There is no GCP equivalent to the aws-signing thing that the aws-signing-proxy uses to proxy all AWS services. Instead, I had to make this proxy GCS-specific and use the Go library for google cloud storage stuff.

So what it does is listen for incoming HTTP GET requests, pulls the object off the URL path, fetches that object from GCS using specified credentials and bucket, and returns an HTTP response with that content-type and payload. That works for arbitrary files I tested it with. I'll test it with a private symbols bucket when Tecken stage is up and has a test private symbols bucket.

Code so far is here:

https://github.com/willkg/gcs-signing-proxy

It's all tested by hand. That seems ok for now.

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

I moved it to mozilla-serivces: https://github.com/mozilla-services/gcs-signing-proxy

It's mostly done. If there are any outstanding issues, they can be tracked in that issue tracker. Marking this as FIXED.