1333165 - Allow DLL and PDB download for privileged users

Status: RESOLVED WONTFIX

(Tecken :: Upload, task)

Description

[Benjamin Smedberg]

Adobe uploads Flash player symbols and binaries, and in particular the DLLs and PDB files associated with Flash releases. It would be very helpful for certain privileged users to be able to download these PDB files for local debugging purposes.

Possible options, from most-great to minimally-sufficient:

* provide a symbol server URL that we can use with an API token to automatically download symbols
* provide a link from the "modules" tab of an individual report to download the binary and PDB for that module, if it's available:
** for example, https://crash-stats.mozilla.com/report/index/a81d373c-5962-47be-9c6e-7621d2170123#tab-modules "NPSWF32_24_0_0_194.dll" links to the public symbols, but this is a private symbol.
* provide a separate API where you submit a debug name and debug ID and get back the matching binary and PDB file. e.g. for the crash above, I'd submit some combination of "NPSWF32_24_0_0_194.dll | 24.0.0.194 | 4EB3710233544687A9723893CD5DBC491 | NPSWF32.pdb" and be able to download the DLL and PDB file.

Comment 1

[(not currently active) Ted Mielczarek]

(In reply to Benjamin Smedberg [:bsmedberg] from comment #0)
> * provide a link from the "modules" tab of an individual report to download
> the binary and PDB for that module, if it's available:
> ** for example,
> https://crash-stats.mozilla.com/report/index/a81d373c-5962-47be-9c6e-
> 7621d2170123#tab-modules "NPSWF32_24_0_0_194.dll" links to the public
> symbols, but this is a private symbol.

Yeah, I totally punted on this when I implemented it:
https://github.com/luser/rust-makecab/commit/d2bc6797648b7a834782714a55d339d2fd4e58c8

Since we only have two symbol buckets (public and private) the links are only ever wrong for locally-cached symbols that were originally from the private bucket. Even if we fixed that to work it still wouldn't be useful, since the private bucket doesn't have public access on by design.

Comment 2

[Peter Bengtsson [:peterbe]]

I've been thinking about some big overhauling changes to accessing symbols. It's going to be similar to the symbols upload as part of Socorrro but more dedicated and easier to add advanced features. 

At the moment the the public symbols store is exposed via its S3 URL. 
We could make it so that the private (or some super private ones like "private-mcafee" hypothetically) could be masked by a web service that checks API tokens. It'd then instead of exposing that private S3 bucket, we'd "channel" them. 
That would definitely solve your "most-great" feature request of "provide a symbol server URL that we can use with an API token to automatically download symbols"

Comment 3

[Peter Bengtsson [:peterbe]]

Tecken now already has the ability to proxy private symbols but obviously we don't want to add a private bucket to the list of buckets that you can download. However, Tecken also has API tokens and permissions and stuff so we could simply add some code in the runtime like this (pseudo'esque code):

  def download_symbol(request, symbol, debug_id, filename):
      symbols_sources = settings.SYMBOL_URLS
      if request.user.has_perm('download_private_symbols'):
          symbol_sources.extend(settings.PRIVATE_SYMBOL_URLS)
      found = find(symbol_sources, symbol, debug_id, filename)
      if found:
          return redirect(found.presigned_url())
      else:
          raise Http404()

The above is largely loud thoughts but my question to you Ted and Benjamin; Is it enough to just have a single permission (technically users belong to groups, and groups have the permissions) that enables users to get access to all private symbol sources/buckets? Or, does it need more fine-tuning, like a permission/group *per* private source/bucket?

In particular, isn't it true that we might soon have some Intel symbols (MacAfee) that are even more sensitive than things like the Adobe Flash ones. After all, I can make 1 permission group per private symbols bucket.

Comment 4

[Benjamin Smedberg]

I don't know anything about Intel symbols. I would have said that a single permission is fine, but I don't know the details.

Comment 5

[Peter Bengtsson [:peterbe]]

(In reply to Benjamin Smedberg [:bsmedberg] from comment #4)
> I don't know anything about Intel symbols. I would have said that a single
> permission is fine, but I don't know the details.

The context is this beast/mess of a bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1293722

Ted's comment was: "I don't believe we have any way of storing them that would both keep them private and allow us to make use of them without making copies."

I.e. even if we make it reachable via Tecken, for special snowflake superusers, it would fundamentally involves the bytes being copied from S3 down to that client. We don't have a server that can engage these symbols and keep the symbolication private. Socorro's symbolication is always quite public. We could do it with Tecken's Symbolication (although today only built for crash pings) because Tecken doesn't store the symbolication result anywhere. 

How about we keep it simple and ignore this  Intel McAfee DLL stuff entirely for now and we can revisit this logic in the future.

Comment 6

[(not currently active) Ted Mielczarek]

I don't think we should complicate things beyond public/private. You can have a single permission for access to private symbols. We should only allow partners to upload symbols if they are comfortable with us using their symbols in this manner.

Comment 7

[Peter Bengtsson [:peterbe]]

Tecken is live now. It doesn't have the ability to download symbols by private buckets yet but it would be quite trivial to add. 

The list of buckets it proxies symbol downloads for is hardcoded but if the client could pass a API Token header, we can turn that into a user and we can match the user against the setting we have that specifies which bucket these people should download from. 

Coding it wouldn't be hard but we'd need to do it gently so we get it right from a security perspective. 
Coupla questions:

* How needed is this feature still? I.e. clients being able to DOWNLOAD private symbols.

* Do we want to advance the security model first? At the moment we maintain a config variable that basically maps email regexes to bucket names for uploads. We *could* move this to the superuser webapp in symbols.mozilla.org so superusers can direct specifically who can download from what specific S3 buckets.

Comment 8

[(not currently active) Ted Mielczarek]

(In reply to Peter Bengtsson [:peterbe] from comment #7)
> * How needed is this feature still? I.e. clients being able to DOWNLOAD
> private symbols.

This is probably not high up the priority list, but I don't know who (if anyone) is stuck dealing with Flash plugin crsahes these days.

Comment 9

[Will Kahn-Greene [:willkg] ET needinfo? me]

Moving Tecken service bugs to Tecken component.

Comment 10

[Will Kahn-Greene [:willkg] ET needinfo? me]

Moving bugs to Tecken product.

Comment 11

[Will Kahn-Greene [:willkg] ET needinfo? me]

I'm going to close this out. If someone is interested in it, we can reopen.