Closed Bug 977218 Opened 10 years ago Closed 10 years ago

Crontabber app to unpack the uploaded symbols archives

Categories

(Socorro :: Backend, task)

Product:
Socorro
Component:
Backend
Platform:
x86
macOS
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: peterbe, Assigned: peterbe)

References

Details

(Whiteboard: [config changes]) 

https://bugzilla.mozilla.org/show_bug.cgi?id=977216 takes care of making it possible to upload symbols archives (aka. zip files with symbols). These files will be stored on NFS mount available to the crontabber app that then unpacks it to another NFS mount according to certain "rules" and validation.
This bug :ted, is where I'm going to need your help on nailing the specifics about that unpacking.
Assignee: nobody → peterbe
Currently the SSH-based uploading from the build slaves is done using this script:
http://mxr.mozilla.org/mozilla-central/source/toolkit/crashreporter/tools/upload_symbols.sh#37

You can ignore POST_SYMBOL_UPLOAD_CMD for now (we may want to implement something later, but that's a lot easier with the webapp architecture). So essentially this just boils down to:
cd ${SYMBOL_SERVER_PATH};
unzip -n '$archive';

SYMBOL_SERVER_PATH is set per-user, so you'll need some configuration for that. It's just a base path (/mnt/netapp/breakpad/ on symbolpush.mozilla.org) + a per-user path (symbols_adobe, symbols_geeksphone, etc).

The `unzip -n` is because symbol archives may contain duplicates from previous archives (this is normal and okay). We account for this in the cleanup script.
That path part very much confuses me. At the moment, in the cron app we're faced with a directory full of zip files. The intention is that it loops over each file, one at a time, unpacks it and if deletes the archive file. 

How do you envision the "need some configuration for that" part?
That path part very much confuses me. At the moment, in the cron app we're faced with a directory full of zip files. The intention is that it loops over each file, one at a time, unpacks it and if deletes the archive file. 

How do you envision the "need some configuration for that" part?
That path part very much confuses me. At the moment, in the cron app we're faced with a directory full of zip files. The intention is that it loops over each file, one at a time, unpacks it and if deletes the archive file. 

How do you envision the "need some configuration for that" part?
That path part very much confuses me. At the moment, in the cron app we're faced with a directory full of zip files. The intention is that it loops over each file, one at a time, unpacks it and if deletes the archive file. 

How do you envision the "need some configuration for that" part?
That path part very much confuses me. At the moment, in the cron app we're faced with a directory full of zip files. The intention is that it loops over each file, one at a time, unpacks it and if deletes the archive file. 

How do you envision the "need some configuration for that" part?
That path part very much confuses me. At the moment, in the cron app we're faced with a directory full of zip files. The intention is that it loops over each file, one at a time, unpacks it and if deletes the archive file. 

How do you envision the "need some configuration for that" part?
That path part very much confuses me. At the moment, in the cron app we're faced with a directory full of zip files. The intention is that it loops over each file, one at a time, unpacks it and if deletes the archive file. 

How do you envision the "need some configuration for that" part?
That path part very much confuses me. At the moment, in the cron app we're faced with a directory full of zip files. The intention is that it loops over each file, one at a time, unpacks it and if deletes the archive file. 

How do you envision the "need some configuration for that" part?
We need configuration somewhere that maps users to subdirectories on the symbol server, so something like

geeksphonesymbols -> symbols_geeksphone
"users"? We don't have users in the cron app. If it needs to depend on the users who do the upload we'd need to step a couple of steps back to the upload part and do some sort of recording there. 

Actually, the way I had envisioned it in the cron job is that all I have is a directory of unpacked files. That part is disconnected from the database and how things were uploaded originally. Not good enough?
Benjamin,

We're currently trying to figure out exactly where to unpack the zip files that get uploaded with HTTP POST through the webapp. 
It's a crontabber app that'll do the unpacking. In front of it it simply has a directory of .zip files with no particular naming scheme and that needs to be placed in /mnt/netapp/breakpad/. 

Is it OK if we just unpack all of these into /mnt/netapp/breakpad/symbols_upload or /mnt/netapp/breakpad/symbols_thirdparty (besides, if you know a better name, let us know)? 


Ted also mentioned a future we'll want more specific mapping from users (e.g. bob@adobe.com) to specific directories (e.g. symbols_adobe) but that's the future.
Flags: needinfo?(benjamin)
In the past we've used separate directories so that unix permissions would handle access control. Since that is no longer an issue, I think we should put all the thirdparty symbols into one big symbols_thirdparty bucket.

Later if we use this same system for Firefox product symbols, we should segregate those a little.
Flags: needinfo?(benjamin)
Oh also please remember that any new directory will need to be added to the processor config.
I think we should be OK as long as we can track every unpacked file back to when it was uploaded by what user (and for what product it is). I'm pretty sure we need vastly different retention/lifetime for, say Flash symbols than, say FxOS releases.
Blocks: 983850
Regarding the [config changes], once this lands we need to point the SymbolsUnpackCronApp to know about two directories:

1. source_directory 
  The NFS mount talked about in https://bugzilla.mozilla.org/show_bug.cgi?id=980082

2. destination_directory 
  The NFS mount talked about in https://bugzilla.mozilla.org/show_bug.cgi?id=983850

These two directories need to be added to our crontabber.ini under SymbolsUnpackCronApp.
Whiteboard: [config changes]
Commit pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/d18f2081ee5f10332f02f6aad5ad1f7bff7a11bf
Merge pull request #1959 from peterbe/bug-977218-crontabber-app-to-unpack-the-uploaded-symbols-archives

fixes bug 977218 - Crontabber app to unpack the uploaded symbols archives
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Target Milestone: --- → 79
As a temporary config change, whilst we wait for the NFS mounts to be set up. 

Add this to our crontabber.ini:
https://gist.github.com/peterbe/9647190
Also, because the crontabber app's job is to unzip any .zip, .tgz, .tar or .tar.gz files in that directory I suggest we don't use `/tmp/` but instead some other empty directory. Instead we should `mkdir /tmp/empty-directory/
However, it'll error if `/tmp/empty-directory/` doesn't exist so if we ever restart the server it'll critical.
This code went out, but the config change has not been made yet. I would much rather disable the job if it isn't useful to run than have it inexplicably reading some empty directory.
(In reply to Chris Lonnen :lonnen from comment #20)
> This code went out, but the config change has not been made yet. I would
> much rather disable the job if it isn't useful to run than have it
> inexplicably reading some empty directory.

That means a code push to disable it and another code push to enable it again. 
But if you prefer that, feel free.
Re-enabling the job so we can start testing this:
https://github.com/mozilla/socorro/pull/2045
Commits pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/3cbe32c6f767c88ba432ba496146125097134c6a
bug 977218 - enable symbolsunpack cron app

https://github.com/mozilla/socorro/commit/3edffc9058fbc034aee0d327e7c887d6831da90c
Merge pull request #2045 from peterbe/bug-977218-enable-symbolsunpack-cron-app

bug 977218 - enable symbolsunpack cron app
You need to log in before you can comment on or make changes to this bug.