717889 - Cron <root@dm-symbolpush01> /mnt/netapp/breakpad/cleanup-breakpad-symbols.sh > /dev/null

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Justin Dow [:jabba]]

Every day, we get a cronmail like this. Not sure what is going on there.

Error removing file:  /mnt/netapp/breakpad/symbols_tbrd/thunderbird/1FC1EADF08A5886D54699F09FBD2CD800/thunderbird.sym
Error removing file:  /mnt/netapp/breakpad/symbols_tbrd/thunderbird/1FC1EADF08A5886D54699F09FBD2CD800/thunderbird.dSYM.tar.bz2
Error removing file:  /mnt/netapp/breakpad/symbols_tbrd/thunderbird/7713F7FB35838010491C4E1C79741A140/thunderbird.sym
Error removing file:  /mnt/netapp/breakpad/symbols_tbrd/thunderbird/7713F7FB35838010491C4E1C79741A140/thunderbird.dSYM.tar.bz2
Error removing file:  /mnt/netapp/breakpad/symbols_tbrd/thunderbird/6BA12D374225E79CF4346DCB950F7F430/thunderbird.sym
Error removing file:  /mnt/netapp/breakpad/symbols_tbrd/thunderbird/6BA12D374225E79CF4346DCB950F7F430/thunderbird.dSYM.tar.bz2
Error removing file:  /mnt/netapp/breakpad/symbols_tbrd/thunderbird/5BBB231A6676D4E5D05ABC29A350A3000/thunderbird.sym
Error removing file:  /mnt/netapp/breakpad/symbols_tbrd/thunderbird/5BBB231A6676D4E5D05ABC29A350A3000/thunderbird.dSYM.tar.bz2

Comment 1

[(not currently active) Ted Mielczarek]

Do those files exist? If so, then perhaps it's a permissions problem. If not, then maybe we just need to fix the cleanup script to not error on trying to remove files that don't exist.

Comment 2

[Justin Dow [:jabba]]

Today's cron has this list:

Error removing file:  /mnt/netapp/breakpad/symbols_tbrd/thunderbird/82EA5F56CE7682D6AA8C6ACEA6120DB80/thunderbird.sym
Error removing file:  /mnt/netapp/breakpad/symbols_tbrd/thunderbird/82EA5F56CE7682D6AA8C6ACEA6120DB80/thunderbird.dSYM.tar.bz2
Error removing file:  /mnt/netapp/breakpad/symbols_tbrd/thunderbird/47C0CFAACC4BA2CD32981C0AFEF92DBE0/thunderbird.sym
Error removing file:  /mnt/netapp/breakpad/symbols_tbrd/thunderbird/47C0CFAACC4BA2CD32981C0AFEF92DBE0/thunderbird.dSYM.tar.bz2

And indeed it appears that 82EA5F56CE7682D6AA8C6ACEA6120DB80 and 47C0CFAACC4BA2CD32981C0AFEF92DBE0 directories do not exist. Any idea why it would try to delete things that don't exist?

Comment 3

[(not currently active) Ted Mielczarek]

It's possible for binaries from different builds to wind up with the same identifiers. The script should handle that properly, but it's possible it screwed up. I don't have time right now to look into this, but if you wanted to make it skip trying to delete files that don't exist, you'd make a change here:
http://hg.mozilla.org/build/tools/file/tip/buildfarm/breakpad/cleanup-breakpad-symbols.py#l97

Comment 4

[Rob Tucker [:rtucker]]

Attachment: Patch applied to cleanup-breakpad-symbols.diff

Check to confirm that the file exists, if not continue through the loop.
If the file exists and cannot be deleted, throw exception as before.

Comment 5

[Rob Tucker [:rtucker]]

Comment on attachment 591541 [details] [diff] [review]
Patch applied to cleanup-breakpad-symbols.diff

This:
if os.path.isfile(path):
Should be:
if os.path.isfile(f):

Comment 6

[Rob Tucker [:rtucker]]

Attachment: Final Patch Revision

Comment 7

[Rob Tucker [:rtucker]]

Ted,
I attached the final patch revision. Please let me know when you've applied it and commited. I'll then set up dm-symbolpush01 to use the version from the repository.

Comment 8

[Justin Dow [:jabba]]

Are we good here? Just waiting for Ted to commit the patch?

Comment 9

[(not currently active) Ted Mielczarek]

http://hg.mozilla.org/build/tools/rev/200974ed3156

Sorry, I lost track of this. I just pushed your change.