Closed Bug 1513497 Opened 6 years ago Closed 6 years ago

Trying to upload symbols for Flashplayer with latest dump_syms, getting error "Unrecognized file pattern."

Categories

(Tecken :: General, task)

Product:
Tecken
Component:
General
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
VERIFIED FIXED

People

(Reporter: viveknegi1, Assigned: peterbe)

Details

Attachments

(3 files)

Screen Shot 2019-01-10 at 10.05.11 AM.png
458.84 KB, image/png
Details
Screen Shot 2019-01-11 at 12.45.20 PM.png
182.98 KB, image/png
Details
Screen Shot 2019-01-14 at 9.28.43 AM.png
137.02 KB, image/png
Details
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.80 Safari/537.36 Steps to reproduce: I am vivek from Adobe. I am trying to upload symbols for Flashplayer on https://symbols.mozilla.org/uploads I am using the latest dump_syms ( dump_syms_vc1800) tool present at here https://hg.mozilla.org/mozilla-central/file/tip/toolkit/crashreporter/tools/win32.When i am trying to upload the .sym file to mozilla symbols repository, it says : " Unrecognized file pattern. Should only be <module>/<hex>/<file> or <name>-symbols.txt and nothing else." Can you help me in this ? -Vivek Actual results: " Unrecognized file pattern. Should only be <module>/<hex>/<file> or <name>-symbols.txt and nothing else." Expected results: Symbols should be uploaded properly.
Component: Untriaged → Symbols
Product: Firefox → Socorro
Summary: dump_syms → Trying to upload symbols for Flashplayer with latest dump_syms, getting error "Unrecognized file pattern."
Version: 66 Branch → unspecified
Component: Symbols → Tecken Integration
Hi, The error isn't really an unexpected error. The .zip file you're trying to upload doesn't have the files inside it in the right way. Here's what you should see when you run `unzip -l` on the file for example: https://gist.github.com/peterbe/e0d81b0a6e53c4c380252fc51c1fee71 So, for example the line `AccessibleMarshal.dll/58DF8E1BC000/AccessibleMarshal.dl_` means there's a "root" folder called "AccessibleMarshal.dll" which containers a folder called "58DF8E1BC000" and that folder contains a file called "AccessibleMarshal.dl_" The code that does this stuff is here: https://github.com/mozilla-services/tecken/blob/3e40fc8c5c7ec77049cefc300b5cfcef5e58f26a/tecken/upload/views.py#L53-L85 One thing I think *I* could do to prevent this from happening again is *include* the line that caused the error. Then the error message would instead be something like: "Unrecognized file pattern. Should only be <module>/<hex>/<file> or <name>-symbols.txt and nothing else. (Not prefix/wrong/010101/suffix.example)" That might help debug it for you. However, more important, **where did you read about how to make a ZIP?** Clearly those instructions didn't help. Or perhaps you simply couldn't find them.
Assignee: nobody → peterbe
Hi Peter, Thanks for the help. This zip file i upload contains the actual file and corresponding .pdb and.sym file. Do I need to upload only .sym file ?
Why don't you email me the .zip file you tried and I'll take a look. I think my email is visible here on Bugzilla, right?
Or, if you can't email the .zip file to me, can you run: `unzip -l` on it and show me what it says?
Hi Peter, I upload files in a ZIP format which includes .dll file and its corresponding .pdb and .sym files. Do all these files need to be uploaded or .sym file is sufficient. I have shared a .sym file for one of the builds at this location: https://adobe.ly/2F9GNbc Please check if it correct. -Vivek
(In reply to Vivek Negi from comment #5) > Hi Peter, > > I upload files in a ZIP format which includes .dll file and its > corresponding .pdb and .sym files. Good. Can you email me (peterbe@mozilla.com) the file you're attempting to upload? > Do all these files need to be uploaded or .sym file is sufficient. All files. 4 months ago you did manage to upload some files. They contained .exe, .pdb, and .sym https://symbols.mozilla.org/uploads/upload/29338 > > I have shared a .sym file for one of the builds at this location: > https://adobe.ly/2F9GNbc > Please check if it correct. I don't know what that is. It appears to be a .sym file displayed as a document or something. Any chance you can email the zip file you're having trouble with to me?
Hi Peter, Just sent you an email. Please have a look. -Vivek
(In reply to Vivek Negi from comment #7) The message was not delivered because attachment was too large. I shared the ZIP file at this location : https://adobe.ly/2F3t1I1 Just download it and have a look. > Hi Peter, > > Just sent you an email. Please have a look. > -Vivek
As you can see from...: ``` ▶ unzip -l NPSWF32_32_0_0_101.zip Archive: NPSWF32_32_0_0_101.zip Length Date Time Name --------- ---------- ----- ---- 97300480 12-10-2018 21:03 NPSWF32.pdb 26919746 01-03-2019 19:29 NPSWF32.sym 19909632 12-10-2018 21:03 NPSWF32_32_0_0_101.dll --------- ------- 144129858 3 files ``` The files are in the root of the zip. Each file is supposed to be a directory. And in each (or, where appropriate) there's supposed to be another directory which is all hex characters and then in there the files are supposed to be. I appreciate that the documentation is sparse about how to make these but here[0] it says: "The final check is that each file path in the zip file matches the pattern <module>/<hex>/<file> or <name>-symbols.txt. All other file paths are rejected." I know you've managed to upload valid .zip files before so you must have seen the module and hex directories (aka. the debugID) before. [0] https://tecken.readthedocs.io/en/latest/upload.html#checks-and-validations

Hi Vivek, do you have any updates on creating a new zip file?

Also, I see that the latest Flash Player version is now 32.0.0.114. Can we please get symbols for both 32.0.0.114 and 32.0.0.101? We're receiving crash reports for both versions because users are slow to update to the latest Flash version. :)

Status: UNCONFIRMED → NEW
Ever confirmed: true

Hi Chris and Peter,

I am still not able to upload the symbols zip. We have not changed any of our scripts ( which creates and compresses the folder ) and still this error is persistent. For your reference I have uploaded all symbols in compressed zip files for 32.0.0.114 release here.
Please have look.
https://adobe.ly/2AH6m0i

-Vivek

(In reply to Vivek Negi from comment #11)

Hi Chris and Peter,

I am still not able to upload the symbols zip. We have not changed any of our scripts ( which creates and compresses the folder ) and still this error is persistent. For your reference I have uploaded all symbols in compressed zip files for 32.0.0.114 release here.

Can't help you much there but the scripts must have broken or something.
The last time you manage to upload symbols was Sept 13 2018 and that was for 31.0.0.108. See attached screenshot.

Please have look.
https://adobe.ly/2AH6m0i

How do you use that? All I see is some sort of listing of files.

If that's the content of your ZIP it demonstrates the same problem as mentioned here: https://bugzilla.mozilla.org/show_bug.cgi?id=1513497#c9 The fact that the files need to be in a directory that is the debug ID (a hex string looking thing) inside a directory (called the library name).
In your zip, all the files are in the "root" of the .zip file.

Hi Peter,

You can download these zip files. Just click on Ellipsis( three dots ) icon and download these files.

Also,earlier too I uploaded the symbols using same directory structure. But I am not able to now.

(In reply to Vivek Negi from comment #14)

Hi Peter,

You can download these zip files. Just click on Ellipsis( three dots ) icon and download these files.

First of all, clicking the ellipsis is for downloading the individual files. Secondly, that requires that you Sign In.

Thirdly, I don't think there's anything wrong with the individual files. I don't even know how to validate them.

The problem is how you've put them into a .zip file. That .zip file needs to contain folders (and folders in those folders). Not just the individual files.

(In reply to Vivek Negi from comment #15)

Also,earlier too I uploaded the symbols using same directory structure. But I am not able to now.

The requirement for symbol files .zip files has always been the same. Look at the screenshot [0]
In September, when you uploaded that. the file (for example) FlashPlayerPlugin.exe wasn't in the root of the .zip file. It was inside a folder called "71CB50E35DF947CDAC94AEC75AAA8E8F1/" and that folder was inside a folder called "FlashPlayerPlugin.pdb/"

I think your scripts have either broken due to some software upgrade or the instructions for how to use your scripts have changed.

Is there any chance you can talk to susingh@ado... for advice?

[0] https://bug1513497.bmoattachments.org/attachment.cgi?id=9035636

Hi Peter,

The link I shared is public link. You don't need to sign-in to download files.

Secondly these are individual zip files containing .sym , .pdb ,.DLL files. We share symbols for three files. These zip contains three different type of files and their symbols.

The earlier symbols I shared had the same directory structure as these zip files.

I would request you download and check these zip files .

Ok. I see now how to download the .zip file. I must have clicked the "Save" (instead of "Download").

I downloaded them and looked inside:

~/Downloads
▶ unzip -l FlashPlayerPlugin.pdb.zip
Archive:  FlashPlayerPlugin.pdb.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
  3454464  01-09-2019 11:13   FlashPlayerPlugin.pdb/symbolpush/FlashPlayerPlugin.exe
 22876160  01-09-2019 11:13   FlashPlayerPlugin.pdb/symbolpush/FlashPlayerPlugin.pdb
  2782679  01-10-2019 08:37   FlashPlayerPlugin.pdb/symbolpush/FlashPlayerPlugin.sym
  3454464  01-09-2019 11:13   FlashPlayerPlugin.pdb/symbolpush/FlashPlayerPlugin_32_0_0_114.exe
---------                     -------
 32567767                     4 files

There's the problem!! The file FlashPlayerPlugin.exe (for example) is inside a folder called symbolpush which is NOT an allowed debugID. That "second folder" needs to be a debugID. E.g. 71CB50E35DF947CDAC94AEC75AAA8E8F1 (from https://symbols.mozilla.org/uploads/upload/29338)

The upload server code can't validate that the debugID is a valid real debugID but it can speculate. The Python code that does that check is here: https://github.com/mozilla-services/tecken/blob/63eb532469449e165a8833f6fe83b7c91d6b40f6/tecken/upload/views.py#L74-L76 and https://github.com/mozilla-services/tecken/blob/63eb532469449e165a8833f6fe83b7c91d6b40f6/tecken/upload/views.py#L50

Hi Peter,
I am still not able to upload symbols. I have updated the second folder with the debug ID(78F28A2E3179412E8BD18D92C36224442).Here is the link of the zip file.
https://adobe.ly/2M5dSqz

-Vivek

The error wasn't very helpful if the .zip file contained something it "didn't like". So I'm making the error clearer. But it won't solve the problem. You have to make a new .zip without the .DS_Store file.

Also, I filed this: https://github.com/mozilla-services/tecken/issues/1477 so next time that won't be so hard to deal with.

Hi Peter,

I have removed the .DS_Store file and issue still persists. The zip i am trying to upload is here:
https://adobe.ly/2M7vZvS

@chris,

Until this issue resolves, I can share all the symbols of current and previous release in ZIP format at adobe assets ( similar to shared folder I am sharing with Peter). Please let me know if it is fine.

-Vivek

As you can see now, the zip contains files that are in an extra folder:

▶ aunpack -l NPSWF64.pdb\(1\).zip
Archive:  NPSWF64.pdb(1).zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  01-14-2019 10:02   NPSWF64.pdb/NPSWF64.pdb/
        0  01-14-2019 10:02   NPSWF64.pdb/NPSWF64.pdb/78F28A2E3179412E8BD18D92C36224442/
 91426816  01-09-2019 11:13   NPSWF64.pdb/NPSWF64.pdb/78F28A2E3179412E8BD18D92C36224442/NPSWF64.pdb
 17065806  01-10-2019 08:33   NPSWF64.pdb/NPSWF64.pdb/78F28A2E3179412E8BD18D92C36224442/NPSWF64.sym
 26873856  01-09-2019 11:13   NPSWF64.pdb/NPSWF64.pdb/78F28A2E3179412E8BD18D92C36224442/NPSWF64_32_0_0_114.dll
---------                     -------
135366478                     5 files

So the last file's path is: NPSWF64.pdb/NPSWF64.pdb/78F28A2E3179412E8BD18D92C36224442/NPSWF64_32_0_0_114.dll when it should be NPSWF64.pdb/78F28A2E3179412E8BD18D92C36224442/NPSWF64_32_0_0_114.dll.

Attached is a screenshot showing what I see when I tried to manually upload it.

I'd love to help with your scripting that produces the .zip files but it's hard to know how they run or what operating system it is. If you want to and are able to, come on to IRC (irc.mozilla.org) and find peterbe in the #breakpad IRC channel.

The error message you get about there being too many sub-folders.

It worked! Yay!
https://symbols.mozilla.org/uploads/upload/41844

I'll leave it to you and Chris Peterson to figure out what other versions of those files need uploading. I think we've done everything we can do here. Thank you for your patience.

Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED

Hi Peter,

Thanks for support.

@Chris,

I have uploaded the symbol for current release 30.0.0.114 and previous release 30.0.0.101. Please let me know if you need
symbols for any there version.

-Vivek

Thanks, Vivek! I see the symbols you uploaded for Flash 32.0.0.101 and 32.0.0.114. I think that covers the Flash versions in our top crash reports.

I will wait a couple days and then confirm that the new symbols are properly symbolicating our crash reports.

Jim, do you have any suggestions why some Flash crash reports for the same plugin file name and version have different DLL "Debug Identifiers"?

Now that new Flash symbols have been uploaded to Socorro, I confirmed some of the Flash crash reports appear to be properly symbolicated (like bp-c5bc4c55-fd4f-4176-a43c-f8efa0190123), but others do not (like bp-42e8b9de-b300-469e-a84b-341480190123).

Those two crash reports have the same plugin file name and version (npswf32_32_0_0_114.dll), but different "Debug Identifiers" according to the reports' Modules tabs:

https://crash-stats.mozilla.com/report/index/c5bc4c55-fd4f-4176-a43c-f8efa0190123#tab-modules
NPSWF32_32_0_0_114.dll 38D1736303E641B28ADCB687990F59912
plugin-container.exe 5D52C8AC2EBB8315F91F455693B7A71E1

https://crash-stats.mozilla.com/report/index/42e8b9de-b300-469e-a84b-341480190123#tab-modules
NPSWF32_32_0_0_114.dll F6939932816D4C259262BEE64F47B2F72
FlashPlayerPlugin_32_0_0_114.exe 22B953EDBC754A89841DFF7A1F0615121

Both reports are for plugin process crashes in x86 Firefox on Windows 7, yet one report's modules includes plugin-container.exe and the other FlashPlayerPlugin_32_0_0_114.exe.

Are the FlashPlayerPlugin_32_0_0_114.exe crashes in Adobe's Protected Mode process?

Status: RESOLVED → VERIFIED
Flags: needinfo?(jmathies)

Jim says that old crash reports won't be retroactively symbolicated just because new symbols were uploaded. We could ask the Socorro team to reprocess the old reports, but I don't think that is necessary.

However, the new symbols were uploaded on January 16 (comment 29), but crash report bp-42e8b9de-b300-469e-a84b-341480190123 is from January 23 and is not symbolicated. bp-c5bc4c55-fd4f-4176-a43c-f8efa0190123, also from January 23, is symbolicated.

(In reply to Chris Peterson [:cpeterson] from comment #32)

Jim says that old crash reports won't be retroactively symbolicated just because new symbols were uploaded. We could ask the Socorro team to reprocess the old reports, but I don't think that is necessary.

That is correct. I happen to know that willkg is out on PTO but there are others who have permission enough to start a reprocess. At least of individual crashes.

However, the new symbols were uploaded on January 16 (comment 29), but crash report bp-42e8b9de-b300-469e-a84b-341480190123 is from January 23 and is not symbolicated. bp-c5bc4c55-fd4f-4176-a43c-f8efa0190123, also from January 23, is symbolicated.

This is mysterious indeed. Not sure what's going on there.

Unfortunately, I don't know to gather the debugID for those npswf32_32_0_0_114.dll symbols. If we knew that we could check if those two crash reports use the same symbol for npswf32_32_0_0_114.dll.
If a crash has been reprocessed it's usually revealed in the "Processor Notes" field under the "Details" tab and that does NOT appear to the explanation for why https://crash-stats.mozilla.com/report/index/c5bc4c55-fd4f-4176-a43c-f8efa0190123#tab-details worked.

We could start a reprocess, right now, of https://crash-stats.mozilla.com/report/index/42e8b9de-b300-469e-a84b-341480190123#tab-details and see if that indeed does fix it. But if we do that we will lose the current state which might be interesting. Perhaps when willkg comes back he can do his magic to copy that crash over to Stage so we can see what happens, there, when we reprocess it.

(In reply to Chris Peterson [:cpeterson] from comment #32)

Jim says that old crash reports won't be retroactively symbolicated just
because new symbols were uploaded. We could ask the Socorro team to
reprocess the old reports, but I don't think that is necessary.

However, the new symbols were uploaded on January 16 (comment 29), but crash
report bp-42e8b9de-b300-469e-a84b-341480190123 is from January 23 and is not
symbolicated. bp-c5bc4c55-fd4f-4176-a43c-f8efa0190123, also from January 23,
is symbolicated.

Not sure about this last one. A test build from adobe? The locale is china, maybe this is one of those weird china specific releases or something?

Flags: needinfo?(jmathies)

Moving from Socorro product to Tecken product.

Component: Tecken → General
Product: Socorro → Tecken
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: