Files larger than 2GiB are rejected
Categories
(Tecken :: Upload, defect, P2)
Tracking
(Not tracked)
People
(Reporter: gsvelto, Unassigned)
Details
I tried uploading a recent Debian-packaged version of libxul.so with full debugging information weighing 2.6GiB. The resulting .zip file was ~900MiB. The symbol server rejected it after the upload was finished. I tried with other files and it seems to reject anything larger than 2GiB.
Comment 1•5 years ago
|
||
What's the rejection error code and response look like?
Comment 3•5 years ago
|
||
That helps a ton. That upload is kicking up this error:
https://sentry.prod.mozaws.net/operations/symbols-prod/issues/6303840/
Symbols has an Uploads dashboard and that says the biggest one it's seen in some period of time is 1.6GB. I don't know the code very well, so I don't know what that number measures nor how far back it's looking.
I wonder if the problem is not the size of the payload, but rather something else about the payload.
Can you use Firefox Send to send me the payload that's kicking up the error?
Reporter | ||
Comment 5•5 years ago
|
||
I'm uploading it right now. The archive itself is less than 2GiB but one of the files it contains is larger.
Comment 7•5 years ago
|
||
I've got the file. I don't know when I can work on this. The Sentry event has 14 instances of it, so it doesn't happen often, but when it does, it completely blocks that symbols file.
Gabriele mentioned that Firefox build symbols are increasing over time, too, so there's a sort of impending doom here.
Reporter | ||
Comment 8•5 years ago
|
||
Note that the file in question is a shared-library with full debug information. Actually symbol files (.sym) are very unlikely to become that large.
Comment 9•5 years ago
|
||
Bumping this over to the Tecken product.
Comment 10•4 years ago
|
||
Gabriele: Do you have a publicly available location to upload the symbols file to? If so, then you can do upload-by-download to get around the 2GiB limit.
https://tecken.readthedocs.io/en/latest/upload.html#how-to-upload-by-download-url
I think that's the best we're going to be able to do unless we rewrite how uploading works.
Reporter | ||
Comment 11•4 years ago
|
||
Yes, I do! I'll give it a spin when I switch to the new dump_syms. I want to include full, unredacted debug information then so it's the right time to try it out. Right now I'm splitting archives in smaller chunks to get around the limit but it would be easier not too. And faster too.
Comment 12•2 years ago
|
||
We don't have any instances of this error in the last 6 months. It's possible things have changed and we don't get it anymore. I'm going to mark it as WORKSFORME. If it happens again, we'll get a sentry error and can look into it then in a new bug.
Description
•