Closed
Bug 695843
Opened 12 years ago
Closed 12 years ago
Nested omni.jar for Android
Categories
(Core Graveyard :: Widget: Android, defect)
Tracking
(Not tracked)
RESOLVED
FIXED
mozilla11
People
(Reporter: glandium, Assigned: glandium)
References
Details
(Whiteboard: [mobilestartupshrink][ts][inbound])
Attachments
(11 files, 17 obsolete files)
3.52 KB,
patch
|
benjamin
:
review+
|
Details | Diff | Splinter Review |
9.21 KB,
patch
|
mrbkap
:
review+
|
Details | Diff | Splinter Review |
7.83 KB,
patch
|
benjamin
:
review+
|
Details | Diff | Splinter Review |
1.26 KB,
patch
|
mrbkap
:
review+
|
Details | Diff | Splinter Review |
16.17 KB,
patch
|
glandium
:
review+
|
Details | Diff | Splinter Review |
3.75 KB,
patch
|
glandium
:
review+
|
Details | Diff | Splinter Review |
2.63 KB,
patch
|
taras.mozilla
:
review+
|
Details | Diff | Splinter Review |
4.59 KB,
patch
|
glandium
:
review+
|
Details | Diff | Splinter Review |
16.38 KB,
patch
|
mwu
:
review+
|
Details | Diff | Splinter Review |
58.49 KB,
patch
|
glandium
:
review+
|
Details | Diff | Splinter Review |
14.34 KB,
patch
|
Details | Diff | Splinter Review |
Around 200ms are wasted on a nexus s at startup because Android does something with the apk zip entries and takes time for this because we have so many entries in the apk. Getting the number of entries down should allow us to avoid wasting these 200ms. We can do that by keeping chrome, components and resources compressed under a single (uncompressed) file in the apk.
Assignee | ||
Comment 1•12 years ago
|
||
(In reply to Mike Hommey [:glandium] from comment #0) > Around 200ms are wasted on a nexus s at startup. Note that it's even more on multi-language builds.
Assignee | ||
Comment 2•12 years ago
|
||
It still needs to be modified such that nsJAR::GetZipArchive is not required.
Assignee | ||
Comment 3•12 years ago
|
||
Attachment #569042 -
Flags: review?(benjamin)
Assignee | ||
Comment 4•12 years ago
|
||
I'm not sure we should keep nsIZipReaders around, here. The impact should definitely be checked.
Assignee | ||
Comment 5•12 years ago
|
||
This part definitely needs more work, to have something proper for the chrome registry and js component loader. The other part that's broken with this patch is that ManifestProcessingContext, both in the component manager and the chrome registry, are using nsIFiles for the base path to resolve from, and thus fail with nested omni.jar. They probably need to store either an nsIFile or an nsIZipReader depending on cases. (I just wish we had fake nsIFiles that would allow to use Zips transparently, like directories, but so many things rely on nsIFiles to actually be nsILocalFiles...)
Updated•12 years ago
|
Attachment #569042 -
Flags: review?(benjamin) → review+
Assignee | ||
Updated•12 years ago
|
Attachment #569041 -
Attachment is obsolete: true
Assignee | ||
Updated•12 years ago
|
Attachment #569043 -
Attachment is obsolete: true
Assignee | ||
Updated•12 years ago
|
Attachment #569044 -
Attachment is obsolete: true
Assignee | ||
Comment 6•12 years ago
|
||
Attachment #572834 -
Flags: review?(mwu)
Assignee | ||
Comment 7•12 years ago
|
||
Attachment #572835 -
Flags: review?(benjamin)
Assignee | ||
Comment 8•12 years ago
|
||
I'm not settled on this one. I may switch to using nsZipHandles instead of nsZipArchives in FileLocation. The class also needs some documentation, and the API may require some tweaking.
Assignee | ||
Updated•12 years ago
|
Attachment #572836 -
Flags: feedback?(mwu)
Assignee | ||
Comment 9•12 years ago
|
||
Attachment #572838 -
Flags: review?(mwu)
Assignee | ||
Comment 10•12 years ago
|
||
Attachment #572840 -
Flags: review?(benjamin)
Assignee | ||
Comment 11•12 years ago
|
||
(In reply to Mike Hommey [:glandium] from comment #10) > Created attachment 572840 [details] [diff] [review] [diff] [details] [review] > part 6 - Don't separate file and JAR modules in component manager Note that the use of net_GetURLSpecFromActualFile goes away in a subsequent patch.
Assignee | ||
Comment 12•12 years ago
|
||
Attachment #572841 -
Flags: review?(mrbkap)
Assignee | ||
Comment 13•12 years ago
|
||
Attachment #572842 -
Flags: review?(mrbkap)
Assignee | ||
Comment 14•12 years ago
|
||
This uses the API from part 4.
Assignee | ||
Updated•12 years ago
|
Attachment #572843 -
Flags: feedback?(benjamin)
Assignee | ||
Comment 15•12 years ago
|
||
Attachment #572844 -
Flags: review?(benjamin)
Assignee | ||
Comment 16•12 years ago
|
||
Attachment #572848 -
Flags: review?(benjamin)
Assignee | ||
Updated•12 years ago
|
Attachment #572840 -
Attachment is obsolete: true
Attachment #572840 -
Flags: review?(benjamin)
Comment 17•12 years ago
|
||
Comment on attachment 572838 [details] [diff] [review] part 5 - Allow nsZipCursor to always use the given buffer Wouldn't it be cleaner to add a nsZipCursor::Copy?
Updated•12 years ago
|
Attachment #572834 -
Flags: review?(mwu) → review+
Comment 18•12 years ago
|
||
Comment on attachment 572838 [details] [diff] [review] part 5 - Allow nsZipCursor to always use the given buffer Taras seems to have more of an opinion on this one than I do.
Attachment #572838 -
Flags: review?(mwu) → review?(tglek)
Assignee | ||
Comment 19•12 years ago
|
||
Try result on this patch queue: https://tbpl.mozilla.org/?noignore=1&tree=Try&rev=cc189b22e21d Summary: - 2 nsZipArchives are leaked - Bootstrap extensions test seems to go in an infinite loop on windows.
Assignee | ||
Comment 20•12 years ago
|
||
(In reply to Mike Hommey [:glandium] from comment #19) > - 2 nsZipArchives are leaked Found the culprit. Will update the relevant patch later. Another issue: jars are likely to be mapped twice in memory with the current patch queue. Bug 504217 should help there.
Assignee | ||
Comment 21•12 years ago
|
||
Attachment #573171 -
Flags: review?(mrbkap)
Assignee | ||
Updated•12 years ago
|
Attachment #572842 -
Attachment is obsolete: true
Attachment #572842 -
Flags: review?(mrbkap)
Assignee | ||
Comment 22•12 years ago
|
||
Same patch, which removes a "mZip.forget()" that was a leftover from the original use of nsAutoPtr.
Assignee | ||
Updated•12 years ago
|
Attachment #572834 -
Attachment is obsolete: true
Assignee | ||
Comment 23•12 years ago
|
||
Comment on attachment 573172 [details] [diff] [review] part 1 - Add Refcounting on nsZipArchives Carrying previous r+, considering the changes.
Attachment #573172 -
Flags: review+
Assignee | ||
Comment 24•12 years ago
|
||
(In reply to Mike Hommey [:glandium] from comment #19) > - Bootstrap extensions test seems to go in an infinite loop on windows. The test goes on waiting for nothing because it doesn't handle failure gracefully. The failure itself is due to the xpi file still being open (iow, a nsZipArchive still existing) when it is removed (NS_ERROR_FILE_ACCESS_DENIED on nsIFile.remove) in test_bootstrap.js line 113, function manuallyUninstall. Obviously that kind of thing only fail on windows.
Comment 25•12 years ago
|
||
Comment on attachment 572838 [details] [diff] [review] part 5 - Allow nsZipCursor to always use the given buffer We decided to add a nsZipCursor::Copy
Attachment #572838 -
Flags: review?(tglek) → review-
Comment 26•12 years ago
|
||
Comment on attachment 572836 [details] [diff] [review] part 4 - Add a FileLocation class to either store a pair (zip, path) or a file, and do some operations on these So, I think FileLocation makes a lot of sense for the component manager code but not as much for nsZipHandle. That being said, this is one of the easier ways for you to generate uris and it doesn't mess with nsZipHandle too much. I suspect that having nsZipHandle directly generate the information you want is a cleaner way to go here, and might be useful for some other code along the way. Up to you though - if moving things to nsZipHandle is too complicated for now, that's fine.
Attachment #572836 -
Flags: feedback?(mwu)
Updated•12 years ago
|
Attachment #573171 -
Flags: review?(mrbkap) → review+
Updated•12 years ago
|
Attachment #572841 -
Flags: review?(mrbkap) → review+
Updated•12 years ago
|
Attachment #572835 -
Flags: review?(benjamin) → review+
Updated•12 years ago
|
Attachment #572843 -
Flags: feedback?(benjamin) → feedback+
Updated•12 years ago
|
Attachment #572844 -
Flags: review?(benjamin) → review+
Comment 27•12 years ago
|
||
Comment on attachment 572848 [details] [diff] [review] part 6 - Don't separate file and JAR modules in component manager I'm not sure that hashing on the URI representation is correct or will work correctly: we've had issues in the past with short and long path names not comparing equal correctly. What is the actual behavior of net_GetURLSpecFromActualFile WRT case differences or things like that?
Assignee | ||
Comment 28•12 years ago
|
||
(In reply to Benjamin Smedberg [:bsmedberg] from comment #27) > Comment on attachment 572848 [details] [diff] [review] [diff] [details] [review] > part 6 - Don't separate file and JAR modules in component manager > > I'm not sure that hashing on the URI representation is correct or will work > correctly: we've had issues in the past with short and long path names not > comparing equal correctly. What is the actual behavior of > net_GetURLSpecFromActualFile WRT case differences or things like that? net_GetURLSpecFromActualFile doesn't ensure a canonical path before turning the path into an url :( The same concern would also apply on part 7. I guess we could canonicalize paths before putting them in the uri string used for keys.
Assignee | ||
Comment 29•12 years ago
|
||
Note that the nsIHashable part of nsLocalFile* doesn't handle case sensitivity on Mac, AFAICS, so we may already have the case sensitivity problem.
Comment 30•12 years ago
|
||
Comment on attachment 573172 [details] [diff] [review] part 1 - Add Refcounting on nsZipArchives >--- a/startupcache/StartupCache.cpp >+++ b/startupcache/StartupCache.cpp >+nsresult >+GetBufferFromZipArchive(nsZipArchive *zip, bool doCRC, const char* id, >+ char** outbuf, PRUint32* length) >+{ >+ if (zip) { >+ nsZipItemPtr<char> zipItem(zip, id, doCRC); >+ if (zipItem) { >+ *outbuf = zipItem.Forget(); >+ *length = zipItem.Length(); >+ return NS_OK; >+ } >+ } >+ return NS_ERROR_NOT_AVAILABLE; >+} Early returns, please. >+ nsresult rv; >+ rv = GetBufferFromZipArchive(mArchive, true, id, outbuf, length); nsresult rv = GetBufferFromZipArchive(mArchive, true, id, outbuf, length); >--- a/xpcom/build/Omnijar.cpp >+++ b/xpcom/build/Omnijar.cpp > Omnijar::InitOne(nsIFile *aPath, Type aType) >+ nsRefPtr<nsZipArchive> zipReader = new nsZipArchive(); >+ if (!zipReader || NS_FAILED(zipReader->OpenArchive(file))) { Remove the '!zipReader' check, 'new' can't fail. >--- a/xpcom/build/Omnijar.h >+++ b/xpcom/build/Omnijar.h >+static inline already_AddRefed<nsZipArchive> > GetReader(Type aType) > { > NS_ABORT_IF_FALSE(IsInitialized(), "Omnijar not initialized"); >+ NS_IF_ADDREF(sReader[aType]); > return sReader[aType]; nsRefPtr<nsZipArchive> reader = sReader[aType]; return reader.forget();
Comment 31•12 years ago
|
||
Comment on attachment 572836 [details] [diff] [review] part 4 - Add a FileLocation class to either store a pair (zip, path) or a file, and do some operations on these >--- /dev/null >+++ b/xpcom/build/FileLocation.cpp >+namespace mozilla { >+ >+nsresult FileLocation::GetData(char *buf, PRUint32 len) >+{ >+ nsZipCursor cursor(item, mBaseZip, (PRUint8 *)buf, len, true); static_cast >+ mozilla::AutoFDClose fd; No need for the 'mozilla::'. >--- /dev/null >+++ b/xpcom/build/FileLocation.h >+ void Init(nsILocalFile *file) >+ { >+ mBaseFile = file; >+ mPath = (char *)NULL; mPath.Truncate()? >+/* This needs to be included at the end because of the cross-references */ >+#include "nsZipArchive.h" Uh, really? Why does this need to be included at all?
Comment 32•12 years ago
|
||
Comment on attachment 572844 [details] [diff] [review] part 10 - Allow nested omni.jar and build one for Android >--- a/xpcom/build/Omnijar.cpp >+++ b/xpcom/build/Omnijar.cpp >@@ -105,16 +106,24 @@ Omnijar::InitOne(nsIFile *aPath, Type aT >+ nsRefPtr<nsZipHandle> handle; >+ if (NS_SUCCEEDED(nsZipHandle::Init(zipReader, "omni.jar", getter_AddRefs(handle)))) { >+ zipReader = new nsZipArchive(); >+ if (!zipReader... And here
Comment 33•12 years ago
|
||
Comment on attachment 572848 [details] [diff] [review] part 6 - Don't separate file and JAR modules in component manager Ugh. I guess that since we're using manifests now it's unlikely that we'll have mixed-case issues; I guess we'll find out later!
Attachment #572848 -
Flags: review?(benjamin) → review+
Assignee | ||
Comment 34•12 years ago
|
||
Updated with Ms2ger comments.
Assignee | ||
Comment 35•12 years ago
|
||
Comment on attachment 579619 [details] [diff] [review] part 1 - Add Refcounting on nsZipArchives. Carrying over r+
Attachment #579619 -
Flags: review+
Assignee | ||
Updated•12 years ago
|
Attachment #573172 -
Attachment is obsolete: true
Assignee | ||
Comment 36•12 years ago
|
||
Refreshed against current m-c
Assignee | ||
Updated•12 years ago
|
Attachment #572835 -
Attachment is obsolete: true
Assignee | ||
Comment 37•12 years ago
|
||
Comment on attachment 579620 [details] [diff] [review] part 3 - Use mozilla::AutoFDClose instead of ad hoc AutoCloseFD classes Carrying over r+
Attachment #579620 -
Flags: review+
Assignee | ||
Comment 38•12 years ago
|
||
Attachment #579621 -
Flags: review?(tglek)
Assignee | ||
Updated•12 years ago
|
Attachment #572838 -
Attachment is obsolete: true
Assignee | ||
Comment 39•12 years ago
|
||
Attachment #579624 -
Flags: review?(mwu)
Assignee | ||
Updated•12 years ago
|
Attachment #572836 -
Attachment is obsolete: true
Assignee | ||
Updated•12 years ago
|
Attachment #579624 -
Flags: review?(benjamin)
Assignee | ||
Comment 40•12 years ago
|
||
Attachment #579626 -
Flags: review?(benjamin)
Assignee | ||
Updated•12 years ago
|
Attachment #572843 -
Attachment is obsolete: true
Assignee | ||
Comment 41•12 years ago
|
||
Basically the same as previous version, but uses OMNIJAR_NAME instead of hardcoding omni.jar.
Assignee | ||
Updated•12 years ago
|
Attachment #572844 -
Attachment is obsolete: true
Assignee | ||
Comment 42•12 years ago
|
||
Comment on attachment 579627 [details] [diff] [review] part 10 - Allow nested omni.jar and build one for Android Carrying over r+
Attachment #579627 -
Flags: review+
Assignee | ||
Comment 43•12 years ago
|
||
In the case we end up having problems with short vs. long path names making uris not stable enough to be hash keys for the component manager and the JS component loader, here is a patch that makes the relevant hash tables use something more canonical. Attaching so that it can be found later if needed, since I already did the work (instead of being lost on my harddrive)
Comment 44•12 years ago
|
||
Comment on attachment 579621 [details] [diff] [review] part 4 - Allow nsZipCursor to always use the given buffer thanks
Attachment #579621 -
Flags: review?(mozilla) → review+
Comment 45•12 years ago
|
||
Comment on attachment 579624 [details] [diff] [review] part 5 - Add a FileLocation class to either store a pair (zip, path) or a file, and do some operations on these Some of the single-line ifs in FileLocation::FileLocation aren't braced and I find this hard to read: please brace them all. Please comment namespace-closing brace in FileLocation.cpp, e.g.: } // namespace mozilla The opening brace of "class FileLocation {" and FileLocation::Data should be on the next line It's unusual to return an nsCString (in FileLocation::GetURIString and FileLocation::GetPath). Normally methods that return a string use: void ReturnAString(nsACString& result); Since we have copy-on-write strings, perhaps this isn't a problem, but I'd prefer to be consistent with other parts of our code. r=me with those changes
Attachment #579624 -
Flags: review?(benjamin) → review+
Updated•12 years ago
|
Attachment #579626 -
Flags: review?(benjamin) → review+
Assignee | ||
Comment 46•12 years ago
|
||
With bsmedberg's comments addressed
Attachment #579972 -
Flags: review?(mwu)
Assignee | ||
Updated•12 years ago
|
Attachment #579624 -
Attachment is obsolete: true
Attachment #579624 -
Flags: review?(mwu)
Assignee | ||
Comment 47•12 years ago
|
||
Refreshed with the new modified GetURIString signature from part 5.
Assignee | ||
Updated•12 years ago
|
Attachment #579626 -
Attachment is obsolete: true
Assignee | ||
Comment 48•12 years ago
|
||
Removed a bit that is meant to be in the "extra" patch.
Attachment #579974 -
Flags: review?(mwu)
Assignee | ||
Updated•12 years ago
|
Attachment #579972 -
Attachment is obsolete: true
Attachment #579972 -
Flags: review?(mwu)
Assignee | ||
Comment 49•12 years ago
|
||
Removed an unused variable
Assignee | ||
Updated•12 years ago
|
Attachment #579973 -
Attachment is obsolete: true
Assignee | ||
Comment 50•12 years ago
|
||
Refreshed against refreshed part 9.
Assignee | ||
Updated•12 years ago
|
Attachment #579628 -
Attachment is obsolete: true
Assignee | ||
Comment 51•12 years ago
|
||
Comment on attachment 579975 [details] [diff] [review] part 9 - Use FileLocations in the component manager. Carrying over r+.
Attachment #579975 -
Flags: review+
Comment 52•12 years ago
|
||
Comment on attachment 579974 [details] [diff] [review] part 5 - Add a FileLocation class to either store a pair (zip, path) or a file, and do some operations on these. Review of attachment 579974 [details] [diff] [review]: ----------------------------------------------------------------- Looks ok overall though I haven't really studied how some of the callers use some of this API. Comments are just nits. ::: xpcom/build/FileLocation.cpp @@ +106,5 @@ > + if (IsZip() && mBaseZip) { > + nsRefPtr<nsZipHandle> handler = mBaseZip->GetFD(); > + if (handler) > + return handler->mFile.GetBaseFile(); > + } else { I think this else could be removed after moving return NULL to the "if (IsZip() && mBaseZip)" block, or maybe invert this if/else. Having the return NULL only apply to one of the if/else blocks is a bit weird. @@ +139,5 @@ > + > +nsresult > +FileLocation::GetData(Data &data) > +{ > + if (IsZip()) { if (!IsZip()) return mBaseFile->OpenNSPRFileDesc(PR_RDONLY, 0444, &data.mFd); ::: xpcom/build/FileLocation.h @@ +132,5 @@ > + > + /** > + * Returns whether the "base file" (see GetBaseFile) is an archive > + */ > + bool IsZip() const { Curly brace style inconsistent here and in the next two functions.
Attachment #579974 -
Flags: review?(mwu) → review+
Assignee | ||
Comment 53•12 years ago
|
||
https://hg.mozilla.org/integration/mozilla-inbound/rev/a31949628a7e https://hg.mozilla.org/integration/mozilla-inbound/rev/8e345e9d93e9 https://hg.mozilla.org/integration/mozilla-inbound/rev/3032fd2bd96a https://hg.mozilla.org/integration/mozilla-inbound/rev/692b6b2207b4 https://hg.mozilla.org/integration/mozilla-inbound/rev/616b0239a9fb https://hg.mozilla.org/integration/mozilla-inbound/rev/c5a971a7dc4d https://hg.mozilla.org/integration/mozilla-inbound/rev/00d65419d422 https://hg.mozilla.org/integration/mozilla-inbound/rev/2dd63b2aa323 https://hg.mozilla.org/integration/mozilla-inbound/rev/502c67d69baa https://hg.mozilla.org/integration/mozilla-inbound/rev/792a9ba2aa26
Whiteboard: [mobilestartupshrink][ts] → [mobilestartupshrink][ts][inbound]
Comment 54•12 years ago
|
||
https://hg.mozilla.org/mozilla-central/rev/a31949628a7e https://hg.mozilla.org/mozilla-central/rev/8e345e9d93e9 https://hg.mozilla.org/mozilla-central/rev/3032fd2bd96a https://hg.mozilla.org/mozilla-central/rev/692b6b2207b4 https://hg.mozilla.org/mozilla-central/rev/616b0239a9fb https://hg.mozilla.org/mozilla-central/rev/c5a971a7dc4d https://hg.mozilla.org/mozilla-central/rev/00d65419d422 https://hg.mozilla.org/mozilla-central/rev/2dd63b2aa323 https://hg.mozilla.org/mozilla-central/rev/502c67d69baa https://hg.mozilla.org/mozilla-central/rev/792a9ba2aa26
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla11
Comment 55•12 years ago
|
||
Note: this seems to have regressed being able to access APK resources from Java using jar: URIs
Updated•2 years ago
|
Product: Core → Core Graveyard
You need to log in
before you can comment on or make changes to this bug.
Description
•