Last Comment Bug 653143 - Detect duplicates in omnijar
: Detect duplicates in omnijar
Status: RESOLVED FIXED
: perf
Product: Core
Classification: Components
Component: Networking: JAR (show other bugs)
: Trunk
: x86 Linux
: -- normal (vote)
: mozilla7
Assigned To: Michael Wu [:mwu]
:
: Patrick McManus [:mcmanus]
Mentors:
Depends on:
Blocks: 647453
  Show dependency treegraph
 
Reported: 2011-04-27 09:02 PDT by (dormant account)
Modified: 2013-09-18 05:47 PDT (History)
12 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
Detect duplicate files in optimizejars.py (1.54 KB, patch)
2011-05-24 13:31 PDT, Michael Wu [:mwu]
no flags Details | Diff | Splinter Review
Detect duplicate files in optimizejars.py, v2 (1.70 KB, patch)
2011-05-24 14:45 PDT, Michael Wu [:mwu]
taras.mozilla: review+
Details | Diff | Splinter Review

Description (dormant account) 2011-04-27 09:02:03 PDT
optimizejars.py should detect duplicate entries in omnijar and instead have multiple central directory items point at the same data.
Comment 1 Mike Hommey [:glandium] 2011-05-24 11:35:22 PDT
What duplicate entries are there? I know there were, but that should have gone with bug 620931.
Comment 2 Michael Wu [:mwu] 2011-05-24 11:39:37 PDT
(In reply to comment #1)
> What duplicate entries are there? I know there were, but that should have
> gone with bug 620931.

This bug was filed in response to the comment at https://bugzilla.mozilla.org/show_bug.cgi?id=647453#c14 which lists a bunch of duplicates.
Comment 3 Michael Wu [:mwu] 2011-05-24 13:31:00 PDT
Created attachment 534873 [details] [diff] [review]
Detect duplicate files in optimizejars.py

This makes us spit out warnings about duplicate files during packaging. Here's the output I get on my linux64 opt local build:

chrome/browser/content/browser/places/bookmarkProperties2.xul
        is a duplicate of
chrome/browser/content/browser/places/bookmarkProperties.xul
---
chrome/toolkit/skin/classic/mozapps/passwordmgr/key.png
        is a duplicate of
chrome/toolkit/skin/classic/mozapps/passwordmgr/key-16.png
---
chrome/toolkit/skin/classic/mozapps/extensions/category-languages.png
        is a duplicate of
chrome/toolkit/skin/classic/mozapps/extensions/localeGeneric.png
---
chrome/toolkit/skin/classic/mozapps/extensions/themeGeneric.png
        is a duplicate of
chrome/toolkit/skin/classic/mozapps/extensions/category-themes.png
---
chrome/toolkit/skin/classic/mozapps/extensions/extensionGeneric.png
        is a duplicate of
chrome/toolkit/skin/classic/mozapps/extensions/category-extensions.png
---
chrome/toolkit/skin/classic/mozapps/update/downloadButtons.png
        is a duplicate of
chrome/toolkit/skin/classic/mozapps/downloads/downloadButtons.png
---
chrome/toolkit/skin/classic/mozapps/plugins/notifyPluginOutdated.png
        is a duplicate of
chrome/toolkit/skin/classic/mozapps/plugins/notifyPluginBlocked.png
---
chrome/toolkit/skin/classic/mozapps/plugins/notifyPluginCrashed.png
        is a duplicate of
chrome/toolkit/skin/classic/mozapps/plugins/pluginGeneric-16.png
---
chrome/toolkit/skin/classic/mozapps/plugins/notifyPluginGeneric.png
        is a duplicate of
chrome/toolkit/skin/classic/mozapps/plugins/pluginGeneric-16.png
---
chrome/toolkit/skin/classic/mozapps/xpinstall/xpinstallItemGeneric.png
        is a duplicate of
chrome/toolkit/skin/classic/mozapps/extensions/category-extensions.png
---
chrome/toolkit/skin/classic/global/icons/tabprompts-bgtexture.png
        is a duplicate of
chrome/toolkit/skin/classic/mozapps/extensions/background-texture.png
---
chrome/toolkit/skin/classic/global/icons/notfound.png
        is a duplicate of
chrome/toolkit/skin/classic/global/icons/information-16.png
---
chrome/toolkit/skin/classic/global/arrow/arrow-rit-hov.gif
        is a duplicate of
chrome/toolkit/skin/classic/global/arrow/arrow-rit.gif
---
chrome/toolkit/skin/classic/global/arrow/arrow-lft-hov.gif
        is a duplicate of
chrome/toolkit/skin/classic/global/arrow/arrow-lft.gif
---
chrome/toolkit/skin/classic/global/console/bullet-question.png
        is a duplicate of
chrome/toolkit/skin/classic/global/icons/information-16.png
---
chrome/toolkit/skin/classic/global/console/bullet-error.png
        is a duplicate of
chrome/toolkit/skin/classic/global/icons/error-16.png
---
chrome/toolkit/skin/classic/global/console/bullet-warning.png
        is a duplicate of
chrome/toolkit/skin/classic/global/icons/warning-16.png
---
chrome/browser/skin/classic/browser/places/tag.png
        is a duplicate of
chrome/toolkit/skin/classic/mozapps/places/tagContainerIcon.png
---
chrome/en-US/locale/en-US/global-platform/win/intl.properties
        is a duplicate of
chrome/en-US/locale/en-US/global-platform/unix/intl.properties
---
chrome/en-US/locale/en-US/global-platform/win/platformKeys.properties
        is a duplicate of
chrome/en-US/locale/en-US/global-platform/unix/platformKeys.properties
---
chrome/en-US/locale/en-US/global-platform/unix/accessible.properties
        is a duplicate of
chrome/en-US/locale/en-US/global-platform/win/accessible.properties
---
chrome/en-US/locale/en-US/global-platform/mac/accessible.properties
        is a duplicate of
chrome/en-US/locale/en-US/global-platform/win/accessible.properties
---
chrome/en-US/locale/en-US/global-platform/mac/intl.properties
        is a duplicate of
chrome/en-US/locale/en-US/global-platform/unix/intl.properties
---
res/table-remove-column.gif
        is a duplicate of
res/table-remove-row.gif
---
res/table-remove-column-hover.gif
        is a duplicate of
res/table-remove-row-hover.gif
---
res/table-remove-row-active.gif
        is a duplicate of
res/table-remove-column-active.gif
---
chrome/browser/skin/classic/browser/feeds/feedIcon.png
        is a duplicate of
chrome/browser/skin/classic/browser/feeds/audioFeedIcon.png
---
chrome/browser/skin/classic/browser/feeds/videoFeedIcon16.png
        is a duplicate of
chrome/browser/skin/classic/browser/feeds/feedIcon16.png
---
chrome/browser/skin/classic/browser/feeds/audioFeedIcon16.png
        is a duplicate of
chrome/browser/skin/classic/browser/feeds/feedIcon16.png
---
chrome/browser/skin/classic/browser/feeds/videoFeedIcon.png
        is a duplicate of
chrome/browser/skin/classic/browser/feeds/audioFeedIcon.png
---
WARNING: Found 30 duplicate files
Comment 4 (dormant account) 2011-05-24 13:52:56 PDT
should sum up total wasted space too
Comment 5 Michael Wu [:mwu] 2011-05-24 14:45:16 PDT
Created attachment 534894 [details] [diff] [review]
Detect duplicate files in optimizejars.py, v2
Comment 6 Michael Wu [:mwu] 2011-05-24 14:46:43 PDT
WARNING: Found 30 duplicate files taking 58018 bytes

Note that about half of it is compressed data, and the other half is overhead from zip metadata.
Comment 7 (dormant account) 2011-05-24 15:25:47 PDT
Comment on attachment 534894 [details] [diff] [review]
Detect duplicate files in optimizejars.py, v2

nice
Comment 8 :Ehsan Akhgari 2011-06-20 12:27:03 PDT
I landed this on mozilla-inbound.
Comment 9 Michael Wu [:mwu] 2011-06-20 15:24:21 PDT
(In reply to comment #8)
> I landed this on mozilla-inbound.

I was waiting for fixes from bug 659407 to land before adding more noise to the build, but hey, why not..
Comment 10 :Ehsan Akhgari 2011-06-20 16:02:10 PDT
(In reply to comment #9)
> (In reply to comment #8)
> > I landed this on mozilla-inbound.
> 
> I was waiting for fixes from bug 659407 to land before adding more noise to the
> build, but hey, why not..

I can back it out if you really want, but let's be honest, our builds are not exactly silent.  ;-)
Comment 11 Mounir Lamouri (:mounir) 2011-06-21 05:37:54 PDT
Pushed:
http://hg.mozilla.org/mozilla-central/rev/634f82bc05fb

Note You need to log in before you can comment on or make changes to this bug.