653143 - Detect duplicates in omnijar

Status: RESOLVED FIXED

(Core :: Networking: JAR, defect)

Description

[(dormant account)]

optimizejars.py should detect duplicate entries in omnijar and instead have multiple central directory items point at the same data.

Comment 1

[Mike Hommey [:glandium]]

What duplicate entries are there? I know there were, but that should have gone with bug 620931.

Comment 2

[Michael Wu [:mwu]]

(In reply to comment #1)
> What duplicate entries are there? I know there were, but that should have
> gone with bug 620931.

This bug was filed in response to the comment at https://bugzilla.mozilla.org/show_bug.cgi?id=647453#c14 which lists a bunch of duplicates.

Comment 3

[Michael Wu [:mwu]]

Attachment: Detect duplicate files in optimizejars.py

This makes us spit out warnings about duplicate files during packaging. Here's the output I get on my linux64 opt local build:

chrome/browser/content/browser/places/bookmarkProperties2.xul
        is a duplicate of
chrome/browser/content/browser/places/bookmarkProperties.xul
---
chrome/toolkit/skin/classic/mozapps/passwordmgr/key.png
        is a duplicate of
chrome/toolkit/skin/classic/mozapps/passwordmgr/key-16.png
---
chrome/toolkit/skin/classic/mozapps/extensions/category-languages.png
        is a duplicate of
chrome/toolkit/skin/classic/mozapps/extensions/localeGeneric.png
---
chrome/toolkit/skin/classic/mozapps/extensions/themeGeneric.png
        is a duplicate of
chrome/toolkit/skin/classic/mozapps/extensions/category-themes.png
---
chrome/toolkit/skin/classic/mozapps/extensions/extensionGeneric.png
        is a duplicate of
chrome/toolkit/skin/classic/mozapps/extensions/category-extensions.png
---
chrome/toolkit/skin/classic/mozapps/update/downloadButtons.png
        is a duplicate of
chrome/toolkit/skin/classic/mozapps/downloads/downloadButtons.png
---
chrome/toolkit/skin/classic/mozapps/plugins/notifyPluginOutdated.png
        is a duplicate of
chrome/toolkit/skin/classic/mozapps/plugins/notifyPluginBlocked.png
---
chrome/toolkit/skin/classic/mozapps/plugins/notifyPluginCrashed.png
        is a duplicate of
chrome/toolkit/skin/classic/mozapps/plugins/pluginGeneric-16.png
---
chrome/toolkit/skin/classic/mozapps/plugins/notifyPluginGeneric.png
        is a duplicate of
chrome/toolkit/skin/classic/mozapps/plugins/pluginGeneric-16.png
---
chrome/toolkit/skin/classic/mozapps/xpinstall/xpinstallItemGeneric.png
        is a duplicate of
chrome/toolkit/skin/classic/mozapps/extensions/category-extensions.png
---
chrome/toolkit/skin/classic/global/icons/tabprompts-bgtexture.png
        is a duplicate of
chrome/toolkit/skin/classic/mozapps/extensions/background-texture.png
---
chrome/toolkit/skin/classic/global/icons/notfound.png
        is a duplicate of
chrome/toolkit/skin/classic/global/icons/information-16.png
---
chrome/toolkit/skin/classic/global/arrow/arrow-rit-hov.gif
        is a duplicate of
chrome/toolkit/skin/classic/global/arrow/arrow-rit.gif
---
chrome/toolkit/skin/classic/global/arrow/arrow-lft-hov.gif
        is a duplicate of
chrome/toolkit/skin/classic/global/arrow/arrow-lft.gif
---
chrome/toolkit/skin/classic/global/console/bullet-question.png
        is a duplicate of
chrome/toolkit/skin/classic/global/icons/information-16.png
---
chrome/toolkit/skin/classic/global/console/bullet-error.png
        is a duplicate of
chrome/toolkit/skin/classic/global/icons/error-16.png
---
chrome/toolkit/skin/classic/global/console/bullet-warning.png
        is a duplicate of
chrome/toolkit/skin/classic/global/icons/warning-16.png
---
chrome/browser/skin/classic/browser/places/tag.png
        is a duplicate of
chrome/toolkit/skin/classic/mozapps/places/tagContainerIcon.png
---
chrome/en-US/locale/en-US/global-platform/win/intl.properties
        is a duplicate of
chrome/en-US/locale/en-US/global-platform/unix/intl.properties
---
chrome/en-US/locale/en-US/global-platform/win/platformKeys.properties
        is a duplicate of
chrome/en-US/locale/en-US/global-platform/unix/platformKeys.properties
---
chrome/en-US/locale/en-US/global-platform/unix/accessible.properties
        is a duplicate of
chrome/en-US/locale/en-US/global-platform/win/accessible.properties
---
chrome/en-US/locale/en-US/global-platform/mac/accessible.properties
        is a duplicate of
chrome/en-US/locale/en-US/global-platform/win/accessible.properties
---
chrome/en-US/locale/en-US/global-platform/mac/intl.properties
        is a duplicate of
chrome/en-US/locale/en-US/global-platform/unix/intl.properties
---
res/table-remove-column.gif
        is a duplicate of
res/table-remove-row.gif
---
res/table-remove-column-hover.gif
        is a duplicate of
res/table-remove-row-hover.gif
---
res/table-remove-row-active.gif
        is a duplicate of
res/table-remove-column-active.gif
---
chrome/browser/skin/classic/browser/feeds/feedIcon.png
        is a duplicate of
chrome/browser/skin/classic/browser/feeds/audioFeedIcon.png
---
chrome/browser/skin/classic/browser/feeds/videoFeedIcon16.png
        is a duplicate of
chrome/browser/skin/classic/browser/feeds/feedIcon16.png
---
chrome/browser/skin/classic/browser/feeds/audioFeedIcon16.png
        is a duplicate of
chrome/browser/skin/classic/browser/feeds/feedIcon16.png
---
chrome/browser/skin/classic/browser/feeds/videoFeedIcon.png
        is a duplicate of
chrome/browser/skin/classic/browser/feeds/audioFeedIcon.png
---
WARNING: Found 30 duplicate files

Comment 4

[(dormant account)]

should sum up total wasted space too

Comment 5

[Michael Wu [:mwu]]

Attachment: Detect duplicate files in optimizejars.py, v2

Comment 6

[Michael Wu [:mwu]]

WARNING: Found 30 duplicate files taking 58018 bytes

Note that about half of it is compressed data, and the other half is overhead from zip metadata.

Comment 7

[(dormant account)]

Comment on attachment 534894 [details] [diff] [review]
Detect duplicate files in optimizejars.py, v2

nice

Comment 8

[(no longer active)]

I landed this on mozilla-inbound.

Comment 9

[Michael Wu [:mwu]]

(In reply to comment #8)
> I landed this on mozilla-inbound.

I was waiting for fixes from bug 659407 to land before adding more noise to the build, but hey, why not..

Comment 10

[(no longer active)]

(In reply to comment #9)
> (In reply to comment #8)
> > I landed this on mozilla-inbound.
> 
> I was waiting for fixes from bug 659407 to land before adding more noise to the
> build, but hey, why not..

I can back it out if you really want, but let's be honest, our builds are not exactly silent.  ;-)

Comment 11

[Mounir Lamouri (:mounir)]

Pushed:
http://hg.mozilla.org/mozilla-central/rev/634f82bc05fb