Open
Bug 1242592
Opened 9 years ago
Updated 2 years ago
JarWriter under Windows creates archives which cause no file permissions if extracted under Linux or Mac
Categories
(Firefox Build System :: General, defect)
Firefox Build System
General
Tracking
(Not tracked)
NEW
People
(Reporter: whimboo, Unassigned)
References
Details
mozfile.extract_zip() fails in setting the correct file permissions for archives which seem to be have created under Windows. You can test this on your Linux machine with the following archive:
http://ftp.mozilla.org/pub/firefox/nightly/latest-mozilla-central/firefox-46.0a1.en-US.win64.common.tests.zip
Just download and execute the following:
> import mozfile
> mozfile.extract_zip(%archive_path%, %target%)
In this case the external_attr and internal_attr property of each file is 0, which also results in a mode value of 0:
17:30:47 INFO - Using ZipFile to extract tests_win32.zip to b
bin/plugins/gmp-fakeopenh264/1.0/fakeopenh264.dll - 0 (0)
bin/plugins/gmp-fakeopenh264/1.0/fakeopenh264.info - 0 (0)
bin/plugins/gmp-fakeopenh264/1.0/fakeopenh264.voucher - 0 (0)
Compared to Linux:
17:30:47 INFO - Using ZipFile to extract tests_linux.zip to b
bin/plugins/gmp-fakeopenh264/1.0/fakeopenh264.info - 420 (2175008768)
bin/plugins/gmp-fakeopenh264/1.0/fakeopenh264.voucher - 420 (2175008768)
bin/plugins/gmp-fakeopenh264/1.0/libfakeopenh264.so - 493 (2179792896)
Especially under Linux and Mac the extracted files are NOT usable and cause our Firefox UI tests to fail, e.g.
> $ pip install -r config/firefox_ui_requirements.txt
> Could not open requirements file: [Errno 13] Permission denied: 'config/firefox_ui_requirements.txt'
Keep in mind that all is working fine for the test archive created under Linux and Mac. So I'm unsure if that is a bug in how we create the archives, which makes it invalid, or if we also have to adjust mozfile to correctly extract the files so it can cope with partial invalid archives.
This is something which you can see if you are trapped by bug 1239808.
My question is if we only set the permissions if the checked external_attr is not 0. This fixes the problem for me whatever platform zip I'm using.
Flags: needinfo?(ahalberstadt)
Comment 1•9 years ago
|
||
Sorry, I don't really know anything about this. Sounds like you have a patch that fixes it though?
Flags: needinfo?(ahalberstadt)
Comment 2•9 years ago
|
||
This feels like a problem with archive generation, which occurs as part of the build system.
We're using a Python-based archiving tool. It should be normalizing the permissions so things are consistent. It sounds like there is a bug there.
Component: Mozbase → Build Config
Product: Testing → Core
Comment 3•9 years ago
|
||
Here is the call stack for dealing with file mode:
https://hg.mozilla.org/mozilla-central/file/bd8bb6298d90/python/mozbuild/mozbuild/action/test_archive.py#l369
https://hg.mozilla.org/mozilla-central/file/bd8bb6298d90/python/mozbuild/mozpack/mozjar.py#l612
We appear to be grabbing and setting the file mode when archiving. I'm not sure why there is an inconsistency between Windows and Linux. Perhaps we should be normalizing the mode more. Let me debug this on my Windows machine to see if I can find anything interesting...
Reporter | ||
Comment 4•9 years ago
|
||
FYI when I move a file inside the archive to another folder, its permissions will be corrected and a following extraction will apply the correct permissions.
Reporter | ||
Updated•9 years ago
|
Summary: mozfile.extract_zip() does not always set the correct file permissions (e.g. archives created under Windows) → JarWriter under Windows creates archives which cause no file permissions if extracted under Linux or Mac
Reporter | ||
Comment 5•9 years ago
|
||
Hopefully bug 1239808 will be fixed soon, so it should not be a blocker for us anymore.
Whiteboard: [qa-automation-blocked]
Reporter | ||
Comment 6•9 years ago
|
||
We are no longer blocked on this issue given that bug 1239808 added platform dependent *.test_packages.json files.
Whiteboard: [qa-automation-blocked]
Updated•7 years ago
|
Product: Core → Firefox Build System
Updated•2 years ago
|
Severity: normal → S3
You need to log in
before you can comment on or make changes to this bug.
Description
•