260174 - xpinstall/packager/Makefile expects gnu tar

Status: RESOLVED INCOMPLETE

(Firefox Build System :: General, defect)

Description

[Ed Pilatowicz]

User-Agent:       Mozilla/5.0 (X11; U; SunOS i86pc; rv:1.7.3) Gecko/20040917 Firefox/0.10
Build Identifier: Mozilla/5.0 (X11; U; SunOS i86pc; rv:1.7.3) Gecko/20040917 Firefox/0.10

my solaris firefox builds break on when i try to build the final packages.

here's the output from my build log that illustrates the problem.
--
Compressing...
cd ../../dist; tar -c --owner=0 --group=0 --numeric-owner --mode="go-w" -f -
firefox | bzip2 -vf > firefox-sparc-sun-solaris2.10.tar.bz2
tar: /dev/rmt/0: Permission denied
  (stdin):  no data compressed.
a bin/ 0K
...
cd ../../dist && tar -c --owner=0 --group=0 --numeric-owner --mode="go-w" -f -
gecko-sdk | bzip2 -vf > gecko-sdk-sparc-sun-solaris2.10.tar.bz2
  (stdin): tar: /dev/rmt/0: Permission denied
 no data compressed.
--

if we look in xpinstall/packager/Makefile we see:
--
CREATE_FINAL_TAR = tar -c --owner=0 --group=0 --numeric-owner --mode="go-w" -f
--

this command line obviously assumse that tar is gnu tar.
on solaris this isn't the case.

it seems like there are a few possible solutions to this problem.

1) may of the other makefiles use tar.  they all make sure to use
   standard tar options (instead of gnu specific ones) that are
   portable.  this line could be made portable.

2) the configuration scripts could be updated to check for the
   presence of gnu tar, possibly with a switch of variable to
   overide the default setting.  (similar to the GMAKE environemnt
   variable.)

i'm sure the are other solutions as well.
but anything is better than my current situation, which is patching
the makefile to call gtar.  (gnu tar on solaris.)



Reproducible: Always
Steps to Reproduce:
1. do a full build and then do a build in xpinstall/packager on solaris
2.
3.

Actual Results:  
i was left with empty build archives.

Expected Results:  
created a valid build archive

Comment 1

[Roland Mainz]

This seems to happen for Mozilla, too.

Comment 2

[Roland Mainz]

Requesting blocking1.8a4 status.
This problem makes it impossible to make/use binary tarballs from Mozilla and
Firefox - the current Makefile makes GNU tar mandatory for packaging (which is
not available on Solaris) - and the resulting binary tarballs (when someone has
replaced the normal tar with GNU tar) cannot be unpacked properly on the user
side either (you will need GNU tar to unpack the tarballs on Solaris. BAD.).

Comment 3

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

If you're willing to provide a properly-ifdef'd patch that includes the
equivalent options for Solaris tar, we'd be happy to take it.  However, having
these options is important.

Comment 4

[Asa Dotzler [:asa]]

not going to block the release for this.

Comment 5

[Steve Zweep]

This affects packaging on FreeBSD as well. A quick fix is to change the line
starting with:

  CREATE_FINAL_TAR = tar -c  ...

to

  CREATE_FINAL_TAR = gtar -c  ...

Comment 6

[Marco Perez [:mpk]]

The last few FreeBSD releases shipped with both gtar and bsdtar.
tar itself was only a symlink pointing to bsdtar by default.
The upcoming FreeBSD 6.0 however will ship without gtar,
so unfortunately there's no easy workaround.

At least HP-UX seems to be affected as well (see bug #293609).

Comment 7

[Marco Perez [:mpk]]

Attachment: proposed patch

With this patch applied, the packager tries to do the following:
 - if tar is GNU tar, then use tar along with the original options 
   - if not, then check if gtar exists and if it is GNU tar
     (if yes, then use gtar along with the original options)
If everything fails, then set the permissions first and then use tar with a
minimum set of options. Unfortunately, no ownership can be set this way.

Comment 8

[Benjamin Smedberg]

Comment on attachment 203733 [details] [diff] [review]
proposed patch

If we need something like this we should be checking and setting TAR in configure tests.

Comment 9

[Marco Perez [:mpk]]

*** Bug 279660 has been marked as a duplicate of this bug. ***

Comment 10

[Marco Perez [:mpk]]

*** Bug 293609 has been marked as a duplicate of this bug. ***

Comment 11

[Marco Perez [:mpk]]

Changing OS to All since apart from Solaris virtually any *nix system where tar
isn't GNU tar (HP-UX and the BSD family come to mind) suffers from this problem.

Comment 12

[Marco Perez [:mpk]]

Attachment: new attempt at creating a suitable patch

I don't know whether we really need this bug fixed, but it would make the work
of some volunteers providing packages for our tier-2 and tier-3 platforms a
whole lot easier. They would just need to have a GNU-compatible tar installed,
but it wouldn't have to be tar. It could be gtar or gnutar just as well.

In order to increase compatibility when extracting our packages on such
platforms, I added the --format=ustar option. This shouldn't be a problem
(see http://www.gnu.org/software/tar/manual/html_node/tar_120.html#SEC120).

For the case where a builder doesn't have a GNU-compatible version of tar
around, we could still implement a routine to check whether our user id is
0, and in this case use chmod, chown and tar with a reduced set of options.
chown will not work if we are not root.

Benjamin, just in case you give the patch a thumbs up, could you please check
it in for me? I don't have cvs write access.

Comment 13

[Benjamin Smedberg]

Comment on attachment 205169 [details] [diff] [review]
new attempt at creating a suitable patch

Please use my official email ;-)

Comment 14

[Benjamin Smedberg]

Comment on attachment 205169 [details] [diff] [review]
new attempt at creating a suitable patch

This is fine except for the --format=ustar change, which is not accepted by tar on MacOS and probably some other systems.

Comment 15

[Marco Perez [:mpk]]

Attachment: same as last but with the format option removed

Then I guess it's best to just remove that option altogether.
(There is an option --portability, but this would impose further restrictions
 and I'm not so sure this would work with any version of tar.)

Comment 16

[Marco Perez [:mpk]]

Attachment: corrected version of the last patch

Oops, my bad. A glitch found its way into the last patch. Sorry for the spam.

Comment 17

[Alfred Peng]

*** Bug 330191 has been marked as a duplicate of this bug. ***

Comment 18

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

I just checked the patch in since nobody had; not sure whether to mark the bug fixed.

(Perhaps configure.in should be checking that the tar it finds is actually GNU tar and giving an error otherwise?)

Comment 19

[Marco Perez [:mpk]]

Thanks a lot for checking the patch in, David. I had somehow forgotten about
it (shame on me). The bug could actually be marked as fixed, as long as nobody
objects against it. But your idea about checking for GNU tar sounds interesting. How about something along the lines of:
    _tar_try=`$TAR --version 2>/dev/null | grep GNU`
    if test ! "$_tar_try"
    then
        echo
        echo "*** $TAR is not GNU tar. \
        You will not be able to build Mozilla without GNU tar."
        echo
        exit 1
    fi
I could whip up a small patch.

Comment 20

[Marco Perez [:mpk]]

BTW David, by checking Bonsai I just noticed that only two of the three files
were correctly modified. Checking in my patch seems to have missed this file:
 /cvsroot/mozilla/toolkit/mozapps/installer/packager.mk
The code must have rotted a bit since it got r+. That doesn't break anything,
but we still don't have the intented benefits. So the bug still ain't fixed.
I'll review and modify my patch. Depending upon any feedback about the GNU tar
check I may implement that as well.

Comment 21

[Dave Lam]

some investigation on Solaris 8/9/10,
1, GNU tar(gtar) was not delivered in Solaris until s9 and later release, s8 user probably installed the unbundled version from website, which might locate in /usr/local/bin/tar or /opt/csw/bin/tar
2, in s9/10, the bundled version, named "gtar", lives in /usr/sfw/bin, which is not in the default path
3, Solaris tar doesn't have equivalent options of "--owner, --group, --mode"

would like to know what's benefit of using '--owner=0 --group=0 --numeric-owner --mode="go-w"'? 

Comment 22

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

(In reply to comment #21)
> would like to know what's benefit of using '--owner=0 --group=0 --numeric-owner
> --mode="go-w"'? 

See bug 254303.

Comment 23

[Gregory Szorc [:gps]]

I'm pretty sure this has been addressed in some form in the 11 years since this bug was updated.