Closed Bug 1401166 Opened 3 years ago Closed 3 years ago

./mach bootstrap fails to install cargo package on BSDs

Categories

(Firefox Build System :: General, defect)

Product:
Firefox Build System
Component:
General
Platform:
Unspecified
FreeBSD
Type:
defect
Priority:
Not set

Tracking

(firefox-esr52 unaffected, firefox55 wontfix, firefox56 fixed, firefox57 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla57
Tracking Flags:
Tracking Status
firefox-esr52 --- unaffected
firefox55 --- wontfix
firefox56 --- fixed
firefox57 --- fixed

People

(Reporter: jbeich, Assigned: jbeich)

Details

(Whiteboard: [npotb])

Attachments

(3 files)

Bug 1401166 - Drop obsolete cargo dependency from ./mach bootstrap on BSDs.
59 bytes, text/x-review-board-request
rillian
: review+
lizzard
: approval-mozilla-release+
Details
Bug 1401166 - Simplify llvm40 dependency for ./mach bootstrap on FreeBSD.
59 bytes, text/x-review-board-request
rillian
: review+
lizzard
: approval-mozilla-release+
Details
Bug 1401166 - Skip Stylo dependency with ./mach bootstrap on OpenBSD. f=gaston
59 bytes, text/x-review-board-request
rillian
: review+
lizzard
: approval-mozilla-release+
Details 
cargo is no longer a separate package but a part of "rust" on FreeBSD and OpenBSD.

$ ./mach bootstrap
[...]
Please choose the version of Firefox you want to build:
1. Firefox for Desktop Artifact Mode
2. Firefox for Desktop
3. Firefox for Android Artifact Mode
4. Firefox for Android
Your choice: 2
Executing as root: su root -c "pkg install autoconf213 cargo gmake gtar mercurial pkgconf rust watchman zip"
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
pkg: No packages available to install matching 'cargo' have been found in the repositories
Error running mach:

    ['bootstrap']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

CalledProcessError: Command '[u'su', u'root', u'-c', u'pkg install autoconf213 cargo gmake gtar mercurial pkgconf rust watchman zip']' returned non-zero exit status 70

  File "python/mozboot/mozboot/mach_commands.py", line 32, in bootstrap
    bootstrapper.bootstrap()
  File "python/mozboot/mozboot/bootstrap.py", line 292, in bootstrap
    self.instance.install_system_packages()
  File "python/mozboot/mozboot/freebsd.py", line 51, in install_system_packages
    self.pkg_install(*self.packages)
  File "python/mozboot/mozboot/freebsd.py", line 48, in pkg_install
    self.run_as_root(command)
  File "python/mozboot/mozboot/base.py", line 300, in run_as_root
    subprocess.check_call(command, stdin=sys.stdin)
  File "/usr/local/lib/python2.7/subprocess.py", line 186, in check_call
    raise CalledProcessError(retcode, cmd)

https://github.com/freebsd/freebsd-ports/commit/0b663e6dddf67c26d69ee9e838a493969939381c
https://github.com/openbsd/ports/commit/5057038dd9dea7c0db2c6727aed89f0e52da61cb
https://github.com/openbsd/ports/commit/da08f3aa7aab21e4c6d96e0679846be6426852aa
Comment on attachment 8909748 [details]
Bug 1401166 - Skip Stylo dependency with ./mach bootstrap on OpenBSD.  f=gaston

Does this unbreak "./mach bootstrap" on OpenBSD?
Attachment #8909748 - Flags: feedback?(landry)
Comment on attachment 8909748 [details]
Bug 1401166 - Skip Stylo dependency with ./mach bootstrap on OpenBSD.  f=gaston

I don't really use mach bootstrap, but probably.
Attachment #8909748 - Flags: feedback?(landry) → feedback+
Comment on attachment 8909746 [details]
Bug 1401166 - Drop obsolete cargo dependency from ./mach bootstrap on BSDs.

https://reviewboard.mozilla.org/r/181244/#review186568

Does this apply to released installs as well as new ones?
Attachment #8909746 - Flags: review?(giles) → review+
Comment on attachment 8909747 [details]
Bug 1401166 - Simplify llvm40 dependency for ./mach bootstrap on FreeBSD.

https://reviewboard.mozilla.org/r/181246/#review186572

Note configure still prefers gcc to clang if it's in the PATH. I assume if llvm40 supplies libclang for stylo is also supplies the `clang` front-end, and developers won't normally have gcc on their system?
Attachment #8909747 - Flags: review?(giles) → review+
Comment on attachment 8909748 [details]
Bug 1401166 - Skip Stylo dependency with ./mach bootstrap on OpenBSD.  f=gaston

https://reviewboard.mozilla.org/r/181248/#review186576
Attachment #8909748 - Flags: review?(giles) → review+
Comment on attachment 8909746 [details]
Bug 1401166 - Drop obsolete cargo dependency from ./mach bootstrap on BSDs.

https://reviewboard.mozilla.org/r/181244/#review186568

Yes. FreeBSD only supports 2 sets of packages per-architecture for each major[1] release: /latest and /quarterly. The latter is rerolled from /latest on each quarter. 2017Q3 still has cargo package but will reach EOL on 2017-10-01. Firefox 56 is scheduled on 2017-09-28, so 2 day window of broken "./mach bootstrap" in some scenarios can be ignored.

[1] Packages are built on the oldest supported minor release e.g., FreeBSD 10.3 and 10.4 or 11.0 and 11.1 currently share packages. Once 11.0 reaches EOL on 2017-10-26, 11.1 packages would be built on 11.1.
Comment on attachment 8909747 [details]
Bug 1401166 - Simplify llvm40 dependency for ./mach bootstrap on FreeBSD.

https://reviewboard.mozilla.org/r/181246/#review186572

FreeBSD doesn't have GCC installed by default on i386 (Tier1), amd64 (Tier1), armv6 (Tier2), aarch64 (Tier2). If GCC is later installed from ports/packages as a dependency "gcc" and "g++" symlinks are often excluded. On powerpc* (Tier2), sparc64 (Tier2), mips* base GCC 4.2.1 is too old to be usable for building Firefox, not to mention, big-endian issues and lack of Rust bootstrap.

Building on FreeBSD 10.* requires explicitly setting CC/CXX/CPP via .mozconfig (or environment) as base Clang 3.4.1 is too old. Ideally, it should be advertised somewhere or auto-tried similar to llvm-config*.
Keywords: checkin-needed
Whiteboard: [npotb]
Comment on attachment 8909746 [details]
Bug 1401166 - Drop obsolete cargo dependency from ./mach bootstrap on BSDs.

Approval Request Comment
[Feature/Bug causing the regression]: Keeping up with downstream packaging
[User impact if declined]: ./mach bootstrap aborts installing dependencies required to build Firefox
[Is this code covered by automated tests?]: NPOTB
[Has the fix been verified in Nightly?]: Verified on FreeBSD via pristine jail.
[Needs manual test from QE? If yes, steps to reproduce]: No
[List of other uplifts needed for the feature/fix]: None
[Is the change risky?]: No
[Why is the change risky/not risky?]: Can only break ./mach bootstrap due to a typo causing syntax error
[String changes made/needed]: None.
Attachment #8909746 - Flags: approval-mozilla-beta?
Comment on attachment 8909748 [details]
Bug 1401166 - Skip Stylo dependency with ./mach bootstrap on OpenBSD.  f=gaston

Approval Request Comment
[Feature/Bug causing the regression]: bug 1310852 regression
[User impact if declined]: ./mach bootstrap always fails on OpenBSD with: NotImplementedError: mozboot.base does not yet implement ensure_stylo_packages()
[Is this code covered by automated tests?]: NPOTB
[Has the fix been verified in Nightly?]: No but tested via similar change done on FreeBSD.
[Needs manual test from QE? If yes, steps to reproduce]: No
[List of other uplifts needed for the feature/fix]: None
[Is the change risky?]: No
[Why is the change risky/not risky?]: Can only break ./mach bootstrap due to a typo causing syntax error.
[String changes made/needed]: None
Attachment #8909748 - Flags: approval-mozilla-beta?
Assignee: nobody → jbeich
Pushed by ryanvm@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/4dbee711378f
Drop obsolete cargo dependency from ./mach bootstrap on BSDs. r=rillian
https://hg.mozilla.org/integration/autoland/rev/65a459d1739b
Simplify llvm40 dependency for ./mach bootstrap on FreeBSD. r=rillian
https://hg.mozilla.org/integration/autoland/rev/991e8f1b7a0a
Skip Stylo dependency with ./mach bootstrap on OpenBSD. r=rillian f=gaston
Keywords: checkin-needed
Comment on attachment 8909747 [details]
Bug 1401166 - Simplify llvm40 dependency for ./mach bootstrap on FreeBSD.

Approval Request Comment
[Feature/Bug causing the regression]: bug 1290647 regression
[User impact if declined]: Startup crashes if libxul.so indirectly pulls libstdc++.so after Firefox was built by GCC. Affects i386, amd64, armv6, aarch64.
[Is this code covered by automated tests?]: NPOTB
[Has the fix been verified in Nightly?]: Applied and verified in a pristine jail.
[Needs manual test from QE? If yes, steps to reproduce]: No
[List of other uplifts needed for the feature/fix]: None
[Is the change risky?]: No
[Why is the change risky/not risky?]: Can only break ./mach bootstrap due to a typo causing syntax error.
[String changes made/needed]: None. FreeBSD uses generic Unix instructions for building via ./mach bootstrap + ./mach build.
Attachment #8909747 - Flags: approval-mozilla-beta?
Comment on attachment 8909746 [details]
Bug 1401166 - Drop obsolete cargo dependency from ./mach bootstrap on BSDs.

https://reviewboard.mozilla.org/r/181244/#review186658
Comment on attachment 8909746 [details]
Bug 1401166 - Drop obsolete cargo dependency from ./mach bootstrap on BSDs.

56 is on mozilla-release now.
Attachment #8909746 - Flags: approval-mozilla-beta? → approval-mozilla-release?
Attachment #8909747 - Flags: approval-mozilla-beta? → approval-mozilla-release?
Attachment #8909748 - Flags: approval-mozilla-beta? → approval-mozilla-release?
Comment on attachment 8909746 [details]
Bug 1401166 - Drop obsolete cargo dependency from ./mach bootstrap on BSDs.

Fix for BSD linux builds only, should be ok for uplift to m-r.
Attachment #8909746 - Flags: approval-mozilla-release? → approval-mozilla-release+
Attachment #8909747 - Flags: approval-mozilla-release? → approval-mozilla-release+
Attachment #8909748 - Flags: approval-mozilla-release? → approval-mozilla-release+
(In reply to Liz Henry (:lizzard) (needinfo? me) from comment #26)

> Fix for BSD linux builds only, should be ok for uplift to m-r.

sigh.. 'BSD linux' isnt a thing, please :)
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.