1780220 - Bootstrap on openSUSE Tumbleweed fails when trying to upgrade Mercurial

Status: RESOLVED FIXED

(Firefox Build System :: Bootstrap Configuration, defect)

Description

[Malte Jürgens [:mjurgens]]

When Mercurial is not installed during ./mach bootstrap, mach tries to install it. But it fails with the following error:

TypeError: 'OpenSUSEBootstrapper' object is not callable

  File "/root/firefox-102.0/python/mozboot/mozboot/mach_commands.py", line 43, in bootstrap
    bootstrapper.bootstrap(command_context.settings)
  File "/root/firefox-102.0/python/mozboot/mozboot/bootstrap.py", line 333, in bootstrap
    hg_installed, hg_modern = self.instance.ensure_mercurial_modern()
  File "/root/firefox-102.0/python/mozboot/mozboot/base.py", line 698, in ensure_mercurial_modern
    if self.upgrade_mercurial(version) is False:
  File "/root/firefox-102.0/python/mozboot/mozboot/opensuse.py", line 96, in upgrade_mercurial
    self(["pip3", "install", "--upgrade", "pip", "--user"])

A possible solution could be to just install mercurial from the openSUSE repositories or to revert the changes of D110960.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1703305

Comment 2

[Nicklas Boman [:smurfd]]

Hey maltejur , thanks for the find.

It is abit strange you get that error.
Because what pip3 install --upgrade pip --user and pip3 install --upgrade Mercurial --user does is upgrade the versions of pip and mercurial (and install those if not installed).
What the --user means, is that instead of overwriting the pip that zypper has installed in like /usr/bin/pip3, it installs a pip in your home folder, like : ~/.local/bin/pip which is wayyyy better than overwriting your system installed pip.

... oh, i see something. please dont run as root :) general linux tip of the day, please use your user and use sudo when you might need to be root.

However you should still be able to, even as root, run : pip3 install --upgrade pip --user and it would install a upgraded pip in /root/.local/bin
... try that from a command prompt.

But, according to the installation instructions, mercurial should be installed before bootstrapping :
https://firefox-source-docs.mozilla.org/setup/linux_build.html

so basicly your steps should be :

sudo zypper install python3 python3-devel
python3 -m pip install --user mercurial
echo "export PATH=\"$(python3 -m site --user-base)/bin:$PATH\"" >> ~/.bashrc

the last line is to add where pip installs stuff to your path, and for that to take effect you need to close the terminal and start another (or another tab)
Then if you dont have the mozilla source, then do:

curl https://hg.mozilla.org/mozilla-central/raw-file/default/python/mozboot/bin/bootstrap.py -O
python3 bootstrap.py

or in your case when you already have the source tree, assuming you are in the mozilla code folder.

./mach bootstrap

But, i tried this on a fresh installed Tumbleweed in a virtual machine. GCC is not installed, by default, which causes issues when trying to install mercurial via pip. And i think its like that for other distributions, like Debian, Ubuntu and fedora, aswell, not sure though.
And for some reason i notice that Watchman has been removed from tumbleweed (it is still in Leap)
you can install that via pip3 install watchman --user

Eh that became long, hopefully you got helped? need info me otherwise.
i might look into if other distributions install GCC by default, if not maby its needed to be added to the instructions.

Comment 3

[Malte Jürgens [:mjurgens]]

Thanks a lot for your length answer. But I know how to install Mercurial, I was more specifically reporting this issue that mach isn't able to do it automatically. You were the one that added the self(["pip3", "install", "--upgrade", "pip", "--user"]) line, but it seems that that syntax is wrong and doesn't even run the pip3 install --upgrade pip --user command. I think the only way to run a command currently is self.run_as_root and that wouldn't work for the command since it, as the name says, always runs the command as root.

... oh, i see something. please dont run as root :) general linux tip of the day, please use your user and use sudo when you might need to be root.

I was just running the build in docker, so thats why I am root here :D

But, according to the installation instructions, mercurial should be installed before bootstrapping

That is true, but if the functionality of installing Mercurial on demand is built into mach I think that should nevertheless work properly. And apart from that, the same issue probably occurs when mach tries to update Mercurial.

Comment 4

[Alex Hochheiden [:ahochheiden] (needinfo? me)]

(In reply to Malte Jürgens from comment #3)

the self(["pip3", "install", "--upgrade", "pip", "--user"]) line, but it seems that that syntax is wrong and doesn't even run the pip3 install --upgrade pip --user command. I think the only way to run a command currently is self.run_as_root and that wouldn't work for the command since it, as the name says, always runs the command as root.

You're right, the syntax is wrong and it will never work as-is. Looks like it just wasn't caught during code review for D110960.

You're also right that there isn't a function for just run, but what you want to use is subprocess.check_call (example here).

I'll also note that line 74 has a syntax error as well, it's just missing the parentheses (it should be self.zypper_update()).

Comment 5

[Malte Jürgens [:mjurgens]]

Thanks for the hint with subprocess.check_call, I'll create a patch using that.

Should I also just include the parentheses in that patch?

Comment 6

[Alex Hochheiden [:ahochheiden] (needinfo? me)]

Yeah, that's fine. Just make the description capture both changes/fixes.

Comment 7

[Malte Jürgens [:mjurgens]]

Ok. One other thing, all the other distros seem to just install Mercurial globally to the system (be it via pip or the system package manager). Is there any reason why things are done differently on openSUSE?

Comment 8

[Alex Hochheiden [:ahochheiden] (needinfo? me)]

I think there isn't any justification as to why the openSUSE bootstrapper is doing things differently from the rest of the bootstrappers. In general, we should aim for consistency. If there's a strong enough argument not to use root on one, the same change should be done to all. I don't have strong feelings one way or the other, but if openSUSE is the odd one out, it should probably follow suit with the rest.

On that note, openSUSE is also the only one that has install_mercurial, while all the rest only have upgrade_mercurial. And it doesn't even look like install_mercurial is ever used? It's probably safe to delete that.

Comment 9

[Malte Jürgens [:mjurgens]]

Attachment: Bug 1780220 - Prompt Mercurial installation on openSUSE like on Debian r?ahochheiden

• Removed install_mercurial, since it doesn't seem to be used.
• Rewrote upgrade_mercurial to act more like on Debian,
  meaning it will prompt the user if Mercurial should be
  installed via zypper or pip, and install via zypper if
  non-interactive. Everything is then installed system-
  wide.
• Added parantheses to self.zypper_update().

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/28becef2fe5d
Prompt Mercurial installation on openSUSE like on Debian r=ahochheiden

Comment 11

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/28becef2fe5d

Comment 12

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-esr102/rev/b8c3d804cabb