1801153 - ./mach doc is broken - "mozlog 7.1.0 requires blessings, which is not installed."

Status: RESOLVED FIXED

(Firefox Build System :: Mach Core, defect)

Description

[Mark Banner (:standard8)]

On latest mozilla-central on Mac, run:

./mach doc

The command starts to run, but then fails with:

mozlog 7.1.0 requires blessings, which is not installed..

Pretty sure this is a regression from bug 1744340 based on the other bugs. Note, I do have the fix for bug 1800056 from the latest pull.

Comment 1

[Mark Banner (:standard8)]

I should have included these errors as well:

mach.site.InstallPipRequirementsException: As part of validation after installing "/Users/mark/dev/gecko/tools/moztreedocs/requirements.txt" into the "docs" site, the site appears to contain installed packages that are incompatible with each other.

  File "/Users/mark/dev/gecko/tools/moztreedocs/mach_commands.py", line 139, in build_docs
    command_context.virtualenv_manager.install_pip_requirements(
  File "/Users/mark/dev/gecko/python/mach/mach/site.py", line 677, in install_pip_requirements
    raise InstallPipRequirementsException(

Sentry event ID: 2cd269ff393a4210880b523bddf975b4

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1744340

:ahochheiden, since you are the author of the regressor, bug 1744340, could you take a look? Also, could you set the severity field?

For more information, please visit auto_nag documentation.

Comment 3

[Alex Hochheiden [:ahochheiden] (needinfo? me)]

Can you provide more of the log? I can't reproduce on either Windows or Mac.

copying static files... done
copying extra files... done
dumping search index in English (code: en)... done
dumping object inventory... done
build succeeded, 4115 warnings.

The HTML pages are in obj-x86_64-pc-mingw32\docs\html.

Also, try ./mach clobber --full and ./mach bootstrap just for sanity.

I'm surprised I can't reproduce, but I also still need to get the new version of mozlog onto PyPi. If you can still reproduce, maybe that will fix it?

Comment 4

[Mark Banner (:standard8)]

Attachment: Full log

I've run bootstrap & the full clobber, and I'm still getting the same. Here is the full log.

Comment 5

[Mark Banner (:standard8)]

I've also just updated to the latest m-c, I'm still also seeing bug 1801153 as well. So maybe that module does need updating?

Comment 6

[Alex Hochheiden [:ahochheiden] (needinfo? me)]

(In reply to Mark Banner (:standard8) from comment #5)

I've also just updated to the latest m-c, I'm still also seeing bug 1801153 as well. So maybe that module does need updating?

That's my best guess. :ahal should be taking care of it tomorrow (He's a maintainer on PyPi and I'm not). I'm just very confused as to why I can't reproduce this on either of my machines...

Comment 7

[Alex Hochheiden [:ahochheiden] (needinfo? me)]

The mozlog module was updated on PyPi, and Barret was still able to reproduce, so we did a long pair debugging session and we think we figured out the problem. Barret has had the same mozilla-unified for a very long time, and it looks like our 'first-party' modules used egg-info directories in the past (but seemingly no longer do?). We had to dig into the internals of pip to figure out that's where it was getting the wrong version number from, but the solution was to just delete the egg-info folder for mozlog (and mozterm) on the system, and everything worked.

That explains why Glandium and I (and most people, really) weren't able to reproduce. Anyone with a very old mozilla-unified directory probably has this problem.

I think the solution to this will probably be to update the .gitignore and .hgignore rules (and having anyone encountering this issue run hg purge afterwards), but I need to investigate a bit further before I make that change. I want to understand why we don't have/need egg-info directories for 'first-party' modules anymore.

For now, you should be able to navigate to mozilla-unified\testing\mozbase\mozlog and delete the egg-info directory there and this issue should go away (let me know if it doesn't).

Comment 8

[Drew Willcoxon :adw]

Using a new clone of mozilla-central fixes this bug for me and also try chooser in bug 1800564. Removing the egg-info dirs from mozlog and mozterm did not fix them, but I have a ton of those dirs elsewhere in my tree, so maybe if I removed them all it would.

Comment 9

[Mark Banner (:standard8)]

I've got it working after deleting some additional directories (see below).

What I actually did was to find all the *.egg-info directories (find . -name '*egg-info') and then delete them, then restored the ones in source control.

Thanks for finding this.

./xpcom/geckoprocesstypes_generator/geckoprocesstypes.egg-info
./python/mozterm/mozterm.egg-info
./testing/mozbase/mozprocess/mozprocess.egg-info
./testing/mozbase/mozrunner/mozrunner.egg-info
./testing/mozbase/mozhttpd/mozhttpd.egg-info
./testing/mozbase/mozfile/mozfile.egg-info
./testing/mozbase/moztest/moztest.egg-info
./testing/mozbase/mozdebug/mozdebug.egg-info
./testing/mozbase/mozversion/mozversion.egg-info
./testing/mozbase/mozprofile/mozprofile.egg-info
./testing/mozbase/mozdevice/mozdevice.egg-info
./testing/mozbase/mozinfo/mozinfo.egg-info
./testing/mozbase/mozproxy/mozproxy.egg-info
./testing/mozbase/mozleak/mozleak.egg-info
./testing/mozbase/moznetwork/moznetwork.egg-info
./testing/mozbase/manifestparser/manifestparser.egg-info
./testing/mozbase/mozgeckoprofiler/mozgeckoprofiler.egg-info
./testing/mozbase/mozpower/mozpower.egg-info
./testing/mozbase/mozscreenshot/mozscreenshot.egg-info
./testing/mozbase/mozcrash/mozcrash.egg-info
./testing/mozbase/mozinstall/mozInstall.egg-info
./testing/mozbase/mozlog/mozlog.egg-info

Comment 10

[Mark Banner (:standard8)]

The only issue I have now is that python seems to re-install the virtualenv on every run.

Comment 11

[Alex Hochheiden [:ahochheiden] (needinfo? me)]

(In reply to Mark Banner (:standard8) from comment #9)

I've got it working after deleting some additional directories (see below).

What I actually did was to find all the *.egg-info directories (find . -name '*egg-info') and then delete them, then restored the ones in source control.

Thanks for finding this.

./xpcom/geckoprocesstypes_generator/geckoprocesstypes.egg-info
./python/mozterm/mozterm.egg-info
./testing/mozbase/mozprocess/mozprocess.egg-info
./testing/mozbase/mozrunner/mozrunner.egg-info
./testing/mozbase/mozhttpd/mozhttpd.egg-info
./testing/mozbase/mozfile/mozfile.egg-info
./testing/mozbase/moztest/moztest.egg-info
./testing/mozbase/mozdebug/mozdebug.egg-info
./testing/mozbase/mozversion/mozversion.egg-info
./testing/mozbase/mozprofile/mozprofile.egg-info
./testing/mozbase/mozdevice/mozdevice.egg-info
./testing/mozbase/mozinfo/mozinfo.egg-info
./testing/mozbase/mozproxy/mozproxy.egg-info
./testing/mozbase/mozleak/mozleak.egg-info
./testing/mozbase/moznetwork/moznetwork.egg-info
./testing/mozbase/manifestparser/manifestparser.egg-info
./testing/mozbase/mozgeckoprofiler/mozgeckoprofiler.egg-info
./testing/mozbase/mozpower/mozpower.egg-info
./testing/mozbase/mozscreenshot/mozscreenshot.egg-info
./testing/mozbase/mozcrash/mozcrash.egg-info
./testing/mozbase/mozinstall/mozInstall.egg-info
./testing/mozbase/mozlog/mozlog.egg-info

Good to know that needing to delete more than just the one is necessary. My plan of updating the .gitignore and .hgignore lists should cover that.

(In reply to Mark Banner (:standard8) from comment #10)

The only issue I have now is that python seems to re-install the virtualenv on every run.

Could you file a separate bug with more details? The "Site out-of-date" reason with the details is important. Though, I might need to expand that to help debug this.

Comment 12

[Mike Hommey [:glandium]]

I don't know whether removing egg-info from the .{git,hg}ignore files is the right move. The main reason they are there is for a hg addremove/git add not to pick them.

Comment 13

[Alex Hochheiden [:ahochheiden] (needinfo? me)]

Attachment: WIP: Bug 1801153 - Do not add `.egg-info` directories to the ignore lists in `.hgignore` and `.gitignore`

Comment 14

[Alex Hochheiden [:ahochheiden] (needinfo? me)]

(In reply to Mike Hommey [:glandium] from comment #12)

I don't know whether removing egg-info from the .{git,hg}ignore files is the right move. The main reason they are there is for a hg addremove/git add not to pick them.

Okay, we can discuss alternatives in our meeting later today.

Comment 15

[Mark Banner (:standard8)]

(In reply to Alex Hochheiden [:ahochheiden] from comment #11)

(In reply to Mark Banner (:standard8) from comment #10)

The only issue I have now is that python seems to re-install the virtualenv on every run.

Could you file a separate bug with more details? The "Site out-of-date" reason with the details is important. Though, I might need to expand that to help debug this.

It seems to have settled down now. I'll keep an eye out for it happening and file if it happens again.

Comment 16

[Alex Hochheiden [:ahochheiden] (needinfo? me)]

Attachment: Bug 1801153 - If "pip check" fails, try purging all first-party ".egg-info" dirs then run "pip check" again r?#build

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7c0cc5c591a7
If "pip check" fails, try purging all first-party ".egg-info" dirs then run "pip check" again r=firefox-build-system-reviewers,glandium

Comment 18

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/7c0cc5c591a7