1505856 - Virtual machine build

Status: UNCONFIRMED

(Firefox Build System :: General, defect)

Description

[Vasco Pereira]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:51.0) Gecko/20100101 Firefox/51.0

Steps to reproduce:

Downloaded the virtual machine for developing in firefox.
Ran the virtual machine using Virtual Box, RAM: 5120MB, Number of processor: 4.
The console opened up and proceded to install sublime 2 and download the source.
O Sublime 2, I clicked on Tools -> Build and the build failed


Actual results:

Adding client.mk options from /home/firefox-dev/mozilla-central/.mozconfig:
    MOZ_CO_PROJECT=browser
    AUTOCLOBBER=1
    MOZ_OBJDIR=/home/firefox-dev/mozilla-central/objdir-frontend
    OBJDIR=/home/firefox-dev/mozilla-central/objdir-frontend
    FOUND_MOZCONFIG=/home/firefox-dev/mozilla-central/.mozconfig
Collecting taskcluster==0.0.32
  Using cached https://files.pythonhosted.org/packages/ff/b1/6fa388ce8fc3340e059d1d3a83692d142a0b4eb1c3c7417e94210a35baa0/taskcluster-0.0.32.tar.gz
Requirement already satisfied (use --upgrade to upgrade): requests<3,>=2.4.3 in /home/firefox-dev/mozilla-central/python/requests (from taskcluster==0.0.32)
Collecting PyHawk_with_a_single_extra_commit==0.1.5 (from taskcluster==0.0.32)
  Could not find a version that satisfies the requirement PyHawk_with_a_single_extra_commit==0.1.5 (from taskcluster==0.0.32) (from versions: )
No matching distribution found for PyHawk_with_a_single_extra_commit==0.1.5 (from taskcluster==0.0.32)
You are using pip version 8.1.2, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Error running mach:

    ['--log-no-times', 'artifact', 'install']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
 
 You should consider filing a bug for this issue.

 If filing a bug, please include the full output of mach, including this error
 message.

The details of the failure are as follows:

CalledProcessError: Command '[u'/home/firefox-dev/mozilla-central/objdir-frontend/_virtualenv/bin/pip', u'install', u'--use-wheel', u'taskcluster==0.0.32']' returned non-zero exit status 1

  File "/home/firefox-dev/mozilla-central/python/mozbuild/mozbuild/mach_commands.py", line 1554, in artifact_install
    artifacts = self._make_artifacts(tree=tree, job=job, skip_cache=skip_cache)
  File "/home/firefox-dev/mozilla-central/python/mozbuild/mozbuild/mach_commands.py", line 1504, in _make_artifacts
    self._install_pip_package(package)
  File "/home/firefox-dev/mozilla-central/python/mozbuild/mozbuild/mach_commands.py", line 1492, in _install_pip_package
    self.virtualenv_manager.install_pip_package(package)
  File "/home/firefox-dev/mozilla-central/python/mozbuild/mozbuild/virtualenv.py", line 489, in install_pip_package
    return self._run_pip(args)
  File "/home/firefox-dev/mozilla-central/python/mozbuild/mozbuild/virtualenv.py", line 525, in _run_pip
    stderr=subprocess.STDOUT)
  File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
    raise CalledProcessError(retcode, cmd)
Makefile:233: recipe for target 'recurse_artifact' failed
make[4]: *** [recurse_artifact] Error 1
/home/firefox-dev/mozilla-central/config/recurse.mk:32: recipe for target 'artifact' failed
make[3]: *** [artifact] Error 2
/home/firefox-dev/mozilla-central/config/rules.mk:522: recipe for target 'default' failed
make[2]: *** [default] Error 2
/home/firefox-dev/mozilla-central/client.mk:419: recipe for target 'realbuild' failed
make[1]: *** [realbuild] Error 2
client.mk:170: recipe for target 'build' failed
make: *** [build] Error 2
0 compiler warnings present.


Expected results:

I would have expected for the build to run without problems, as I have not changed any code.

Comment 1

[Andre Klapper]

> Downloaded the virtual machine for developing in firefox.

Which one, from where exactly? Steps to reproduce very welcome. :)

Comment 2

[laszlo.bialis]

Hi,

Can you please give us the details requested in comment 1? Also it would be a lot of help if you would follow the steps suggested in this guideline https://developer.mozilla.org/en-US/docs/Mozilla/QA/Bug_writing_guidelines

Comment 3

[Vasco Pereira]

Hello,
Sorry for taking this long to reply.
I've followed the exact steps detailed in the following guide https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Using_the_VM 

1. Install the most recent VirtualBox. 
2. Download the Firefox development environment VM. 
3. Adjust to settings of the VM to 5120MB of RAM and 4 processor cores.
4. Open the VM in VirtualBox
5. Let the setup script run, which downloads the source code and also installs and sets up Sublime 2.
6. Sublime 2 opens up by itself 
7. In sublime 2 go to Tools -> Build
8. It doesn't build

I also updated the OS afterwards, but it didn't change a thing.

Comment 4

[haotian xiang]

I got the same problems

```
 0:00.18 /usr/bin/make -f client.mk -s
 0:00.94 Adding client.mk options from /home/firefox-dev/mozilla-central/.mozconfig-debug:
 0:00.94     MOZ_CO_PROJECT=browser
 0:00.94     AUTOCLOBBER=1
 0:00.94     MOZ_OBJDIR=/home/firefox-dev/mozilla-central/objdir-frontend-debug
 0:00.94     OBJDIR=/home/firefox-dev/mozilla-central/objdir-frontend-debug
 0:00.94     FOUND_MOZCONFIG=/home/firefox-dev/mozilla-central/.mozconfig-debug
 0:02.06 Collecting taskcluster==0.0.32
 0:02.12   Using cached https://files.pythonhosted.org/packages/ff/b1/6fa388ce8fc3340e059d1d3a83692d142a0b4eb1c3c7417e94210a35baa0/taskcluster-0.0.32.tar.gz
 0:02.47 Requirement already satisfied (use --upgrade to upgrade): requests<3,>=2.4.3 in /home/firefox-dev/mozilla-central/python/requests (from taskcluster==0.0.32)
 0:02.49 Collecting PyHawk_with_a_single_extra_commit==0.1.5 (from taskcluster==0.0.32)
 0:02.68   Could not find a version that satisfies the requirement PyHawk_with_a_single_extra_commit==0.1.5 (from taskcluster==0.0.32) (from versions: )
 0:02.68 No matching distribution found for PyHawk_with_a_single_extra_commit==0.1.5 (from taskcluster==0.0.32)
 0:02.87 You are using pip version 8.1.2, however version 18.1 is available.
 0:02.87 You should consider upgrading via the 'pip install --upgrade pip' command.
 0:02.91 Error running mach:
 0:02.91 
 0:02.91     ['--log-no-times', 'artifact', 'install']
 0:02.91 
 0:02.91 The error occurred in code that was called by the mach command. This is either
 0:02.91 a bug in the called code itself or in the way that mach is calling it.
 0:02.91 
 0:02.91 You should consider filing a bug for this issue.
 0:02.91 
 0:02.91 If filing a bug, please include the full output of mach, including this error
 0:02.91 message.
 0:02.91 
 0:02.91 The details of the failure are as follows:
 0:02.91 
 0:02.91 CalledProcessError: Command '[u'/home/firefox-dev/mozilla-central/objdir-frontend-debug/_virtualenv/bin/pip', u'install', u'--use-wheel', u'taskcluster==0.0.32']' returned non-zero exit status 1
 0:02.91 
 0:02.91   File "/home/firefox-dev/mozilla-central/python/mozbuild/mozbuild/mach_commands.py", line 1554, in artifact_install
 0:02.91     artifacts = self._make_artifacts(tree=tree, job=job, skip_cache=skip_cache)
 0:02.91   File "/home/firefox-dev/mozilla-central/python/mozbuild/mozbuild/mach_commands.py", line 1504, in _make_artifacts
 0:02.91     self._install_pip_package(package)
 0:02.91   File "/home/firefox-dev/mozilla-central/python/mozbuild/mozbuild/mach_commands.py", line 1492, in _install_pip_package
 0:02.91     self.virtualenv_manager.install_pip_package(package)
 0:02.91   File "/home/firefox-dev/mozilla-central/python/mozbuild/mozbuild/virtualenv.py", line 489, in install_pip_package
 0:02.91     return self._run_pip(args)
 0:02.91   File "/home/firefox-dev/mozilla-central/python/mozbuild/mozbuild/virtualenv.py", line 525, in _run_pip
 0:02.91     stderr=subprocess.STDOUT)
 0:02.91   File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
 0:02.91     raise CalledProcessError(retcode, cmd)
 0:03.00 Makefile:233: recipe for target 'recurse_artifact' failed
 0:03.00 make[4]: *** [recurse_artifact] Error 1
 0:03.00 /home/firefox-dev/mozilla-central/config/recurse.mk:32: recipe for target 'artifact' failed
 0:03.00 make[3]: *** [artifact] Error 2
 0:03.00 /home/firefox-dev/mozilla-central/config/rules.mk:522: recipe for target 'default' failed
 0:03.01 make[2]: *** [default] Error 2
 0:03.01 /home/firefox-dev/mozilla-central/client.mk:419: recipe for target 'realbuild' failed
 0:03.01 make[1]: *** [realbuild] Error 2
 0:03.02 client.mk:170: recipe for target 'build' failed
 0:03.02 make: *** [build] Error 2
 0:03.02 0 compiler warnings present.
[Finished in 3.3s with exit code 2]
```

I just downloaded a new virtual machine and did what tutorial told.

Comment 5

[Jonathan Larmour]

I am trying to rebuild the firefox 52 esr release and have also hit this problem.

The problem seems to be because the python module pyhawk-with-a-single-extra-commit does not appear to exist on pypi.org any more. Google found me a transcript of a build which got past this, at https://readthedocs.org/projects/mozilla-ci-tools/builds/3403420/ and that mentions the link to get the module: https://files.pythonhosted.org/packages/source/P/PyHawk-with-a-single-extra-commit/PyHawk-with-a-single-extra-commit-0.1.5.tar.gz

But there is now nothing at that URL. Do stable releases really depend on external websites whose files can come and go? I thought builds should be fully reproducible as a matter of good practice?

How do we now get hold of (and then use in the build) this module?

Thanks

J