1115978 - Error building Firefox from mozilla-central

Status: RESOLVED DUPLICATE of bug 1072521

(Firefox Build System :: General, defect)

Description

[Projjol [:dorsatum]]

Hi, I'd pulled in from m-c recently and on running ./mach build, I received the following error:

Error running mach:

    ['build']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0: ordinal not in range(128)

  File "/home/dorsatum/mozilla-central/python/mozbuild/mozbuild/mach_commands.py", line 387, in build
    silent=not verbose)
  File "/home/dorsatum/mozilla-central/python/mozbuild/mozbuild/base.py", line 419, in _run_make
    args = self._make_path()
  File "/home/dorsatum/mozilla-central/python/mozbuild/mozbuild/base.py", line 528, in _make_path
    make = which.which(test)
  File "/home/dorsatum/mozilla-central/python/which/which.py", line 246, in which
    match = whichgen(command, path, verbose, exts).next()
  File "/home/dorsatum/mozilla-central/python/which/which.py", line 201, in whichgen
    os.path.normpath(os.path.join(dirName, command+ext)))
  File "/usr/lib/python2.7/posixpath.py", line 71, in join
    path += '/' + b
^CError in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
Process Process-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    func(*targs, **kargs)
  File "/usr/lib/python2.7/multiprocessing/util.py", line 295, in _exit_function
    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/dorsatum/mozilla-central/testing/mozbase/mozsystemmonitor/mozsystemmonitor/resourcemonitor.py", line 55, in _collect
    p.join()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 145, in join
    res = self._popen.wait(timeout)
  File "/usr/lib/python2.7/multiprocessing/forking.py", line 148, in wait
    while not pipe.poll(sleep_interval):
KeyboardInterrupt
    return self.poll(0)
  File "/usr/lib/python2.7/multiprocessing/forking.py", line 133, in poll
    pid, sts = os.waitpid(self.pid, flag)
KeyboardInterrupt
Error in sys.exitfunc:
Traceback (most recent call last):
  File "/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/usr/lib/python2.7/multiprocessing/util.py", line 295, in _exit_function
    p.join()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 145, in join
    res = self._popen.wait(timeout)
  File "/usr/lib/python2.7/multiprocessing/forking.py", line 148, in wait
    return self.poll(0)
  File "/usr/lib/python2.7/multiprocessing/forking.py", line 133, in poll
    pid, sts = os.waitpid(self.pid, flag)
KeyboardInterrupt

The ^C and KeyboardInterrupt errors are because ./mach build hangs and I stopped it manually.

Comment 1

[Gregory Szorc [:gps]]

This smells like one of {gmake, make, mozmake, gnumake} exists in a path that has Unicode characters. There are a number of bugs in the build system where paths containing Unicode characters fail. This one smells like a bug in the 3rd party "which" Python package.

Try moving binaries out of paths with Unicode characters as a workaround. Installing a symlink in /usr/local/bin, etc should be sufficient.

Comment 2

[Projjol [:dorsatum]]

Hi :gps , thank you for looking into this. I am not very clear as to how I should proceed with this. Should I make a symbolic link for the 'which' package into /usr/local/bin ?

Comment 3

[Gregory Szorc [:gps]]

No, it is your make binary that appears to reside in a path with unicode characters. If that's not it, then it is your source or object directory that is to blame. Read comment 1. Manually run `which` from the shell until you find the binary being reported in a path with unicode. Make a symlink to that from a non-unicode path.

Comment 4

[Projjol [:dorsatum]]

Thank you for clearing that out. Steps that I have taken are:
a)Used which to see the directory where make,gnumake, mozmake and gmake are present. Currently my machine only has make, and it resides in /usr/bin, so that should be fine.
b) My source directory for m-c is ~/mozilla-central and the object directory is ~/mozilla-central/obj-x86_64-unknown-linux-gnu. ~ expands to /home/dorsatum. This should be fine too, right? Or am I looking at the wrong things. 
Pardon me for extending what should be a simple fix into something longer.

Comment 5

[Gregory Szorc [:gps]]

I'm at a loss to understand what's going on here.

If you know how to debug Python, it would really help to see what the raw value being manipulated is.

Try this:

1) mach --debug-command build
2) Inside the pdb terminal, type:

b /home/dorsatum/mozilla-central/python/which/which.py:201
c

This will set a breakpoint on the line that is failing.

When the breakpoint is hit, type |p dirName| |p command| to print the raw path components. Type |c| to continue the debugger until the next point (I'm not sure how many times this function will get called before it throws an error).

If you can capture the values containing Unicode/triggering an exception, it would help immensely.

Comment 6

[Projjol [:dorsatum]]

Is there an IRC channel you're in where I could speak to you about this?

Comment 7

[Projjol [:dorsatum]]

Pairs of p dirName and p command uptil the point where the error is thrown: https://pastebin.mozilla.org/8142731

Comment 8

[Mike Hommey [:glandium]]

Does your $PATH contain unicode characters?

Comment 9

[Mike Hommey [:glandium]]

'\xe2\x80\x9d\xe2\x80\x9c/home/dorsatum/adt-bundle-linux/sdk/tools\xe2\x80\x9d'

That's '”“/home/dorsatum/adt-bundle-linux/sdk/tools”' Note how both ” and “ are not ". Remove them from your $PATH.

Comment 11

[Daniel Miller]

A sincere thanks to Gregory Szorc and Mike Hommey. I was having this exact issue I've spent over 6 hours this past week trying to fix it! And then I read this, and now I can stop building and start contributing to Firefox. So thank-you.
- Dan