Open Bug 1015570 Opened 10 years ago Updated 2 years ago

"ERROR PROCESSING MOZBUILD FILE [...] The error appears to be part of the mozbuild.frontend.reader Python module itself!" --> mozilla/media/libyuv/moz.build

Categories

(Firefox Build System :: General, defect)

Product:
Firefox Build System
Component:
General
Platform:
x86_64
Windows XP
Type:
defect

Tracking

(Not tracked)

People

(Reporter: Taraman, Unassigned)

References

Details

Building COmm-Central on WInXP I get the following error:

 3:56.44 Reticulating splines...
 3:56.44 d:\mozilla\objdir-tb\mozilla\_virtualenv\Scripts\python.exe: can't open
 file './build/dir_exists.py': [Errno 2] No such file or directory
 3:56.45
 3:56.45 Traceback (most recent call last):
 3:56.47   File "./config.status", line 918, in <module>
 3:56.47     config_status(**args)
 3:56.48   File "d:\mozilla\comm-central\mozilla\python\mozbuild\mozbuild\config
_status.py", line 148, in config_status
 3:56.50     summary = the_backend.consume(definitions)
 3:56.50   File "d:\mozilla\comm-central\mozilla\python\mozbuild\mozbuild\backen
d\base.py", line 186, in consume
 3:56.52     for obj in objs:
 3:56.53   File "d:\mozilla\comm-central\mozilla\python\mozbuild\mozbuild\fronte
nd\emitter.py", line 105, in emit
 3:56.55     for out in output:
 3:56.56   File "d:\mozilla\comm-central\mozilla\python\mozbuild\mozbuild\fronte
nd\reader.py", line 726, in read_mozbuild
 3:56.58     raise bre
 3:56.58 mozbuild.frontend.reader.BuildReaderError: ============================
==
 3:56.59 ERROR PROCESSING MOZBUILD FILE
 3:56.61 ==============================
 3:56.61
 3:56.62 The error occurred while processing the following file:
 3:56.62
 3:56.64     d:/mozilla/comm-central/mozilla/media/libyuv/moz.build
 3:56.64
 3:56.66 The error appears to be part of the mozbuild.frontend.reader Python mod
ule itself! It is possible you have stumbled across a legitimate bug.
 3:56.67
 3:56.69 Traceback (most recent call last):
 3:56.70   File "d:\mozilla\comm-central\mozilla\python\mozbuild\mozbuild\fronte
nd\reader.py", line 722, in read_mozbuild
 3:56.72     descend=descend, metadata=metadata):
 3:56.73   File "d:\mozilla\comm-central\mozilla\python\mozbuild\mozbuild\fronte
nd\reader.py", line 819, in _read_mozbuild
 3:56.75     non_unified_sources = non_unified_sources):
 3:56.77   File "d:\mozilla\comm-central\mozilla\python\mozbuild\mozbuild\fronte
nd\gyp_reader.py", line 111, in read_from_gyp
 3:56.80     params=params)
 3:56.81   File "d:\mozilla\comm-central\mozilla\media\webrtc\trunk\tools\gyp\py
lib\gyp\__init__.py", line 132, in Load
 3:56.84     params['parallel'])
 3:56.84   File "d:\mozilla\comm-central\mozilla\media\webrtc\trunk\tools\gyp\py
lib\gyp\input.py", line 2560, in Load
 3:56.88     variables, includes, depth, check, True)
 3:56.89   File "d:\mozilla\comm-central\mozilla\media\webrtc\trunk\tools\gyp\py
lib\gyp\input.py", line 391, in LoadTargetBuildFile
 3:56.91     build_file_data, PHASE_EARLY, variables, build_file_path)
 3:56.92   File "d:\mozilla\comm-central\mozilla\media\webrtc\trunk\tools\gyp\py
lib\gyp\input.py", line 1142, in ProcessVariablesAndConditionsInDict
 3:56.95     variables, build_file, 'variables')
 3:56.95   File "d:\mozilla\comm-central\mozilla\media\webrtc\trunk\tools\gyp\py
lib\gyp\input.py", line 1149, in ProcessVariablesAndConditionsInDict
 3:56.97     expanded = ExpandVariables(value, phase, variables, build_file)
 3:56.98   File "d:\mozilla\comm-central\mozilla\media\webrtc\trunk\tools\gyp\py
lib\gyp\input.py", line 881, in ExpandVariables
 3:56.98     (contents, p.returncode))
 3:57.00 GypError: Call to 'd:\mozilla\objdir-tb\mozilla\_virtualenv\Scripts\pyt
hon.exe ./build/dir_exists.py ./third_party/platformsdk_win7/files/redist/x86' r
eturned exit status 2. while trying to load d:\mozilla\comm-central\mozilla\medi
a\libyuv\libyuv.gyp
 3:57.02
 3:57.03 configure: error: d:/mozilla/comm-central/mozilla/configure failed for
mozilla
 3:57.05 *** Fix above errors and then restart with               "c:/mozilla-bu
ild/python/python2.7.exe d:/mozilla/comm-central/mozilla/build/pymake/pymake/../
make.py -f client.mk build"
 3:57.08 d:\mozilla\comm-central\client.mk:351:0: command 'cd d:/mozilla/objdir-
tb &&  MAKE="c:/mozilla-build/python/python2.7.exe d:/mozilla/comm-central/mozil
la/build/pymake/pymake/../make.py"  d:/mozilla/comm-central/configure  \
 3:57.11   || ( echo "*** Fix above errors and then restart with\
 3:57.11                \"c:/mozilla-build/python/python2.7.exe d:/mozilla/comm-
central/mozilla/build/pymake/pymake/../make.py -f client.mk build\"" && exit 1 )
' failed, return code 1
 3:57.14 d:\mozilla\comm-central\client.mk:363:0: command 'c:/mozilla-build/pyth
on/python2.7.exe d:/mozilla/comm-central/mozilla/build/pymake/pymake/../make.py
-f d:/mozilla/comm-central/client.mk configure' failed, return code 2
 3:57.17 Traceback (most recent call last):
 3:57.19   File "d:/mozilla/comm-central/build/pymake/make.py", line 35, in <mod
ule>
 3:57.20     main(sys.argv[1:])
 3:57.22   File "d:/mozilla/comm-central/build/pymake/make.py", line 32, in main

 3:57.23     subprocess.check_call([sys.executable, PYMAKE] + args)
 3:57.25   File "c:\mozilla-build\python\lib\subprocess.py", line 542, in check_
call
 3:57.27     raise CalledProcessError(retcode, cmd)
 3:57.28 subprocess.CalledProcessError: Command '['c:\\mozilla-build\\python\\py
thon2.7.exe', 'd:\\mozilla\\comm-central\\mozilla\\build\\pymake\\make.py', '-f'
, 'client.mk', '-s']' returned non-zero exit status 2


My System: Intel I5 processor, 4GB RAM
Build Environment: WinXP, Mozilla-Build 1.9 in "C:\mozilla-build", MSVC10, c-c source in "D:\mozilla\comm-central", Win7 SDK

I tried loads of different mozconfig options, but none made any difference except throwing errors earlier in the build process.
How old is the tree you're building?
Which age do you mean?
Last client.py checkout was yesterday.
First clone of the current tree must have been sometime in autumn 2013.
Last week I deleted the local directories except .hg and rebuilt from local repo.
I think he means, what is the revision of your mozilla-central clone? (i.e. what's the output of "hg identify" in your mozilla-central directory?)
(or your comm-central clone, I suppose. I forget how comm-central manages mozilla-central -- whether it's a separate clone vs. a direct import)
c-c maintains a clone of the m-c repo, so the version is the same as for the c-c.
This is done with the command "client.py checkout"
Hi Taraman,

I tried to build comm-central and it does not have the problem(It fails in the later stage that there is no thunderbird.exe. I think it should be almost done.). I'm not sure if my steps is correct. 
* hg clone http://hg.mozilla.org/comm-central 
* python client.py checkout
* ./mozilla/mach build

Here is my .mozconfig
ac_add_options --enable-debug
ac_add_options --disable-optimize
mk_add_options --AUTOCLOBBER=1
ac_add_options --disable-gamepad
ac_add_options --disable-webrtc
Hi Steven,

that means your build succeeded. The .exe is not found, because mach expects to be called from ./ and not from ./mozilla --> This is a diferent bug.

I will try to completely re-clone my repo to see if it helps tonight.
...which did not help. :-(
Note: media/libyuv/build/dir_exists.py requires that media/webrtc/trunk/build/dir_exists.py exist; if the c-c checkout process doesn't have that in the build tree, this will fail.  From Steven Lee's comments, it sounds like it should be there.  

Perhaps this is a Python/MSYS difference from Linux/mac (Steven, what are you using?)  Ted, any ideas about this idea?
Flags: needinfo?(ted)
Flags: needinfo?(slee)
I don't think that's it, since otherwise the Thunderbird builders would be broken, right? There must be something odd about the current working directory when this gyp file is being evaluated that it can't find that script.
Flags: needinfo?(ted)
I think the line:
> GypError: Call to 'd:\mozilla\objdir-tb\mozilla\_virtualenv\Scripts\python.exe
> ./build/dir_exists.py ./third_party/platformsdk_win7/files/redist/x86'

may be a hint.
The directory "./third_party/platformsdk_win7/files/redist/x86" is not existing, thus the error message makes sense:
d:\mozilla\objdir-tb\mozilla\_virtualenv\Scripts\python.exe: can't open file './build/dir_exists.py': [Errno 2] No such file or directory
(In reply to Markus Adrario [:Taraman] from comment #11)
> I think the line:
> > GypError: Call to 'd:\mozilla\objdir-tb\mozilla\_virtualenv\Scripts\python.exe
> > ./build/dir_exists.py ./third_party/platformsdk_win7/files/redist/x86'
> 
> may be a hint.
> The directory "./third_party/platformsdk_win7/files/redist/x86" is not
> existing, thus the error message makes sense:
> d:\mozilla\objdir-tb\mozilla\_virtualenv\Scripts\python.exe: can't open file
> './build/dir_exists.py': [Errno 2] No such file or directory

No, the point of the script is simply to test if that directory exists.  The problem is it can't find the script (for various reasons, the dir_exists.py script in libyuv simply calls the one in media/webrtc; if there's some showstopper reason not to we can re-implement (though note on mac this same issue may arise for the SDK check; that script is more complex to reverse-engineer - though with a trivial license.html patch we could copy it.)
Hi Taraman,

My build environment is just download from the link provided by mdn. I don't modify anything. :(
Flags: needinfo?(slee)
Product: Core → Firefox Build System
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.