Closed Bug 1177634 Opened 9 years ago Closed 9 years ago

error occurred while processing the following file: access denied to suite/app.mozbuild

Categories

(Firefox Build System :: General, defect)

Unspecified
Windows Server 2008
defect
Not set
normal

Tracking

(firefox39 wontfix, firefox40 fixed, firefox41 fixed, firefox42 fixed, firefox-esr38 fixed)

RESOLVED FIXED
mozilla42
Tracking Status
firefox39 --- wontfix
firefox40 --- fixed
firefox41 --- fixed
firefox42 --- fixed
firefox-esr38 --- fixed

People

(Reporter: ewong, Assigned: ewong)

References

Details

Attachments

(1 file, 1 obsolete file)

js\src\ctypes\libffi> checking command to parse dumpbin -symbols output from /c/builds/slave3/c-c/build/mozilla/js/src/ctypes/libffi/msvcc.sh object... ok
js\src\ctypes\libffi> checking for sysroot... no
js\src\ctypes\libffi> checking for i686-pc-mingw32-mt... no
js\src\ctypes\libffi> checking for mt... mt
js\src\ctypes\libffi> checking if mt is a manifest tool... yes
js\src\ctypes\libffi> checking how to run the C preprocessor... cl -nologo -EP
js\src\ctypes\libffi> checking for ANSI C header files... yes
js\src\ctypes\libffi> checking for sys/types.h... yes
js\src\ctypes\libffi> checking for sys/stat.h... yes
js\src\ctypes\libffi> checking for stdlib.h... yes
js\src\ctypes\libffi> checking for string.h... yes
js\src\ctypes\libffi> checking for memory.h... yes
js\src\ctypes\libffi> checking for strings.h... no
js\src\ctypes\libffi> checking for inttypes.h... yes
js\src\ctypes\libffi> checking for stdint.h... yes
js\src\ctypes\libffi> checking for unistd.h... no
js\src\ctypes\libffi> checking for dlfcn.h... no
js\src\ctypes\libffi> checking for objdir... .libs
js\src\ctypes\libffi> checking for /c/builds/slave3/c-c/build/mozilla/js/src/ctypes/libffi/msvcc.sh option to produce PIC... -DDLL_EXPORT -DPIC
js\src\ctypes\libffi> checking if /c/builds/slave3/c-c/build/mozilla/js/src/ctypes/libffi/msvcc.sh PIC flag -DDLL_EXPORT -DPIC works... yes
js\src\ctypes\libffi> checking if /c/builds/slave3/c-c/build/mozilla/js/src/ctypes/libffi/msvcc.sh static flag  works... yes
js\src\ctypes\libffi> checking if /c/builds/slave3/c-c/build/mozilla/js/src/ctypes/libffi/msvcc.sh supports -c -o file.obj... yes
js\src\ctypes\libffi> checking if /c/builds/slave3/c-c/build/mozilla/js/src/ctypes/libffi/msvcc.sh supports -c -o file.obj... (cached) yes
js\src\ctypes\libffi> checking whether the /c/builds/slave3/c-c/build/mozilla/js/src/ctypes/libffi/msvcc.sh linker (link) supports shared libraries... yes
js\src\ctypes\libffi> checking dynamic linker characteristics... Win32 ld.exe
js\src\ctypes\libffi> checking how to hardcode library paths into programs... immediate
js\src\ctypes\libffi> checking whether stripping libraries is possible... no
js\src\ctypes\libffi> checking if libtool supports shared libraries... yes
js\src\ctypes\libffi> checking whether to build shared libraries... no
js\src\ctypes\libffi> checking whether to build static libraries... yes
js\src\ctypes\libffi> checking how to run the C++ preprocessor... cl -nologo -EP
js\src\ctypes\libffi> checking whether the /c/builds/slave3/c-c/build/mozilla/js/src/ctypes/libffi/msvcc.sh linker (link) supports shared libraries... no
js\src\ctypes\libffi> checking for /c/builds/slave3/c-c/build/mozilla/js/src/ctypes/libffi/msvcc.sh option to produce PIC... -DDLL_EXPORT -DPIC
js\src\ctypes\libffi> checking if /c/builds/slave3/c-c/build/mozilla/js/src/ctypes/libffi/msvcc.sh PIC flag -DDLL_EXPORT -DPIC works... yes
js\src\ctypes\libffi> checking if /c/builds/slave3/c-c/build/mozilla/js/src/ctypes/libffi/msvcc.sh static flag  works... yes
js\src\ctypes\libffi> checking if /c/builds/slave3/c-c/build/mozilla/js/src/ctypes/libffi/msvcc.sh supports -c -o file.obj... yes
js\src\ctypes\libffi> checking if /c/builds/slave3/c-c/build/mozilla/js/src/ctypes/libffi/msvcc.sh supports -c -o file.obj... (cached) yes
js\src\ctypes\libffi> checking whether the /c/builds/slave3/c-c/build/mozilla/js/src/ctypes/libffi/msvcc.sh linker (link) supports shared libraries... no
js\src\ctypes\libffi> checking dynamic linker characteristics... Win32 ld.exe
js\src\ctypes\libffi> checking how to hardcode library paths into programs... immediate
js\src\ctypes\libffi> checking size of size_t... 4
js\src\ctypes\libffi> checking for C compiler vendor... microsoft
js\src\ctypes\libffi> 
js\src\ctypes\libffi> ********************************************************
js\src\ctypes\libffi> * WARNING: Don't know the best CFLAGS for this system  *
js\src\ctypes\libffi> * Use ./configure CFLAGS=... to specify your own flags *
js\src\ctypes\libffi> * (otherwise, a default of CFLAGS=-O3 will be used)    *
js\src\ctypes\libffi> ********************************************************
js\src\ctypes\libffi> 
js\src\ctypes\libffi> checking whether C compiler accepts -O3... yes
js\src\ctypes\libffi> checking CFLAGS for maximum warnings... -Wall
js\src\ctypes\libffi> checking whether to enable maintainer-specific portions of Makefiles... no
js\src\ctypes\libffi> checking sys/mman.h usability... no
js\src\ctypes\libffi> checking sys/mman.h presence... no
js\src\ctypes\libffi> checking for sys/mman.h... no
js\src\ctypes\libffi> checking for mmap... no
js\src\ctypes\libffi> checking for sys/mman.h... (cached) no
js\src\ctypes\libffi> checking for mmap... (cached) no
js\src\ctypes\libffi> checking for ANSI C header files... (cached) yes
js\src\ctypes\libffi> checking for memcpy... no
js\src\ctypes\libffi> checking for size_t... yes
js\src\ctypes\libffi> checking for working alloca.h... no
js\src\ctypes\libffi> checking for alloca... yes
js\src\ctypes\libffi> checking size of double... 8
js\src\ctypes\libffi> checking size of long double... 8
js\src\ctypes\libffi> checking whether byte ordering is bigendian... no
js\src\ctypes\libffi> checking assembler .cfi pseudo-op support... no
js\src\ctypes\libffi> checking assembler supports pc related relocs... yes
js\src\ctypes\libffi> checking assembler .ascii pseudo-op support... yes
js\src\ctypes\libffi> checking assembler .string pseudo-op support... yes
js\src\ctypes\libffi> checking for _ prefix in compiled symbols... yes
js\src\ctypes\libffi> configure: updating cache c:/builds/slave3/c-c/build/objdir/js/src/ctypes/libffi/config.cache
js\src\ctypes\libffi> checking that generated files are newer than configure... done
js\src\ctypes\libffi> configure: creating ./config.status
js\src\ctypes\libffi> config.status: creating include/Makefile
js\src\ctypes\libffi> config.status: creating include/ffi.h
js\src\ctypes\libffi> config.status: creating Makefile
js\src\ctypes\libffi> config.status: creating testsuite/Makefile
js\src\ctypes\libffi> config.status: creating man/Makefile
js\src\ctypes\libffi> config.status: creating libffi.pc
js\src\ctypes\libffi> config.status: creating fficonfig.h
js\src\ctypes\libffi> config.status: linking C:/builds/slave3/c-c/build/mozilla/js/src/ctypes/libffi/src/x86/ffitarget.h to include/ffitarget.h
js\src\ctypes\libffi> config.status: executing buildir commands
js\src\ctypes\libffi> config.status: skipping top_srcdir/Makefile - not created
js\src\ctypes\libffi> config.status: executing depfiles commands
js\src\ctypes\libffi> config.status: executing libtool commands
js\src\ctypes\libffi> config.status: executing include commands
js\src\ctypes\libffi> config.status: executing src commands

Reticulating splines...
Traceback (most recent call last):
  File "./config.status", line 971, in <module>
    config_status(**args)
  File "c:\builds\slave3\c-c\build\mozilla\python\mozbuild\mozbuild\config_status.py", line 149, in config_status
    summary = the_backend.consume(definitions)
  File "c:\builds\slave3\c-c\build\mozilla\python\mozbuild\mozbuild\backend\base.py", line 180, in consume
    for obj in objs:
  File "c:\builds\slave3\c-c\build\mozilla\python\mozbuild\mozbuild\frontend\emitter.py", line 147, in emit
    for out in output:
  File "c:\builds\slave3\c-c\build\mozilla\python\mozbuild\mozbuild\frontend\reader.py", line 1051, in read_mozbuild
    sys.exc_info()[2], sandbox_load_error=sle)
mozbuild.frontend.reader.BuildReaderError: 
==============================
ERROR PROCESSING MOZBUILD FILE
==============================

The error occurred while processing the following file:

    c:/builds/slave3/c-c/build/mozilla/moz.build

The underlying problem is an illegal file access. This is likely due to trying to access a file outside of the top source directory.

The path whose access was denied is:

    c:/builds/slave3/c-c/build/suite/app.mozbuild

Modify the script to not access this file and try again.

*** Fix above errors and then restart with\
               "c:/builds/slave3/c-c/build/mozmake.exe -f client.mk build"
C:/builds/slave3/c-c/build/client.mk:361: recipe for target 'configure' failed
mozmake.exe[1]: *** [configure] Error 1
mozmake.exe[1]: Leaving directory 'C:/builds/slave3/c-c/build'
client.mk:375: recipe for target 'C:/builds/slave3/c-c/build/objdir/Makefile' failed
mozmake.exe: *** [C:/builds/slave3/c-c/build/objdir/Makefile] Error 2

This is with the following mozconfig:

. "$topsrcdir/build/mozconfig.common"

ac_add_options --enable-application=suite
ac_add_options --enable-update-channel=${MOZ_UPDATE_CHANNEL}
ac_add_options --enable-update-packaging
ac_add_options --enable-jemalloc
ac_add_options --enable-tests
ac_add_options --enable-profiling

# Needed to enable breakpad in application.ini
export MOZILLA_OFFICIAL=1

# Bug 1155007 temporary workaround. Needs to be
# backed out once we get a Socorro token generated.
unset SOCORRO_SYMBOL_UPLOAD_TOKEN_FILE

if test "$PROCESSOR_ARCHITECTURE" = "AMD64" -o "$PROCESSOR_ARCHITEW6432" = "AMD64"; then
  . $topsrcdir/build/win32/mozconfig.vs2013-win64
else
  . $topsrcdir/build/win32/mozconfig.vs2010
fi

# Set up mapi includes (must be done after visual studio setup)
export INCLUDE=$INCLUDE:/c/Office\ 2010\ Developer\ Resources/Outlook\ 2010\ MAPI\ Headers
mk_export_correct_style INCLUDE
mk_add_options MOZ_OBJDIR=c:/builds/slave3/c-c/build/objdir
OS: Unspecified → Windows Server 2008
Also, this isn't done in a mozbuild start.bat environment.  It's done
from a command prompt with environment variables copied from a working
Thunderbird nightly build.
Having done some testing, I have found out where the problem lies.

The problem I had was with the EXTERNAL_SOURCE_DIR as retrieved in
http://mxr.mozilla.org/comm-central/source/mozilla/python/mozbuild/mozbuild/backend/configenvironment.py#146

In the environment that I was building with, EXTERNAL_SOURCE_DIR was
C:/builds/slave/c-c/build.

whereas for the other checks in the system, specifically in
http://mxr.mozilla.org/comm-central/source/mozilla/python/mozbuild/mozbuild/frontend/reader.py#149,
the other paths were of the format "c:/builds/slave/c-c/build..."

So since C: != c:  in python but in Windows it is, this is where the
check fails.

I'm not exactly sure what the proper fix should be.

I tried:

1) for http://mxr.mozilla.org/comm-central/source/mozilla/python/mozbuild/mozbuild/backend/configenvironment.py#151,
   self.external_source_dir = mozpath.normpath(external).lower()

2) somehow even during the config.status processing, EXTERNAL_SOURCE_DIR
could be lowercased.

But this could potentially screw up case-sensitive paths.

So I'm going to guess that the proper way of doing this might be to
check if the first part of the path is a Windows drive, i.e. C:, and
just lowercase it. C: -> c:

glandium, what should be done?
Flags: needinfo?(mh+mozilla)
Use os.path.normcase in is_read_allowed in reader.py.
Flags: needinfo?(mh+mozilla)
Attached patch proposed patch (v1) (obsolete) — Splinter Review
Assignee: nobody → ewong
Status: NEW → ASSIGNED
Attachment #8629294 - Flags: review?(mh+mozilla)
Comment on attachment 8629294 [details] [diff] [review]
proposed patch (v1)

Review of attachment 8629294 [details] [diff] [review]:
-----------------------------------------------------------------

::: python/mozbuild/mozbuild/frontend/reader.py
@@ +147,5 @@
>          return True
>  
> +    if config.external_source_dir:
> +        external_dir = os.path.normcase(config.external_source_dir)
> +        if mozpath.basedir(path, [external_dir]):

you need to normcase path as well.
Attachment #8629294 - Flags: review?(mh+mozilla) → feedback+
Attachment #8629294 - Attachment is obsolete: true
Attachment #8630288 - Flags: review?(mh+mozilla)
I'm running into this problem too, and unfortunately the v2 patch here doesn't help.  I get this error: https://pastebin.mozilla.org/8838761.  I also tried with my objdir explicitly set (to c:/comm-central/obj) but got the same error.

These are the variable values I get at this point:

norm_path: c:\comm-central\suite\app.mozbuild
external_dir: \c\comm-central

I was playing around with various os.path methods but couldn't find one that would normalize the Windows-style paths to msys ones or vice versa.

If I just force is_read_allowed to return True, I run into another error: https://pastebin.mozilla.org/8838763
Oh, I realized I was using make rather than mozmake.  Using mozmake and the v2 patch here works.
Attachment #8630288 - Flags: review?(mh+mozilla) → review+
https://hg.mozilla.org/mozilla-central/rev/6b30ec9f6580
Status: ASSIGNED → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Target Milestone: --- → seamonkey2.38
Product: SeaMonkey → Core
Target Milestone: seamonkey2.38 → mozilla44
Comment on attachment 8630288 [details] [diff] [review]
proposed patch (v2)

Approval Request Comment
[Feature/regressing bug #]:
[User impact if declined]: c-a, c-b, and c-r cannot be built.
[Describe test coverage new/current, TreeHerder]:
[Risks and why]: 
[String/UUID change made/needed]: none
Attachment #8630288 - Flags: approval-mozilla-release?
Attachment #8630288 - Flags: approval-mozilla-beta?
Attachment #8630288 - Flags: approval-mozilla-aurora?
That's NPOTB for m-* automation. That only affects seamonkey, thunderbird and comm-* related stuff.
Comment on attachment 8630288 [details] [diff] [review]
proposed patch (v2)

Approving for beta/aurora, it has been on m-c for several days with no known issues. For release, please on the train to arrive.
Attachment #8630288 - Flags: approval-mozilla-beta?
Attachment #8630288 - Flags: approval-mozilla-beta+
Attachment #8630288 - Flags: approval-mozilla-aurora?
Attachment #8630288 - Flags: approval-mozilla-aurora+
(In reply to Kate Glazko from comment #14)
> Comment on attachment 8630288 [details] [diff] [review]
> proposed patch (v2)
> 
> Approving for beta/aurora, it has been on m-c for several days with no known
> issues. For release, please on the train to arrive.

Thanks Kate!  As for m-r,  I also need it on it; but iiuc, I can
always push it to the SEAMONKEY_2_35_RELEASE_BRANCH, right?  That
shouldn't affect m-r proper, right?
Indeed, please use the seamonkey relbranch for release.
Attachment #8630288 - Flags: approval-mozilla-release? → approval-mozilla-release-
Setting c-n for the m-a and m-b version of the patches.
Keywords: checkin-needed
No need to set checkin-needed for uplifts if you've got the other flags set correctly ;)
Keywords: checkin-needed
Target Milestone: mozilla44 → mozilla42
Comment on attachment 8630288 [details] [diff] [review]
proposed patch (v2)

[Approval Request Comment]
If this is not a sec:{high,crit} bug, please state case for ESR consideration:
User impact if declined: SeaMonkey 2.35 is to be based on mozilla-esr38.  This
patch is needed in order to get it built. (NPOTB since this patch only
is used by comm-release)
Fix Landed on Version:
Risk to taking this patch (and alternatives if risky): 
String or UUID changes made by this patch:  none

See https://wiki.mozilla.org/Release_Management/ESR_Landing_Process for more info.
Attachment #8630288 - Flags: approval-mozilla-esr38?
Blocks: SM2.35
(In reply to Edmund Wong (:ewong) from comment #21)
> Comment on attachment 8630288 [details] [diff] [review]
> proposed patch (v2)
> 
> [Approval Request Comment]
> If this is not a sec:{high,crit} bug, please state case for ESR
> consideration:
> User impact if declined: SeaMonkey 2.35 is to be based on mozilla-esr38. 
> This
> patch is needed in order to get it built. (NPOTB since this patch only
> is used by comm-release)
Fix Landed on Version: 41
Risk to taking this patch (and alternatives if risky):  None
> String or UUID changes made by this patch:  none
> 
> See https://wiki.mozilla.org/Release_Management/ESR_Landing_Process for more
> info.
Comment on attachment 8630288 [details] [diff] [review]
proposed patch (v2)

This seems like a mozbuild related patch. Should be safe.
Attachment #8630288 - Flags: approval-mozilla-esr38? → approval-mozilla-esr38+
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.