Closed Bug 859349 Opened 11 years ago Closed 11 years ago

"mach build" fails with "No such file or directory: .mozbuild/warnings.json" after a clobber fails

Categories

(Firefox Build System :: Mach Core, enhancement)

enhancement
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED
mozilla23

People

(Reporter: mbrubeck, Assigned: mbrubeck)

References

(Depends on 1 open bug)

Details

Attachments

(1 file, 2 obsolete files)

This happened after clobber.py failed with an "Access is denied" error.  It got far enough to clobber the .mozbuild directory before it aborted, so save_to_warnings then failed too, making the real error harder to see:

mbrub_000@X1:~/src/elm$ mach build
 0:00.38 C:/mozilla-build/msys/bin/sh.exe -c c:/Users/mbrub_000/src/elm/build/pymake/make.py -f client.mk -s
 0:01.78 Adding client.mk options from c:/Users/mbrub_000/src/elm/.mozconfig:
 0:01.78     MOZ_OBJDIR=$(TOPSRCDIR)/obj-metro
 0:02.04 TEST-PASS | check-sync-dirs.py | c:\Users\mbrub_000\src\elm\js\src\build <= c:\Users\mbrub_000\src\elm\build
 0:02.05 Generating c:/Users/mbrub_000/src/elm/configure using autoconf
 0:02.84 TEST-PASS | check-sync-dirs.py | c:\Users\mbrub_000\src\elm\js\src\build <= c:\Users\mbrub_000\src\elm\build
 0:02.85 Generating c:/Users/mbrub_000/src/elm/js/src/configure using autoconf
 0:06.67 Automatically clobbering c:\Users\mbrub_000\src\elm\obj-metro
 0:06.67 Traceback (most recent call last):
 0:06.67   File "c:/Users/mbrub_000/src/elm/python/mozbuild/mozbuild/controller/clobber.py", line 193, in <module>
 0:06.67     sys.exit(main(sys.argv[1:], os.environ, os.getcwd(), sys.stdout))
 0:06.67   File "c:/Users/mbrub_000/src/elm/python/mozbuild/mozbuild/controller/clobber.py", line 183, in main
 0:06.67     required, performed, message = clobber.maybe_do_clobber(cwd, auto, fh)
 0:06.68   File "c:/Users/mbrub_000/src/elm/python/mozbuild/mozbuild/controller/clobber.py", line 152, in maybe_do_clobber
 0:06.68     rmtree(self.topobjdir)
 0:06.68   File "c:\mozilla-build\python\lib\shutil.py", line 244, in rmtree
 0:06.68     rmtree(fullname, ignore_errors, onerror)
 0:06.68   File "c:\mozilla-build\python\lib\shutil.py", line 244, in rmtree
 0:06.68     rmtree(fullname, ignore_errors, onerror)
 0:06.68   File "c:\mozilla-build\python\lib\shutil.py", line 244, in rmtree
 0:06.68     rmtree(fullname, ignore_errors, onerror)
 0:06.68   File "c:\mozilla-build\python\lib\shutil.py", line 249, in rmtree
 0:06.68     onerror(os.remove, fullname, sys.exc_info())
 0:06.68   File "c:\mozilla-build\python\lib\shutil.py", line 247, in rmtree
 0:06.68     os.remove(fullname)
 0:06.68 WindowsError: [Error 5] Access is denied: 'c:\\Users\\mbrub_000\\src\\elm\\obj-metro\\dist\\include\\dbm\\cdefs.h'
 0:06.68 c:\Users\mbrub_000\src\elm\client.mk:309:0: command 'python c:/Users/mbrub_000/src/elm/python/mozbuild/mozbuild/controller/clobber.py c:/Users/mbrub_000/src/elm c:/Users/mbrub_000/src/elm/obj-metro' failed, return code 1
 0:06.68 c:\Users\mbrub_000\src\elm\client.mk:339:0: command 'c:/mozillabuild/python/python.exe c:/Users/mbrub_000/src/elm/build/pymake/pymake/../make.py -f c:/Users/mbrub_000/src/elm/client.mk configure' failed, return code
 2 0:06.68 c:\Users\mbrub_000\src\elm\client.mk:163:0: command 'c:/mozilla-build/python/python.exe c:/Users/mbrub_000/src/elm/build/pymake/pymake/../make.py -f c:/Users/mbrub_000/src/elm/client.mk realbuild' failed, return code 2
 0:06.75 1962 compiler warnings present.
Error running mach:

    ['build']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

IOError: [Errno 2] No such file or directory: u'c:\\Users\\mbrub_000\\src\\elm/obj-metro\\.mozbuild\\warnings.json'

  File "c:\Users\mbrub_000\src\elm\python/mozbuild/mozbuild/mach_commands.py", line 115, in build
    warnings_database.save_to_file(warnings_path)
  File "c:\Users\mbrub_000\src\elm\python/mozbuild\mozbuild\compilation\warnings.py", line 268, in save_to_file
    with open(filename, 'wb') as fh:
Attached patch patch (obsolete) — Splinter Review
This patch contains two changes:
* warnings.save_to_file will create the parent directory if it doesn't exist.
* Don't bother writing the warnings file if the build failed.

Either change on its own would fix this bug, but each seemed worthwhile for other reasons, so I kept both.  It would be reasonable to accept just the second change, if you think the first one adds too much implicit behavior.
Assignee: nobody → mbrubeck
Status: NEW → ASSIGNED
Attachment #734664 - Flags: review?(gps)
Depends on: 859178
Comment on attachment 734664 [details] [diff] [review]
patch

Review of attachment 734664 [details] [diff] [review]:
-----------------------------------------------------------------

::: python/mozbuild/mozbuild/compilation/warnings.py
@@ +266,5 @@
>      def save_to_file(self, filename):
>          """Save the database to a file."""
> +        dirname = os.path.dirname(filename)
> +        if not os.path.exists(dirname):
> +            os.makedirs(dirname)

As I relearned recently, this isn't good enough. We need to do:

try:
    os.makedirs(path)
except OSError as e:
    if e.errno != errno.EEXIST:
        raise

::: python/mozbuild/mozbuild/mach_commands.py
@@ +112,5 @@
>                  '{count} compiler warnings present.')
>  
> +        if status == 0:
> +            warnings_database.prune()
> +            warnings_database.save_to_file(warnings_path)

We want to save the warnings database for failed builds. Otherwise warnings from before the failure point will be lost until the next recompile and we will under report total warnings.
Attachment #734664 - Flags: review?(gps)
Attached patch patch v2 (obsolete) — Splinter Review
Attachment #734664 - Attachment is obsolete: true
Attachment #734694 - Flags: review?(gps)
Attached patch patch v3Splinter Review
Added a missing import.
Attachment #734694 - Attachment is obsolete: true
Attachment #734694 - Flags: review?(gps)
Attachment #734721 - Flags: review?(gps)
Attachment #734721 - Attachment is patch: true
Attachment #734721 - Flags: review?(gps) → review+
https://hg.mozilla.org/mozilla-central/rev/3250333a5e6c
Status: ASSIGNED → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla23
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: