Closed Bug 1328184 Opened 5 years ago Closed 5 years ago

build-clang.py doesn't properly handle being run with multiple config files on the same machine

Categories

(Firefox Build System :: Source Code Analysis, defect)

defect
Not set
normal

Tracking

(firefox53 fixed)

RESOLVED FIXED
mozilla53
Tracking Status
firefox53 --- fixed

People

(Reporter: ehsan.akhgari, Assigned: ehsan.akhgari)

References

Details

Attachments

(2 files)

First push: https://treeherder.mozilla.org/#/jobs?repo=try&revision=863c56f0e29bd73497a5ff1dc8070f6997d85ff0
Second push based on the same base revision: https://treeherder.mozilla.org/#/jobs?repo=try&revision=6280de6910c87e6365f24e5dad6b61c671579d18

Error example: https://public-artifacts.taskcluster.net/I7q41zyLSZuENx7PqXwDxg/0/public/logs/live_backing.log
BUILD_CLANG_DIR=build/src/build/build-clang
MOZCONFIG=${MOZCONFIG} build/src/mach python ${BUILD_CLANG_DIR}/build-clang.py -c ${BUILD_CLANG_DIR}/clang-static-analysis-win64.json
New python executable in Z:\z\task_1483397805\objdir\_virtualenv\Scripts\python2.7.exe
Also creating executable in Z:\z\task_1483397805\objdir\_virtualenv\Scripts\python.exe
Installing setuptools, pip, wheel...done.
running build_ext

building 'psutil._psutil_windows' extension

error: Microsoft Visual C++ 9.0 is required. Get it from http://aka.ms/vcpython27


Error processing command. Ignoring because optional. (optional:setup.py:python/psutil:build_ext:--inplace)
Z:\task_1483397805\build\src\python/mozbuild\mozbuild\virtualenv.py:376: UserWarning: Hacking environment to allow binary Python extensions to build. You can make this warning go away by installing Visual Studio 2008. You can download the Express Edition installer from http://go.microsoft.com/?linkid=7729279
  warnings.warn('Hacking environment to allow binary Python '
[0/1] Re-running CMake...
FAILED: build.ninja 
z:\task_1483395109\cmake\bin\cmake.exe -HZ:\task_1483395109\llvm-sources\src\llvm -BZ:\task_1483395109\llvm-sources\build\stage1\build
CreateProcess failed: The system cannot find the file specified.
ninja: error: rebuilding 'build.ninja': subcommand failed
Traceback (most recent call last):
  File "build/src/build/build-clang/build-clang.py", line 496, in <module>
  File "build/src/build/build-clang/build-clang.py", line 247, in build_one_stage
  File "build/src/build/build-clang/build-clang.py", line 75, in build_package
  File "build/src/build/build-clang/build-clang.py", line 48, in run_in
  File "build/src/build/build-clang/build-clang.py", line 39, in check_run
AssertionError


I suspect there is something wrong with the caches set up for this job...
Assignee: nobody → ehsan
After looking more into this, firstly the error message about MSVC 9 being required is benign.  What's happening here is that build-clang.py mostly assumes it will only be re-invoked with the same config file.  I made a couple of changes to it to make it more robust in the face of an existing LLVM checkout and an existing build directory which I'll upload shortly.  Those patches seem to mostly fix this issue.
Component: General → Rewriting and Analysis
Product: Taskcluster → Core
Summary: "error: Microsoft Visual C++ 9.0 is required" when using shared resources in Windows clang-cl builds → build-clang.py doesn't properly handle being run with multiple config files on the same machine
Since individual config files have different source repos declared,
it's better to deal with each individual source directory separately.
Also make sure to revert any of the existing changes in each directory
so that attempts to apply patches to the source directory or import
our static analysis checks into clang-tidy are guaranteed to always
succeed.
Attachment #8823383 - Flags: review?(michael)
Instead of relying on the assumption that a previous run of CMake was
using the same arguments, remove the CMake cache file and re-run it.
This way the script is robust no matter what kind of build directory
existed from before.
Attachment #8823384 - Flags: review?(michael)
Attachment #8823383 - Flags: review?(michael) → review+
Attachment #8823384 - Flags: review?(michael) → review+
Pushed by eakhgari@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/5441cfb69cdc
Part 1: Deal better with an existing source checkout; r=mystor
https://hg.mozilla.org/integration/mozilla-inbound/rev/1bba98f1d809
Part 2: Deal better with an existing build directory being found; r=mystor
https://hg.mozilla.org/mozilla-central/rev/5441cfb69cdc
https://hg.mozilla.org/mozilla-central/rev/1bba98f1d809
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla53
Bug 1329306 finishes fixing what part 2 here was fixing.
Depends on: 1329306
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.