Closed
Bug 1490427
Opened 6 years ago
Closed 5 years ago
jsshell bench runs v8 shell as 'd8', AWFY custom built d8 with is_debug=false
Categories
(Testing :: Raptor, enhancement)
Testing
Raptor
Tracking
(firefox66 fixed)
RESOLVED
FIXED
mozilla66
Tracking | Status | |
---|---|---|
firefox66 | --- | fixed |
People
(Reporter: jmaher, Assigned: sparky)
References
Details
Attachments
(1 file)
we need to build a custom d8 version instead of the prebuilt version. We should be able to do this with: https://github.com/v8/v8/wiki/Building-from-Source you can see a comment referencing the old build process: https://bugzilla.mozilla.org/show_bug.cgi?id=1488689#c14 and references to some build settings: https://github.com/mozilla/arewefastyet/blob/34d7366a4a479fcf88624ee75b13597095eb7821/slave/build.py#L285 I suspect we can build a docker image that has all the tools for a build and can build a simple script that runs and builds d8 with an upload as an artifact.
Comment 1•6 years ago
|
||
In the shorter term, it would be easier to just build 'd8' locally and upload that to the perf-automation repo (to be used in the fetch task). I agree having automation do the build is more ideal, but it also seems like a waste to re-build the exact same version of d8 over and over again. I guess we can figure out the taskgraph magic to re-use the same artifact over and over again so we don't always need to re-build it. I think implementing this as a "toolchain" task might be a good fit, but I would have to do some investigation to be sure.
Comment 2•6 years ago
|
||
Just noting that I did originally try to build 'd8' locally, but their build process doesn't support Fedora, so that's why I ended up using the pre-built binary.
Reporter | ||
Comment 3•6 years ago
|
||
so I think the best bet here is to create a docker image that does all the custom building- then we can download that locally as well as use it in CI. I would prefer that over figuring out how to get all the bits installed on my local windows machine.
Reporter | ||
Comment 4•6 years ago
|
||
here is an example docker file used to build d8: https://github.com/REMnux/docker/blob/master/v8/Dockerfile I need to figure out if there is an example build script I could use to add to the tree so we could: 1) create a new build job in-tree for v8, ideally using an existing dockerfile or a new one 2) upload the output file as an artifact that we can use :ted, do you have suggestions on where I could start looking?
Flags: needinfo?(ted)
Comment 5•6 years ago
|
||
If you look at our existing toolchain tasks we have a bunch that build random third-party code. As an arbitrary example, here's where we build libdmg for the dmg tools for our mac cross-builds: https://dxr.mozilla.org/mozilla-central/rev/c291143e24019097d087f9307e59b49facaf90cb/taskcluster/ci/toolchain/linux.yml#379-392 It calls into this script which is pretty straightforward: https://dxr.mozilla.org/mozilla-central/source/taskcluster/scripts/misc/build-libdmg-hfsplus.sh Toolchain tasks all run in the toolchain-build Docker image which is defined here: https://dxr.mozilla.org/mozilla-central/source/taskcluster/docker/toolchain-build/Dockerfile Odds are it already has everything you need to build v8, we're building a lot of projects in that image.
Flags: needinfo?(ted)
Reporter | ||
Comment 6•6 years ago
|
||
the docker image from comment 4 is outdated, but I still think with little effort we can reuse a toolchain build. The docs have been updated: https://v8.dev/docs/build-gn
Assignee | ||
Comment 7•6 years ago
|
||
I've spent some time looking into this and the toolchain-build uses debian-7 which has some missing packages that seem to primarily exist for debian-8 (jessie) and debian-9 (stretch) with no existing backports: g++-arm-linux-gnueabihf libc6-dev-armhf-cross libffi6 libwayland-egl1-mesa libxkbcommon-dev linux-libc-dev-armhf-cross Here's a try run that shows this: https://treeherder.mozilla.org/#/jobs?repo=try&selectedJob=213145527&revision=bde63b7bec6ba7a16f0f0cf8ad4a920e79135d0a This error occurs when I follow the instructions here [1]. I'm going to test the instructions here [1] now to see if those commands along with the options mentioned here [2] would solve this. (Maybe [1] is building things we don't need). If this doesn't work, and we still have missing packages, we could either use a debian-8/9 docker image for this or perhaps fallback onto an older v8 version which might not require them (I imagine that there are other options as well). [1]: https://v8.dev/docs/build [2]: https://v8.dev/docs/build-gn [3]: https://github.com/mozilla/arewefastyet/blob/34d7366a4a479fcf88624ee75b13597095eb7821/slave/build.py#L285
Assignee | ||
Updated•6 years ago
|
Assignee: nobody → gmierz2
Assignee | ||
Comment 8•6 years ago
|
||
(In reply to Greg Mierzwinski [:sparky] from comment #7) > > This error occurs when I follow the instructions here [1]. > I'm going to test the instructions here [2] now to see if those commands > along with the options mentioned here [3] would solve this. (Maybe [1] is Mixed up the reference numbers - this quote is the corrected version.
Assignee | ||
Comment 9•6 years ago
|
||
I've got a successful custom v8 build here: https://treeherder.mozilla.org/#/jobs?repo=try&revision=6486059d850e15f4e3b68ea206f77d2a64b251a8 It's not possible to build this on debian7 so I made a debian9-base docker image for the custom-v8 task to do this. (More specifically, `libwayland-egl1-mesa` is a common requirement in the build dependencies but is not available in debian7). Also, both methods I mentioned in comment 7 ([1], and [2]) requires us to have installed the build dependencies so using one method instead of the other didn't help. After I add the extra build settings and upload the artifact in the patch, I'll post a patch for review.
Assignee | ||
Comment 10•6 years ago
|
||
:jmaher, is there a specific OS/CPU that we should be targeting in this build or would it be used for more than just one variation like the AWFY builder is set up [1]? [1]: https://github.com/mozilla/arewefastyet/blob/34d7366a4a479fcf88624ee75b13597095eb7821/slave/build.py#L285
Flags: needinfo?(jmaher)
Reporter | ||
Comment 11•6 years ago
|
||
we are targeting linux only- but writing code to be more generic is a good idea. using debian9 is just fine from my perspective- do we have a try push or evidence that the generated v8 build is able to run the shell tests?
Flags: needinfo?(jmaher)
Assignee | ||
Comment 12•6 years ago
|
||
Here's a test run that has the jsshell tests use the d8 version that was built on debian9: https://treeherder.mozilla.org/#/jobs?repo=try&revision=b504a8e5761d7eda3a4ccc7fee45f4ef6729eeb1 This is the task which built the d8 that was used: https://treeherder.mozilla.org/#/jobs?repo=try&revision=2bde819ab10de2dfa7ca7eb887636ab3d4c24a1f
Assignee | ||
Comment 13•5 years ago
|
||
This patch adds a toolchain task for building d8 with customized build settings and uses it in jsshell benchmark tests. A customized image with a debian9-base ('custom-v8') is added by this patch as well and is required to build the tool.
Assignee | ||
Comment 14•5 years ago
|
||
For the patch that was submitted for review, here's a test run with it: https://treeherder.mozilla.org/#/jobs?repo=try&revision=a68449743d6eadb9ed4787f2dc8a0c29531de11c The docker image was built in this test run (the broken tests have been fixed): https://treeherder.mozilla.org/#/jobs?repo=try&revision=9fd77c4304d17ba80f590d83455dd63ec7bd7da5
Comment 15•5 years ago
|
||
Pushed by jmaher@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/ad264a713556 Build and use custom d8 from a toolchain task. r=jmaher,ahal
Comment 16•5 years ago
|
||
Backed out changeset ad264a713556 (bug 1490427) for build bustage. CLOSED TREE Log: https://treeherder.mozilla.org/logviewer.html#?job_id=217870374&repo=autoland Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=ad264a7135561eaa4179235fbd59790e25e38a0c Backout: https://hg.mozilla.org/integration/autoland/rev/4e14149b1dffce2b7f400ae7b9a6932445213c12
Flags: needinfo?(gmierz2)
Assignee | ||
Comment 17•5 years ago
|
||
This should be fixed in the latest patch, I can't reproduce this failure on try but I can remove the setup_packages.sh file which is causing the issue. Here's a test run using this patch: https://treeherder.mozilla.org/#/jobs?repo=try&revision=c4dc54c8366c2c4771ff64e5a15575013a49fd38
Flags: needinfo?(gmierz2)
Comment 18•5 years ago
|
||
Pushed by gmierz2@outlook.com: https://hg.mozilla.org/integration/autoland/rev/49dfcbef9786 Build and use custom d8 from a toolchain task. r=jmaher,ahal
Comment 19•5 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/49dfcbef9786
Status: NEW → RESOLVED
Closed: 5 years ago
status-firefox66:
--- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla66
You need to log in
before you can comment on or make changes to this bug.
Description
•