Run Valgrind builds on every push

RESOLVED FIXED

Status

Release Engineering
General Automation
P3
normal
RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: njn, Assigned: catlee)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [valgrind])

Attachments

(1 attachment)

(Reporter)

Description

4 years ago
For Valgrind builds to be unhidden, they need to run on every push.
(Reporter)

Comment 1

4 years ago
Additional info:  they only need to run on Linux64, which is our most scalable platform.  They currently take about 45 minutes.
(Assignee)

Updated

4 years ago
Assignee: nobody → catlee
(Assignee)

Comment 2

4 years ago
Created attachment 8343715 [details] [diff] [review]
valgrind-buildbotcustom.diff

do valgrind builds per push rather than nightly
Attachment #8343715 - Flags: review?(jhopkins)
Attachment #8343715 - Flags: review?(jhopkins) → review+
(Assignee)

Comment 3

4 years ago
Comment on attachment 8343715 [details] [diff] [review]
valgrind-buildbotcustom.diff

https://hg.mozilla.org/build/buildbotcustom/rev/c6536c16d218

this will take effect after the next reconfig
Attachment #8343715 - Flags: checked-in+
(Reporter)

Comment 4

4 years ago
Thanks, catlee!  I give you that gold star now :)

> this will take effect after the next reconfig

When will that be, and is there a way to be notified?
(Reporter)

Comment 5

4 years ago
> They currently take about 45 minutes.

FWIW, I did some measurements (with debugging code on the TBPL machines) and found that about 38 minutes of that is setup and build time, and only about 7 minutes is the actual Valgrind run.
In production
(Reporter)

Comment 7

4 years ago
\o/
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
(Assignee)

Comment 8

4 years ago
we ended up busting the trees because these jobs time out and leave processes running, and don't reboot. Landed http://hg.mozilla.org/build/buildbotcustom/rev/9c71ca2b0e0b to try and fix.
Specifically:

[cltbld@bld-linux64-ec2-376.build.releng.usw2.mozilla.com ~]$ ps uxwww
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
...
cltbld    1412 17.0  6.4 1459264 990024 ?      Sl   16:28   5:19 /usr/bin/valgrind --error-exitcode=1 --smc-check=all-non-file --gen-suppressions=all --leak-check=full --num-callers=15 --show-possibly-lost=no --track-origins=yes --suppressions=/builds/slave/fx-team-l64-valgrind-000000000/objdir/_valgrind/cross-architecture.sup --suppressions=/builds/slave/fx-team-l64-valgrind-000000000/objdir/_valgrind/x86_64-redhat-linux-gnu.sup /builds/slave/fx-team-l64-valgrind-000000000/objdir/dist/firefox/firefox http://localhost:8888/index.html -profile /tmp/tmpB7OAsw
(Reporter)

Comment 10

4 years ago
I'm investigating the timeouts in bug 948145.  They only started happening yesterday, I don't know why.
in production
You need to log in before you can comment on or make changes to this bug.