1404547 - [meta] Enable 'enum' UBSan check

Status: RESOLVED FIXED

(Core :: Sanitizers, enhancement)

Description

[Arthur Edelstein [:arthur]]

There are only a few more -fsanitize=enum in Mozilla's automated tests on the gcc Linux build. (See the bugs this bug depends on.) Once those are all fixed, we should be able to add the flags
`-fsanitize=enum -fno-sanitize-recover=enum`
to the debug build (or maybe the ASAN) build so that any future UBSAN enum errors are caught before they land.

Comment 1

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

This isn't a bmo bug, moving it somewhere slightly more appropriate.

Comment 2

[Arthur Edelstein [:arthur]]

I ran an fsanitize=enum on a gcc build to see what errors are remaining.
https://treeherder.mozilla.org/#/jobs?repo=try&revision=d464e6aade11&selectedJob=134174002

I used the following script to download the logs files and extract and enumerate the runtime errors:
https://github.com/arthuredelstein/firefox-ubsan-errors

The results are as follows:
https://docs.google.com/spreadsheets/d/1yB5TR8TYBa8zNkNLo8UhsjjkzypVDCEtnNVAkE23yLo/edit?usp=sharing

Comment 3

[Arthur Edelstein [:arthur]]

Running -fsanitize=enum on linux64-asan (clang) detected 5 more enum error locations that gcc didn't detect. :P

Comment 4

[Tyson Smith [:tsmith]]

Things have changed a bit since this bug was initially opened. Instead of adding the check to debug builds we will enable it on the ASan + UBSan builds.

Comment 5

[Tyson Smith [:tsmith]]

Attachment: Bug 1404547 - Enable 'enum' UBSan check. r=froydnj

Comment 6

[Tyson Smith [:tsmith]]

Oops wrong component.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4af4d32ee984
Enable 'enum' UBSan check. r=froydnj

Comment 8

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/4af4d32ee984

Comment 9

[Alexandru Michis [:malexandru]]

Backed out changeset 4af4d32ee984 for causing wpt leaks.

Backout link: https://hg.mozilla.org/integration/autoland/rev/b686373c4c6da27d6fd2516900463df5af709159

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&searchStr=linux%2C18.04%2Cx64%2Casan%2Copt%2Cweb%2Cplatform%2Ctests%2Ctest-linux1804-64-asan%2Fopt-web-platform-tests-e10s-22%2Cw%28wpt22%29&tochange=122b34044596c640b64f7d93bc22dd421de83095&fromchange=c26f12bfc75c306cfb235392b4143198a977bb7e&selectedJob=291584252

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=291593371&repo=autoland&lineNumber=29708

Comment 10

[Tyson Smith [:tsmith]]

Attachment: Bug 1404547 - Enable 'enum' UBSan check. r=froydnj,mccr8

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4a007d28149d
Enable 'enum' UBSan check. r=froydnj,mccr8

Comment 12

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/4a007d28149d