1548059 - Analyze if PVS-Studio is useful as a static analyses tool

Status: RESOLVED WONTFIX

(Developer Infrastructure :: Source Code Analysis, task, P3)

Description

[Mayank Bansal]

Found this blog post : https://www.viva64.com/en/b/0629/
Sounded interesting.
Maybe we should try to analyze if it is useful for Firefox source code.

Comment 1

[Mayank Bansal]

It is most probably not free

Comment 2

[Sylvestre Ledru [:Sylvestre]]

Last time I checked, they required to update the headers to add some advertisement for the tool.
However, maybe we could pay if it finds different/more stuff than coverity..
Anyway, we had a meta bug about it: bug 710966

Comment 3

[Sylvestre Ledru [:Sylvestre]]

Still the case:
https://www.viva64.com/en/b/0457/

Make edits in all the compilable files of your project. I.e. in all the files with the extensions c, cc, cpp, cs, and so on.

Comment 4

[Sylvestre Ledru [:Sylvestre]]

Attachment: project.tasks

They sent me a trial license to evaluate the product and here is a run on Firefox code.
Found 12051 defects (including thirdparty code)
on b45f9acbdfe4 / Sat Aug 10 01:17:41 2019 +0300

Comment 5

[Sylvestre Ledru [:Sylvestre]]

Super easy to test
$ ./mach build-backend -b CompileDB
=> generate the compile db
$ pvs-studio-analyzer analyze -l pvs.lic -o project.log -j30
=> run it
$ plog-converter -a GA:1,2 -t tasklist -o project.tasks project.log
=> generate the list

Comment 6

[Sylvestre Ledru [:Sylvestre]]

Attachment: project-without-obj-defect.tasks

Here are the results without:

• thirdparty code (including nss or nspr)
• the /obj-*/ dir content

Comment 7

[Sylvestre Ledru [:Sylvestre]]

Attachment: project-obj-defect.tasks

Here, only the defects found in /obj-*/

Comment 8

[Andrey Karpov]

https://www.viva64.com/en/b/0457/

This is old article.
Ways to Get a Free PVS-Studio License: https://www.viva64.com/en/b/0614/

Comment 9

[Andrey Karpov]

Found 12051 defects (including thirdparty code)

How to quickly check out interesting warnings given by the PVS-Studio analyzer for C and C++ code? https://www.viva64.com/en/b/0633/

Comment 10

[Andi [:andi]]

Looking other the results generated by PVS-Studio and doing my own scans for the past two days I can say that the results have improved since last time when we've used it, and the ratio of false-positives decreased.
Most of the checkers that PVS has are also part, or can be implemented, in the clang-tidy suit. Some of them, that are not currently present in the clang tooling will be very easy to implement after the new framework for AST-Matchers will be in place.

Since we already have Coverity as a static-analysis tool I feel the need to make a comparison with PVS-Studio. The current results strongly continue to sustain the idea that, for the time being, Coverity is the best choice, specially because of it cross compilation unit support and stack deduction for a defect, making the decision if the issue is true of false much easier. Also a another major key-point that I see for Coverity is it's symbolic analysis capabilities and the branching of multiple nested if statements.

Also I feel the need to raise one of the major benefits of PVS-Studio and in the same time the drawbacks of Coverity, it's pipeline for static-analysis is very similar to the framework that we use for clang-tidy, since it used the compilation database, this makes the integration of the analysis and not branching the logic a lot easier. For the moment we have two separate branches for the same C/C++ code for static-analysis, clang-tidy and Coverity.

As final thoughts, we should continue to monitor the evolution of PVS-Studio since it has the potential to be of interest to us in the future.

Comment 11

[Sylvestre Ledru [:Sylvestre]]

Yeah, we won't be using PVS for now. The cost is similar to Coverity with less features (except that it is way easier to run). And the open source access of PVS is way too strict (we won't be updating Firefox headers to use a software) and we only have mirror on github/bitbucket (so, we cannot use the new PVS option) . I am sure we would use it without such restriction.

A bunch of features are also available in clang-tidy.