1873296 - Switch to JSON for NIST AES KWP test vectors

Status: RESOLVED FIXED

(NSS :: Test, enhancement, P3)

Description

[John Schanck [:jschanck]]

Large static arrays are known to cause slow builds and gtests/common/testvectors/kwp-vectors.h has a real doozy in it. Optimized asan builds of NSS are taking around 12 minutes on my 64 core machine, and it seems to be getting worse with new versions of clang. Using post_build_ninja_summary.py I found that almost all of this time is spent in building gtests/pk11_gtest/pk11_gtest.pk11_aeskeywrapkwp_unittest.o.

$ ./build.sh -c -o --clang --asan
[...]
$ python post_build_ninja_summary.py -C out/Release/
    Longest build steps:
           0.9 weighted s to build obj/gtests/ssl_gtest/ssl_gtest.ssl_loopback_unittest.o (11.2 s elapsed time)
           0.9 weighted s to build obj/gtests/pk11_gtest/pk11_gtest.pk11_hkdf_unittest.o (12.6 s elapsed time)
           1.0 weighted s to build obj/gtests/ssl_gtest/ssl_gtest.tls_ech_unittest.o (11.5 s elapsed time)
           1.3 weighted s to build obj/gtests/pk11_gtest/pk11_gtest.pk11_dsa_unittest.o (15.0 s elapsed time)
           1.5 weighted s to build obj/gtests/ssl_gtest/ssl_gtest.ssl_extension_unittest.o (13.1 s elapsed time)
           1.9 weighted s to build obj/gtests/mozpkix_gtest/mozpkix_gtest.pkixnames_tests.o (18.3 s elapsed time)
           3.6 weighted s to build obj/gtests/ssl_gtest/ssl_gtest.ssl_auth_unittest.o (17.3 s elapsed time)
           6.3 weighted s to build obj/gtests/pk11_gtest/pk11_gtest.pk11_hmac_unittest.o (30.1 s elapsed time)
          10.2 weighted s to build obj/lib/sqlite/sqlite.sqlite3.o (42.1 s elapsed time)
         679.4 weighted s to build obj/gtests/pk11_gtest/pk11_gtest.pk11_aeskeywrapkwp_unittest.o (721.6 s elapsed time)

We should read these tests from JSON instead.

Comment 1

[John Schanck [:jschanck]]

Attachment: Bug 1873296 - read KWP testvectors from JSON. r=#nss-reviewers

Comment 2

[John Schanck [:jschanck]]

With the patch:

    Longest build steps:
           0.7 weighted s to build obj/gtests/pk11_gtest/pk11_gtest.pk11_aes_gcm_unittest.o (11.4 s elapsed time)
           0.9 weighted s to build obj/gtests/ssl_gtest/ssl_gtest.ssl_loopback_unittest.o (10.2 s elapsed time)
           0.9 weighted s to build obj/gtests/pk11_gtest/pk11_gtest.pk11_hkdf_unittest.o (12.3 s elapsed time)
           1.4 weighted s to build obj/gtests/ssl_gtest/ssl_gtest.tls_ech_unittest.o (11.8 s elapsed time)
           1.6 weighted s to build obj/gtests/pk11_gtest/pk11_gtest.pk11_dsa_unittest.o (15.1 s elapsed time)
           1.7 weighted s to build obj/gtests/ssl_gtest/ssl_gtest.ssl_extension_unittest.o (12.4 s elapsed time)
           2.2 weighted s to build obj/gtests/mozpkix_gtest/mozpkix_gtest.pkixnames_tests.o (18.3 s elapsed time)
           5.8 weighted s to build obj/gtests/ssl_gtest/ssl_gtest.ssl_auth_unittest.o (16.5 s elapsed time)
           8.8 weighted s to build obj/gtests/pk11_gtest/pk11_gtest.pk11_hmac_unittest.o (29.6 s elapsed time)
          16.0 weighted s to build obj/lib/sqlite/sqlite.sqlite3.o (41.3 s elapsed time)

Comment 3

[John Schanck [:jschanck]]

https://hg.mozilla.org/projects/nss/rev/a7a87585b7c41eeddc790e3bbc883cf207f357f2