Closed Bug 1553797 Opened 1 year ago Closed 1 year ago

Perma - late beta - TEST-UNEXPECTED-FAIL | services/common/tests/unit/test_blocklist_onecrl.js | xpcshell return code: 0 when Gecko 69 merges to Beta on 19-07-01

Categories

(Cloud Services :: Firefox: Common, defect, P1)

defect

Tracking

(firefox67 unaffected, firefox67.0.1 unaffected, firefox68 unaffected, firefox69+ verified)

VERIFIED FIXED
Firefox 69
Tracking Status
firefox67 --- unaffected
firefox67.0.1 --- unaffected
firefox68 --- unaffected
firefox69 + verified

People

(Reporter: dvarga, Assigned: leplatrem)

References

(Regression)

Details

Attachments

(1 file)

[Tracking Requested - why for this release]:

Central-as-beta simulation: https://treeherder.mozilla.org/#/jobs?repo=try&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel%2Crunnable&revision=e4badc6e8243638ecba464c3ae8b49a6626a502f&selectedJob=247949834

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=247949834&repo=try&lineNumber=2460

[task 2019-05-23T11:42:56.897Z] 11:42:56     INFO -  TEST-START | services/common/tests/unit/test_blocklist_onecrl.js
[task 2019-05-23T11:42:58.795Z] 11:42:58  WARNING -  TEST-UNEXPECTED-FAIL | services/common/tests/unit/test_blocklist_onecrl.js | xpcshell return code: 0
[task 2019-05-23T11:42:58.796Z] 11:42:58     INFO -  TEST-INFO took 1896ms
[task 2019-05-23T11:42:58.796Z] 11:42:58     INFO -  >>>>>>>
[task 2019-05-23T11:42:58.797Z] 11:42:58     INFO -  PID 16398 | [16398, Main Thread] WARNING: Couldn't get the user appdata directory. Crash events may not be produced.: file /builds/worker/workspace/build/src/toolkit/crashreporter/nsExceptionHandler.cpp, line 2561
[task 2019-05-23T11:42:58.798Z] 11:42:58     INFO -  PID 16398 | [16398, Main Thread] WARNING: NS_ENSURE_SUCCESS(rv, kKnownEsrVersion) failed with result 0x80004002: file /builds/worker/workspace/build/src/toolkit/components/resistfingerprinting/nsRFPService.cpp, line 662
[task 2019-05-23T11:42:58.798Z] 11:42:58     INFO -  PID 16398 | [16398, Main Thread] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80520012: file /builds/worker/workspace/build/src/extensions/permissions/nsPermissionManager.cpp, line 2906
[task 2019-05-23T11:42:58.799Z] 11:42:58     INFO -  (xpcshell/head.js) | test MAIN run_test pending (1)
[task 2019-05-23T11:42:58.799Z] 11:42:58     INFO -  (xpcshell/head.js) | test run_next_test 0 pending (2)
[task 2019-05-23T11:42:58.800Z] 11:42:58     INFO -  (xpcshell/head.js) | test MAIN run_test finished (2)
[task 2019-05-23T11:42:58.800Z] 11:42:58     INFO -  running event loop
[task 2019-05-23T11:42:58.800Z] 11:42:58     INFO -  services/common/tests/unit/test_blocklist_onecrl.js | Starting test_uses_a_custom_signer
[task 2019-05-23T11:42:58.800Z] 11:42:58     INFO -  (xpcshell/head.js) | test test_uses_a_custom_signer pending (2)
[task 2019-05-23T11:42:58.801Z] 11:42:58     INFO -  TEST-PASS | services/common/tests/unit/test_blocklist_onecrl.js | test_uses_a_custom_signer - [test_uses_a_custom_signer : 11] "onecrl.content-signature.mozilla.org" != "remote-settings.content-signature.mozilla.org"
[task 2019-05-23T11:42:58.801Z] 11:42:58     INFO -  (xpcshell/head.js) | test run_next_test 0 finished (2)
[task 2019-05-23T11:42:58.801Z] 11:42:58     INFO -  (xpcshell/head.js) | test run_next_test 1 pending (2)
[task 2019-05-23T11:42:58.802Z] 11:42:58     INFO -  (xpcshell/head.js) | test test_uses_a_custom_signer finished (2)
[task 2019-05-23T11:42:58.803Z] 11:42:58     INFO -  services/common/tests/unit/test_blocklist_onecrl.js | Starting test_has_initial_dump
[task 2019-05-23T11:42:58.803Z] 11:42:58     INFO -  (xpcshell/head.js) | test test_has_initial_dump pending (2)
[task 2019-05-23T11:42:58.804Z] 11:42:58     INFO -  (xpcshell/head.js) | test run_next_test 1 finished (2)
[task 2019-05-23T11:42:58.804Z] 11:42:58     INFO -  TEST-PASS | services/common/tests/unit/test_blocklist_onecrl.js | test_has_initial_dump - [test_has_initial_dump : 15] true == true
[task 2019-05-23T11:42:58.808Z] 11:42:58     INFO -  (xpcshell/head.js) | test run_next_test 2 pending (2)
[task 2019-05-23T11:42:58.809Z] 11:42:58     INFO -  (xpcshell/head.js) | test test_has_initial_dump finished (2)
[task 2019-05-23T11:42:58.809Z] 11:42:58     INFO -  services/common/tests/unit/test_blocklist_onecrl.js | Starting test_default_jexl_filter_is_used
[task 2019-05-23T11:42:58.810Z] 11:42:58     INFO -  (xpcshell/head.js) | test test_default_jexl_filter_is_used pending (2)
Flags: needinfo?(mathieu)

This test fails because of this:
https://hg.mozilla.org/mozilla-central/rev/ee7e04b12dd3#l4.290

The test assumes that revocations.txt exists in the profile, and apparently in this case it doesn't.

If it makes sense, we could add a safety check, but I'd like to get a confirmation from the PSM team that it's acceptable for this file to be missing in some situations.

Flags: needinfo?(mathieu) → needinfo?(jjones)

I could reproduce the failure locally by forcing MOZ_NEW_CERT_STORAGE
https://searchfox.org/mozilla-central/rev/952521e6164ddffa3f34bc8cfa5a81afc5b859c4/old-configure.in#1695-1698

But the test is supposed to be skipped when MOZ_NEW_CERT_STORAGE is true:
https://hg.mozilla.org/mozilla-central/rev/ee7e04b12dd3#l4.285

I'm not entirely sure it's related to my patch. I think I would need some help from Dana and Myk, who introduced this in https://bugzilla.mozilla.org/show_bug.cgi?id=1547877

Thanks 🙏

Flags: needinfo?(myk)
Flags: needinfo?(jjones)
Flags: needinfo?(dkeeler)

That looks correct.

Assignee: nobody → mathieu
Flags: needinfo?(dkeeler)
Priority: -- → P1

Is there a way to run a try build for this patch on the beta tree?

Flags: needinfo?(dvarga)

When I try to compile Firefox locally by forcing MOZ_NEW_CERT_STORAGE=0 in old-configure.in, I get this error NSSCertDBTrustDomain.h:17:12: fatal error: 'nsICertBlocklist.h' file not found.

I'd like to at least be able to test the patch somewhere before requesting landing :)

(In reply to Mathieu Leplatre [:leplatrem] from comment #6)

When I try to compile Firefox locally by forcing MOZ_NEW_CERT_STORAGE=0 in old-configure.in, I get this error NSSCertDBTrustDomain.h:17:12: fatal error: 'nsICertBlocklist.h' file not found.

I'd like to at least be able to test the patch somewhere before requesting landing :)

This would be because nsICertBlocklist.h isn't generated; if you've done previous builds there might be old stuff hanging around (see https://searchfox.org/mozilla-beta/source/security/manager/ssl/moz.build#184) maybe you need to clobber?

(In reply to Mathieu Leplatre [:leplatrem] from comment #6)

When I try to compile Firefox locally by forcing MOZ_NEW_CERT_STORAGE=0 in old-configure.in, I get this error NSSCertDBTrustDomain.h:17:12: fatal error: 'nsICertBlocklist.h' file not found.

I built beta, made this change, then built again:

diff --git a/old-configure.in b/old-configure.in
index ae37bffd5153d..60adaae960d24 100644
--- a/old-configure.in
+++ b/old-configure.in
@@ -1687,20 +1687,20 @@ if test "$BUILDING_RELEASE"; then
   # Override value in defines.sh, if any
   EARLY_BETA_OR_EARLIER=
 elif test "$EARLY_BETA_OR_EARLIER"; then
   AC_DEFINE(EARLY_BETA_OR_EARLIER)
 fi
 AC_SUBST(EARLY_BETA_OR_EARLIER)
 
 
-if test "$EARLY_BETA_OR_EARLIER"; then
-    MOZ_NEW_CERT_STORAGE=1
-    AC_DEFINE(MOZ_NEW_CERT_STORAGE)
-fi
+# if test "$EARLY_BETA_OR_EARLIER"; then
+#     MOZ_NEW_CERT_STORAGE=1
+#     AC_DEFINE(MOZ_NEW_CERT_STORAGE)
+# fi
 AC_SUBST(MOZ_NEW_CERT_STORAGE)
 
 # Allow someone to change MOZ_APP_NAME and MOZ_APP_BASENAME in mozconfig
 MOZ_ARG_WITH_STRING(app-name,
 [--with-app-name=APPNAME sets MOZ_APP_NAME to APPNAME],
 WITH_APP_NAME=$withval,
 )

And I didn't see the same failure, although I did see this one:

Undefined symbols for architecture x86_64:
  "construct_cert_storage(nsISupports*, nsID const&, void**)", referenced from:
      mozilla::xpcom::CreateInstanceImpl(mozilla::xpcom::ModuleID, nsISupports*, nsID const&, void**) in StaticComponents.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

However, after clobbering and reverting the change, I was unable to reproduce the problem, even after reapplying/reverting the change and rebuilding repeatedly (without clobbering in between builds). So I don't know why the change originally failed, nor why the build failure I saw is different from the one you saw.

Nevertheless, as @mgoodwin notes, you might resolve it by clobbering.

Flags: needinfo?(myk)

I tried this morning to modify old-configure.in on trunk, clobber and build, and still fatal error: 'nsICertBlocklist.h' file not found.

If I checkout FIREFOX_BETA_68_BASE, there's no MOZ_NEW_CERT_STORAGE in old-configure.in. So I modified toolkit/moz.configure, clobbered and recompiled. Then had other errors like use of undeclared identifier 'SK_R16_MASK'

I added this test to be thorough, but start to regret it...

If you have a local build with the old storage, could you please try the attached patch please?

Flags: needinfo?(myk)
Flags: needinfo?(dvarga)

(In reply to Mathieu Leplatre [:leplatrem] from comment #9)

I tried this morning to modify old-configure.in on trunk, clobber and build, and still fatal error: 'nsICertBlocklist.h' file not found.

It'd be easier to reproduce this failure if you provided the specific changes you're making to old-configure.in.

If I checkout FIREFOX_BETA_68_BASE, there's no MOZ_NEW_CERT_STORAGE in old-configure.in. So I modified toolkit/moz.configure, clobbered and recompiled. Then had other errors like use of undeclared identifier 'SK_R16_MASK'

I added this test to be thorough, but start to regret it...

If you have a local build with the old storage, could you please try the attached patch please?

If I apply the patch in comment 8 to the tip of mozilla-central and also apply your patch in https://phabricator.services.mozilla.com/D32331, then build and run the test services/common/tests/unit/test_blocklist_onecrl.js, it succeeds.

Flags: needinfo?(myk)

It'd be easier to reproduce this failure if you provided the specific changes you're making to old-configure.in.

Oh sorry :/ Basically I commented the few lines that sets MOZ_NEW_CERT_STORAGE to 1, like you did.

If I apply the patch in comment 8 to the tip of mozilla-central and also apply your patch in https://phabricator.services.mozilla.com/D32331, then build and run the test services/common/tests/unit/test_blocklist_onecrl.js, it succeeds.

Ok, excellent news :) Let's land this then!

Thank you!

Pushed by mleplatre@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/7c8a87a7e190
Fix OneCRL tests when MOZ_NEW_CERT_STORAGE is disabled r=keeler
Status: NEW → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 69
You need to log in before you can comment on or make changes to this bug.