1262909 - Directory Server segfaults caused by CHECK_FORK calls in {C|NSC}_GetFunctionList

Status: RESOLVED FIXED

(NSS :: Libraries, defect)

Description

[Elio Maldonado]

The Red Hat Directory server segfaults. This was reported by Mark Sauton and I slightly edited it for clarity.

[:error] [pid 19707:tid 139825548642368] NSS_Initialize failed. Certificate database: /etc/dirsrv/admin-serv.
[:error] [pid 19707:tid 139825548642368] SSL Library Error: -8038 SEC_ERROR_NOT_INITIALIZED
[core:notice] [pid 19705:tid 139825548642368] AH00052: child pid 19707 exit signal Segmentation fault (11)
[:error] [pid 19708:tid 139825548642368] NSS_Initialize failed. Certificate database: /etc/dirsrv/admin-serv.
[:error] [pid 19708:tid 139825548642368] SSL Library Error: -8038 SEC_ERROR_NOT_INITIALIZED
[core:notice] [pid 19705:tid 139825548642368] AH00052: child pid 19708 exit signal Segmentation fault (11)

I'm 100% able to reproduce this:
1. Install RHDS10 and configure SSL on the admin server
2. Install the updates NSS packages
3. Stop dirsrv and admin server
   # systemctl stop dirsrv.target
   # systemctl stop dirsrv-admin
4. Start dirsrv
   # systemctl start dirsrv.target
5. Wait for the dirsrv is initialized and running
   # sleep 5
6. Start admin server
   # systemctl start dirsrv-admin
7. Check logs
   # tail -f /var/log/dirsrv/admin-serv/error
8. Notice the "NSS_Initialize failed" error msg.

So if everything is stopped and the admin server is started first, the errors
don't appear and the console and dirsrv are operating fine.

The reason was determined to be CHECK_FORK() calls in nss-softokn in pkcs11.c. 

The fix was to remove the CHECK_FORK() calls at the start of 
CK_RV NSC_GetFunctionList(CK_FUNCTION_LIST_PTR *pFunctionList) and
CK_RV C_GetFunctionList(CK_FUNCTION_LIST_PTR *pFunctionList)

Comment 1

[Ryan Sleevi]

Elio, did you mean to file this in directory server? The API contract preventing forking is a core part, as explained in https://bugzilla.mozilla.org/show_bug.cgi?id=331096

Comment 2

[Elio Maldonado]

Attachment: Removes CHECK_FORK from CK_RV NSC_GetFunctionList and CK_RV C_GetFunctionList

Comment 3

[Elio Maldonado]

(In reply to Ryan Sleevi from comment #1)
> Elio, did you mean to file this in directory server? The API contract
> preventing forking is a core part, as explained in
> https://bugzilla.mozilla.org/show_bug.cgi?id=331096

Thank you Ryan for the pointer https://bugzilla.mozilla.org/show_bug.cgi?id=331096 it explains a lot.
To answer you question, it was meant for NSS not DS. There is something I hadn't mentioned and that's that problem that the patch was applied downstream years ago on RHEL-6 days and the problem resurfaced on latest RHEL-7 as the customer was doing a massive migration from 6 to 7 where the patch wasn't brought forward. I was wandering how come I never brought upstream, as we normally do, and reading the other bug tells me why.

Comment 4

[Robert Relyea]

The problem is when an application opens NSS and forks, and another application or library tries to initialize NSS. The fix for reseting softoken in the application is to call C_Initialize again, but you can't call C_Initialize until you get the function list. The function list currently has the fork check in it (which it shouldn't), so the patch removes that fork check so the application can call C_GetFunctionList and C_Initialize().

bob

Comment 5

[Kai Engert [:KaiE:]]

It looks like this patch was r+'ed but then forgotten...

Elio, please set need-info to me, if you want me to check it in.

Comment 6

[Elio Maldonado]

Yes, please check it in. Thank you.

Comment 7

[Kai Engert [:KaiE:]]

needs target milestone 3.25 when available.

https://hg.mozilla.org/projects/nss/rev/e2b3ef6ac27b