Closed Bug 1560486 Opened 2 years ago Closed 9 months ago

firefox closes suddenly without crash (Estonian e-Residency: onepin-opensc-pkcs11.dll)

Categories

(Core :: Security: PSM, defect, P3)

Product:
Core
Component:
Security: PSM
Version:
68 Branch
Platform:
x86_64
Windows 10
Type:
defect

Tracking

()

Status:
RESOLVED INVALID
Tracking Flags:
Tracking Status
thunderbird_esr68 --- affected
firefox68 --- affected

People

(Reporter: leifeld, Unassigned, NeedInfo)

References

(Depends on 1 open bug,
URL
)

Details

(Keywords: crash, Whiteboard: [psm-smartcard])

Crash Data

Attachments

(4 files)

about.support.txt
20.67 KB, text/plain
Details
new about.support.txt
20.59 KB, text/plain
Details
memory.png
9.84 KB, image/png
Details
Report contents from Mozilla Bug Reporter
17.85 KB, text/plain
Details

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0

Steps to reproduce:

While surfing firefox closes randomly, maybe one time in an hour.
I did not find a way to reproduce it faster.

Webrender is enabled.

Actual results:

Firefox closes without anything in about:crashes
One time there was a dialog, that an memory error occurred.
I tried debugging firefox with windbg, but i didn't see something helpful.

Expected results:

It should not close randomly.

I can try to build everything from source, if that would be helpful.

Component: Untriaged → Graphics: WebRender
Product: Firefox → Core

leifeld, thanks for reporting. Can you upload about:support?
Open about:support, click on "Copy text to clipboard", paste it into a text file and attach it to the report. Thanks!

Flags: needinfo?(leifeld)

I wonder if Bug Bug 1559681 might be related to this bug.

See Also: → 1560490
See Also: 1560490 → 1559681
Attached file about.support.txt —

The about:support output

Flags: needinfo?(leifeld)

Version: 68.0b12

Compositing: WebRender

Description: Radeon(TM) RX 460 Graphics
Vendor ID: 0x1002
Device ID: 0x67ef

Qualified device by bug 1559375, therefore marking 68 as affected.

Failure Log
(#0): GP+[GFX1-]: shader-cache: Timed out before finishing loads
(#1): CP+[GFX1-]: Unexpected BufferProvider over-production.
(#2): CP+[GFX1-]: Unexpected BufferProvider over-production.

Blocks: wr-amd
status-firefox68: --- → affected
Keywords: crash
OS: Unspecified → Windows 10
Hardware: Unspecified → x86_64
Blocks: wr-stability
Severity: normal → critical

Background: Graphics problems usually don't close Firefox anymore. In case of a GPU process crash the window should only flash white for a second.

Name: Firefox PKCS11 loader
Version: 1.0.1

Name: IDEMIA PKCS11 loader
Version: 2.0.6

You are using two smartcard plugins that are using the WebExtension API provided by bug 1357391.
It's not unlikely that they could cause trouble. (bug 1396030 comment 0)

Please try disabling all these addons to see if that fixes the problem. Thanks!

I disabeld the addons and after an hour I got a new crash, this time with an call stack. It is uploaded with id
1fef7e06-3433-4002-93e1-f35fb0190625
https://crash-stats.mozilla.org/report/index/1fef7e06-3433-4002-93e1-f35fb0190625#tab-details

i guess, that it is quite common to load these two pkcs11 plugins, since it is the recocmended way to connect the estonian e residency.
see https://e-resident.gov.ee/welcome/ ⇢ install link ⇢ Firefox

Attached file new about.support.txt —

Thanks! Moving to the right component.

No longer blocks: wr-stability, wr-amd
URL: https://e-resident.gov.ee/welcome/
Crash Signature: [@ onepin-opensc-pkcs11.dll | BaseAllocator::malloc | SECMOD_UpdateSlotList ]
Component: Graphics: WebRender → Security: PSM
See Also: 1559681 → 1396030

The two PKCS11 loader Plugins are supposed to be enabled at once for the e residency: https://id.ee/index.php?id=34392

Strange that I got the crash, even with both plugins disabled (see second about.support.txt)

The PKCS11 WebExtension API is used to install PKCS#11 modules. Afterwards, I assume, these addons are only needed for updates.

Depends on: 1396030
See Also: 1396030 →
Summary: firefox closes suddenly without crash → firefox closes suddenly without crash (Estonian e-Residency: onepin-opensc-pkcs11.dll)
Attached image memory.png —

Firefox being terminated by Windows

Crashed again without anything in about:crashes
(In reply to Jan Andre Ikenmeyer [:darkspirit] from comment #11)

The PKCS11 WebExtension API is used to install PKCS#11 modules. Afterwards, I assume, these addons are only needed for updates.

Thank you for the explanation.

Got a new crash, which could be reported 8e936a70-394f-4445-b2ac-a142f0190701

(In reply to leifeld from comment #14)

Got a new crash, which could be reported bp-8e936a70-394f-4445-b2ac-a142f0190701

That one is empty :/

Priority: -- → P3
Whiteboard: [psm-smartcard]
Crash Signature: [@ onepin-opensc-pkcs11.dll | BaseAllocator::malloc | SECMOD_UpdateSlotList ] → [@ onepin-opensc-pkcs11.dll | BaseAllocator::malloc | SECMOD_UpdateSlotList ] [@ onepin-opensc-pkcs11.dll | SECMOD_UpdateSlotList ]

Bugbug thinks this bug is a regression, but please revert this change in case of error.

Keywords: regression

https://support.mozilla.org/en-US/questions/1283019#answer-1301119
User: I don't know if it's too early to say (been using FF for 3 hours) but removing the e-Recidency
software (onepin-opensc-pkcs11.dll) seems to have solved the problem.

I've seen about 50 crashes in the past week, and I finally got it under windbg.
I've got a rough, not particularly well-symbolicated backtrace:

(34b4.2ea8): C++ EH exception - code e06d7363 (first chance)
(34b4.2ea8): C++ EH exception - code e06d7363 (first chance)
(34b4.2ea8): C++ EH exception - code e06d7363 (first chance)
(34b4.2ea8): C++ EH exception - code e06d7363 (first chance)
(34b4.2ffc): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
opensc_pkcs11!C_GetTokenInfo+0x1c151d:
00007fff`ebfa6769 f3a4            rep movs byte ptr [rdi],byte ptr [rsi]

0:078> k
 # Child-SP          RetAddr           Call Site
00 000000a7`042be5f8 000001e2`75da51e0 opensc_pkcs11!C_GetTokenInfo+0x1c151d
01 000000a7`042be600 000001e2`6cc51720 0x000001e2`75da51e0
02 000000a7`042be608 00007fff`ebe7c950 0x000001e2`6cc51720
03 000000a7`042be610 00007fff`ebdea395 opensc_pkcs11!C_GetTokenInfo+0x97704
04 000000a7`042be6a0 00007fff`ebdda176 opensc_pkcs11!C_GetTokenInfo+0x5149
05 000000a7`042be6d0 00007fff`f86ad296 opensc_pkcs11!C_GetSlotList+0x7e
06 000000a7`042be720 00007fff`e12e7434 nss3!SECMOD_UpdateSlotList+0x66
07 000000a7`042be790 00007fff`de4eb496 xul!soundtouch::SoundTouch::operator=+0x13d1e34
08 000000a7`042be810 00007fff`de4eccf3 xul!workerlz4_compress+0x29ba26
09 000000a7`042beb10 00007fff`e12c0fa1 xul!workerlz4_compress+0x29d283

This may be relevant to this issue. I'll work on symbols next.

symbolicated.

0:000> k
 # Child-SP          RetAddr           Call Site
00 00000061`1ddfc090 00007fff`d996df9c opensc_pkcs11!_invoke_watson+0x17 [f:\dd\vctools\crt\crtw32\misc\invarg.c @ 132] 
01 00000061`1ddfc0c0 00007fff`d996dfb9 opensc_pkcs11!_invalid_parameter+0x64 [f:\dd\vctools\crt\crtw32\misc\invarg.c @ 85] 
02 00000061`1ddfc100 00007fff`d9972f65 opensc_pkcs11!_invalid_parameter_noinfo+0x19 [f:\dd\vctools\crt\crtw32\misc\invarg.c @ 97] 
03 00000061`1ddfc140 00007fff`d997ed23 opensc_pkcs11!strcpy_s+0x29 [f:\dd\vctools\crt\crtw32\h\tcscpy_s.inl @ 18] 
04 00000061`1ddfc170 00007fff`d983c31c opensc_pkcs11!_strdup+0x3f [f:\dd\vctools\crt\crtw32\string\strdup.c @ 80] 
05 00000061`1ddfc1b0 00007fff`d983c99d opensc_pkcs11!pcsc_add_reader+0x90 [c:\projects\opensc\src\libopensc\reader-pcsc.c @ 1274] 
06 00000061`1ddfc1e0 00007fff`d97aa395 opensc_pkcs11!pcsc_detect_readers+0x27d [c:\projects\opensc\src\libopensc\reader-pcsc.c @ 1403] 
07 00000061`1ddfc270 00007fff`d979a176 opensc_pkcs11!sc_ctx_detect_readers+0x39 [c:\projects\opensc\src\libopensc\ctx.c @ 733] 
08 00000061`1ddfc2a0 00007fff`f86ad296 opensc_pkcs11!C_GetSlotList+0x7e [c:\projects\opensc\src\pkcs11\pkcs11-global.c @ 407] 
09 00000061`1ddfc2f0 00007fff`e12e7434 nss3!SECMOD_UpdateSlotList+0x66 [/builds/worker/checkouts/gecko/security/nss/lib/pk11wrap/pk11util.c @ 1028] 
0a 00000061`1ddfc360 00007fff`e12f0351 xul!nsNSSComponent::CheckForSmartCardChanges+0xe4 [/builds/worker/checkouts/gecko/security/manager/ssl/nsNSSComponent.cpp @ 849] 
0b (Inline Function) --------`-------- xul!CheckForSmartCardChanges+0x3b [/builds/worker/checkouts/gecko/security/manager/ssl/nsNSSComponent.h @ 149] 
0c 00000061`1ddfc3e0 00007fff`e29313b2 xul!nsPKCS11Module::ListSlots+0x61 [/builds/worker/checkouts/gecko/security/manager/ssl/nsPKCS11Slot.cpp @ 250] 
0d 00000061`1ddfc470 00007fff`ddccf573 xul!XPTC__InvokebyIndex+0x72

from pcsc_add_reader

0:000> dv
            ctx = 0x000001a2`cefff5d0
    reader_name = 0x00000061`1ddfc228 " \i???"
reader_name_len = 0x000001a2`c680a74f
     out_reader = 0x000001a2`cedd1840

reader_name and reader_name_len appear to be bogus. This is likely an upstream issue that Firefox can't necessarily fix.

Can you try:

  1. Downloading Firefox Beta (75)
  2. Ensure the e-residency software is not installed in Firefox Beta
  3. Navigate to about:config and search for security.osclientcerts.autoload
  4. Set security.osclientcerts.autoload to true
  5. Try to use a website that still needs the e-residency Smart Card and see if it works?

If so, then I think the realistic resolution here is to recommend setting security.osclientcerts.autoload to true for Firefox users instead of using OpenSC. I suspect you're right that this is an upstream issue.

Flags: needinfo?(dustin)

With security.osclientcerts.autoload on true on 75 I can log in with the e-Residency passport!
With it on false I cannot, and I removed the dll of the software some time ago.

But on the e-residency website the they still recommend the installation of the software.

This looks like an OpenSC issue. Debug symbols are available here. Despite the trace above (thanks!), I cannot immediately see what's going wrong. It would be interesting to see what the value of the internal reader_buf is, which is returned from SCardListReaders. If you wish we can discuss this in the OpenSC issue.

See Also: → 1621804, 1636100
See Also: → 1639537

I'm reporter of 1621804.

Installed an updated ID-card SW. https://id.ee/?id=30519&read=39806

Same crashes on FF78b.

It's the plugin that crashes. It doesn't seem to be fixed yet:

(J.C. Jones [:jcj] (he/him) from bug 1636100 comment 9)

The upstream issue at OpenSC is https://github.com/OpenSC/OpenSC/issues/1999.

They ask affected users for debug information:
https://github.com/OpenSC/OpenSC/issues/1999#issuecomment-627297400
https://github.com/OpenSC/OpenSC/issues/1999#issuecomment-628531648

FYI we've unblocked the most recent release of OpenSC upon request from the team developing the Estonian e-Residency software which explains the spike in crashes in the last month. They claimed a new version - integrating the recent OpenSC fixes - would be available soon. I'll ping them to know what's going on. We're in a really bad spot here: either our users experience crashes or we have to disable some important functionality they rely on.

Status: UNCONFIRMED → NEW
Ever confirmed: true
Duplicate of this bug: 1643312

The latest version is still crashing:
ID-software version: 20.05, DigiDoc4 client version: 4.2.5.76, released 08.06.2020

https://installer.id.ee/?lang=eng

Also crashes Thunderbird

QA Whiteboard: [tbird crash]
status-thunderbird_esr68: --- → affected

Win10 64-bit Build 19042 ,FF 77.0.1 and DigiDoc4 client 4.2.5.76, 20.05 release- no crash for me. Tried with 2 different pc with same version.
Only difference is that I'm using ID-card, not e-resident one.

If anyone can reproduce the issue, please share debug information:
https://github.com/OpenSC/OpenSC/issues/1999#issuecomment-627297400
https://github.com/OpenSC/OpenSC/issues/1999#issuecomment-628531648

(In reply to Eugene Savitsky from comment #31)

The latest version is still crashing:
ID-software version: 20.05, DigiDoc4 client version: 4.2.5.76, released 08.06.2020

https://installer.id.ee/?lang=eng

I would recommend writing to the software developer at help@ria.ee and send OpenSC logs there as well. Might speed up troubleshoot & fixes.

This isn't a bug in Firefox, so I'm going to close this. For users experiencing this, please turn on osclientcerts instead by setting security.osclientcerts.autoload to true in about:config.

Status: NEW → RESOLVED
Closed: 9 months ago
Resolution: --- → INVALID

Note that thanks to bug 1633052 this type of crashes will never be silent again. The next time we run into something like this Firefox will generate a proper crash report which should help both identify the issue and let upstream projects develop a fix.

See Also: → 1560052

Hello,
I updated my Windows 10 to the 2004 version recently I started getting these sudden crashes (no crash report). One even took place in the middle of writing this message. I tried using Dana's solution of changing the osclientcerts to true (is false by default) but to no avail. I also tried to disable the Estonian ID plugins, but that did not help either.

Mozilla Firefox: 79.0 (64-bit)
Windows 10 Professional: Version 2004, 19041.450
Windows Feature Experience Pack: 120.2212.31.0
Estonian ID plugins installed:

PKSC11 Loader: version 1.0.5
Token signing: version 0.0.31

Please provide assistance/ideas how to prevent Firefox from crashing.

Thanks in advance,
Rasmus

Seems that there is no way to edit my posts here so adding some extra. The issues is happening several times in an hour and is getting very annoying. Is there any way to reopen this case or direct me to a new one?

This is from the bug reporter.

Submitted the crash report as well and it does look that the same issue has indeed resurfaced: https://crash-stats.mozilla.org/report/index/ced71081-7b15-4bde-b44d-4f7b60200816

Please reopen this bug or provide a working workaround.

(In reply to Rasmus from comment #39)

Submitted the crash report as well and it does look that the same issue has indeed resurfaced: https://crash-stats.mozilla.org/report/index/ced71081-7b15-4bde-b44d-4f7b60200816

Please reopen this bug or provide a working workaround.

As Dana said, this isn't Firefox bug rather coming from OpenSC https://github.com/OpenSC/OpenSC/issues/1999#issuecomment-627297400.
Seems like the problem found a solution and needs to be implemented to releases https://github.com/OpenSC/OpenSC/projects/7.
I would suggest using alternative browsers or run FF in safe mode before OpenSC and Estonian ID release a newer version.

Yes decided to go for that, mostly because I do not use the actual ID software features that much. So I reinstalled the client without the Firefox plugins and will use other browsers when necessary.

If you use osclientcerts in Firefox Beta (80) (without the Estonian ID plugins), does it work for you? (as in, can you authenticate to the websites you need to authenticate to?)

Flags: needinfo?(rasmus.rahnu)

That is a lot of testing you want me to do without clear and simple instructions how to do it. I already applied the workaround of re-installing the ID software without Firefox plugins (the only way to remove them apparently) and will use Chrome/Edge to do any ID plugin required tasks if they come up in the future. My case can be considered as closed.

Flags: needinfo?(rasmus.rahnu)
  1. Download and install Firefox Beta: https://www.mozilla.org/en-US/firefox/channel/desktop/
  2. Open Firefox Beta
  3. Navigate to about:config
  4. Search for security.osclientcerts.autoload
  5. Click the toggle button on the far right, which should set the value of this preference to true
  6. Attempt to do something that would have required the ID plugin
Flags: needinfo?(rasmus.rahnu)

hi to you all, apologize my poor english and skills
i only if this bug with the impossibility of loading asepkcs.dll
has been fixed
i'm now running the stable last ff vers 80 but i'm still unable to load that module in order to have my Athena smart card running
thxs so much indeed for yr kind attention and patience too
cheers

Still happening in Firefox Nightly... removed onepin-opensc-pkcs11.dll file and all good now.

so also in vers stable 80 latest this trick is working and permitt to load the asepkcs.dll...??

(In reply to Giusy from comment #45)

hi to you all, apologize my poor english and skills
i only if this bug with the impossibility of loading asepkcs.dll
has been fixed
i'm now running the stable last ff vers 80 but i'm still unable to load that module in order to have my Athena smart card running
thxs so much indeed for yr kind attention and patience too
cheers

The module is not blocked anymore and it should load correctly. We removed the block at the request of the Estonian Information System Authority so the most recent version of the plug-in will load with Firefox 80. I do not know if the crash was also resolved in the most recent version of the Estonian ID-card software; it was resolved in upstream OpenSC though. Unfortunately when the ID-card software picks up the latest version of OpenSC is outside of our control, what you can do in the meantime is follow the instructions in comment 44.

hi Gabriele,
first let me thank you so much for yr kind reply,
now i have finally be able to re-load the asepkcs.dll!
i have the oldest vers 6.5.0.5
otherwise the method of post 44 did not work for me...
i mean that if i've do not have the dll loaded also if i set in about config the parameter from false to true i'm really not able to navigate the site for which i've my athena smart card
about the Estonian asepkcs.dll how to have it?
cause is surely newest then mine and i'm afraid tha probably in next future ff will block again the dll probably starting from the oldest vers like mine
another user: Alcr in bugzilla n. 1629002 talks about:
"@Alcr. in yr post 3 you state to have the :

I think that I found a more recent asepkcs.dll from the Colegio de Abogados La Plata website:

http://www.calp.org.ar/download/controladores-token

(Controlador para Athena ID-Protect para Windows de 64 bits.)
asepkcs.dll version is 7.0.2.0"

...but how he did..??...if you dl the file is a zipped that require a password...:

Controlador para Athena ID-Protect en Windows

https://www.macroseguridad.net/soporte/download/detectar.php

to sum up now finally my ff works like in the past with the .dll loaded but is a very old vers 6.5.0.5 2016 so i'm afraid that this situation may vary in the very netx time..

cheers

The issue in Estonian ID SW still persist.

They plan to release a new version in September. But will it resolve the issue - is I understand it should be first fixed in OpenSC module:
https://github.com/OpenSC/OpenSC/projects/7
Which will happen in ver. 0.21, but it has yet to be released.

https://www.id.ee/en/article/the-next-version-of-the-id-card-software-will-introduce-several-significant-changes/

You need to log in before you can comment on or make changes to this bug.