1924439 - Perma ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)

Status: NEW

(Testing :: Talos, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: imoraru [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=478111323&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/TyznSr2gSE6nnq2GVhmgIA/runs/0/artifacts/public/logs/live_backing.log

---

[task 2024-10-14T06:35:22.685Z] fetch-talos-pdf: Gathering PDF pdfs/pdkids.pdf...
[task 2024-10-14T06:35:22.685Z] fetch-talos-pdf: Downloading from PDF link: https://www.archive.org/download/PdKids/PdKids.pdf
[task 2024-10-14T06:35:22.685Z] Traceback (most recent call last):
[task 2024-10-14T06:35:22.685Z]   File "/builds/worker/workspace/build/src/third_party/python/urllib3/urllib3/connectionpool.py", line 714, in urlopen
[task 2024-10-14T06:35:22.685Z]     httplib_response = self._make_request(
[task 2024-10-14T06:35:22.685Z]                        ^^^^^^^^^^^^^^^^^^^
[task 2024-10-14T06:35:22.685Z]   File "/builds/worker/workspace/build/src/third_party/python/urllib3/urllib3/connectionpool.py", line 403, in _make_request
[task 2024-10-14T06:35:22.685Z]     self._validate_conn(conn)
[task 2024-10-14T06:35:22.685Z]   File "/builds/worker/workspace/build/src/third_party/python/urllib3/urllib3/connectionpool.py", line 1053, in _validate_conn
[task 2024-10-14T06:35:22.685Z]     conn.connect()
[task 2024-10-14T06:35:22.685Z]   File "/builds/worker/workspace/build/src/third_party/python/urllib3/urllib3/connection.py", line 419, in connect
[task 2024-10-14T06:35:22.685Z]     self.sock = ssl_wrap_socket(
[task 2024-10-14T06:35:22.685Z]                 ^^^^^^^^^^^^^^^^
[task 2024-10-14T06:35:22.685Z]   File "/builds/worker/workspace/build/src/third_party/python/urllib3/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
[task 2024-10-14T06:35:22.685Z]     ssl_sock = _ssl_wrap_socket_impl(
[task 2024-10-14T06:35:22.685Z]                ^^^^^^^^^^^^^^^^^^^^^^
[task 2024-10-14T06:35:22.685Z]   File "/builds/worker/workspace/build/src/third_party/python/urllib3/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
[task 2024-10-14T06:35:22.685Z]     return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
[task 2024-10-14T06:35:22.685Z]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[task 2024-10-14T06:35:22.685Z]   File "/usr/lib/python3.11/ssl.py", line 517, in wrap_socket
[task 2024-10-14T06:35:22.686Z]     return self.sslsocket_class._create(
[task 2024-10-14T06:35:22.686Z]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[task 2024-10-14T06:35:22.686Z]   File "/usr/lib/python3.11/ssl.py", line 1075, in _create
[task 2024-10-14T06:35:22.686Z]     self.do_handshake()
[task 2024-10-14T06:35:22.686Z]   File "/usr/lib/python3.11/ssl.py", line 1346, in do_handshake
[task 2024-10-14T06:35:22.686Z]     self._sslobj.do_handshake()
[task 2024-10-14T06:35:22.686Z] ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)
[task 2024-10-14T06:35:22.686Z] 
[task 2024-10-14T06:35:22.686Z] During handling of the above exception, another exception occurred:
[task 2024-10-14T06:35:22.686Z] 
[task 2024-10-14T06:35:22.686Z] Traceback (most recent call last):
[task 2024-10-14T06:35:22.686Z]   File "/builds/worker/workspace/build/src/third_party/python/requests/requests/adapters.py", line 486, in send
[task 2024-10-14T06:35:22.686Z]     resp = conn.urlopen(
[task 2024-10-14T06:35:22.686Z]            ^^^^^^^^^^^^^
[task 2024-10-14T06:35:22.686Z]   File "/builds/worker/workspace/build/src/third_party/python/urllib3/urllib3/connectionpool.py", line 798, in urlopen
[task 2024-10-14T06:35:22.686Z]     retries = retries.increment(
[task 2024-10-14T06:35:22.686Z]               ^^^^^^^^^^^^^^^^^^
[task 2024-10-14T06:35:22.686Z]   File "/builds/worker/workspace/build/src/third_party/python/urllib3/urllib3/util/retry.py", line 592, in increment
[task 2024-10-14T06:35:22.686Z]     raise MaxRetryError(_pool, url, error or ResponseError(cause))
[task 2024-10-14T06:35:22.686Z] urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.archive.org', port=443): Max retries exceeded with url: /download/PdKids/PdKids.pdf (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)')))
[task 2024-10-14T06:35:22.686Z] 
[task 2024-10-14T06:35:22.686Z] During handling of the above exception, another exception occurred:
[task 2024-10-14T06:35:22.686Z] 
[task 2024-10-14T06:35:22.686Z] Traceback (most recent call last):
[task 2024-10-14T06:35:22.686Z]   File "/builds/worker/workspace/build/src/taskcluster/scripts/misc/fetch-talos-pdfs.py", line 120, in <module>
[task 2024-10-14T06:35:22.686Z]     gather_talos_pdfs(pdf_js_repo, output_dir_path)
[task 2024-10-14T06:35:22.686Z]   File "/builds/worker/workspace/build/src/taskcluster/scripts/misc/fetch-talos-pdfs.py", line 96, in gather_talos_pdfs
[task 2024-10-14T06:35:22.686Z]     gather_talos_pdf(test_folder, pdf_info, output_dir)
[task 2024-10-14T06:35:22.687Z]   File "/builds/worker/workspace/build/src/taskcluster/scripts/misc/fetch-talos-pdfs.py", line 70, in gather_talos_pdf
[task 2024-10-14T06:35:22.687Z]     fetch_talos_pdf_link(
[task 2024-10-14T06:35:22.687Z]   File "/builds/worker/workspace/build/src/taskcluster/scripts/misc/fetch-talos-pdfs.py", line 50, in fetch_talos_pdf_link
[task 2024-10-14T06:35:22.687Z]     fetch_file(pdf_link, output_file)
[task 2024-10-14T06:35:22.687Z]   File "/builds/worker/workspace/build/src/third_party/python/redo/redo/__init__.py", line 230, in _retriable_wrapper
[task 2024-10-14T06:35:22.687Z]     return retry(func, args=args, kwargs=kwargs, *retry_args, **retry_kwargs)
[task 2024-10-14T06:35:22.687Z]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[task 2024-10-14T06:35:22.687Z]   File "/builds/worker/workspace/build/src/third_party/python/redo/redo/__init__.py", line 185, in retry
[task 2024-10-14T06:35:22.687Z]     return action(*args, **kwargs)
[task 2024-10-14T06:35:22.687Z]            ^^^^^^^^^^^^^^^^^^^^^^^
[task 2024-10-14T06:35:22.687Z]   File "/builds/worker/workspace/build/src/taskcluster/scripts/misc/fetch-talos-pdfs.py", line 34, in fetch_file
[task 2024-10-14T06:35:22.687Z]     r = requests.get(url, stream=True)
[task 2024-10-14T06:35:22.687Z]         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[task 2024-10-14T06:35:22.687Z]   File "/builds/worker/workspace/build/src/third_party/python/requests/requests/api.py", line 73, in get
[task 2024-10-14T06:35:22.687Z]     return request("get", url, params=params, **kwargs)
[task 2024-10-14T06:35:22.687Z]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[task 2024-10-14T06:35:22.687Z]   File "/builds/worker/workspace/build/src/third_party/python/requests/requests/api.py", line 59, in request
[task 2024-10-14T06:35:22.687Z]     return session.request(method=method, url=url, **kwargs)
[task 2024-10-14T06:35:22.687Z]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[task 2024-10-14T06:35:22.687Z]   File "/builds/worker/workspace/build/src/third_party/python/requests/requests/sessions.py", line 589, in request
[task 2024-10-14T06:35:22.687Z]     resp = self.send(prep, **send_kwargs)
[task 2024-10-14T06:35:22.687Z]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[task 2024-10-14T06:35:22.687Z]   File "/builds/worker/workspace/build/src/third_party/python/requests/requests/sessions.py", line 703, in send
[task 2024-10-14T06:35:22.687Z]     r = adapter.send(request, **kwargs)
[task 2024-10-14T06:35:22.687Z]         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[task 2024-10-14T06:35:22.687Z]   File "/builds/worker/workspace/build/src/third_party/python/requests/requests/adapters.py", line 517, in send
[task 2024-10-14T06:35:22.687Z]     raise SSLError(e, request=request)
[task 2024-10-14T06:35:22.687Z] requests.exceptions.SSLError: HTTPSConnectionPool(host='www.archive.org', port=443): Max retries exceeded with url: /download/PdKids/PdKids.pdf (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)')))
[task 2024-10-14T06:35:22.715Z] Creating default state directory: /builds/worker/.mozbuild
[task 2024-10-14T06:35:22.715Z] Creating local state directory: /builds/worker/.mozbuild/srcdirs/src-6db0bb8b1edd
[taskcluster 2024-10-14 06:35:23.098Z] === Task Finished ===
[taskcluster 2024-10-14 06:35:23.115Z] Artifact "public/build" not found at "/builds/worker/artifacts/": (HTTP code 404) no such container - Could not find the file /builds/worker/artifacts/ in container 96b7235b8ada544515f648a8f2f3abbf9a208c36916dc8fa42772b594e0a6fe2 
[taskcluster 2024-10-14 06:35:23.203Z] Unsuccessful task run with exit code: 1 completed in 581.755 seconds

Comment 1

[Iulian Moraru]

Hi Alex! Can you please take a look at this? I'm not sure this is a talos issue per se, so if it is not, feel free to change the component.
Thank you!

Comment 2

[Mike Hommey [:glandium]]

The link in https://github.com/mozilla/pdf.js/blob/master/test/pdfs/pdkids.pdf.link is dead.

Comment 3

[Intermittent Failures Robot]

28 failures in 644 pushes (0.043 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 28

Platform and build breakdown:

• toolchains: 28
  • opt: 28

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1924439&startday=2024-10-14&endday=2024-10-14&tree=all

Comment 4

[Calixte Denizet (:calixte)]

We're used to "store" some of our pdfs for our tests on webarchive: we're looking for a better solution.

Comment 5

[Intermittent Failures Robot]

30 failures in 4579 pushes (0.007 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 28
• mozilla-esr128: 1
• try: 1

Platform and build breakdown:

• toolchains: 30
  • opt: 30

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1924439&startday=2024-10-14&endday=2024-10-20&tree=all