intermittent git failure: git.code.sf.net failure tracker for mingw-w64
Categories
(Webtools :: Searchfox, task)
Tracking
(Not tracked)
People
(Reporter: asuth, Unassigned)
References
Details
release4 failed today with the line:
fatal: unable to access 'https://git.code.sf.net/p/mingw-w64/mingw-w64/': gnutls_handshake() failed: The TLS connection was non-properly terminated.
Filing this bug to track incidences of this.
|
Reporter | |
Comment 1•4 years ago
•
|
||
Another git failure for mingw-w64
+ git pull origin master
fatal: unable to access 'https://git.code.sf.net/p/mingw-w64/mingw-w64/': Failed to connect to git.code.sf.net port 443: Connection timed out
Reasonable steps if this keeps happening are probably:
- Add some (time-)bounded retry on the git operations which generates a warning when we need to retry. It would be nice if this indicated how long the overall process took (failures and any success) in case the failure modes involve very long timeouts.
- We would want to know about slowness so we could revisit having searchfox pull from a mirror as originally proposed in bug 1671917 which could utilize a cron job that wouldn't impact indexer runtime.
- Treat git checkout/update failures for the mingw jobs as non-fatal warnings. It's likely not the end of the world for the results to be stale for a day, and it seems preferable to have stale results than having the repo be skipped for the day. (And we'd still have the state of the tarball we maintain which is effectively a non-public git mirror...)
|
||
Comment 2•4 years ago
|
||
(In reply to Andrew Sutherland [:asuth] (he/him) from comment #1)
- Treat git checkout/update failures for the mingw jobs as non-fatal warnings. It's likely not the end of the world for the results to be stale for a day, and it seems preferable to have stale results than having the repo be skipped for the day. (And we'd still have the state of the tarball we maintain which is effectively a non-public git mirror...)
Sounds fine to me!
|
|
Reporter | |
Comment 3•4 years ago
•
|
||
mingw failed again, but this could potentially be due to changes in the CA infrastructure with root certificates expiring that might require us to re-provision, so I'm not sure this counts as a ding against sourceforge:
fatal: unable to access 'https://git.code.sf.net/p/mingw-w64/mingw-w64/': server certificate verification failed. CAfile: none CRLfile: none
edit: I just checked https://git.code.sf.net/ and it is secured by Let's Encrypt, so yeah, it was exactly that.
|
|
Reporter | |
Comment 4•4 years ago
|
||
The re-provisioning of the servers in Bug 1721037 seems to have fixed the certificate problem.
|
||
Comment 5•4 years ago
|
||
Thanks for taking care of that! Sorry I've been so absent from searchfox stuff lately, life things have eaten up much of my free time :/
|
|
Reporter | |
Comment 6•4 years ago
|
||
(In reply to Kartikaya Gupta (email:kats@mozilla.staktrace.com) from comment #5)
Thanks for taking care of that! Sorry I've been so absent from searchfox stuff lately, life things have eaten up much of my free time :/
No worries! I am continually appreciative of the time you do spend on searchfox, and the reality is that other life stuff should come first!
Also, it's worth calling out that it's known and supported within my management chain that I do spend some time on searchfox as part of my job, so if anyone should be dealing with the less glamorous stuff, it's probably me!
|
|
Reporter | |
Comment 7•4 years ago
|
||
This happened again today:
fatal: unable to access 'https://git.code.sf.net/p/mingw-w64/mingw-w64/': Failed to connect to git.code.sf.net port 443 after 130052 ms: Connection timed out
My comment 1 proposal of "just use what's already in the tarball" as a fallback for having this be non-fatal still makes a lot of sense and is not too dramatic, so if this happens again tomorrow, we'll do that.
Description
•