Closed Bug 1689040 Opened 3 years ago Closed 3 years ago

Intermittent FV FAILURE 1: Update xml file not available

Categories

(Release Engineering :: Release Automation: Updates, defect)

Product:
Release Engineering
Component:
Release Automation: Updates
Type:
defect

Tracking

(firefox-esr78 fixed, firefox86 fixed, firefox87 fixed, firefox88 fixed)

Status:
RESOLVED FIXED
Tracking Flags:
Tracking Status
firefox-esr78 --- fixed
firefox86 --- fixed
firefox87 --- fixed
firefox88 --- fixed

People

(Reporter: bogdan_tara, Assigned: jcristau)

Details

(Keywords: intermittent-failure)

Attachments

(5 files)

Bug 1689040 - Don't silence curl errors in update-verify. r?#releng
48 bytes, text/x-phabricator-request
Details | Review
[task 2021-02-23T13:
14.00 KB, text/plain
Details
[task 2021-02-23T13:
14.00 KB, text/plain
Details
Bug 1689040 - support installing mercurial on ubuntu 20.04 docker images. r?bhearsum
48 bytes, text/x-phabricator-request
Details | Review
Bug 1689040 - bump update-verify docker image to ubuntu 20.04. r?bhearsum
48 bytes, text/x-phabricator-request
Details | Review

https://firefox-ci-tc.services.mozilla.com/tasks/DYzIQdvURMK2StovAjZMUg/runs/5
Log

...
[task 2021-01-27T01:49:00.826Z] Wed Jan 27 01:48:55 UTC 2021:  Mar file https://download.mozilla.org/?product=devedition-86.0b2-partial-86.0b1&os=win64&lang=hi-IN => https://download-installer.cdn.mozilla.net/pub/devedition/releases/86.0b2/update/win64/hi-IN/firefox-86.0b1-86.0b2.partial.mar available with correct size (6084944 bytes)
[task 2021-01-27T01:49:00.827Z] Wed Jan 27 01:49:00 UTC 2021:  
[task 2021-01-27T01:49:00.828Z] Wed Jan 27 01:49:00 UTC 2021:  Stopping stopwatch...
[task 2021-01-27T01:49:08.473Z] Wed Jan 27 01:49:08 UTC 2021:  
[task 2021-01-27T01:49:08.474Z] Wed Jan 27 01:49:08 UTC 2021:  ====================================
[task 2021-01-27T01:49:08.475Z] Wed Jan 27 01:49:08 UTC 2021:  107 FAILURES
[task 2021-01-27T01:49:11.592Z] Wed Jan 27 01:49:11 UTC 2021:  ====================================
[task 2021-01-27T01:49:11.593Z] Wed Jan 27 01:49:11 UTC 2021:  
[task 2021-01-27T01:49:11.594Z] Wed Jan 27 01:49:11 UTC 2021:  FAILURE 1: Update xml file not available
[task 2021-01-27T01:49:11.594Z] Wed Jan 27 01:49:11 UTC 2021:  
[task 2021-01-27T01:49:11.595Z] Wed Jan 27 01:49:11 UTC 2021:      Download url: https://aus5.mozilla.org/update/3/Firefox/56.0/20170819205558/WINNT_x86_64-msvc-x64/ja/aurora-cdntest/default/default/default/update.xml?force=1
[task 2021-01-27T01:49:11.596Z] Wed Jan 27 01:49:11 UTC 2021:      Curl returned exit code: 16
[task 2021-01-27T01:49:11.596Z] Wed Jan 27 01:49:11 UTC 2021:  
[task 2021-01-27T01:49:11.597Z] Wed Jan 27 01:49:11 UTC 2021:      The HTTP headers were:
[task 2021-01-27T01:49:11.599Z] Wed Jan 27 01:49:11 UTC 2021:          HTTP/2 200 
[task 2021-01-27T01:49:11.599Z] Wed Jan 27 01:49:11 UTC 2021:          server: nginx/1.17.9
[task 2021-01-27T01:49:11.599Z] Wed Jan 27 01:49:11 UTC 2021:          date: Wed, 27 Jan 2021 01:30:49 GMT
[task 2021-01-27T01:49:11.599Z] Wed Jan 27 01:49:11 UTC 2021:          content-type: text/xml; charset=utf-8
[task 2021-01-27T01:49:11.599Z] Wed Jan 27 01:49:11 UTC 2021:          content-length: 529
[task 2021-01-27T01:49:11.599Z] Wed Jan 27 01:49:11 UTC 2021:          vary: Accept-Encoding
[task 2021-01-27T01:49:11.599Z] Wed Jan 27 01:49:11 UTC 2021:          cache-control: public, max-age=90
[task 2021-01-27T01:49:11.599Z] Wed Jan 27 01:49:11 UTC 2021:          strict-transport-security: max-age=31536000;
[task 2021-01-27T01:49:11.599Z] Wed Jan 27 01:49:11 UTC 2021:          x-content-type-options: nosniff
[task 2021-01-27T01:49:11.599Z] Wed Jan 27 01:49:11 UTC 2021:          content-security-policy: default-src 'none'; frame-ancestors 'none'
[task 2021-01-27T01:49:11.599Z] Wed Jan 27 01:49:11 UTC 2021:          x-proxy-cache-status: MISS
[task 2021-01-27T01:49:11.599Z] Wed Jan 27 01:49:11 UTC 2021:          via: 1.1 google
[task 2021-01-27T01:49:11.599Z] Wed Jan 27 01:49:11 UTC 2021:          alt-svc: clear
[task 2021-01-27T01:49:11.599Z] Wed Jan 27 01:49:11 UTC 2021:          
...

So green after 13 reruns, that doesn't look good. The Firefox one is still failing after 15-17 reruns. The XML actually exists, the error seems to be somewhere around the curl calls.

Curl returned exit code: 16
    16     HTTP/2 error. A problem was detected in the HTTP2 framing layer. This is somewhat generic and can be one out of several problems, see  the  error  message
              for details.

XML that Balrog returns seems fine on both :jcristau's machine and mine.
Since we're silenting the errors, we can't really know what's going on.

:jcristau suggested we apply

diff --git a/tools/update-verify/release/get-update-xml.sh b/tools/update-verify/release/get-update-xml.sh
--- a/tools/update-verify/release/get-update-xml.sh
+++ b/tools/update-verify/release/get-update-xml.sh
@@ -1,16 +1,16 @@
 #!/bin/bash
 
 update_xml_url="${1}"
 patch_types="${2}"
 update_xml="$(mktemp -t update.xml.XXXXXXXXXX)"
 update_xml_headers="$(mktemp -t update.xml.headers.XXXXXXXXXX)"
 update_xml_debug="$(mktemp -t update.xml.debug.XXXXXXXXXX)"
-curl --retry 50 --retry-max-time 300 -s -D "${update_xml_headers}" -L -v "${update_xml_url}" > "${update_xml}" 2>"${update_xml_debug}"
+curl --retry 50 --retry-max-time 300 -s --show-error -D "${update_xml_headers}" -L -v "${update_xml_url}" > "${update_xml}" 2>"${update_xml_debug}"
 update_xml_curl_exit_code=$?
 if [ "${update_xml_curl_exit_code}" == 0 ]
 then
     update_xml_actual_url="$(sed -e "s/$(printf '\r')//" -n -e 's/^Location: //p' "${update_xml_headers}" | tail -1)"
     [ -n "${update_xml_actual_url}" ] && update_xml_url_with_redirects="${update_xml_url} => ${update_xml_actual_url}" || update_xml_url_with_redirects="${update_xml_url}"
     echo "$(date):  Downloaded update.xml file from ${update_xml_url_with_redirects}" > "$(mktemp -t log.XXXXXXXXXX)"
     for patch_type in ${patch_types//,/ }
     do

for the next betas to try to get a better feeling of these errors next time they occur.

Flags: needinfo?(mtabara)

I've tried to reproduce on try but the task ran successfully, so still no idea what happened. :(

That's really weird. We could also try an interactive task to try to reproduce the failure with the logging enhancements. It's tough to say much about this without more information.

Keywords: leave-open

This should make errors easier to diagnose, since what we seem to be
getting now is just the exit code, which in the recent instance was 16,
aka:

HTTP/2 error. A problem was detected in the HTTP2 framing layer. This
is somewhat generic and can be one out of several problems, see the
error message for details.

So hopefully future errors will include the error message.

Pushed by jcristau@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/8fd48aa75dfc
Don't silence curl errors in update-verify. r=releng-reviewers,mtabara DONTBUILD

We haven't seen this in the last beta releases, I'm thinking we might have been unlucky with the workers? Should we close or wait for another cycle to confirm?

Flags: needinfo?(jcristau)

Yeah, let's close.

Status: NEW → RESOLVED
Closed: 3 years ago
Flags: needinfo?(jcristau)
Resolution: --- → WORKSFORME

This hit release-final-verify-{firefox-devedition} for 87.0b1 today.

Status: RESOLVED → REOPENED
Resolution: WORKSFORME → ---
Attached file [task 2021-02-23T13: — 
Here's a sample error with increased verbosity from curl:
```
Attached file [task 2021-02-23T13: — 
Here's a sample error with increased verbosity from curl:
```

Here's a sample error with increased verbosity from curl:

[task 2021-02-23T13:41:52.930Z] Tue Feb 23 13:41:52 UTC 2021:  FAILURE 1: Update xml file not available
[task 2021-02-23T13:41:52.931Z] Tue Feb 23 13:41:52 UTC 2021:  
[task 2021-02-23T13:41:52.931Z] Tue Feb 23 13:41:52 UTC 2021:      Download url: https://aus5.mozilla.org/update/3/Firefox/57.0/20171030163911/Linux_x86_64-gcc3/sq/beta-cdntest/default/default/default/update.xml?force=1
[task 2021-02-23T13:41:52.932Z] Tue Feb 23 13:41:52 UTC 2021:      Curl returned exit code: 16
[task 2021-02-23T13:41:52.933Z] Tue Feb 23 13:41:52 UTC 2021:  
[task 2021-02-23T13:41:52.933Z] Tue Feb 23 13:41:52 UTC 2021:      The HTTP headers were:
[task 2021-02-23T13:41:52.935Z] Tue Feb 23 13:41:52 UTC 2021:          HTTP/2 200 
[task 2021-02-23T13:41:52.935Z] Tue Feb 23 13:41:52 UTC 2021:          server: nginx/1.17.9
[task 2021-02-23T13:41:52.935Z] Tue Feb 23 13:41:52 UTC 2021:          date: Tue, 23 Feb 2021 12:14:51 GMT
[task 2021-02-23T13:41:52.935Z] Tue Feb 23 13:41:52 UTC 2021:          content-type: text/xml; charset=utf-8
[task 2021-02-23T13:41:52.935Z] Tue Feb 23 13:41:52 UTC 2021:          content-length: 635
[task 2021-02-23T13:41:52.935Z] Tue Feb 23 13:41:52 UTC 2021:          vary: Accept-Encoding
[task 2021-02-23T13:41:52.935Z] Tue Feb 23 13:41:52 UTC 2021:          cache-control: public, max-age=90
[task 2021-02-23T13:41:52.935Z] Tue Feb 23 13:41:52 UTC 2021:          strict-transport-security: max-age=31536000;
[task 2021-02-23T13:41:52.935Z] Tue Feb 23 13:41:52 UTC 2021:          x-content-type-options: nosniff
[task 2021-02-23T13:41:52.935Z] Tue Feb 23 13:41:52 UTC 2021:          content-security-policy: default-src 'none'; frame-ancestors 'none'
[task 2021-02-23T13:41:52.935Z] Tue Feb 23 13:41:52 UTC 2021:          x-proxy-cache-status: MISS
[task 2021-02-23T13:41:52.935Z] Tue Feb 23 13:41:52 UTC 2021:          via: 1.1 google
[task 2021-02-23T13:41:52.935Z] Tue Feb 23 13:41:52 UTC 2021:          alt-svc: clear
[task 2021-02-23T13:41:52.935Z] Tue Feb 23 13:41:52 UTC 2021:          
[task 2021-02-23T13:41:52.936Z] Tue Feb 23 13:41:52 UTC 2021:  
[task 2021-02-23T13:41:52.937Z] Tue Feb 23 13:41:52 UTC 2021:      The full curl debug output was:
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          *   Trying 35.244.181.201...
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          * TCP_NODELAY set
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          * Connected to aus5.mozilla.org (35.244.181.201) port 443 (#0)
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          * ALPN, offering h2
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          * ALPN, offering http/1.1
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          * successfully set certificate verify locations:
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          *   CAfile: /etc/ssl/certs/ca-certificates.crt
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:            CApath: /etc/ssl/certs
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          } [5 bytes data]
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          * TLSv1.3 (OUT), TLS handshake, Client hello (1):
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          } [512 bytes data]
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          * TLSv1.3 (IN), TLS handshake, Server hello (2):
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          { [122 bytes data]
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          * TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          { [1 bytes data]
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          * TLSv1.3 (IN), TLS handshake, Unknown (8):
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          { [15 bytes data]
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          * TLSv1.3 (IN), TLS handshake, Certificate (11):
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          { [3018 bytes data]
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          * TLSv1.3 (IN), TLS handshake, CERT verify (15):
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          { [264 bytes data]
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          * TLSv1.3 (IN), TLS handshake, Finished (20):
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          { [52 bytes data]
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          * TLSv1.3 (OUT), TLS change cipher, Client hello (1):
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          } [1 bytes data]
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          * TLSv1.3 (OUT), TLS Unknown, Certificate Status (22):
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          } [1 bytes data]
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          * TLSv1.3 (OUT), TLS handshake, Finished (20):
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          } [52 bytes data]
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          * SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
[task 2021-02-23T13:41:52.938Z] Tue Feb 23 13:41:52 UTC 2021:          * ALPN, server accepted to use h2
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          * Server certificate:
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          *  subject: C=US; ST=California; L=Mountain View; O=Mozilla Corporation; OU=Cloud Services; CN=aus5.mozilla.org
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          *  start date: Jun 27 00:00:00 2019 GMT
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          *  expire date: Jun 16 12:00:00 2021 GMT
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          *  subjectAltName: host "aus5.mozilla.org" matched cert's "aus5.mozilla.org"
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          *  issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          *  SSL certificate verify ok.
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          * Using HTTP2, server supports multi-use
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          * Connection state changed (HTTP/2 confirmed)
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          } [5 bytes data]
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          * TLSv1.3 (OUT), TLS Unknown, Unknown (23):
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          } [1 bytes data]
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          * TLSv1.3 (OUT), TLS Unknown, Unknown (23):
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          } [1 bytes data]
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          * TLSv1.3 (OUT), TLS Unknown, Unknown (23):
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          } [1 bytes data]
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          * Using Stream ID: 1 (easy handle 0x55b0cbd115c0)
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          } [5 bytes data]
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          * TLSv1.3 (OUT), TLS Unknown, Unknown (23):
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          } [1 bytes data]
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          > GET /update/3/Firefox/57.0/20171030163911/Linux_x86_64-gcc3/sq/beta-cdntest/default/default/default/update.xml?force=1 HTTP/2
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          > Host: aus5.mozilla.org
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          > User-Agent: curl/7.58.0
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          > Accept: */*
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          > 
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          { [5 bytes data]
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          * TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          { [1 bytes data]
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          { [264 bytes data]
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          { [264 bytes data]
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          * TLSv1.3 (IN), TLS Unknown, Unknown (23):
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          { [1 bytes data]
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          * Connection state changed (MAX_CONCURRENT_STREAMS updated)!
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          } [5 bytes data]
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          * TLSv1.3 (OUT), TLS Unknown, Unknown (23):
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          } [1 bytes data]
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          * TLSv1.3 (IN), TLS Unknown, Unknown (23):
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          { [1 bytes data]
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          * TLSv1.3 (IN), TLS Unknown, Unknown (23):
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          { [1 bytes data]
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          < HTTP/2 200 
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          < server: nginx/1.17.9
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          < date: Tue, 23 Feb 2021 12:14:51 GMT
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          < content-type: text/xml; charset=utf-8
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          < content-length: 635
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          < vary: Accept-Encoding
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          < cache-control: public, max-age=90
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          < strict-transport-security: max-age=31536000;
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          < x-content-type-options: nosniff
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          < content-security-policy: default-src 'none'; frame-ancestors 'none'
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          < x-proxy-cache-status: MISS
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          < via: 1.1 google
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          < alt-svc: clear
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          < 
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          { [5 bytes data]
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          * TLSv1.3 (IN), TLS Unknown, Unknown (23):
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          { [1 bytes data]
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          * TLSv1.3 (IN), TLS Unknown, Unknown (23):
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          { [1 bytes data]
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          * Connection #0 to host aus5.mozilla.org left intact
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:          curl: (16) Error in the HTTP2 framing layer
[task 2021-02-23T13:41:52.939Z] Tue Feb 23 13:41:52 UTC 2021:  
[task 2021-02-23T13:41:52.940Z] Tue Feb 23 13:41:52 UTC 2021:      The returned update.xml file was:
[task 2021-02-23T13:41:52.941Z] Tue Feb 23 13:41:52 UTC 2021:          <?xml version="1.0"?>
[task 2021-02-23T13:41:52.941Z] Tue Feb 23 13:41:52 UTC 2021:          <updates>
[task 2021-02-23T13:41:52.941Z] Tue Feb 23 13:41:52 UTC 2021:              <update actions="showURL" appVersion="87.0" buildID="20210222154334" detailsURL="https://www.mozilla.org/sq/firefox/87.0/releasenotes/" displayVersion="87.0 Beta 1" openURL="https://www.mozilla.org/sq/firefox/87.0beta/whatsnew/?oldversion=%OLD_VERSION%" type="minor">
[task 2021-02-23T13:41:52.942Z] Tue Feb 23 13:41:52 UTC 2021:                  <patch type="complete" URL="https://download.mozilla.org/?product=firefox-87.0b1-complete&amp;os=linux64&amp;lang=sq" hashFunction="sha512" hashValue="3e58df32dd27bd17767d941275e88f10ef5a936b08f0d3d5882e2ca152a563c6d337328a33f190d7a4a427bcac21d1fa757a473ecfe0add40c7355d51b7d5e1f" size="61365669"/>
[task 2021-02-23T13:41:52.942Z] Tue Feb 23 13:41:52 UTC 2021:              </update>
[task 2021-02-23T13:41:52.942Z] Tue Feb 23 13:41:52 UTC 2021:          </updates>
[task 2021-02-23T13:41:52.942Z] Tue Feb 23 13:41:52 UTC 2021:  
[task 2021-02-23T13:41:52.943Z] Tue Feb 23 13:41:52 UTC 2021:      This url was tested because of the following cfg file entries:
[task 2021-02-23T13:41:53.101Z] Tue Feb 23 13:41:53 UTC 2021:          /tmp/tmp.D7RvAtBbSj line 375: release="57.0" product="Firefox" platform="Linux_x86_64-gcc3" build_id="20171030163911" locales="ach af an ar ast az be bg br bs ca cak cs cy da de dsb el en-GB en-US eo es-AR es-CL es-ES es-MX et eu fa ff fi fr fy-NL ga-IE gd gl gn gu-IN he hi-IN hr hsb hu hy-AM id is it ja ka kab kk km kn ko lij lt lv mk mr ms my nb-NO nl nn-NO pa-IN pl pt-BR pt-PT rm ro ru si sk sl son sq sr sv-SE ta te th tr uk ur uz vi xh zh-CN zh-TW" channel="beta-localtest" patch_types="complete"
[task 2021-02-23T13:41:53.102Z] Tue Feb 23 13:41:53 UTC 2021:

What's interesting is that despite getting error 16 from curl, we're still getting an xml file - which I find really weird.

https://github.com/curl/curl/issues/4267 and https://github.com/curl/curl/issues/3750 suggest that there was some bugs in curl similar to or the same as this in the past. It looks like we're running a version of curl that was released in 2018, so getting that up to date is probably a good next step here:

curl 7.58.0 (x86_64-pc-linux-gnu) libcurl/7.58.0 OpenSSL/1.1.1 zlib/1.2.11 libidn2/2.0.4 libpsl/0.19.1 (+libidn2/2.0.4) nghttp2/1.30.0 librtmp/2.3
Release-Date: 2018-01-24

FTR, :Aryx reported something similar for curl in Fenix as well in https://github.com/mozilla-mobile/fenix/issues/18067#issue-811508254.

ubuntu focal comes with mercurial 5.3.1 which is new enough for us right
now, so install it directly from the distro instead of fetching the
older 4.8.1 from tooltool.

Use python3 in more places so we don't have to explicitly install
the python 2.x package.

The newer version of curl in 20.04 will hopefully fix the intermittent
http2 framing errors we've been seeing recently.

Pushed by jcristau@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/9d7ba6ad83be
support installing mercurial on ubuntu 20.04 docker images. r=bhearsum
https://hg.mozilla.org/integration/autoland/rev/4adde2676286
bump update-verify docker image to ubuntu 20.04. r=bhearsum

Tentatively marking as fixed. We'll get this in the next beta, and see how that goes...

Assignee: nobody → jcristau
Status: REOPENED → RESOLVED
Closed: 3 years ago → 3 years ago
status-firefox88: --- → fixed
Keywords: leave-open
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: