1948276 - verizon.com - Images missing, design breaking with each page refresh (cloudfront)

Status: VERIFIED FIXED

(Web Compatibility :: Site Reports, defect, P2)

Description

[Calin Tanase]

Attachment: verizon FF vs Chrome.mp4

Environment:
Operating system: Windows 11/10
Firefox version: Firefox 135.0 (release)/137

Preconditions:

• Clean profile

Steps to reproduce:

1. Navigate to: https://www.verizon.com/business/en-sg/resources/reports/dbir/
2. Scroll down to "Top takeaways".
3. Observe the page.

Expected Behavior:
The page loads correctly.

Actual Behavior:
The images are missing. (refreshing the page, breaks the design even further)

Notes:

• Reproducible on the latest Firefox Release and Nightly
• Reproducible regardless of the ETP setting
• Works as expected using Chrome

---

Created from webcompat-user-report:4ff2c953-715c-4c1d-b1a3-961b173437ef

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Since nightly and release are affected, beta will likely be affected too.
For more information, please visit BugBot documentation.

Comment 2

[Jeff Muizelaar [:jrmuizel]]

Looks like we're getting NS_ERROR_NET_INTERRUPT from cloudflare some of the time.

Comment 3

[Jeff Muizelaar [:jrmuizel]]

Changing to affects:some as this is only on one part of verizon's site as far as we know

Comment 4

[Kershaw Chang [:kershaw]]

The issue appears to be caused by the server resetting the stream when using HTTP/3. For example, when accessing the resource:

https://d23uobcja6cuc.cloudfront.net/business/etc.clientlibs/ves/clientlibs/main.css

The server resets the stream with an H3_REQUEST_CANCELLED error.

This behavior can also be reproduced using curl with HTTP/3. The curl logs (available via this link) confirm that the stream is reset by the server.

Chrome does not show this issue because it appears to always use HTTP/2 when loading resources from d23uobcja6cuc.cloudfront.net, although the reason for this behavior remains unclear to me.

Comment 5

[Randell Jesup [:jesup] (needinfo me)]

Since this also repros with curl, likely we need to talk to cloudfront about this. Do we have any contacts there?

Comment 6

[Lars Eggert [:lars]]

I'll raise it on mozilla-amazon-discuss.

Comment 7

[Lars Eggert [:lars]]

We had mentioned this to Amazon on the quicdev Slack. Their reply:

on firefox, I see that the asset is not loading because of the CORS policy
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://d23uobcja6cuc.cloudfront.net/business/etc.clientlibs/ves/clientlibs/main.css. (Reason: CORS request did not succeed). Status code: (null).

They also asked whether we have logs or pcaps?

Comment 8

[Kershaw Chang [:kershaw]]

Attachment: bug_1948276_pcap_netlog.zip

I attached a wireshark trace and a net log captured with Chrome.
Both logs show that the stream was cancelled by the server.

Comment 9

[Randell Jesup [:jesup] (needinfo me)]

Lars, can you ask them to check out the trace (if you haven't already)? Thanks

Comment 10

[Randell Jesup [:jesup] (needinfo me)]

Lars - any updates?

Comment 11

[Lars Eggert [:lars]]

No. I pinged them again on the quicdev Slack.

Comment 12

[Lars Eggert [:lars]]

(In reply to Kershaw Chang [:kershaw] from comment #4)

The curl logs (available via this link) confirm that the stream is reset by the server.

@kershaw that link is dead?

Comment 13

[Kershaw Chang [:kershaw]]

Attachment: Curl log

(In reply to Lars Eggert [:lars] from comment #12)
> (In reply to Kershaw Chang [:kershaw] from comment #4)
> > The curl logs (available via this [link](https://pastebin.mozilla.org/DXdkSDZ1)) confirm that the stream is reset by the server.
> 
> @kershaw that link is dead?

I've uploaded the log from curl.

Comment 14

[Jeff Muizelaar [:jrmuizel]]

Cloudfront may have deployed a fix for this. Kershaw, does it still reproduce for you?

Comment 15

[Kershaw Chang [:kershaw]]

(In reply to Jeff Muizelaar [:jrmuizel] from comment #14)

Cloudfront may have deployed a fix for this. Kershaw, does it still reproduce for you?

Unfortunately, no. I still see the same reset error.
This is my STR with Chrome:

1. I launch Chrome with this: ./Google\ Chrome --origin-to-force-quic-on=d23uobcja6cuc.cloudfront.net:443
2. Open https://d23uobcja6cuc.cloudfront.net/business/etc.clientlibs/ves/clientlibs/main.css
3. Got an ERR_QUIC_PROTOCOL_ERROR error page.

This is the log from Chrome's log:

t= 5979 [st= 618]    HTTP3_HEADERS_SENT
                     --> :method: GET
                         :authority: d23uobcja6cuc.cloudfront.net
                         :scheme: https
                         :path: /business/etc.clientlibs/ves/clientlibs/main.css
                         sec-ch-ua: "Not)A;Brand";v="8", "Chromium";v="138", "Google Chrome";v="138"
                         sec-ch-ua-mobile: ?0
                         sec-ch-ua-platform: "macOS"
                         upgrade-insecure-requests: 1
                         user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36
                         sec-purpose: prefetch;prerender
                         purpose: prefetch
                         accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
                         sec-fetch-site: none
                         sec-fetch-mode: navigate
                         sec-fetch-user: ?1
                         sec-fetch-dest: document
                         accept-encoding: gzip, deflate, br, zstd
                         accept-language: en-GB,en-US;q=0.9,en;q=0.8
                         priority: u=0, i
                     --> stream_id = 0
t= 5979 [st= 618]    QUIC_SESSION_STREAM_FRAME_SENT
                     --> fin = true
                     --> length = 520
                     --> offset = 0
                     --> stream_id = 0
t= 5979 [st= 618]    QUIC_SESSION_PACKET_SENT
                     --> batch_id = 0
                     --> encryption_level = "ENCRYPTION_FORWARD_SECURE"
                     --> packet_number = 17
                     --> sent_time_us = 998133153715
                     --> size = 576
                     --> transmission_type = "NOT_RETRANSMISSION"
t= 6027 [st= 666]    QUIC_SESSION_POOL_ATTACH_HTTP_STREAM_JOB_TO_EXISTING_SESSION
                     --> source_dependency = 662 (HTTP_STREAM_JOB)
t= 6034 [st= 673]    QUIC_SESSION_PACKET_RECEIVED
                     --> peer_address = "[2600:9000:21c7:6600:3:801f:2100:21]:443"
                     --> self_address = "[2a02:3100:23fd:c300:34e9:998b:21ca:655a]:50887"
                     --> size = 43
t= 6034 [st= 673]    QUIC_SESSION_UNAUTHENTICATED_PACKET_HEADER_RECEIVED
                     --> connection_id = "38980800088d0e650c5a2e1beee4fbaa16149336"
                     --> header_format = "IETF_QUIC_SHORT_HEADER_PACKET"
                     --> packet_number = 17
t= 6034 [st= 673]    QUIC_SESSION_PACKET_AUTHENTICATED
t= 6034 [st= 673]    QUIC_SESSION_ACK_FRAME_RECEIVED
                     --> delta_time_largest_observed_us = 25000
                     --> largest_observed = 17
                     --> missing_packets = []
                     --> received_packet_times = []
                     --> smallest_observed = 12
t= 6034 [st= 673]    QUIC_SESSION_PADDING_FRAME_RECEIVED
                     --> num_padding_bytes = 19
t= 6238 [st= 877]    QUIC_SESSION_PING_FRAME_SENT
t= 6238 [st= 877]    QUIC_SESSION_PACKET_SENT
                     --> batch_id = 0
                     --> encryption_level = "ENCRYPTION_FORWARD_SECURE"
                     --> packet_number = 18
                     --> sent_time_us = 998133412816
                     --> size = 41
                     --> transmission_type = "NOT_RETRANSMISSION"
t= 6294 [st= 933]    QUIC_SESSION_PACKET_RECEIVED
                     --> peer_address = "[2600:9000:21c7:6600:3:801f:2100:21]:443"
                     --> self_address = "[2a02:3100:23fd:c300:34e9:998b:21ca:655a]:50887"
                     --> size = 43
t= 6294 [st= 933]    QUIC_SESSION_UNAUTHENTICATED_PACKET_HEADER_RECEIVED
                     --> connection_id = "38980800088d0e650c5a2e1beee4fbaa16149336"
                     --> header_format = "IETF_QUIC_SHORT_HEADER_PACKET"
                     --> packet_number = 18
t= 6295 [st= 934]    QUIC_SESSION_PACKET_AUTHENTICATED
t= 6295 [st= 934]    QUIC_SESSION_ACK_FRAME_RECEIVED
                     --> delta_time_largest_observed_us = 25000
                     --> largest_observed = 18
                     --> missing_packets = []
                     --> received_packet_times = []
                     --> smallest_observed = 18
t= 6295 [st= 934]    QUIC_SESSION_PADDING_FRAME_RECEIVED
                     --> num_padding_bytes = 19
t= 6499 [st=1138]    QUIC_SESSION_PING_FRAME_SENT
t= 6499 [st=1138]    QUIC_SESSION_PACKET_SENT
                     --> batch_id = 0
                     --> encryption_level = "ENCRYPTION_FORWARD_SECURE"
                     --> packet_number = 19
                     --> sent_time_us = 998133673413
                     --> size = 41
                     --> transmission_type = "NOT_RETRANSMISSION"
t= 6554 [st=1193]    QUIC_SESSION_PACKET_RECEIVED
                     --> peer_address = "[2600:9000:21c7:6600:3:801f:2100:21]:443"
                     --> self_address = "[2a02:3100:23fd:c300:34e9:998b:21ca:655a]:50887"
                     --> size = 43
t= 6554 [st=1193]    QUIC_SESSION_UNAUTHENTICATED_PACKET_HEADER_RECEIVED
                     --> connection_id = "38980800088d0e650c5a2e1beee4fbaa16149336"
                     --> header_format = "IETF_QUIC_SHORT_HEADER_PACKET"
                     --> packet_number = 19
t= 6554 [st=1193]    QUIC_SESSION_PACKET_AUTHENTICATED
t= 6554 [st=1193]    QUIC_SESSION_ACK_FRAME_RECEIVED
                     --> delta_time_largest_observed_us = 24976
                     --> largest_observed = 19
                     --> missing_packets = []
                     --> received_packet_times = []
                     --> smallest_observed = 19
t= 6554 [st=1193]    QUIC_SESSION_PADDING_FRAME_RECEIVED
                     --> num_padding_bytes = 19
t= 6646 [st=1285]    QUIC_SESSION_PACKET_RECEIVED
                     --> peer_address = "[2600:9000:21c7:6600:3:801f:2100:21]:443"
                     --> self_address = "[2a02:3100:23fd:c300:34e9:998b:21ca:655a]:50887"
                     --> size = 43
t= 6646 [st=1285]    QUIC_SESSION_UNAUTHENTICATED_PACKET_HEADER_RECEIVED
                     --> connection_id = "38980800088d0e650c5a2e1beee4fbaa16149336"
                     --> header_format = "IETF_QUIC_SHORT_HEADER_PACKET"
                     --> packet_number = 20
t= 6646 [st=1285]    QUIC_SESSION_PACKET_AUTHENTICATED
t= 6646 [st=1285]    QUIC_SESSION_RST_STREAM_FRAME_RECEIVED
                     --> ietf_error_code = 268
                     --> offset = 0
                     --> quic_rst_stream_error = 6 (QUIC_STREAM_CANCELLED)
                     --> stream_id = 0

Comment 16

[Kershaw Chang [:kershaw]]

This should be fixed on the server side. The resource at https://d23uobcja6cuc.cloudfront.net/business/etc.clientlibs/ves/clientlibs/main.css can now be loaded via HTTP/3.
I think we can close this bug.

Comment 17

[Raul Bucata]

Verified as FIXED using the RC Build

Tested with:

Browser / Version: Firefox 142.0-candidate build 1
Operating System: Windows 10 PRO x64