Open Bug 1789775 Opened 2 years ago Updated 4 months ago

Intermittent <talos> | application crashed [@ mozilla::net::CheckForBrokenChromeURL]

Categories

(Core :: Networking, defect, P5)

defect

Tracking

()

People

(Reporter: intermittent-bug-filer, Unassigned)

References

Details

(Keywords: crash, intermittent-failure, Whiteboard: [necko-triaged] [necko-monitor])

Crash Data

Filed by: ccozmuta [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=389815378&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/WiTD33UjSVKhzNpJS5whBQ/runs/0/artifacts/public/logs/live_backing.log


[task 2022-09-08T03:35:06.495Z] 03:35:06     INFO -  PID 3240 | Exiting due to channel error.
[task 2022-09-08T03:35:06.495Z] 03:35:06     INFO -  PID 3240 | Exiting due to channel error.
[task 2022-09-08T03:35:06.495Z] 03:35:06     INFO -  PID 3240 | Exiting due to channel error.
[task 2022-09-08T03:35:06.605Z] 03:35:06     INFO -  Terminating psutil.Process(pid=3240, name='firefox', status='zombie', started='03:28:08')
[task 2022-09-08T03:35:06.605Z] 03:35:06     INFO -  TEST-INFO | 3240: exit 1
[task 2022-09-08T03:35:06.607Z] 03:35:06     INFO -  mozcrash checking /var/folders/km/vp1xqc9565l_rzflvf70skjc000014/T/tmpo3082j0k/profile/minidumps for minidumps...
[task 2022-09-08T03:35:06.607Z] 03:35:06     INFO -  mozcrash Downloading symbols from: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/VrBm3qPOTsy4n6GBZ5oRng/artifacts/public/build/target.crashreporter-symbols.zip
[task 2022-09-08T03:35:44.640Z] 03:35:44     INFO -  mozcrash Copy/paste: /opt/worker/tasks/task_166260344393811/fetches/minidump-stackwalk/minidump-stackwalk --symbols-url=https://symbols.mozilla.org/ --human /var/folders/km/vp1xqc9565l_rzflvf70skjc000014/T/tmpo3082j0k/profile/minidumps/29F614C8-AFFB-4017-98BD-4FCC958A853A.dmp /var/folders/km/vp1xqc9565l_rzflvf70skjc000014/T/tmplawpbj_0
[task 2022-09-08T03:35:51.432Z] 03:35:51     INFO -  mozcrash Saved minidump as /opt/worker/tasks/task_166260344393811/build/blobber_upload_dir/29F614C8-AFFB-4017-98BD-4FCC958A853A.dmp
[task 2022-09-08T03:35:51.432Z] 03:35:51     INFO -  mozcrash Saved app info as /opt/worker/tasks/task_166260344393811/build/blobber_upload_dir/29F614C8-AFFB-4017-98BD-4FCC958A853A.extra
[task 2022-09-08T03:35:51.433Z] 03:35:51     INFO -  PROCESS-CRASH | basic_compositor_video | application crashed [@ MOZ_Crash(char const*, int, char const*)]
[task 2022-09-08T03:35:51.433Z] 03:35:51     INFO -  Crash dump filename: /var/folders/km/vp1xqc9565l_rzflvf70skjc000014/T/tmpo3082j0k/profile/minidumps/29F614C8-AFFB-4017-98BD-4FCC958A853A.dmp
[task 2022-09-08T03:35:51.434Z] 03:35:51     INFO -  Mozilla crash reason: Missing chrome or resource URLs: chrome://pageloader/content/utils.js
[task 2022-09-08T03:35:51.434Z] 03:35:51     INFO -  Operating system: Mac OS X
[task 2022-09-08T03:35:51.434Z] 03:35:51     INFO -                    10.15.7 19H524
[task 2022-09-08T03:35:51.434Z] 03:35:51     INFO -  CPU: amd64
[task 2022-09-08T03:35:51.434Z] 03:35:51     INFO -       family 6 model 158 stepping 10
[task 2022-09-08T03:35:51.435Z] 03:35:51     INFO -       12 CPUs
[task 2022-09-08T03:35:51.435Z] 03:35:51     INFO -  Crash reason:  EXC_BAD_ACCESS / KERN_INVALID_ADDRESS
[task 2022-09-08T03:35:51.435Z] 03:35:51     INFO -  Crash address: 0x0
[task 2022-09-08T03:35:51.435Z] 03:35:51     INFO -  Mac Crash Info:
[task 2022-09-08T03:35:51.436Z] 03:35:51     INFO -  Process uptime: 418 seconds
[task 2022-09-08T03:35:51.436Z] 03:35:51     INFO -  Thread 0 MainThread (crashed)
[task 2022-09-08T03:35:51.436Z] 03:35:51     INFO -   0  XUL!MOZ_Crash(char const*, int, char const*) [Assertions.h:2df511a16e4194617632cce3f261312c8e068620 : 261]
[task 2022-09-08T03:35:51.436Z] 03:35:51     INFO -      Found by: inlining
[task 2022-09-08T03:35:51.437Z] 03:35:51     INFO -   1  XUL!mozilla::net::CheckForBrokenChromeURL(nsILoadInfo*, nsIURI*) [nsNetUtil.cpp:2df511a16e4194617632cce3f261312c8e068620 : 3898 + 0xa]
[task 2022-09-08T03:35:51.437Z] 03:35:51     INFO -       rax = 0x00000001093da560    rdx = 0xfffffffffffffff4
[task 2022-09-08T03:35:51.437Z] 03:35:51     INFO -       rcx = 0x00000001093d8588    rbx = 0x00007ffee68bfdb8
[task 2022-09-08T03:35:51.437Z] 03:35:51     INFO -       rsi = 0x000000011b46f2a8    rdi = 0x00007ffee68bf820
[task 2022-09-08T03:35:51.438Z] 03:35:51     INFO -       rbp = 0x00007ffee68bff00    rsp = 0x00007ffee68bfda0
[task 2022-09-08T03:35:51.438Z] 03:35:51     INFO -        r8 = 0x0000000000000000     r9 = 0x0000000000000001
[task 2022-09-08T03:35:51.438Z] 03:35:51     INFO -       r10 = 0x0000000000000000    r11 = 0xffffffffedf6b2f9
[task 2022-09-08T03:35:51.438Z] 03:35:51     INFO -       r12 = 0x0003001100000000    r13 = 0x0002002100000017
[task 2022-09-08T03:35:51.439Z] 03:35:51     INFO -       r14 = 0x000000010968a860    r15 = 0x0000000128f885e0
[task 2022-09-08T03:35:51.439Z] 03:35:51     INFO -       rip = 0x000000010bd1b187
[task 2022-09-08T03:35:51.439Z] 03:35:51     INFO -      Found by: given as instruction pointer in context
[task 2022-09-08T03:35:51.439Z] 03:35:51     INFO -   2  XUL!nsFileChannel::MakeFileInputStream(nsIFile*, nsCOMPtr<nsIInputStream>&, nsTString<char>&, bool) [nsFileChannel.cpp:2df511a16e4194617632cce3f261312c8e068620 : 296 + 0x4]
[task 2022-09-08T03:35:51.440Z] 03:35:51     INFO -       rbx = 0x0000000080520012    rbp = 0x00007ffee68bff70
[task 2022-09-08T03:35:51.440Z] 03:35:51     INFO -       rsp = 0x00007ffee68bff10    r12 = 0x00000001301b2f50
[task 2022-09-08T03:35:51.440Z] 03:35:51     INFO -       r13 = 0x00007ffee68bffc8    r14 = 0x0000000000000000
[task 2022-09-08T03:35:51.441Z] 03:35:51     INFO -       r15 = 0x0000000109673080    rip = 0x000000010bed4097
[task 2022-09-08T03:35:51.441Z] 03:35:51     INFO -      Found by: call frame info
[task 2022-09-08T03:35:51.441Z] 03:35:51     INFO -   3  XUL!nsFileChannel::OpenContentStream(bool, nsIInputStream**, nsIChannel**) [nsFileChannel.cpp:2df511a16e4194617632cce3f261312c8e068620 : 387 + 0x19]
[task 2022-09-08T03:35:51.441Z] 03:35:51     INFO -       rbx = 0x0000000125b62b40    rbp = 0x00007ffee68c0070
[task 2022-09-08T03:35:51.442Z] 03:35:51     INFO -       rsp = 0x00007ffee68bff80    r12 = 0x0000000000000000
[task 2022-09-08T03:35:51.442Z] 03:35:51     INFO -       r13 = 0x00007ffee68c0088    r14 = 0x00007ffee68c0170
[task 2022-09-08T03:35:51.442Z] 03:35:51     INFO -       r15 = 0x0000000109673080    rip = 0x000000010bed448f
[task 2022-09-08T03:35:51.442Z] 03:35:51     INFO -      Found by: call frame info
[task 2022-09-08T03:35:51.443Z] 03:35:51     INFO -   4  XUL!nsBaseChannel::Open(nsIInputStream**) [nsBaseChannel.cpp:2df511a16e4194617632cce3f261312c8e068620 : 628 + 0x10]
[task 2022-09-08T03:35:51.443Z] 03:35:51     INFO -       rbx = 0x0000000109673080    rbp = 0x00007ffee68c00c0
[task 2022-09-08T03:35:51.443Z] 03:35:51     INFO -       rsp = 0x00007ffee68c0080    r12 = 0x00007ffee68c0170
[task 2022-09-08T03:35:51.444Z] 03:35:51     INFO -       r13 = 0x0000000125620e00    r14 = 0x00000001096730b8
[task 2022-09-08T03:35:51.444Z] 03:35:51     INFO -       r15 = 0x00000000804b000f    rip = 0x0000000109d0f144
[task 2022-09-08T03:35:51.444Z] 03:35:51     INFO -      Found by: call frame info
[task 2022-09-08T03:35:51.445Z] 03:35:51     INFO -   5  XUL!nsMessageManagerScriptExecutor::TryCacheLoadAndCompileScript(nsTSubstring<char16_t> const&, bool, JS::Handle<JSObject*>) [nsFrameMessageManager.cpp:2df511a16e4194617632cce3f261312c8e068620 : 1288]
[task 2022-09-08T03:35:51.445Z] 03:35:51     INFO -      Found by: inlining
Component: Talos → Networking
Product: Testing → Core
See Also: → 1745531
Summary: Intermittent basic_compositor_video | application crashed [@ MOZ_Crash(char const*, int, char const*)] → Intermittent <random_test> | application crashed [@ mozilla::net::CheckForBrokenChromeURL(nsILoadInfo*, nsIURI*)]
Crash Signature: [@ MOZ_Crash(char const*, int, char const*)] → [@ mozilla::net::CheckForBrokenChromeURL(nsILoadInfo*, nsIURI*)]
Duplicate of this bug: 1792132
Summary: Intermittent <random_test> | application crashed [@ mozilla::net::CheckForBrokenChromeURL(nsILoadInfo*, nsIURI*)] → Intermittent <talos> | application crashed [@ mozilla::net::CheckForBrokenChromeURL(nsILoadInfo*, nsIURI*)]

Copying crash signatures from duplicate bugs.

Crash Signature: [@ mozilla::net::CheckForBrokenChromeURL(nsILoadInfo*, nsIURI*)] → [@ mozilla::net::CheckForBrokenChromeURL(nsILoadInfo*, nsIURI*)] [@ MOZ_Crash(char const*, int, char const*)]

The severity field for this bug is set to S4. However, the following bug duplicate has higher severity:

:jesup, could you consider increasing the severity of this bug to S3?

For more information, please visit auto_nag documentation.

Flags: needinfo?(rjesup)
Crash Signature: [@ mozilla::net::CheckForBrokenChromeURL(nsILoadInfo*, nsIURI*)] [@ MOZ_Crash(char const*, int, char const*)] → [@ mozilla::net::CheckForBrokenChromeURL(nsILoadInfo*, nsIURI*)] [@ MOZ_Crash(char const*, int, char const*)]
Flags: needinfo?(rjesup)
Priority: -- → P5
Whiteboard: [necko-triaged]
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → INCOMPLETE
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
Status: REOPENED → RESOLVED
Closed: 2 years ago2 years ago
Resolution: --- → INCOMPLETE
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
Crash Signature: [@ mozilla::net::CheckForBrokenChromeURL(nsILoadInfo*, nsIURI*)] [@ MOZ_Crash(char const*, int, char const*)] → [@ mozilla::net::CheckForBrokenChromeURL] [@ MOZ_Crash]
Summary: Intermittent <talos> | application crashed [@ mozilla::net::CheckForBrokenChromeURL(nsILoadInfo*, nsIURI*)] → Intermittent <talos> | application crashed [@ mozilla::net::CheckForBrokenChromeURL]
Duplicate of this bug: 1745531

Copying crash signatures from duplicate bugs.

Crash Signature: [@ mozilla::net::CheckForBrokenChromeURL] [@ MOZ_Crash] → [@ mozilla::net::CheckForBrokenChromeURL] [@ MOZ_Crash] [@ CheckForBrokenChromeURL]

Looking. I think the pageloader utils.js case is already on file elsewhere.

Meta question: why do these failures not have the MOZ_CRASH reason show up as the failure message / bug summary? That would make triage a lot more straightforward.

Crash Signature: [@ mozilla::net::CheckForBrokenChromeURL] [@ MOZ_Crash] [@ CheckForBrokenChromeURL] → [@ mozilla::net::CheckForBrokenChromeURL] [@ MOZ_Crash] [@ CheckForBrokenChromeURL]
Flags: needinfo?(gijskruitbosch+bugs) → needinfo?(jmaher)

bug 1734525 was what I was thinking of. I expect, given failures dried up there, that the stacks changed and then we ended up with new bugs. As noted there, Nika and I already did a bunch of triage/digging in that bug. We didn't understand then (and I still don't understand now) how it's possible for these requests to end up as "file not found". If there is some way of getting more information about the load by dumping more info in the crashing code, that may be how we'd need to debug this. But I don't know how to do that, or what information would help elucidate where the NS_ERROR_FILE_NOT_FOUND would be coming from. Valentin?

We could simply squash these errors by adding exceptions for these URLs (like these existing ones) but I think the raciness here and the not understanding why/how it happens, means that that feels wrong - and given the symptoms are not entirely dissimilar to e.g. bug 1639821, I'd prefer understanding this more before resorting to just ignoring the error for these files.

Flags: needinfo?(valentin.gosu)
See Also: → 1734525

:gijs, do you think all crashes should have the reason:

[task 2023-02-03T11:13:28.894Z] 11:13:28     INFO -  mozcrash Saved app info as C:\Users\task_1675405360\build\blobber_upload_dir\d8feb897-24e6-4f89-908d-0a95ab26b5aa.extra
[task 2023-02-03T11:13:28.901Z] 11:13:28     INFO -  PROCESS-CRASH | glterrain | application crashed [@ mozilla::net::CheckForBrokenChromeURL(nsILoadInfo*, nsIURI*)]
[task 2023-02-03T11:13:28.901Z] 11:13:28     INFO -  Crash dump filename: C:\Users\task_1675405360\AppData\Local\Temp\tmp9zvma8tk\profile\minidumps\d8feb897-24e6-4f89-908d-0a95ab26b5aa.dmp
[task 2023-02-03T11:13:28.902Z] 11:13:28     INFO -  Mozilla crash reason: Missing chrome or resource URLs: chrome://pageloader/content/utils.js
[task 2023-02-03T11:13:28.902Z] 11:13:28     INFO -  Operating system: Windows NT
[task 2023-02-03T11:13:28.902Z] 11:13:28     INFO -                    10.0.19045
[task 2023-02-03T11:13:28.902Z] 11:13:28     INFO -  CPU: amd64

in this case the reason is Missing chrome or resource URLs: chrome://pageloader/content/utils.js as the primary error message surfaced? Assuming it exists that can be done- or just for Talos?

Flags: needinfo?(jmaher)

(In reply to Joel Maher ( :jmaher ) (UTC -8) from comment #25)

:gijs, do you think all crashes should have the reason:

[task 2023-02-03T11:13:28.894Z] 11:13:28     INFO -  mozcrash Saved app info as C:\Users\task_1675405360\build\blobber_upload_dir\d8feb897-24e6-4f89-908d-0a95ab26b5aa.extra
[task 2023-02-03T11:13:28.901Z] 11:13:28     INFO -  PROCESS-CRASH | glterrain | application crashed [@ mozilla::net::CheckForBrokenChromeURL(nsILoadInfo*, nsIURI*)]
[task 2023-02-03T11:13:28.901Z] 11:13:28     INFO -  Crash dump filename: C:\Users\task_1675405360\AppData\Local\Temp\tmp9zvma8tk\profile\minidumps\d8feb897-24e6-4f89-908d-0a95ab26b5aa.dmp
[task 2023-02-03T11:13:28.902Z] 11:13:28     INFO -  Mozilla crash reason: Missing chrome or resource URLs: chrome://pageloader/content/utils.js
[task 2023-02-03T11:13:28.902Z] 11:13:28     INFO -  Operating system: Windows NT
[task 2023-02-03T11:13:28.902Z] 11:13:28     INFO -                    10.0.19045
[task 2023-02-03T11:13:28.902Z] 11:13:28     INFO -  CPU: amd64

in this case the reason is Missing chrome or resource URLs: chrome://pageloader/content/utils.js as the primary error message surfaced? Assuming it exists that can be done- or just for Talos?

I'll get a second opinion off :gsvelto but generally I think any MOZ_CRASH which I think gets logged with Mozilla crash reason: ... should have that Mozilla crash reason surfaced, yes.

Flags: needinfo?(gsvelto)

(In reply to :Gijs (he/him) from comment #26)

I'll get a second opinion off :gsvelto but generally I think any MOZ_CRASH which I think gets logged with Mozilla crash reason: ... should have that Mozilla crash reason surfaced, yes.

Yes that's the contents of MOZ_CRASH()'s message. mozcrash.py surfaces it when it encounters it.

Flags: needinfo?(gsvelto)
Whiteboard: [necko-triaged] → [necko-triaged] [necko-monitor]

I also found bug 1631828 while looking at this, I added a comment in there to how we might want to approach this.

Status: REOPENED → NEW
Flags: needinfo?(valentin.gosu)
See Also: → 1870627
You need to log in before you can comment on or make changes to this bug.