Open Bug 1836556 Opened 2 years ago Updated 2 years ago

Intermittent runtime error: invalid memory address or nil pointer dereference

Categories

(Taskcluster :: Workers, defect)

Product:
Taskcluster
Component:
Workers
Type:
defect

Tracking

(Not tracked)

People

(Reporter: imoraru, Unassigned)

Details

(Keywords: intermittent-failure)

This failed as an exception.

[taskcluster 2023-06-02T18:53:26.390Z] Worker Type (gecko-t/win11-64-2009) settings:
[taskcluster 2023-06-02T18:53:26.390Z]   {
[taskcluster 2023-06-02T18:53:26.390Z]     "config": {
[taskcluster 2023-06-02T18:53:26.390Z]       "deploymentId": ""
[taskcluster 2023-06-02T18:53:26.390Z]     },
[taskcluster 2023-06-02T18:53:26.390Z]     "generic-worker": {
[taskcluster 2023-06-02T18:53:26.390Z]       "engine": "multiuser",
[taskcluster 2023-06-02T18:53:26.390Z]       "go-arch": "amd64",
[taskcluster 2023-06-02T18:53:26.390Z]       "go-os": "windows",
[taskcluster 2023-06-02T18:53:26.390Z]       "go-version": "go1.18.1",
[taskcluster 2023-06-02T18:53:26.390Z]       "release": "https://github.com/taskcluster/taskcluster/releases/tag/v44.13.6",
[taskcluster 2023-06-02T18:53:26.390Z]       "revision": "122c8f40b61779f71b1fa49ceca70e2b5609b45b",
[taskcluster 2023-06-02T18:53:26.390Z]       "source": "https://github.com/taskcluster/taskcluster/commits/122c8f40b61779f71b1fa49ceca70e2b5609b45b",
[taskcluster 2023-06-02T18:53:26.390Z]       "version": "44.13.6"
[taskcluster 2023-06-02T18:53:26.390Z]     },
[taskcluster 2023-06-02T18:53:26.390Z]     "instance-type": "Standard_F8s_v2",
[taskcluster 2023-06-02T18:53:26.390Z]     "local-ipv4": "10.0.0.113",
[taskcluster 2023-06-02T18:53:26.390Z]     "public-ipv4": "",
[taskcluster 2023-06-02T18:53:26.390Z]     "region": "westus2",
[taskcluster 2023-06-02T18:53:26.390Z]     "vm-id": "e4bc9465-d80d-4c4f-b124-e32cadf0bc13"
[taskcluster 2023-06-02T18:53:26.390Z]   }
[taskcluster 2023-06-02T18:53:26.394Z] Task ID: DYv9Iu7vQN-ly0BBlOBqXw
[taskcluster 2023-06-02T18:53:26.394Z] === Task Starting ===
[taskcluster 2023-06-02T18:53:26.814Z] Uploading redirect artifact public/logs/live.log to URL https://firefoxci-websocktunnel.services.mozilla.com/westus2.vm-r3k0etzuq0onnssznsusbqwvdnhuxmteobn.60099/log/KLRr2tI7TtitF9dGY7qbZA with mime type "text/plain; charset=utf-8" and expiry 2023-06-02T21:08:26.770Z
[taskcluster 2023-06-02T18:53:26.844Z] [mounts] Downloading task VV2QPo4xTxuzGmFKWnX_Hg artifact public/build/mozharness.zip to Z:\downloads\f8JXHZ3cRTqZZi-9C3nbng
[taskcluster 2023-06-02T19:08:37.093Z] Uploading link artifact public/logs/live.log to artifact public/logs/live_backing.log with expiry 2024-06-01T18:07:29.593Z
[taskcluster:error] goroutine 1 [running]:
[taskcluster:error] runtime/debug.Stack()
[taskcluster:error] 	/task_165210836896924/go/go/src/runtime/debug/stack.go:24 +0x65
[taskcluster:error] main.(*TaskRun).Run.func2()
[taskcluster:error] 	/task_165210836896924/taskcluster/workers/generic-worker/main.go:895 +0xb3
[taskcluster:error] panic({0x144d500, 0x19392a0})
[taskcluster:error] 	/task_165210836896924/go/go/src/runtime/panic.go:844 +0x258
[taskcluster:error] github.com/taskcluster/taskcluster/v44/clients/client-go/internal.GetURL(0xc00051c1b0?, {0xc0005221e0?, 0x90?}, {0x1606268?, 0xc000006310?})
[taskcluster:error] 	/task_165210836896924/taskcluster/clients/client-go/internal/geturl.go:65 +0x161
[taskcluster:error] github.com/taskcluster/taskcluster/v44/clients/client-go/tcqueue.(*Queue).DownloadArtifactToWriteSeeker(0xc0002f8150, {0xc0005160d8?, 0x242?}, 0xc000540100?, {0xc0002f4120?, 0x14e1f0a?}, {0x1606268, 0xc000006310})
[taskcluster:error] 	/task_165210836896924/taskcluster/clients/client-go/tcqueue/download.go:86 +0x4a5
[taskcluster:error] github.com/taskcluster/taskcluster/v44/clients/client-go/tcqueue.(*Queue).DownloadArtifactToFile(0xc00029a480?, {0xc0005160d8, 0x16}, 0xc0001b91e8?, {0xc0002f4120, 0x1b}, {0xc00029a480?, 0xc000364340?})
[taskcluster:error] 	/task_165210836896924/taskcluster/clients/client-go/tcqueue/download.go:47 +0x165
[taskcluster:error] main.(*ArtifactContent).Download(0xc000514660, 0xc0001c4000)
[taskcluster:error] 	/task_165210836896924/taskcluster/workers/generic-worker/mounts.go:736 +0x168
[taskcluster:error] main.ensureCached({0x1608580, 0xc000514660}, 0x22?)
[taskcluster:error] 	/task_165210836896924/taskcluster/workers/generic-worker/mounts.go:621 +0x3a7
[taskcluster:error] main.extract({0x1608580?, 0xc000514660?}, {0xc0001eb0f0, 0x3}, {0xc00029a450, 0x22}, 0xc00032e160?)
[taskcluster:error] 	/task_165210836896924/taskcluster/workers/generic-worker/mounts.go:651 +0x65
[taskcluster:error] main.(*ReadOnlyDirectory).Mount(0xc000550d40, 0x1000?)
[taskcluster:error] 	/task_165210836896924/taskcluster/workers/generic-worker/mounts.go:536 +0x12c
[taskcluster:error] main.(*TaskMount).Start(0xc0001c2150)
[taskcluster:error] 	/task_165210836896924/taskcluster/workers/generic-worker/mounts.go:356 +0x1ba
[taskcluster:error] main.(*TaskRun).Run(0xc0001c4000)
[taskcluster:error] 	/task_165210836896924/taskcluster/workers/generic-worker/main.go:971 +0x995
[taskcluster:error] main.RunWorker()
[taskcluster:error] 	/task_165210836896924/taskcluster/workers/generic-worker/main.go:426 +0xb59
[taskcluster:error] main.main()
[taskcluster:error] 	/task_165210836896924/taskcluster/workers/generic-worker/main.go:158 +0x54a
[taskcluster:error] 
[taskcluster:error] "invalid memory address or nil pointer dereference"
[taskcluster:error] runtime error: invalid memory address or nil pointer dereference

This is a very bizarre error.

This happened on line 65 below:

 49     var resp *http.Response
 50     // HTTP status codes handled here automatically
 51     client := httpBackoffClient
 52     if client == nil {
 53         client = &httpbackoff.Client{
 54             BackOffSettings: backoff.NewExponentialBackOff(),
 55         }
 56     }
 57     var attempts int
 58     resp, attempts, err = client.Retry(retryFunc)
 59     if err != nil {
 60         err = HTTPRetryError{
 61             Attempts: attempts,
 62             Err:      err,
 63         }
 64     }
 65     defer resp.Body.Close()

from the stack trace:

[taskcluster:error] github.com/taskcluster/taskcluster/v44/clients/client-go/internal.GetURL(0xc00051c1b0?, {0xc0005221e0?, 0x90?}, {0x1606268?, 0xc000006310?})
[taskcluster:error] 	/task_165210836896924/taskcluster/clients/client-go/internal/geturl.go:65 +0x161

I don't expect to see this happen a lot, I wonder if it might have been a one-off random hardware failure.

You need to log in before you can comment on or make changes to this bug.