Logs are not live on `standard_gcp_docker_worker` tasks
Categories
(Taskcluster :: Workers, defect)
Tracking
(Not tracked)
People
(Reporter: SimonSapin, Assigned: bstack)
Details
For example, this task links to this log file. Before the task finished, the log viewer showed:
NetworkError when attempting to fetch resource.
An error occurred attempting to load the provided log.
Please check the URL and ensure it is reachable.
https://community-tc.services.mozilla.com/api/queue/v1/task/WMUJHen2QyGYTEevhxi9og/runs/0/artifacts/public/logs/live.log
|
Reporter | |
Comment 1•5 years ago
|
||
That last URL redirects to https://proj-servo-docker-oy5aes4orzebkdskvi9ayw.c.community-tc-workers.internal:32771/log/ihHgF5YySRqvjpJ09TmJjQ
The .internal TLD does not resolve on public DNS.
|
||
Comment 2•5 years ago
|
||
This is because the hosts don't have public IPs. We need to figure out how to configure that in GCP and update the worker pool definitions.
|
|
Reporter | |
Comment 3•5 years ago
|
||
Is it easier to configure public IP addresses in GCP, or to (maybe temporarily) switch back to AWS and use taskcluster.net’s docker-worker image?
|
|
||
Updated•5 years ago
|
|
|
||
Comment 4•5 years ago
|
||
Ah, actually, the IP config is fine here, and we just need the secrets set correctly. I think Brian can do that.
|
Assignee | |
Comment 5•5 years ago
|
||
I set up the secrets last night. Hopefully anything running today will have livelogs. I'll check back in with this later today.
|
|
Reporter | |
Comment 6•5 years ago
|
||
I’ve verified that logs were live on https://community-tc.services.mozilla.com/tasks/Nq3MJv7HRgWcC5dOVB1g6w while it was running. Thanks!
|
|
Reporter | |
Comment 7•5 years ago
|
||
Logs were not live in https://community-tc.services.mozilla.com/tasks/ciL0gzGYSQSLT3RDZJFM6Q/runs/0. The only notable difference is that it ran with a different worker pool ID, with a worker pool definition that I copied from proj-servo/docker.
Brian, are the secrets you mentioned in comment 5 tied to a given pool ID?
|
|
Assignee | |
Comment 8•5 years ago
|
||
Yeah, each pool needs its own secret. The value is actually identical so it should be quick/easy for someone to copy over. If you let me know what the new workerPoolId is, I can copy it in a sec.
|
|
Reporter | |
Comment 9•5 years ago
|
||
Looks like I have scopes to read and write https://community-tc.services.mozilla.com/secrets/worker-pool%3Aproj-servo%2Fdocker
This was a temporary worker pool to verify the fix for bug 1593543. If I need more pools like this in the future I’ll try to remember to copy-paste the secret. Thanks!
|
|
Assignee | |
Comment 10•5 years ago
|
||
Once we're through this rush to the finish line we're going to automate this for all workerpools so that you won't need to do this manually. Thanks for getting it set up!
Description
•