1175034 - "curl: (22) The requested URL returned error: 404 Not Found" even in successful builds

Status: RESOLVED WORKSFORME

(Taskcluster :: General, defect)

Description

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

see https://s3-us-west-2.amazonaws.com/taskcluster-public-artifacts/Z9yUh9RBTI2-FeclT26QJA/0/public/logs/live_backing.log

Comment 1

[Pete Moore [:pmoore][:pete]]

Greg,

Is this just an issue with eventual consistency of amazon artifacts? e.g. we try to pull down an artifact, but it is not available yet, so we retry with backoff, and eventually it appears?

If this is the case, I'm not sure there is a lot we can do here. Although I can see maybe it can be confusing for users. Maybe we could log a message to say something like "This 404 error indicates that this artifact is not yet available, which is often to be expected due to time taken for artifacts to upload/replicate across regions. Will keep retrying for some time until it becomes available, or will give up."

Another alternative might be to not log the 404 unless all attempts fail (i.e. to buffer up error messages, and only output them if the situation does not resolve).

These are just ideas, what are your thoughts? And is this the reason for the 404's?

I can see that for us the 404 in the log is not concerning, but I can imagine Naoki might not be the only person that is disconcerted by it.

Thanks,
Pete

Comment 2

[Greg Arndt [:garndt]]

Pete, the artifacts that are being downloaded here were created by the tc-vcs cloning 24 days ago so everything should have been there when this task ran.  Also those artifacts are in us-west-2 which is read-after-write consistent.

I'm not sure what's going pulled down, but if 404's don't cause the task to fail, then I say that's a bug even if in this case it doesn't look like it caused an issue.  If a package wasn't retrieved that would change the build if it was or was not present, then that's not good.

Comment 3

[Selena Deckelmann :selenamarie :selena]

Hasn't come up again in 8 months. Closing.