Bug 1400258 Comment 28 Edit History

Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.

Recent failure log: 
2019-03-17T02:50:55     INFO - build_task_dependencies signing cSUzbTErSxCBNkm8wXeMZA
2019-03-17T02:50:55     INFO - find_sorted_task_dependencies signing cSUzbTErSxCBNkm8wXeMZA
2019-03-17T02:50:55     INFO - found dependencies: [('signing:parent', 'MUW-S9mASkOV71jDnCH9Qg'), ('signing:build', 'JrgR_f8qTduQW8cBW1wZlA'), ('signing:build', 'JrgR_f8qTduQW8cBW1wZlA')]
2019-03-17T02:50:56    DEBUG -   signing:parent MUW-S9mASkOV71jDnCH9Qg is docker-worker
2019-03-17T02:50:56    DEBUG -   makedirs(/builds/scriptworker/work/cot/MUW-S9mASkOV71jDnCH9Qg)
2019-03-17T02:50:56     INFO - build_task_dependencies signing:parent MUW-S9mASkOV71jDnCH9Qg
2019-03-17T02:50:56     INFO - find_sorted_task_dependencies signing:parent MUW-S9mASkOV71jDnCH9Qg
2019-03-17T02:50:56     INFO - found dependencies: [('signing:parent:parent', 'cKGHzJR4QP684BoKhmQBlQ')]
2019-03-17T02:50:56    DEBUG -   signing:parent:parent cKGHzJR4QP684BoKhmQBlQ is docker-worker
2019-03-17T02:50:56    DEBUG -   makedirs(/builds/scriptworker/work/cot/cKGHzJR4QP684BoKhmQBlQ)
2019-03-17T02:50:56     INFO - build_task_dependencies signing:parent:parent cKGHzJR4QP684BoKhmQBlQ
2019-03-17T02:50:56     INFO - find_sorted_task_dependencies signing:parent:parent cKGHzJR4QP684BoKhmQBlQ
2019-03-17T02:50:56     INFO - found dependencies: [('signing:parent:parent:parent', 'cKGHzJR4QP684BoKhmQBlQ')]
2019-03-17T02:50:56    DEBUG -   signing:build JrgR_f8qTduQW8cBW1wZlA is generic-worker
2019-03-17T02:50:56    DEBUG -   makedirs(/builds/scriptworker/work/cot/JrgR_f8qTduQW8cBW1wZlA)
2019-03-17T02:50:56     INFO - build_task_dependencies signing:build JrgR_f8qTduQW8cBW1wZlA
2019-03-17T02:50:56     INFO - find_sorted_task_dependencies signing:build JrgR_f8qTduQW8cBW1wZlA
2019-03-17T02:50:56     INFO - found dependencies: [('signing:build:parent', 'cKGHzJR4QP684BoKhmQBlQ')]
2019-03-17T02:50:56     INFO - Downloading https://queue.taskcluster.net/v1/task/MUW-S9mASkOV71jDnCH9Qg/artifacts/public%2FchainOfTrust.json.asc
2019-03-17T02:50:56     INFO - Downloading https://queue.taskcluster.net/v1/task/MUW-S9mASkOV71jDnCH9Qg/artifacts/public%2Fchain-of-trust.json
2019-03-17T02:50:56     INFO - Downloading https://queue.taskcluster.net/v1/task/cKGHzJR4QP684BoKhmQBlQ/artifacts/public%2FchainOfTrust.json.asc
2019-03-17T02:50:56     INFO - Downloading https://queue.taskcluster.net/v1/task/cKGHzJR4QP684BoKhmQBlQ/artifacts/public%2Fchain-of-trust.json
2019-03-17T02:50:56     INFO - Downloading https://queue.taskcluster.net/v1/task/JrgR_f8qTduQW8cBW1wZlA/artifacts/public%2FchainOfTrust.json.asc
2019-03-17T02:50:56     INFO - Downloading https://queue.taskcluster.net/v1/task/JrgR_f8qTduQW8cBW1wZlA/artifacts/public%2Fchain-of-trust.json
2019-03-17T02:50:56    DEBUG -   404 downloading https://queue.taskcluster.net/v1/task/MUW-S9mASkOV71jDnCH9Qg/artifacts/public%2FchainOfTrust.json.asc: 404; body={
  "code": "ResourceNotFound",
  "message": "Artifact not found\n\n---\n\n* method:     getLatestArtifact\n* errorCode:  ResourceNotFound\n* statusCode: 404\n* time:       2019-03-17T02:50:56.919Z",
  "requestInfo": {
    "method": "getLatestArtifact",
    "params": {
      "0": "public/chainOfTrust.json.asc",
      "taskId": "MUW-S9mASkOV71jDnCH9Qg",
      "name": "public/chainOfTrust.json.asc"
    },
    "payload": {},
    "time": "2019-03-17T02:50:56.919Z"
  }
}
2019-03-17T02:50:57    DEBUG -   404 downloading https://queue.taskcluster.net/v1/task/MUW-S9mASkOV71jDnCH9Qg/artifacts/public%2Fchain-of-trust.json: 404; body={
  "code": "ResourceNotFound",
  "message": "Artifact not found\n\n---\n\n* method:     getLatestArtifact\n* errorCode:  ResourceNotFound\n* statusCode: 404\n* time:       2019-03-17T02:50:57.232Z",
  "requestInfo": {
    "method": "getLatestArtifact",
    "params": {
      "0": "public/chain-of-trust.json",
      "taskId": "MUW-S9mASkOV71jDnCH9Qg",
      "name": "public/chain-of-trust.json"
    },
    "payload": {},
    "time": "2019-03-17T02:50:57.232Z"
  }
}
2019-03-17T02:50:57    DEBUG -   makedirs(/builds/scriptworker/work/cot/JrgR_f8qTduQW8cBW1wZlA/public)
2019-03-17T02:50:57     INFO - Done
2019-03-17T02:50:57    DEBUG -   makedirs(/builds/scriptworker/work/cot/cKGHzJR4QP684BoKhmQBlQ/public)
2019-03-17T02:50:57     INFO - Done
2019-03-17T02:50:57     INFO - Done
2019-03-17T02:50:57     INFO - Done
2019-03-17T02:50:57 CRITICAL - Chain of Trust verification error!
Traceback (most recent call last):

  File "/builds/scriptworker/lib/python3.6/site-packages/scriptworker/cot/verify.py", line 2197, in verify_chain_of_trust
    await download_cot(chain)
  File "/builds/scriptworker/lib/python3.6/site-packages/scriptworker/cot/verify.py", line 646, in download_cot
    mandatory_artifacts_paths = await raise_future_exceptions(mandatory_artifact_tasks)
  File "/builds/scriptworker/lib/python3.6/site-packages/scriptworker/utils.py", line 327, in raise_future_exceptions
    succeeded_results, _ = await _process_future_exceptions(tasks, raise_at_first_error=True)
  File "/builds/scriptworker/lib/python3.6/site-packages/scriptworker/utils.py", line 361, in _process_future_exceptions
    raise exc
  File "/builds/scriptworker/lib/python3.6/site-packages/scriptworker/artifacts.py", line 308, in download_artifacts
    await raise_future_exceptions(tasks)
  File "/builds/scriptworker/lib/python3.6/site-packages/scriptworker/utils.py", line 327, in raise_future_exceptions
    succeeded_results, _ = await _process_future_exceptions(tasks, raise_at_first_error=True)
  File "/builds/scriptworker/lib/python3.6/site-packages/scriptworker/utils.py", line 361, in _process_future_exceptions
    raise exc
  File "/builds/scriptworker/lib/python3.6/site-packages/scriptworker/utils.py", line 261, in retry_async
    return await func(*args, **kwargs)
  File "/builds/scriptworker/lib/python3.6/site-packages/scriptworker/utils.py", line 558, in download_file
    raise Download404("{} status {}!".format(loggable_url, resp.status))
scriptworker.exceptions.Download404: https://queue.taskcluster.net/v1/task/MUW-S9mASkOV71jDnCH9Qg/artifacts/public%2FchainOfTrust.json.asc status 404!
2019-03-17T02:50:57    ERROR - Hit ScriptWorkerException: 'https://queue.taskcluster.net/v1/task/MUW-S9mASkOV71jDnCH9Qg/artifacts/public%2FchainOfTrust.json.asc status 404!'
2019-03-17T02:50:57    DEBUG -   "/builds/scriptworker/artifacts/public/logs/chain_of_trust.log" is encoded with "None" and has mime/type "text/plain"
2019-03-17T02:50:57     INFO - "/builds/scriptworker/artifacts/public/logs/chain_of_trust.log" can be gzip'd. Compressing...
https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&resultStatus=testfailed%2Cbusted%2Cexception&selectedJob=234386319&searchStr=windows%2C2012%2Cdebug%2Cbuild-signing-win32%2Fdebug%2C%28bs%29

Recent failure log: 
2019-03-17T02:50:55     INFO - build_task_dependencies signing cSUzbTErSxCBNkm8wXeMZA
2019-03-17T02:50:55     INFO - find_sorted_task_dependencies signing cSUzbTErSxCBNkm8wXeMZA
2019-03-17T02:50:55     INFO - found dependencies: [('signing:parent', 'MUW-S9mASkOV71jDnCH9Qg'), ('signing:build', 'JrgR_f8qTduQW8cBW1wZlA'), ('signing:build', 'JrgR_f8qTduQW8cBW1wZlA')]
2019-03-17T02:50:56    DEBUG -   signing:parent MUW-S9mASkOV71jDnCH9Qg is docker-worker
2019-03-17T02:50:56    DEBUG -   makedirs(/builds/scriptworker/work/cot/MUW-S9mASkOV71jDnCH9Qg)
2019-03-17T02:50:56     INFO - build_task_dependencies signing:parent MUW-S9mASkOV71jDnCH9Qg
2019-03-17T02:50:56     INFO - find_sorted_task_dependencies signing:parent MUW-S9mASkOV71jDnCH9Qg
2019-03-17T02:50:56     INFO - found dependencies: [('signing:parent:parent', 'cKGHzJR4QP684BoKhmQBlQ')]
2019-03-17T02:50:56    DEBUG -   signing:parent:parent cKGHzJR4QP684BoKhmQBlQ is docker-worker
2019-03-17T02:50:56    DEBUG -   makedirs(/builds/scriptworker/work/cot/cKGHzJR4QP684BoKhmQBlQ)
2019-03-17T02:50:56     INFO - build_task_dependencies signing:parent:parent cKGHzJR4QP684BoKhmQBlQ
2019-03-17T02:50:56     INFO - find_sorted_task_dependencies signing:parent:parent cKGHzJR4QP684BoKhmQBlQ
2019-03-17T02:50:56     INFO - found dependencies: [('signing:parent:parent:parent', 'cKGHzJR4QP684BoKhmQBlQ')]
2019-03-17T02:50:56    DEBUG -   signing:build JrgR_f8qTduQW8cBW1wZlA is generic-worker
2019-03-17T02:50:56    DEBUG -   makedirs(/builds/scriptworker/work/cot/JrgR_f8qTduQW8cBW1wZlA)
2019-03-17T02:50:56     INFO - build_task_dependencies signing:build JrgR_f8qTduQW8cBW1wZlA
2019-03-17T02:50:56     INFO - find_sorted_task_dependencies signing:build JrgR_f8qTduQW8cBW1wZlA
2019-03-17T02:50:56     INFO - found dependencies: [('signing:build:parent', 'cKGHzJR4QP684BoKhmQBlQ')]
2019-03-17T02:50:56     INFO - Downloading https://queue.taskcluster.net/v1/task/MUW-S9mASkOV71jDnCH9Qg/artifacts/public%2FchainOfTrust.json.asc
2019-03-17T02:50:56     INFO - Downloading https://queue.taskcluster.net/v1/task/MUW-S9mASkOV71jDnCH9Qg/artifacts/public%2Fchain-of-trust.json
2019-03-17T02:50:56     INFO - Downloading https://queue.taskcluster.net/v1/task/cKGHzJR4QP684BoKhmQBlQ/artifacts/public%2FchainOfTrust.json.asc
2019-03-17T02:50:56     INFO - Downloading https://queue.taskcluster.net/v1/task/cKGHzJR4QP684BoKhmQBlQ/artifacts/public%2Fchain-of-trust.json
2019-03-17T02:50:56     INFO - Downloading https://queue.taskcluster.net/v1/task/JrgR_f8qTduQW8cBW1wZlA/artifacts/public%2FchainOfTrust.json.asc
2019-03-17T02:50:56     INFO - Downloading https://queue.taskcluster.net/v1/task/JrgR_f8qTduQW8cBW1wZlA/artifacts/public%2Fchain-of-trust.json
2019-03-17T02:50:56    DEBUG -   404 downloading https://queue.taskcluster.net/v1/task/MUW-S9mASkOV71jDnCH9Qg/artifacts/public%2FchainOfTrust.json.asc: 404; body={
  "code": "ResourceNotFound",
  "message": "Artifact not found\n\n---\n\n* method:     getLatestArtifact\n* errorCode:  ResourceNotFound\n* statusCode: 404\n* time:       2019-03-17T02:50:56.919Z",
  "requestInfo": {
    "method": "getLatestArtifact",
    "params": {
      "0": "public/chainOfTrust.json.asc",
      "taskId": "MUW-S9mASkOV71jDnCH9Qg",
      "name": "public/chainOfTrust.json.asc"
    },
    "payload": {},
    "time": "2019-03-17T02:50:56.919Z"
  }
}
2019-03-17T02:50:57    DEBUG -   404 downloading https://queue.taskcluster.net/v1/task/MUW-S9mASkOV71jDnCH9Qg/artifacts/public%2Fchain-of-trust.json: 404; body={
  "code": "ResourceNotFound",
  "message": "Artifact not found\n\n---\n\n* method:     getLatestArtifact\n* errorCode:  ResourceNotFound\n* statusCode: 404\n* time:       2019-03-17T02:50:57.232Z",
  "requestInfo": {
    "method": "getLatestArtifact",
    "params": {
      "0": "public/chain-of-trust.json",
      "taskId": "MUW-S9mASkOV71jDnCH9Qg",
      "name": "public/chain-of-trust.json"
    },
    "payload": {},
    "time": "2019-03-17T02:50:57.232Z"
  }
}
2019-03-17T02:50:57    DEBUG -   makedirs(/builds/scriptworker/work/cot/JrgR_f8qTduQW8cBW1wZlA/public)
2019-03-17T02:50:57     INFO - Done
2019-03-17T02:50:57    DEBUG -   makedirs(/builds/scriptworker/work/cot/cKGHzJR4QP684BoKhmQBlQ/public)
2019-03-17T02:50:57     INFO - Done
2019-03-17T02:50:57     INFO - Done
2019-03-17T02:50:57     INFO - Done
2019-03-17T02:50:57 CRITICAL - Chain of Trust verification error!
Traceback (most recent call last):

  File "/builds/scriptworker/lib/python3.6/site-packages/scriptworker/cot/verify.py", line 2197, in verify_chain_of_trust
    await download_cot(chain)
  File "/builds/scriptworker/lib/python3.6/site-packages/scriptworker/cot/verify.py", line 646, in download_cot
    mandatory_artifacts_paths = await raise_future_exceptions(mandatory_artifact_tasks)
  File "/builds/scriptworker/lib/python3.6/site-packages/scriptworker/utils.py", line 327, in raise_future_exceptions
    succeeded_results, _ = await _process_future_exceptions(tasks, raise_at_first_error=True)
  File "/builds/scriptworker/lib/python3.6/site-packages/scriptworker/utils.py", line 361, in _process_future_exceptions
    raise exc
  File "/builds/scriptworker/lib/python3.6/site-packages/scriptworker/artifacts.py", line 308, in download_artifacts
    await raise_future_exceptions(tasks)
  File "/builds/scriptworker/lib/python3.6/site-packages/scriptworker/utils.py", line 327, in raise_future_exceptions
    succeeded_results, _ = await _process_future_exceptions(tasks, raise_at_first_error=True)
  File "/builds/scriptworker/lib/python3.6/site-packages/scriptworker/utils.py", line 361, in _process_future_exceptions
    raise exc
  File "/builds/scriptworker/lib/python3.6/site-packages/scriptworker/utils.py", line 261, in retry_async
    return await func(*args, **kwargs)
  File "/builds/scriptworker/lib/python3.6/site-packages/scriptworker/utils.py", line 558, in download_file
    raise Download404("{} status {}!".format(loggable_url, resp.status))
scriptworker.exceptions.Download404: https://queue.taskcluster.net/v1/task/MUW-S9mASkOV71jDnCH9Qg/artifacts/public%2FchainOfTrust.json.asc status 404!
2019-03-17T02:50:57    ERROR - Hit ScriptWorkerException: 'https://queue.taskcluster.net/v1/task/MUW-S9mASkOV71jDnCH9Qg/artifacts/public%2FchainOfTrust.json.asc status 404!'
2019-03-17T02:50:57    DEBUG -   "/builds/scriptworker/artifacts/public/logs/chain_of_trust.log" is encoded with "None" and has mime/type "text/plain"
2019-03-17T02:50:57     INFO - "/builds/scriptworker/artifacts/public/logs/chain_of_trust.log" can be gzip'd. Compressing...

Back to Bug 1400258 Comment 28