I noticed we had coverage yesterday in searchfox. It appears this happened because the nightly was built off of a Mozilla-central commit that was significantly older than usual (Tuesday at 10:49pm ET for Wednesday rather than the normal 4:30am+ ET). Because the manual process of comment 5 was laborious and maybe wrong and I need to figure out how to implement the logic in comment 4, I created a terrifying bash script at https://gist.github.com/asutherland/0f820de77856e459d7e1d4196be6f7ae that provides us with the helpful output at the bottom of the comment with the headings. There's 3 passes in the script as displayed below: 1. "Searchfox rev artifact creator times": - Find the recent mozilla-central searchfox treeherder linux64 jobs and use that to figure out the revisions that searchfox indexed. - Lookup the task that corresponds to the given artifact index root for the given tree and revision. - If the was a task, get its info, then output it. 2. "Hook Recent Runs for code-coverage-cron-production" - This just shows the completion and start times from jobs for this hook. These tasks don't have any revision information in them. They also seem to only produce precursor artifacts and not the actual aggregation we would assume if I run `taskcluster task def en9AhwDYQGSSWZ32PF30Tw` for example on the 12-04 cron task's id. 3. "Hook Results for code-coverage-repo-production": - Find the recent hook runs - Filter the runs down to those that correspond to mozilla-central and a revision for which searchfox indexing run (as determined in the first pass) - Output the info. What the output and my investigation seems to be showing is that: - Maybe the cron job is currently required to make things happen. - The cron job is actually running at midnight UTC time instead of when we want it to? :marco, thoughts? ## Output ### Searchfox rev artifact creator times - d4b21131a094a0819b8856fbf4165cfb32d671c6 artifact project.relman.code-coverage.production.repo.mozilla-central.d4b21131a094a0819b8856fbf4165cfb32d671c6 - 2020-12-03T10:44:38.906Z - indexing completed in job IZjIJ121TMO3s4OwZFMl2Q - 2020-12-04T01:40:41.803Z - artifact task completed in job VrUpFqafShWpEv5-DCqHQA started at 2020-12-04T00:03:31.464Z - b836812230d1274efa92eb0a0d38601ef4715cda artifact project.relman.code-coverage.production.repo.mozilla-central.b836812230d1274efa92eb0a0d38601ef4715cda - 2020-12-04T10:45:32.316Z - indexing completed in job WQ8-4NBtTWqaXTA74wjWRg - 2020-12-05T01:50:36.159Z - artifact task completed in job bsAQop-BTfGXOa-Mp-C9wQ started at 2020-12-05T00:05:40.519Z - 7ce95b6cde2643be60ec53d8c743db28aec30884 artifact project.relman.code-coverage.production.repo.mozilla-central.7ce95b6cde2643be60ec53d8c743db28aec30884 - 2020-12-05T10:42:01.895Z - indexing completed in job TrEpg3-2RneHzhzUVccmxw - 2020-12-06T01:40:05.383Z - artifact task completed in job OLpXEJx7RDeOuTSkrSuKOA started at 2020-12-06T00:06:45.971Z - 5a17a9893fe09fd912f1972b204d0f9ca1184b28 artifact project.relman.code-coverage.production.repo.mozilla-central.5a17a9893fe09fd912f1972b204d0f9ca1184b28 - 2020-12-06T10:45:04.382Z - indexing completed in job BTicAleYQ2uOBMbZXS_gPw - 2020-12-07T01:33:25.315Z - artifact task completed in job RBIL-Rv-RNG34DHcz-LM0g started at 2020-12-07T00:05:12.411Z - 7167ab3f6e8d7b77439ac1cc7d9356bbaff02510 artifact project.relman.code-coverage.production.repo.mozilla-central.7167ab3f6e8d7b77439ac1cc7d9356bbaff02510 - 2020-12-07T10:37:57.384Z - indexing completed in job Yy6RvYLzQ9y7QF3TFD7Clw - 2020-12-08T01:46:59.147Z - artifact task completed in job d3yZDYm2R9St4Eiwn4_P2g started at 2020-12-08T00:05:12.404Z - 1a164819adefcdb42bdba8a24976b62ee8d03c80 artifact project.relman.code-coverage.production.repo.mozilla-central.1a164819adefcdb42bdba8a24976b62ee8d03c80 - 2020-12-08T10:43:20.898Z - indexing completed in job TC6zbrHwR8S_dJ7DyBO-eg - 2020-12-09T01:41:03.400Z - artifact task completed in job QxoUycv-TAaXLVAn54gn1Q started at 2020-12-09T00:09:07.369Z - b4cd29abb74ae321be6176130fcd2130c7179d6d artifact project.relman.code-coverage.production.repo.mozilla-central.b4cd29abb74ae321be6176130fcd2130c7179d6d - 2020-12-09T10:56:35.407Z - indexing completed in job V7vA_AsxQiGPH1_2aGgB9g - 2020-12-09T11:10:07.410Z - artifact task completed in job L6wue_-7TPug_39vNRaEAA started at 2020-12-09T08:59:40.346Z - 73f050da7d204fcfb59aa2528f3d22665bc043d3 artifact project.relman.code-coverage.production.repo.mozilla-central.73f050da7d204fcfb59aa2528f3d22665bc043d3 - 2020-12-10T10:46:45.880Z - indexing completed in job erxttPsYRDSqf8YWwDr5Cg - No such artifact! ### Hook Recent Runs for code-coverage-cron-production - 2020-12-04T00:28:22.567Z completed after starting at 2020-12-04T00:00:53.062Z for task en9AhwDYQGSSWZ32PF30Tw - 2020-12-05T00:42:28.150Z completed after starting at 2020-12-05T00:02:45.325Z for task ZccIOU_jT6ylGydRTjQQIA - 2020-12-06T01:09:56.020Z completed after starting at 2020-12-06T00:01:58.761Z for task cIjBJx1bQ42IdsuBiuq8WQ - 2020-12-07T00:37:33.837Z completed after starting at 2020-12-07T00:03:03.062Z for task KRPevd9WRs2iXICruGPFLQ - 2020-12-08T00:22:34.855Z completed after starting at 2020-12-08T00:02:14.910Z for task KJ1Iwt9lTu2sJkY4gwP_DQ - 2020-12-09T00:21:27.892Z completed after starting at 2020-12-09T00:03:04.293Z for task dbWUaFy0RmKMbJId8fHRhA - 2020-12-10T00:27:41.105Z completed after starting at 2020-12-10T00:01:59.470Z for task WxOg4C3mSRa4YXhwhIKanw ### Hook Results for code-coverage-repo-production - rev d4b21131a094a0819b8856fbf4165cfb32d671c6 - 2020-12-10T11:48:40.246Z - indexing completed in job IZjIJ121TMO3s4OwZFMl2Q - 2020-12-04T01:40:41.803Z - aggregation completed in job VrUpFqafShWpEv5-DCqHQA - rev b836812230d1274efa92eb0a0d38601ef4715cda - 2020-12-10T11:48:40.246Z - indexing completed in job WQ8-4NBtTWqaXTA74wjWRg - 2020-12-05T01:50:36.159Z - aggregation completed in job bsAQop-BTfGXOa-Mp-C9wQ - rev 7ce95b6cde2643be60ec53d8c743db28aec30884 - 2020-12-10T11:48:40.246Z - indexing completed in job TrEpg3-2RneHzhzUVccmxw - 2020-12-06T01:40:05.383Z - aggregation completed in job OLpXEJx7RDeOuTSkrSuKOA - rev 5a17a9893fe09fd912f1972b204d0f9ca1184b28 - 2020-12-10T11:48:40.246Z - indexing completed in job BTicAleYQ2uOBMbZXS_gPw - 2020-12-07T01:33:25.315Z - aggregation completed in job RBIL-Rv-RNG34DHcz-LM0g - rev 7167ab3f6e8d7b77439ac1cc7d9356bbaff02510 - 2020-12-10T11:48:40.246Z - indexing completed in job Yy6RvYLzQ9y7QF3TFD7Clw - 2020-12-08T01:46:59.147Z - aggregation completed in job d3yZDYm2R9St4Eiwn4_P2g - rev 1a164819adefcdb42bdba8a24976b62ee8d03c80 - 2020-12-10T11:48:40.246Z - indexing completed in job TC6zbrHwR8S_dJ7DyBO-eg - 2020-12-09T01:41:03.400Z - aggregation completed in job QxoUycv-TAaXLVAn54gn1Q - rev b4cd29abb74ae321be6176130fcd2130c7179d6d - 2020-12-10T11:48:40.246Z - indexing completed in job V7vA_AsxQiGPH1_2aGgB9g - 2020-12-09T11:10:07.410Z - aggregation completed in job L6wue_-7TPug_39vNRaEAA - rev b4cd29abb74ae321be6176130fcd2130c7179d6d - 2020-12-10T11:48:40.246Z - indexing completed in job V7vA_AsxQiGPH1_2aGgB9g - 2020-12-10T06:33:31.604Z - aggregation completed in job cCIzobjoTuu6BI3iNIBYdw
Bug 1677903 Comment 6 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
I noticed we had coverage yesterday in searchfox. It appears this happened because the nightly was built off of a Mozilla-central commit that was significantly older than usual (Tuesday at 10:49pm ET for Wednesday rather than the normal 4:30am+ ET). Because the manual process of comment 5 was laborious and maybe wrong and I need to figure out how to implement the logic in comment 4, I created a terrifying bash script at https://gist.github.com/asutherland/0f820de77856e459d7e1d4196be6f7ae that provides us with the helpful output at the bottom of the comment with the headings. There's 3 passes in the script as displayed below: 1. "Searchfox rev artifact creator times": - Find the recent mozilla-central searchfox treeherder linux64 jobs and use that to figure out the revisions that searchfox indexed. - Lookup the task that corresponds to the given artifact index root for the given tree and revision. - If the was a task, get its info, then output it. 2. "Hook Recent Runs for code-coverage-cron-production" - This just shows the completion and start times from jobs for this hook. These tasks don't have any revision information in them. They also seem to only produce precursor artifacts and not the actual aggregation we would assume if I run `taskcluster task def en9AhwDYQGSSWZ32PF30Tw` for example on the 12-04 cron task's id. 3. "Hook Results for code-coverage-repo-production": - Find the recent hook runs - Filter the runs down to those that correspond to mozilla-central and a revision for which searchfox indexing run (as determined in the first pass) - Output the info. What the output and my investigation seems to be showing is that: - Maybe the cron job is currently required to make things happen. - The cron job is actually running at midnight UTC time instead of when we want it to? :marco, thoughts? ## Output ### Searchfox rev artifact creator times - d4b21131a094a0819b8856fbf4165cfb32d671c6 artifact project.relman.code-coverage.production.repo.mozilla-central.d4b21131a094a0819b8856fbf4165cfb32d671c6 - 2020-12-03T10:44:38.906Z - indexing completed in job IZjIJ121TMO3s4OwZFMl2Q - 2020-12-04T01:40:41.803Z - artifact task completed in job VrUpFqafShWpEv5-DCqHQA started at 2020-12-04T00:03:31.464Z - b836812230d1274efa92eb0a0d38601ef4715cda artifact project.relman.code-coverage.production.repo.mozilla-central.b836812230d1274efa92eb0a0d38601ef4715cda - 2020-12-04T10:45:32.316Z - indexing completed in job WQ8-4NBtTWqaXTA74wjWRg - 2020-12-05T01:50:36.159Z - artifact task completed in job bsAQop-BTfGXOa-Mp-C9wQ started at 2020-12-05T00:05:40.519Z - 7ce95b6cde2643be60ec53d8c743db28aec30884 artifact project.relman.code-coverage.production.repo.mozilla-central.7ce95b6cde2643be60ec53d8c743db28aec30884 - 2020-12-05T10:42:01.895Z - indexing completed in job TrEpg3-2RneHzhzUVccmxw - 2020-12-06T01:40:05.383Z - artifact task completed in job OLpXEJx7RDeOuTSkrSuKOA started at 2020-12-06T00:06:45.971Z - 5a17a9893fe09fd912f1972b204d0f9ca1184b28 artifact project.relman.code-coverage.production.repo.mozilla-central.5a17a9893fe09fd912f1972b204d0f9ca1184b28 - 2020-12-06T10:45:04.382Z - indexing completed in job BTicAleYQ2uOBMbZXS_gPw - 2020-12-07T01:33:25.315Z - artifact task completed in job RBIL-Rv-RNG34DHcz-LM0g started at 2020-12-07T00:05:12.411Z - 7167ab3f6e8d7b77439ac1cc7d9356bbaff02510 artifact project.relman.code-coverage.production.repo.mozilla-central.7167ab3f6e8d7b77439ac1cc7d9356bbaff02510 - 2020-12-07T10:37:57.384Z - indexing completed in job Yy6RvYLzQ9y7QF3TFD7Clw - 2020-12-08T01:46:59.147Z - artifact task completed in job d3yZDYm2R9St4Eiwn4_P2g started at 2020-12-08T00:05:12.404Z - 1a164819adefcdb42bdba8a24976b62ee8d03c80 artifact project.relman.code-coverage.production.repo.mozilla-central.1a164819adefcdb42bdba8a24976b62ee8d03c80 - 2020-12-08T10:43:20.898Z - indexing completed in job TC6zbrHwR8S_dJ7DyBO-eg - 2020-12-09T01:41:03.400Z - artifact task completed in job QxoUycv-TAaXLVAn54gn1Q started at 2020-12-09T00:09:07.369Z - b4cd29abb74ae321be6176130fcd2130c7179d6d artifact project.relman.code-coverage.production.repo.mozilla-central.b4cd29abb74ae321be6176130fcd2130c7179d6d - 2020-12-09T10:56:35.407Z - indexing completed in job V7vA_AsxQiGPH1_2aGgB9g - 2020-12-09T11:10:07.410Z - artifact task completed in job L6wue_-7TPug_39vNRaEAA started at 2020-12-09T08:59:40.346Z - 73f050da7d204fcfb59aa2528f3d22665bc043d3 artifact project.relman.code-coverage.production.repo.mozilla-central.73f050da7d204fcfb59aa2528f3d22665bc043d3 - 2020-12-10T10:46:45.880Z - indexing completed in job erxttPsYRDSqf8YWwDr5Cg - No such artifact! ### Hook Recent Runs for code-coverage-cron-production - 2020-12-04T00:28:22.567Z completed after starting at 2020-12-04T00:00:53.062Z for task en9AhwDYQGSSWZ32PF30Tw - 2020-12-05T00:42:28.150Z completed after starting at 2020-12-05T00:02:45.325Z for task ZccIOU_jT6ylGydRTjQQIA - 2020-12-06T01:09:56.020Z completed after starting at 2020-12-06T00:01:58.761Z for task cIjBJx1bQ42IdsuBiuq8WQ - 2020-12-07T00:37:33.837Z completed after starting at 2020-12-07T00:03:03.062Z for task KRPevd9WRs2iXICruGPFLQ - 2020-12-08T00:22:34.855Z completed after starting at 2020-12-08T00:02:14.910Z for task KJ1Iwt9lTu2sJkY4gwP_DQ - 2020-12-09T00:21:27.892Z completed after starting at 2020-12-09T00:03:04.293Z for task dbWUaFy0RmKMbJId8fHRhA - 2020-12-10T00:27:41.105Z completed after starting at 2020-12-10T00:01:59.470Z for task WxOg4C3mSRa4YXhwhIKanw ### Hook Results for code-coverage-repo-production - rev d4b21131a094a0819b8856fbf4165cfb32d671c6 - 2020-12-10T11:48:40.246Z - indexing completed in job IZjIJ121TMO3s4OwZFMl2Q - 2020-12-04T01:40:41.803Z - aggregation completed in job VrUpFqafShWpEv5-DCqHQA - rev b836812230d1274efa92eb0a0d38601ef4715cda - 2020-12-10T11:48:40.246Z - indexing completed in job WQ8-4NBtTWqaXTA74wjWRg - 2020-12-05T01:50:36.159Z - aggregation completed in job bsAQop-BTfGXOa-Mp-C9wQ - rev 7ce95b6cde2643be60ec53d8c743db28aec30884 - 2020-12-10T11:48:40.246Z - indexing completed in job TrEpg3-2RneHzhzUVccmxw - 2020-12-06T01:40:05.383Z - aggregation completed in job OLpXEJx7RDeOuTSkrSuKOA - rev 5a17a9893fe09fd912f1972b204d0f9ca1184b28 - 2020-12-10T11:48:40.246Z - indexing completed in job BTicAleYQ2uOBMbZXS_gPw - 2020-12-07T01:33:25.315Z - aggregation completed in job RBIL-Rv-RNG34DHcz-LM0g - rev 7167ab3f6e8d7b77439ac1cc7d9356bbaff02510 - 2020-12-10T11:48:40.246Z - indexing completed in job Yy6RvYLzQ9y7QF3TFD7Clw - 2020-12-08T01:46:59.147Z - aggregation completed in job d3yZDYm2R9St4Eiwn4_P2g - rev 1a164819adefcdb42bdba8a24976b62ee8d03c80 - 2020-12-10T11:48:40.246Z - indexing completed in job TC6zbrHwR8S_dJ7DyBO-eg - 2020-12-09T01:41:03.400Z - aggregation completed in job QxoUycv-TAaXLVAn54gn1Q - rev b4cd29abb74ae321be6176130fcd2130c7179d6d - 2020-12-10T11:48:40.246Z - indexing completed in job V7vA_AsxQiGPH1_2aGgB9g - 2020-12-09T11:10:07.410Z - aggregation completed in job L6wue_-7TPug_39vNRaEAA - rev b4cd29abb74ae321be6176130fcd2130c7179d6d - 2020-12-10T11:48:40.246Z - indexing completed in job V7vA_AsxQiGPH1_2aGgB9g - 2020-12-10T06:33:31.604Z - aggregation completed in job cCIzobjoTuu6BI3iNIBYdw
I noticed we had coverage yesterday in searchfox. It appears this happened because the nightly was built off of a Mozilla-central commit that was significantly older than usual (Tuesday at 10:49pm ET for Wednesday rather than the normal 4:30am+ ET). Because the manual process of comment 5 was laborious and maybe wrong and I need to figure out how to implement the logic in comment 4, I created a terrifying bash script at https://gist.github.com/asutherland/0f820de77856e459d7e1d4196be6f7ae that provides us with the helpful output at the bottom of the comment with the headings. There's 3 passes in the script as displayed below: 1. "Searchfox rev artifact creator times": - Find the recent mozilla-central searchfox treeherder linux64 jobs and use that to figure out the revisions that searchfox indexed. - Lookup the task that corresponds to the given revision's coverage artifact index root for the given tree and revision. - If the was a task, get its info, then output it. 2. "Hook Recent Runs for code-coverage-cron-production" - This just shows the completion and start times from jobs for this hook. These tasks don't have any revision information in them. They also seem to only produce precursor artifacts and not the actual aggregation we would assume if I run `taskcluster task def en9AhwDYQGSSWZ32PF30Tw` for example on the 12-04 cron task's id. 3. "Hook Results for code-coverage-repo-production": - Find the recent hook runs - Filter the runs down to those that correspond to mozilla-central and a revision for which searchfox indexing run (as determined in the first pass) - Output the info. What the output and my investigation seems to be showing is that: - Maybe the cron job is currently required to make things happen. - The cron job is actually running at midnight UTC time instead of when we want it to? :marco, thoughts? ## Output ### Searchfox rev artifact creator times - d4b21131a094a0819b8856fbf4165cfb32d671c6 artifact project.relman.code-coverage.production.repo.mozilla-central.d4b21131a094a0819b8856fbf4165cfb32d671c6 - 2020-12-03T10:44:38.906Z - indexing completed in job IZjIJ121TMO3s4OwZFMl2Q - 2020-12-04T01:40:41.803Z - artifact task completed in job VrUpFqafShWpEv5-DCqHQA started at 2020-12-04T00:03:31.464Z - b836812230d1274efa92eb0a0d38601ef4715cda artifact project.relman.code-coverage.production.repo.mozilla-central.b836812230d1274efa92eb0a0d38601ef4715cda - 2020-12-04T10:45:32.316Z - indexing completed in job WQ8-4NBtTWqaXTA74wjWRg - 2020-12-05T01:50:36.159Z - artifact task completed in job bsAQop-BTfGXOa-Mp-C9wQ started at 2020-12-05T00:05:40.519Z - 7ce95b6cde2643be60ec53d8c743db28aec30884 artifact project.relman.code-coverage.production.repo.mozilla-central.7ce95b6cde2643be60ec53d8c743db28aec30884 - 2020-12-05T10:42:01.895Z - indexing completed in job TrEpg3-2RneHzhzUVccmxw - 2020-12-06T01:40:05.383Z - artifact task completed in job OLpXEJx7RDeOuTSkrSuKOA started at 2020-12-06T00:06:45.971Z - 5a17a9893fe09fd912f1972b204d0f9ca1184b28 artifact project.relman.code-coverage.production.repo.mozilla-central.5a17a9893fe09fd912f1972b204d0f9ca1184b28 - 2020-12-06T10:45:04.382Z - indexing completed in job BTicAleYQ2uOBMbZXS_gPw - 2020-12-07T01:33:25.315Z - artifact task completed in job RBIL-Rv-RNG34DHcz-LM0g started at 2020-12-07T00:05:12.411Z - 7167ab3f6e8d7b77439ac1cc7d9356bbaff02510 artifact project.relman.code-coverage.production.repo.mozilla-central.7167ab3f6e8d7b77439ac1cc7d9356bbaff02510 - 2020-12-07T10:37:57.384Z - indexing completed in job Yy6RvYLzQ9y7QF3TFD7Clw - 2020-12-08T01:46:59.147Z - artifact task completed in job d3yZDYm2R9St4Eiwn4_P2g started at 2020-12-08T00:05:12.404Z - 1a164819adefcdb42bdba8a24976b62ee8d03c80 artifact project.relman.code-coverage.production.repo.mozilla-central.1a164819adefcdb42bdba8a24976b62ee8d03c80 - 2020-12-08T10:43:20.898Z - indexing completed in job TC6zbrHwR8S_dJ7DyBO-eg - 2020-12-09T01:41:03.400Z - artifact task completed in job QxoUycv-TAaXLVAn54gn1Q started at 2020-12-09T00:09:07.369Z - b4cd29abb74ae321be6176130fcd2130c7179d6d artifact project.relman.code-coverage.production.repo.mozilla-central.b4cd29abb74ae321be6176130fcd2130c7179d6d - 2020-12-09T10:56:35.407Z - indexing completed in job V7vA_AsxQiGPH1_2aGgB9g - 2020-12-09T11:10:07.410Z - artifact task completed in job L6wue_-7TPug_39vNRaEAA started at 2020-12-09T08:59:40.346Z - 73f050da7d204fcfb59aa2528f3d22665bc043d3 artifact project.relman.code-coverage.production.repo.mozilla-central.73f050da7d204fcfb59aa2528f3d22665bc043d3 - 2020-12-10T10:46:45.880Z - indexing completed in job erxttPsYRDSqf8YWwDr5Cg - No such artifact! ### Hook Recent Runs for code-coverage-cron-production - 2020-12-04T00:28:22.567Z completed after starting at 2020-12-04T00:00:53.062Z for task en9AhwDYQGSSWZ32PF30Tw - 2020-12-05T00:42:28.150Z completed after starting at 2020-12-05T00:02:45.325Z for task ZccIOU_jT6ylGydRTjQQIA - 2020-12-06T01:09:56.020Z completed after starting at 2020-12-06T00:01:58.761Z for task cIjBJx1bQ42IdsuBiuq8WQ - 2020-12-07T00:37:33.837Z completed after starting at 2020-12-07T00:03:03.062Z for task KRPevd9WRs2iXICruGPFLQ - 2020-12-08T00:22:34.855Z completed after starting at 2020-12-08T00:02:14.910Z for task KJ1Iwt9lTu2sJkY4gwP_DQ - 2020-12-09T00:21:27.892Z completed after starting at 2020-12-09T00:03:04.293Z for task dbWUaFy0RmKMbJId8fHRhA - 2020-12-10T00:27:41.105Z completed after starting at 2020-12-10T00:01:59.470Z for task WxOg4C3mSRa4YXhwhIKanw ### Hook Results for code-coverage-repo-production - rev d4b21131a094a0819b8856fbf4165cfb32d671c6 - 2020-12-10T11:48:40.246Z - indexing completed in job IZjIJ121TMO3s4OwZFMl2Q - 2020-12-04T01:40:41.803Z - aggregation completed in job VrUpFqafShWpEv5-DCqHQA - rev b836812230d1274efa92eb0a0d38601ef4715cda - 2020-12-10T11:48:40.246Z - indexing completed in job WQ8-4NBtTWqaXTA74wjWRg - 2020-12-05T01:50:36.159Z - aggregation completed in job bsAQop-BTfGXOa-Mp-C9wQ - rev 7ce95b6cde2643be60ec53d8c743db28aec30884 - 2020-12-10T11:48:40.246Z - indexing completed in job TrEpg3-2RneHzhzUVccmxw - 2020-12-06T01:40:05.383Z - aggregation completed in job OLpXEJx7RDeOuTSkrSuKOA - rev 5a17a9893fe09fd912f1972b204d0f9ca1184b28 - 2020-12-10T11:48:40.246Z - indexing completed in job BTicAleYQ2uOBMbZXS_gPw - 2020-12-07T01:33:25.315Z - aggregation completed in job RBIL-Rv-RNG34DHcz-LM0g - rev 7167ab3f6e8d7b77439ac1cc7d9356bbaff02510 - 2020-12-10T11:48:40.246Z - indexing completed in job Yy6RvYLzQ9y7QF3TFD7Clw - 2020-12-08T01:46:59.147Z - aggregation completed in job d3yZDYm2R9St4Eiwn4_P2g - rev 1a164819adefcdb42bdba8a24976b62ee8d03c80 - 2020-12-10T11:48:40.246Z - indexing completed in job TC6zbrHwR8S_dJ7DyBO-eg - 2020-12-09T01:41:03.400Z - aggregation completed in job QxoUycv-TAaXLVAn54gn1Q - rev b4cd29abb74ae321be6176130fcd2130c7179d6d - 2020-12-10T11:48:40.246Z - indexing completed in job V7vA_AsxQiGPH1_2aGgB9g - 2020-12-09T11:10:07.410Z - aggregation completed in job L6wue_-7TPug_39vNRaEAA - rev b4cd29abb74ae321be6176130fcd2130c7179d6d - 2020-12-10T11:48:40.246Z - indexing completed in job V7vA_AsxQiGPH1_2aGgB9g - 2020-12-10T06:33:31.604Z - aggregation completed in job cCIzobjoTuu6BI3iNIBYdw
I noticed we had coverage yesterday in searchfox. It appears this happened because the nightly was built off of a Mozilla-central commit that was significantly older than usual (Tuesday at 10:49pm ET for Wednesday rather than the normal 4:30am+ ET). Because the manual process of comment 5 was laborious and maybe wrong and I need to figure out how to implement the logic in comment 4, I created a terrifying bash script at https://gist.github.com/asutherland/0f820de77856e459d7e1d4196be6f7ae that provides us with the helpful output at the bottom of the comment with the headings. There's 3 passes in the script as displayed below: 1. "Searchfox rev artifact creator times": - Find the recent mozilla-central searchfox treeherder linux64 jobs and use that to figure out the revisions that searchfox indexed. - Lookup the task that corresponds to the coverage artifact index root for the given tree and revision. - If the was a task, get its info, then output it. 2. "Hook Recent Runs for code-coverage-cron-production" - This just shows the completion and start times from jobs for this hook. These tasks don't have any revision information in them. They also seem to only produce precursor artifacts and not the actual aggregation we would assume if I run `taskcluster task def en9AhwDYQGSSWZ32PF30Tw` for example on the 12-04 cron task's id. 3. "Hook Results for code-coverage-repo-production": - Find the recent hook runs - Filter the runs down to those that correspond to mozilla-central and a revision for which searchfox indexing run (as determined in the first pass) - Output the info. What the output and my investigation seems to be showing is that: - Maybe the cron job is currently required to make things happen. - The cron job is actually running at midnight UTC time instead of when we want it to? :marco, thoughts? ## Output ### Searchfox rev artifact creator times - d4b21131a094a0819b8856fbf4165cfb32d671c6 artifact project.relman.code-coverage.production.repo.mozilla-central.d4b21131a094a0819b8856fbf4165cfb32d671c6 - 2020-12-03T10:44:38.906Z - indexing completed in job IZjIJ121TMO3s4OwZFMl2Q - 2020-12-04T01:40:41.803Z - artifact task completed in job VrUpFqafShWpEv5-DCqHQA started at 2020-12-04T00:03:31.464Z - b836812230d1274efa92eb0a0d38601ef4715cda artifact project.relman.code-coverage.production.repo.mozilla-central.b836812230d1274efa92eb0a0d38601ef4715cda - 2020-12-04T10:45:32.316Z - indexing completed in job WQ8-4NBtTWqaXTA74wjWRg - 2020-12-05T01:50:36.159Z - artifact task completed in job bsAQop-BTfGXOa-Mp-C9wQ started at 2020-12-05T00:05:40.519Z - 7ce95b6cde2643be60ec53d8c743db28aec30884 artifact project.relman.code-coverage.production.repo.mozilla-central.7ce95b6cde2643be60ec53d8c743db28aec30884 - 2020-12-05T10:42:01.895Z - indexing completed in job TrEpg3-2RneHzhzUVccmxw - 2020-12-06T01:40:05.383Z - artifact task completed in job OLpXEJx7RDeOuTSkrSuKOA started at 2020-12-06T00:06:45.971Z - 5a17a9893fe09fd912f1972b204d0f9ca1184b28 artifact project.relman.code-coverage.production.repo.mozilla-central.5a17a9893fe09fd912f1972b204d0f9ca1184b28 - 2020-12-06T10:45:04.382Z - indexing completed in job BTicAleYQ2uOBMbZXS_gPw - 2020-12-07T01:33:25.315Z - artifact task completed in job RBIL-Rv-RNG34DHcz-LM0g started at 2020-12-07T00:05:12.411Z - 7167ab3f6e8d7b77439ac1cc7d9356bbaff02510 artifact project.relman.code-coverage.production.repo.mozilla-central.7167ab3f6e8d7b77439ac1cc7d9356bbaff02510 - 2020-12-07T10:37:57.384Z - indexing completed in job Yy6RvYLzQ9y7QF3TFD7Clw - 2020-12-08T01:46:59.147Z - artifact task completed in job d3yZDYm2R9St4Eiwn4_P2g started at 2020-12-08T00:05:12.404Z - 1a164819adefcdb42bdba8a24976b62ee8d03c80 artifact project.relman.code-coverage.production.repo.mozilla-central.1a164819adefcdb42bdba8a24976b62ee8d03c80 - 2020-12-08T10:43:20.898Z - indexing completed in job TC6zbrHwR8S_dJ7DyBO-eg - 2020-12-09T01:41:03.400Z - artifact task completed in job QxoUycv-TAaXLVAn54gn1Q started at 2020-12-09T00:09:07.369Z - b4cd29abb74ae321be6176130fcd2130c7179d6d artifact project.relman.code-coverage.production.repo.mozilla-central.b4cd29abb74ae321be6176130fcd2130c7179d6d - 2020-12-09T10:56:35.407Z - indexing completed in job V7vA_AsxQiGPH1_2aGgB9g - 2020-12-09T11:10:07.410Z - artifact task completed in job L6wue_-7TPug_39vNRaEAA started at 2020-12-09T08:59:40.346Z - 73f050da7d204fcfb59aa2528f3d22665bc043d3 artifact project.relman.code-coverage.production.repo.mozilla-central.73f050da7d204fcfb59aa2528f3d22665bc043d3 - 2020-12-10T10:46:45.880Z - indexing completed in job erxttPsYRDSqf8YWwDr5Cg - No such artifact! ### Hook Recent Runs for code-coverage-cron-production - 2020-12-04T00:28:22.567Z completed after starting at 2020-12-04T00:00:53.062Z for task en9AhwDYQGSSWZ32PF30Tw - 2020-12-05T00:42:28.150Z completed after starting at 2020-12-05T00:02:45.325Z for task ZccIOU_jT6ylGydRTjQQIA - 2020-12-06T01:09:56.020Z completed after starting at 2020-12-06T00:01:58.761Z for task cIjBJx1bQ42IdsuBiuq8WQ - 2020-12-07T00:37:33.837Z completed after starting at 2020-12-07T00:03:03.062Z for task KRPevd9WRs2iXICruGPFLQ - 2020-12-08T00:22:34.855Z completed after starting at 2020-12-08T00:02:14.910Z for task KJ1Iwt9lTu2sJkY4gwP_DQ - 2020-12-09T00:21:27.892Z completed after starting at 2020-12-09T00:03:04.293Z for task dbWUaFy0RmKMbJId8fHRhA - 2020-12-10T00:27:41.105Z completed after starting at 2020-12-10T00:01:59.470Z for task WxOg4C3mSRa4YXhwhIKanw ### Hook Results for code-coverage-repo-production - rev d4b21131a094a0819b8856fbf4165cfb32d671c6 - 2020-12-10T11:48:40.246Z - indexing completed in job IZjIJ121TMO3s4OwZFMl2Q - 2020-12-04T01:40:41.803Z - aggregation completed in job VrUpFqafShWpEv5-DCqHQA - rev b836812230d1274efa92eb0a0d38601ef4715cda - 2020-12-10T11:48:40.246Z - indexing completed in job WQ8-4NBtTWqaXTA74wjWRg - 2020-12-05T01:50:36.159Z - aggregation completed in job bsAQop-BTfGXOa-Mp-C9wQ - rev 7ce95b6cde2643be60ec53d8c743db28aec30884 - 2020-12-10T11:48:40.246Z - indexing completed in job TrEpg3-2RneHzhzUVccmxw - 2020-12-06T01:40:05.383Z - aggregation completed in job OLpXEJx7RDeOuTSkrSuKOA - rev 5a17a9893fe09fd912f1972b204d0f9ca1184b28 - 2020-12-10T11:48:40.246Z - indexing completed in job BTicAleYQ2uOBMbZXS_gPw - 2020-12-07T01:33:25.315Z - aggregation completed in job RBIL-Rv-RNG34DHcz-LM0g - rev 7167ab3f6e8d7b77439ac1cc7d9356bbaff02510 - 2020-12-10T11:48:40.246Z - indexing completed in job Yy6RvYLzQ9y7QF3TFD7Clw - 2020-12-08T01:46:59.147Z - aggregation completed in job d3yZDYm2R9St4Eiwn4_P2g - rev 1a164819adefcdb42bdba8a24976b62ee8d03c80 - 2020-12-10T11:48:40.246Z - indexing completed in job TC6zbrHwR8S_dJ7DyBO-eg - 2020-12-09T01:41:03.400Z - aggregation completed in job QxoUycv-TAaXLVAn54gn1Q - rev b4cd29abb74ae321be6176130fcd2130c7179d6d - 2020-12-10T11:48:40.246Z - indexing completed in job V7vA_AsxQiGPH1_2aGgB9g - 2020-12-09T11:10:07.410Z - aggregation completed in job L6wue_-7TPug_39vNRaEAA - rev b4cd29abb74ae321be6176130fcd2130c7179d6d - 2020-12-10T11:48:40.246Z - indexing completed in job V7vA_AsxQiGPH1_2aGgB9g - 2020-12-10T06:33:31.604Z - aggregation completed in job cCIzobjoTuu6BI3iNIBYdw