I'm working on a task to discontinue the storage of job artifacts in the Treeherder database. I'd like to switch to retrieving artifacts in the UI directly from a taskcluster API but we're currently only able to retrieve "artifact uploaded" artifacts. A blocker to fully implementing this is `TinderboxPrint`. Treeherder currently parses[2] raw text logs for `TinderboxPrint` lines and stores them as artifacts for use in treeherder's job details panel. ``` [vcs 2020-03-25T18:26:51.244Z] TinderboxPrint:<a href=https://hg.mozilla.org/mozilla-central/rev/c5112a7573ac8a9d388e6253e3305061654b123e title='Built from mozilla-central revision c5112a7573ac8a9d388e6253e3305061654b123e'>c5112a7573ac8a9d388e6253e3305061654b123e</a> ``` ``` [task 2020-03-25T08:27:34.766Z] 08:27:34 INFO - TinderboxPrint: xpcshell-xpcshell<br/>852/<em class="testfail">1</em>/4 [task 2020-03-25T08:27:34.850Z] 08:27:34 INFO - TinderboxPrint: CPU usage<br/>42.0% [task 2020-03-25T08:27:34.851Z] 08:27:34 INFO - TinderboxPrint: I/O read bytes / time<br/>133,754,880 / 3,204 [task 2020-03-25T08:27:34.851Z] 08:27:34 INFO - TinderboxPrint: I/O write bytes / time<br/>3,524,530,176 / 514,556 [task 2020-03-25T08:27:34.852Z] 08:27:34 INFO - TinderboxPrint: CPU idle<br/>921.4 (57.6%) [task 2020-03-25T08:27:34.852Z] 08:27:34 INFO - TinderboxPrint: CPU system<br/>132.4 (8.3%) [task 2020-03-25T08:27:34.853Z] 08:27:34 INFO - TinderboxPrint: CPU user<br/>539.1 (33.7%) [task 2020-03-25T08:27:34.853Z] 08:27:34 INFO - TinderboxPrint: Swap in / out<br/>0 / 0 ``` We format them in our API like so: ``` { "title": "Swap in / out", "value": "0 / 0", "url": null }, ``` Tom.Prince suggested a structured artifact could be created that supplies that information (ideally for both the "Built by" link and the log lines). [1] https://searchfox.org/mozilla-central/rev/4d9cd186767978a99dafe77eb536a9525980e118/taskcluster/scripts/run-task#490 [2] https://github.com/mozilla/treeherder/blob/master/treeherder/log_parser/parsers.py#L258 Hopefully some of the existing code in the two links provided could be reused.
Bug 1625033 Comment 0 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
I'm working on a task to discontinue the storage of job artifacts in the Treeherder database. I'd like to switch to retrieving artifacts in the UI directly from a taskcluster API but we're currently only able to retrieve "artifact uploaded" artifacts. A blocker to fully implementing this is `TinderboxPrint`. Treeherder currently parses[2] raw text logs for `TinderboxPrint` lines and stores them as artifacts for use in treeherder's job details panel. ``` [vcs 2020-03-25T18:26:51.244Z] TinderboxPrint:<a href=https://hg.mozilla.org/mozilla-central/rev/c5112a7573ac8a9d388e6253e3305061654b123e title='Built from mozilla-central revision c5112a7573ac8a9d388e6253e3305061654b123e'>c5112a7573ac8a9d388e6253e3305061654b123e</a> ``` ``` [task 2020-03-25T08:27:34.766Z] 08:27:34 INFO - TinderboxPrint: xpcshell-xpcshell<br/>852/<em class="testfail">1</em>/4 [task 2020-03-25T08:27:34.850Z] 08:27:34 INFO - TinderboxPrint: CPU usage<br/>42.0% [task 2020-03-25T08:27:34.851Z] 08:27:34 INFO - TinderboxPrint: I/O read bytes / time<br/>133,754,880 / 3,204 [task 2020-03-25T08:27:34.851Z] 08:27:34 INFO - TinderboxPrint: I/O write bytes / time<br/>3,524,530,176 / 514,556 [task 2020-03-25T08:27:34.852Z] 08:27:34 INFO - TinderboxPrint: CPU idle<br/>921.4 (57.6%) [task 2020-03-25T08:27:34.852Z] 08:27:34 INFO - TinderboxPrint: CPU system<br/>132.4 (8.3%) [task 2020-03-25T08:27:34.853Z] 08:27:34 INFO - TinderboxPrint: CPU user<br/>539.1 (33.7%) [task 2020-03-25T08:27:34.853Z] 08:27:34 INFO - TinderboxPrint: Swap in / out<br/>0 / 0 ``` We format them in our API like so: ``` { "title": "Built from mozilla-central revision c5112a7573ac8a9d388e6253e3305061654b123e", "value": "c5112a7573ac8a9d388e6253e3305061654b123e", "url": "https://hg.mozilla.org/mozilla-central/rev/c5112a7573ac8a9d388e6253e3305061654b123e " } ``` ``` { "title": "Swap in / out", "value": "0 / 0", "url": null }, ``` Tom.Prince suggested a structured artifact could be created that supplies that information (ideally for both the "Built by" link and the log lines). [1] https://searchfox.org/mozilla-central/rev/4d9cd186767978a99dafe77eb536a9525980e118/taskcluster/scripts/run-task#490 [2] https://github.com/mozilla/treeherder/blob/master/treeherder/log_parser/parsers.py#L258 Hopefully some of the existing code in the two links provided could be reused.