1993275 - Document a process for verifying new hardware or hardware updates

Status: NEW

(Testing :: Performance, task, P3)

Description

[Greg Mierzwinski [:sparky]]

We currently don't have a process that is documented for verifying new hardware, new OSes, or hardware updates. This bug is for developing one.

Comment 1

[Kash Shampur [:kshampur] ⌚EST]

Just going to (loosely) compile the things we have done (incomplete list, and in no specific order)

• allow for a 2 weeks transition period between old version of OS/hardware to new one.
  • In some cases, keep old hardware/OS for even longer (e.g. like in case of windows 10, even a51's)
  • this will require the timing of a couple of patches with taskgraph transforms and config file changes to target machines/OSes
• do a perfcompare to examine old vs new baseline values (large before/after deltas are likely, but stable/low variance values is important)
• coordinate with other teams that may use the same hardware (e.g. necko, ML, audio, graphics, relops)
• general idea for tests:
  • green up as many as we can
  • shortlist the problematic ones and file bugs to independently investigate, potentially even permanently disabling some