Open Bug 1712308 Opened 4 years ago Updated 3 years ago

[meta] Review performance benchmarks in automation

Categories

(Testing :: Performance, task)

Product:
Testing
Component:
Performance
Version:
Default
Type:
task

Tracking

(Not tracked)

People

(Reporter: Bebe, Unassigned)

References

(Blocks 1 open bug)

Details

(Keywords: meta)

Browsertime is using some custom scripts(benchmarks) to measure various performance data points.

These scripts are years old and need to be upodated.
Develop a process and update/review these benchmarks.

The first step should be to discuss with the performance team and draft a process for reviewing the benchmarks.
Things that come to mind:

Is this benchmark still relevant?
Which platforms are appropriate for the benchmark?
Which metrics/suites within the benchmark should we run?
Should we sheriff the results for the benchmark?
Which team is responsible for the benchmark?
This could be something we review annually

Depends on: 1689537
Summary: Update browsertime benchmarks → [meta]Update browsertime benchmarks
Summary: [meta]Update browsertime benchmarks → [meta]Update benchmarks tests
Summary: [meta]Update benchmarks tests → [meta]Update/review benchmarks tests
Blocks: 1689537, 1631558
No longer depends on: 1689537
Keywords: meta
Summary: [meta]Update/review benchmarks tests → [meta] Review performance benchmarks in automation

We should also consider adding benchmarks as part of the review.

The meta keyword is there, the bug doesn't depend on other bugs and there is no activity for 12 months.
:davehunt, maybe it's time to close this bug?

Flags: needinfo?(dave.hunt)
No longer blocks: 1689537
Depends on: 1689537
Flags: needinfo?(dave.hunt)
You need to log in before you can comment on or make changes to this bug.