Closed Bug 1832974 Opened 1 year ago Closed 1 year ago

"That's a lot of tests selected" should give actual number of tests selected, so that I can know by how much to reduce --rebuild

Categories

(Testing :: Performance, enhancement, P2)

Default
enhancement

Tracking

(firefox117 fixed)

RESOLVED FIXED
117 Branch
Tracking Status
firefox117 --- fixed

People

(Reporter: mstange, Assigned: aglavic)

References

(Blocks 1 open bug)

Details

(Whiteboard: [fxp])

Attachments

(1 file)

I wanted to run ./mach try perf --rebuild 20 with "Pageload (essential) desktop firefox" and "Speedometer 3 desktop firefox" but I got the following message:

That's a lot of tests selected (300)!
These tests won't be triggered. If this was unexpected, please file a bug in Testing :: Performance.

I would like to know by how much I need to reduce the rebuild count in order to go below the limit.

However, this message always prints 300, rather than the actual number of tests selected:

https://searchfox.org/mozilla-central/rev/0accfbfb0381e431651b40065c0e08c249100fba/tools/tryselect/selectors/perf.py#1130-1136

if (len(selected_tasks) * rebuild) > MAX_PERF_TASKS:
    print(
        "That's a lot of tests selected (%s)!\n"
        "These tests won't be triggered. If this was unexpected, "
        "please file a bug in Testing :: Performance." % MAX_PERF_TASKS
    )
    return None

Can we make it print the number of tests selected and the max? And maybe also the number of rebuilds that would make it go below the limit?

Summary: "That's a lot of tests selected" should give actual number of tests selected, so that I can now by how much to reduce --rebuild → "That's a lot of tests selected" should give actual number of tests selected, so that I can know by how much to reduce --rebuild
Severity: -- → S3
Priority: -- → P2
Whiteboard: [fxp]
Assignee: nobody → aglavic
Status: NEW → ASSIGNED
Attachment #9344263 - Attachment description: 'Bug 1832974 - Changing unclear output message when too many tests are selected. r?#perftest → Bug 1832974 - Changing unclear output message when too many tests are selected. r?#perftest
Pushed by aglavic@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/e19f10f18cc2
Changing unclear output message when too many tests are selected. r=perftest-reviewers,afinder
Status: ASSIGNED → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
Target Milestone: --- → 117 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: