(In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #16) > :bc, could you figure out the cost of fuzzy builds on autoland via a raw query? I had a bug in my costs script which ignored run_on_projects ["all"]. I've fixed that and here are the results for asan builds (Bof, Bf) on autoland all tiers for the firefox 72 release cycle. ``` | provisionerId | workerType | project | tier | suite | groupSymbol | symbol | collection | cost | label | |---------------|------------|----------|------|-------|-------------|--------|------------|----------|--------------------------------------------------------------------------------------------------------| | gecko-3 | b-linux | autoland | 1 | | nil | Bf | debug | 1,024.16 | build-android-x86-fuzzing/debug, build-linux64-fuzzing/debug, build-macosx64-fuzzing/debug | | gecko-3 | b-linux | autoland | 1 | | nil | Bof | asan | 963.31 | build-android-x86_64-asan-fuzzing/opt, build-linux64-asan-fuzzing/opt, build-macosx64-asan-fuzzing/opt | | gecko-3 | b-linux | autoland | 1 | | nil | Bof | tsan | 247.52 | build-linux64-tsan-fuzzing/opt | | gecko-3 | b-win2012 | autoland | 1 | | nil | Bof | asan | 678.72 | build-win64-asan-fuzzing/opt | | gecko-3 | b-win2012 | autoland | 1 | | nil | Bf | debug | 627.79 | build-win64-fuzzing/debug | | | | | | | | | | | | | | | | | | | | | 3541.50 | | ``` Note this cost is for the full Dec 2 - Jan 6 release cycle. It may be that the added aggravation and developer and sheriff time to deal with a special handling of asan builds might be more costly that just doing the builds. PS. I'm going to redo my earlier cost analysis with the fixed script and review my previous results.
Bug 1608427 Comment 22 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
Updated with moar data. (In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #16) > :bc, could you figure out the cost of fuzzy builds on autoland via a raw query? I had a bug in my costs script which ignored run_on_projects ["all"]. I've fixed that and here are the results for asan builds (Bof, Bf) on autoland all tiers for the firefox 72 release cycle. ``` | provisionerId | workerType | project | tier | suite | groupSymbol | symbol | collection | cost | label | |---------------|------------|----------|------|-------|-------------|--------|------------|----------|--------------------------------------------------------------------------------------------------------| | gecko-3 | b-linux | autoland | 1 | | nil | Bf | debug | 1,024.16 | build-android-x86-fuzzing/debug, build-linux64-fuzzing/debug, build-macosx64-fuzzing/debug | | gecko-3 | b-linux | autoland | 1 | | nil | Bof | asan | 963.31 | build-android-x86_64-asan-fuzzing/opt, build-linux64-asan-fuzzing/opt, build-macosx64-asan-fuzzing/opt | | gecko-3 | b-linux | autoland | 1 | | nil | Bd | asan | 323.64 | build-linux64-asan/debug | | gecko-3 | b-linux | autoland | 1 | | nil | Bo | asan | 276.36 | build-linux64-asan/opt | | gecko-3 | b-linux | autoland | 1 | | nil | Bof | tsan | 247.52 | build-linux64-tsan-fuzzing/opt | | gecko-3 | b-linux | autoland | 1 | | nil | Bo | tsan | 241.55 | build-linux64-tsan/opt | | gecko-3 | b-linux | autoland | 2 | | WMC32 | Bd | all | 321.19 | build-win32-mingwclang/debug | | gecko-3 | b-linux | autoland | 2 | | WMC32 | Bo | all | 447.31 | build-win32-mingwclang/opt | | gecko-3 | b-win2012 | autoland | 1 | | nil | Bof | asan | 678.72 | build-win64-asan-fuzzing/opt | | gecko-3 | b-win2012 | autoland | 1 | | nil | Bd | asan | 737.82 | build-win64-asan/debug | | gecko-3 | b-win2012 | autoland | 1 | | nil | Bo | asan | 682.70 | build-win64-asan/opt | | gecko-3 | b-win2012 | autoland | 1 | | nil | Bf | debug | 627.79 | build-win64-fuzzing/debug | | gecko-3 | b-linux | autoland | 2 | | WMC64 | Bd | all | 319.84 | build-win64-mingwclang/debug | | gecko-3 | b-linux | autoland | 2 | | WMC64 | Bo | all | 433.54 | build-win64-mingwclang/opt | | | | | | | | | | | | | | | | | | | | | 7325.44 | | ``` Note this cost is for the full Dec 2 - Jan 6 release cycle. It may be that the added aggravation and developer and sheriff time to deal with a special handling of asan builds might be more costly that just doing the builds. PS. I'm going to redo my earlier cost analysis with the fixed script and review my previous results.
Updated again with autoland and mozilla-central and with only fuzzing and not asan builds. (In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #16) > :bc, could you figure out the cost of fuzzy builds on autoland via a raw query? I had a bug in my costs script which ignored run_on_projects ["all"]. I've fixed that and here are the results for asan builds (Bof, Bf) on autoland all tiers for the firefox 72 release cycle. ``` | provisionerId | workerType | project | tier | suite | groupSymbol | symbol | collection | cost | label | |---------------|------------|-----------------|------|-------|-------------|--------|------------|---------|--------------------------------------------------------------------------------------------------------| | gecko-3 | b-linux | autoland | 1 | | nil | Bf | debug | 1024.16 | build-android-x86-fuzzing/debug, build-linux64-fuzzing/debug, build-macosx64-fuzzing/debug | | gecko-3 | b-win2012 | autoland | 1 | | nil | Bf | debug | 627.79 | build-win64-fuzzing/debug | | gecko-3 | b-linux | autoland | 1 | | nil | Bof | asan | 963.31 | build-android-x86_64-asan-fuzzing/opt, build-linux64-asan-fuzzing/opt, build-macosx64-asan-fuzzing/opt | | gecko-3 | b-win2012 | autoland | 1 | | nil | Bof | asan | 678.72 | build-win64-asan-fuzzing/opt | | gecko-3 | b-linux | autoland | 1 | | nil | Bof | tsan | 247.52 | build-linux64-tsan-fuzzing/opt | | gecko-3 | b-linux | autoland | 1 | | SM | f | opt | 66.27 | spidermonkey-sm-fuzzing-linux64/opt | | | | autoland | | | | | | 3607.77 | | | gecko-3 | b-linux | mozilla-central | 1 | | nil | Bf | debug | 44.14 | build-android-x86-fuzzing/debug, build-linux64-fuzzing/debug, build-macosx64-fuzzing/debug | | gecko-3 | b-win2012 | mozilla-central | 1 | | nil | Bf | debug | 30.23 | build-win64-fuzzing/debug | | gecko-3 | b-linux | mozilla-central | 1 | | nil | Bocf | asan | 31.08 | build-linux64-asan-fuzzing-ccov/opt | | gecko-3 | b-linux | mozilla-central | 1 | | nil | Bocf | opt | 27.16 | build-linux64-fuzzing-ccov/opt | | gecko-3 | b-linux | mozilla-central | 1 | | nil | Bof | asan | 42.96 | build-android-x86_64-asan-fuzzing/opt, build-linux64-asan-fuzzing/opt, build-macosx64-asan-fuzzing/opt | | gecko-3 | b-win2012 | mozilla-central | 1 | | nil | Bof | asan | 31.50 | build-win64-asan-fuzzing/opt | | gecko-3 | b-linux | mozilla-central | 1 | | nil | Bof | tsan | 9.63 | build-linux64-tsan-fuzzing/opt | | gecko-3 | b-linux | mozilla-central | 1 | | SM | f | opt | 10.53 | spidermonkey-sm-fuzzing-linux64/opt | | | | mozilla-central | | | | | | 227.23 | | | | | | | | | | | 3835.00 | | ``` Note this cost is for the full Dec 2 - Jan 6 release cycle. PS. I'm going to redo my earlier cost analysis with the fixed script and review my previous results.