Running the existing system tests against stage 10 times in a row with no breaks yields this set of timings: ``` round 0: elapsed: 0:01:04.864894 round 1: elapsed: 0:02:35.779171 round 2: elapsed: 0:02:35.855127 round 3: elapsed: 0:02:38.264483 round 4: elapsed: 0:02:36.296327 round 5: elapsed: 0:02:36.322767 round 6: elapsed: 0:02:05.452292 round 7: elapsed: 0:02:35.818059 round 8: elapsed: 0:02:36.022828 round 9: elapsed: 0:02:37.009649 ``` The first round is 1 minute. The others all hit rate-limiting and take between 2 and 2.5 minutes to run. If I understand correctly, the systemtests (skipping large_files tests) are doing: * 2 uploads * 1 upload with a bad token * 1 upload-by-download That's 4 uploads. When the tests hit rate-limiting, they sleep for 30 seconds and retry. It's probably the case that each test run is retrying 3 times. We have the upload API dashboard. Here's the period where I ran 10 rounds of the systemtests: https://earthangel-b40313e5.influxcloud.net/d/6gimTZ6Vz/tecken-upload-api-metrics?orgId=1&var-env=stage&from=1722366144401&to=1722367919744 I'll use that for a baseline to measure changes against.
Bug 1910613 Comment 4 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
Running the existing system tests against stage 10 times in a row with no breaks yields this set of timings: ``` round 0: elapsed: 0:01:04.864894 round 1: elapsed: 0:02:35.779171 round 2: elapsed: 0:02:35.855127 round 3: elapsed: 0:02:38.264483 round 4: elapsed: 0:02:36.296327 round 5: elapsed: 0:02:36.322767 round 6: elapsed: 0:02:05.452292 round 7: elapsed: 0:02:35.818059 round 8: elapsed: 0:02:36.022828 round 9: elapsed: 0:02:37.009649 ``` The first round is 1 minute. The others all hit rate-limiting and take between 2 and 2.5 minutes to run. If I understand correctly, the systemtests (skipping large_files tests) are doing: * 2 uploads * 1 upload with a bad token * 1 upload-by-download That's 4 uploads. When the tests hit rate-limiting, they sleep for 30 seconds and retry. It's probably the case that each test run is retrying 3 times. We have the upload API dashboard. Here's the period where I ran 10 rounds of the systemtests: https://earthangel-b40313e5.influxcloud.net/d/6gimTZ6Vz/tecken-upload-api-metrics?orgId=1&var-env=stage&from=1722366144401&to=1722367919744 I'll use that for a baseline to measure changes against. I can use the new systemtest bits to put together a rough load test.