1858709 - profiles not collected for sunspider (potentially other benchmarks?)

Status: RESOLVED WONTFIX

(Testing :: Raptor, defect, P3)

Description

[Kash Shampur [:kshampur] ⌚EST]

you can see from any sunspider task e.g. you can look at any sunspider one. e.g. https://treeherder.mozilla.org/jobs?repo=mozilla-central&tier=1%2C2%2C3&searchStr=sunspider&revision=03e0b5134ce57c0f8e1bf1302648ec13eb765d82&selectedTaskRun=Ee8F7ocpRD2eiKJgA512fg.0

[task 2023-10-12T11:56:06.099Z] 11:56:06     INFO -  raptor-profiling Info: Found 1 profiles: [{'path': '/home/cltbld/tasks/task_169711138423909/build/blobber_upload_dir/browsertime-results/sunspider/profiling/pages/127_0_0_1/SunSpider/sunspider-1.0.1/sunspider-1.0.1/driver.html/query-8e057622/data/geckoProfile-1.json.gz', 'type': 'benchmark'}]
[task 2023-10-12T11:56:06.100Z] 11:56:06     INFO -  profiler Retrieving symbol zip from https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/YWSFXAEJTyS658KxGDvyqg/artifacts/public/build/target.crashreporter-symbols.zip...
[task 2023-10-12T11:56:18.285Z] 11:56:18     INFO -  raptor-gecko-profile Info: Opening profile at /home/cltbld/tasks/task_169711138423909/build/blobber_upload_dir/browsertime-results/sunspider/profiling/pages/127_0_0_1/SunSpider/sunspider-1.0.1/sunspider-1.0.1/driver.html/query-8e057622/data/geckoProfile-1.json.gz
[task 2023-10-12T11:56:18.285Z] 11:56:18     INFO -  raptor-gecko-profile Info: Profile not found on extra profiler run.

and under the artifacts you'll see the profiler is empty e.g. https://profiler.firefox.com/from-url/https%3A%2F%2Ffirefox-ci-tc.services.mozilla.com%2Fapi%2Fqueue%2Fv1%2Ftask%2FEe8F7ocpRD2eiKJgA512fg%2Fruns%2F0%2Fartifacts%2Fpublic%2Ftest_info%2Fprofile_sunspider.zip/calltree/?v=10

it seems this has been happening either: recently or all the time. Has not been noticed until recently because no one has expressed any interest in sunspider profiles

worth mentioning chrome trace + sunspider produces profiles, and gecko profiler + other benchmarks seems to be fine (though I haven't checked every single one, just most)

Comment 1

[Kash Shampur [:kshampur] ⌚EST]

:canova has pointed out

[task 2023-10-12T11:56:00.324Z] 11:56:00     INFO -  raptor-browsertime Info: Collecting Gecko profile from /home/cltbld/tasks/task_169711138423909/build/blobber_upload_dir/browsertime-results/sunspider/profiling/pages/127_0_0_1/SunSpider/sunspider-1.0.1/sunspider-1.0.1/results.html/query-d4244b24/data/geckoProfile-1.json to /home/cltbld/tasks/task_169711138423909/build/blobber_upload_dir/browsertime-results/sunspider/profiling/pages/127_0_0_1/SunSpider/sunspider-1.0.1/sunspider-1.0.1/results.html/query-d4244b24/data/geckoProfile-1.json
[task 2023-10-12T11:56:06.099Z] 11:56:06     INFO -  raptor-profiling Info: Found 1 profiles: [{'path': '/home/cltbld/tasks/task_169711138423909/build/blobber_upload_dir/browsertime-results/sunspider/profiling/pages/127_0_0_1/SunSpider/sunspider-1.0.1/sunspider-1.0.1/driver.html/query-8e057622/data/geckoProfile-1.json.gz', 'type': 'benchmark'}]

these dont match, one is looking for results.html and the other driver.html

Comment 2

[Kash Shampur [:kshampur] ⌚EST]

:canova has pointed out we get the url at the end of the profiling session: https://github.com/sitespeedio/browsertime/blob/e3e5a46f6c9070c0ab66e543b85e60d021edc848/lib/core/engine/command/geckoProfiler.js#L28-L30 in sunspider we start with driver.html and navigate later to results.html at the end

so it may be better to get the url at the start rather than end

Comment 3

[Kash Shampur [:kshampur] ⌚EST]

it seems assorted dom benchmark tests are also not being collected

Comment 4

[Kash Shampur [:kshampur] ⌚EST]

(but the rest of the current benchmarks seem fine)

Comment 5

[Nazım Can Altınova [:canova][:canaltinova on phabricator]]

We spent some more time to figure this out with Kash and realized that reading the url on GeckoProfiler.start is not enough either. Since we navigate to about:blank first, it was always showing it instead of the real url.

Instead we needed to get the url from another array that contains the url field. Created a PR for the fix: https://github.com/sitespeedio/browsertime/pull/1999

Comment 6

[Kash Shampur [:kshampur] ⌚EST]

Closing this in favour of Bug 1859417