1518085 - ignore missing perfherder summary for task if there were issues earlier during test execution

Status: RESOLVED INVALID

(Testing :: Talos, defect, P3)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

See e.g. https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=219976481&repo=mozilla-inbound&lineNumber=1059

Failure summary:

14:55:37 ERROR - Traceback (most recent call last):
1074	14:55:37 INFO - mozdevice.adb.ADBTimeoutError: args: adb -s HT85S1A02429 wait-for-device logcat -v time -d dalvikvm:I ConnectivityService:S WifiMonitor:S WifiStateTracker:S wpa_supplicant:S NetworkStateTracker:S EmulatedCamera_Camera:S EmulatedCamera_Device:S EmulatedCamera_FakeCamera:S EmulatedCamera_FakeDevice:S EmulatedCamera_CallbackNotifier:S GnssLocationProvider:S Hyphenator:S BatteryStats:S, exitcode: None, stdout:
1075	14:55:38 ERROR - Return code: 1
1078	14:55:38 CRITICAL - PERFHERDER_DATA was seen 0 times, expected 1.
1082	14:55:38 CRITICAL - Error copying results /builds/worker/workspace/build/raptor.json to upload dir /builds/worker/workspace/build/blobber_upload_dir/perfherder-data.json

Those perfherder failures at the end only occurred because of the earlier ADB issue and thus should be ignored.

Comment 1

[Ed Morley [:emorley]]

These come from the test harness here:
https://dxr.mozilla.org/mozilla-central/rev/c2593a3058afdfeaac5c990e18794ee8257afe99/testing/mozharness/mozharness/mozilla/testing/talos.py#665
https://dxr.mozilla.org/mozilla-central/rev/c2593a3058afdfeaac5c990e18794ee8257afe99/testing/mozharness/mozharness/mozilla/testing/raptor.py#464

To work around these in Treeherder would require a complete re-architecture of the log parser to handle multi-line scenarios.

It would be best to make the test harness handle this case itself, which would be simpler, and also mean the test run could finish earlier - saving money and giving the result back to the developer/sheriffs sooner.