1471803 - Intermittent talos T-e10s(s) tart [taskcluster:error] Task aborted - max run time exceeded

Status: RESOLVED DUPLICATE of bug 1439979

(Testing :: Talos, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: apavel [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=185159322&repo=try

https://queue.taskcluster.net/v1/task/SW8kJyCmQvGCC8xPgm_ryQ/runs/0/artifacts/public/logs/live_backing.log

07:27:35     INFO -  Adding profile profile_tsvgr_opacity/big-optimizable-group-opacity-2500.svg_pagecycle_1/cycle_0.profile to archive /Users/cltbld/tasks/task_1530109464/build/blobber_upload_dir/profile_tsvgr_opacity.zip
07:27:37     INFO -  Adding profile profile_tsvgr_opacity/big-optimizable-group-opacity-2500.svg_pagecycle_2/cycle_0.profile to archive /Users/cltbld/tasks/task_1530109464/build/blobber_upload_dir/profile_tsvgr_opacity.zip
07:27:40     INFO -  Adding profile profile_tsvgr_opacity/small-group-opacity-2500.svg_pagecycle_1/cycle_0.profile to archive /Users/cltbld/tasks/task_1530109464/build/blobber_upload_dir/profile_tsvgr_opacity.zip
07:27:42     INFO -  Adding profile profile_tsvgr_opacity/small-group-opacity-2500.svg_pagecycle_2/cycle_0.profile to archive /Users/cltbld/tasks/task_1530109464/build/blobber_upload_dir/profile_tsvgr_opacity.zip
07:27:42     INFO -  TEST-OK | tsvgr_opacity | took 21205ms
07:27:42     INFO -  TEST-START | tart
07:27:42     INFO -  Gecko profiling is enabled so talos is reducing the number of cycles, please disregard reported numbers
07:27:42     INFO -  Initialising browser for tart test...
07:27:42     INFO -  Cloning profile located at /Users/cltbld/tasks/task_1530109464/build/tests/talos/talos/base_profile
07:27:42     INFO -  Merging profile: /Users/cltbld/tasks/task_1530109464/build/tests/talos/talos/profile_data/common
07:27:42     INFO -  Merging profile: /Users/cltbld/tasks/task_1530109464/build/tests/talos/talos/profile_data/perf
07:27:42     INFO -  Installing Add-ons:
07:27:42     INFO -  ['/Users/cltbld/tasks/task_1530109464/build/tests/talos/talos/talos-powers', '/Users/cltbld/tasks/task_1530109464/build/tests/talos/talos/pageloader', '/Users/cltbld/tasks/task_1530109464/build/tests/talos/talos/tests/tart/addon']
07:27:42     INFO -  Application command: /Users/cltbld/tasks/task_1530109464/build/application/Firefox.app/Contents/MacOS/firefox  http://localhost:50394/getInfo.html -foreground -profile /var/folders/3f/y5bycmyx5bqg0dqthr5vd_1r00000w/T/tmpFDqI2E/profile
07:27:42     INFO -  TEST-INFO | started process 2458 (/Users/cltbld/tasks/task_1530109464/build/application/Firefox.app/Contents/MacOS/firefox  http://localhost:50394/getInfo.html -foreground)
07:27:44     INFO -  2018-06-27 07:27:44.855 plugin-container[2460:10516] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x8f2b, name = 'com.apple.tsm.portname'
07:27:44     INFO -  See /usr/include/servers/bootstrap_defs.h for the error codes.
07:27:45     INFO -  2018-06-27 07:27:45.369 plugin-container[2461:10531] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x913b, name = 'com.apple.tsm.portname'
07:27:45     INFO -  See /usr/include/servers/bootstrap_defs.h for the error codes.
07:27:45     INFO -  2018-06-27 07:27:45.620 plugin-container[2462:10565] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x8937, name = 'com.apple.tsm.portname'
07:27:45     INFO -  See /usr/include/servers/bootstrap_defs.h for the error codes.
07:27:46     INFO -  __metrics	Screen width/height:1600/1200
07:27:46     INFO -  	colorDepth:24
07:27:46     INFO -  	Browser inner width/height: 1280/935
07:27:46     INFO -  __metrics
07:27:46     INFO -  WebGL(0x1300ed000)::ForceLoseContext
07:27:46     INFO -  WebGL(0x1300f6000)::ForceLoseContext
07:27:46     INFO -  2018-06-27 07:27:46.182 plugin-container[2463:10614] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x8453, name = 'com.apple.tsm.portname'
07:27:46     INFO -  See /usr/include/servers/bootstrap_defs.h for the error codes.
07:27:47     INFO -  about:support	{"application":{"name":"Firefox","osVersion":"Darwin 14.5.0","version":"63.0b0","buildID":"20180627124150","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:63.0) Gecko/20100101 Firefox/63.0","safeMode":false,"updateChannel":"beta","supportURL":"https://support.mozilla.org/1/firefox/63.0/Darwin/en-US/","numTotalWindows":2,"numRemoteWindows":2,"remoteAutoStart":true,"currentContentProcesses":5,"maxContentProcesses":4,"autoStartStatus":1,"policiesStatus":0,"keyGoogleFound":true,"keyMozillaFound":true},"securitySoftware":{"registeredAntiVirus":"","registeredAntiSpyware":"","registeredFirewall":""},"modifiedPreferences":{"browser.cache.disk.filesystem_reported":1,"browser.cache.disk.smart_size.first_run":false,"browser.cache.disk.smart_size.enabled":false,"browser.link.open_newwindow":2,"browser.places.smartBookmarksVersion":0,"browser.sessionstore.resume_from_crash":false,"browser.startup.homepage_override.mstone":"63.0","browser.startup.homepage_override.buildID":"20180627124150","browser.tabs.warnOnClose":false,"browser.tabs.remote.autostart":true,"browser.urlbar.placeholderName":"Google","browser.urlbar.userMadeSearchSuggestionsChoice":true,"dom.send_after_paint_to_content":false,"dom.disable_window_move_resize":true,"dom.disable_open_during_load":false,"dom.allow_scripts_to_close_windows":true,"dom.max_chrome_script_run_time":0,"dom.max_script_run_time":0,"dom.disable_window_flip":true,"dom.forms.autocomplete.formautofill":true,"extensions.checkCompatibility":false,"extensions.lastAppVersion":"63.0","general.useragent.updates.enabled":false,"media.gmp-manager.updateEnabled":false,"media.gmp-manager.url":"http://127.0.0.1/gmpmanager-dummy/update.xml","media.peerconnection.enabled":true,"media.gmp.storage.version.observed":1,"media.capturestream_hints.enabled":true,"media.navigator.permission.disabled":true,"media.libavcodec.allow-obsolete":true,"media.navigator.enabled":true,"network.http.speculative-parallel-limit":0,"network.cookie.prefsMigrated":true,"places.history.expiration.transient_current_max_pages":112348,"places.database.lastMaintenance":2147483647,"plugin.state.flash":0,"plugins.flashBlock.enabled":false,"privacy.trackingprotection.pbmode.enabled":false,"privacy.trackingprotection.introURL":"http://127.0.0.1/trackingprotection/tour","privacy.sanitize.pending":"[{\"id\":\"newtab-container\",\"itemsToClear\":[],\"options\":{}}]","privacy.trackingprotection.enabled":false,"privacy.reduceTimerPrecision":false,"privacy.trackingprotection.annotate_channels":false,"security.notification_enable_delay":0,"security.turn_off_all_security_so_that_viruses_can_take_over_this_computer":true,"security.sandbox.content.tempDirSuffix":"36cc16de-a67f-b447-8845-5e89800e3689","security.enable_java":false,"security.fileuri.strict_origin_policy":false,"services.sync.engine.addresses.available":true},"lockedPreferences":{},"media":{"currentAudioBackend":"audiounit","currentMaxAudioChannels":2,"currentPreferredSampleRate":44100,"audioOutputDevices":[{"name":"Internal Speakers","groupId":"AppleHDAEngineOutput:1B,0,1,2:0","vendor":"Apple Inc.","type":2,"state":2,"preferred":15,"supportedFormat":12336,"defaultFormat":4096,"maxChannels":2,"defaultRate":44100,"maxRate":96000,"minRate":44100,"maxLatency":4736,"minLatency":654}],"audioInputDevices":[]},"javaScript":{"incrementalGCEnabled":true},"accessibility":{"isActive":false,"forceDisabled":0,"handlerUsed":false,"instantiator":""},"libraryVersions":{"NSPR":{"minVersion":"4.19","version":"4.19"},"NSS":{"minVersion":"3.39 Beta","version":"3.39 Beta"},"NSSUTIL":{"minVersion":"3.39 Beta","version":"3.39 Beta"},"NSSSSL":{"minVersion":"3.39 Beta","version":"3.39 Beta"},"NSSSMIME":{"minVersion":"3.39 Beta","version":"3.39 Beta"}},"userJS":{"exists":true},"intl":{"localeService":{"requested":["en-US"],"available":["en-US"],"supported":["en-US"],"regionalPrefs":["en"],"defaultLocale":"en-US"},"osPrefs":{"systemLocales":["en"],"regionalPrefsLocales":["en"]}},"crashes":{"submitted":[],"pending":0},"sandbox":{"contentSandboxLevel":3,"effectiveContentSandboxLevel":3},"graphics":{"numTotalWindows":3,"numAcceleratedWindows":3,"windowLayerManagerType":"OpenGL","windowLayerManagerRemote":true,"windowUsingAdvancedLayers":false,"adapterDescription":"","adapterVendorID":"0x8086","adapterDeviceID":"0x0a2e","adapterRAM":"","adapterDrivers":"","driverVersion":"","driverDate":"","usesTiling":true,"contentUsesTiling":true,"offMainThreadPaintEnabled":true,"offMainThreadPaintWorkerCount":1,"webgl1Renderer":"Intel Inc. -- Intel Iris OpenGL Engine","webgl1Version":"4.1 INTEL-10.6.33","webgl1DriverExtensions":"GL_ARB_blend_func_extended GL_ARB_draw_buffers_blend GL_ARB_draw_indirect GL_ARB_ES2_compatibility GL_ARB_explicit_attrib_location GL_ARB_gpu_shader_fp64 GL_ARB_gpu_shader5 GL_ARB_instanced_arrays GL_ARB_internalformat_query GL_ARB_occlusion_query2 GL_ARB_sample_shading GL_ARB_sampler_objects GL_ARB_separate_shader_objects GL_ARB_shader_bit_encoding GL_ARB_shader_subroutine GL_ARB_shading_language_include GL_ARB_tessellation_shader GL_ARB_texture_buffer_object_rgb32 GL_ARB_texture_cube_map_array GL_ARB_texture_gather GL_ARB_texture_query_lod GL_ARB_texture_rgb10_a2ui GL_ARB_texture_storage GL_ARB_texture_swizzle GL_ARB_timer_query GL_ARB_transform_feedback2 GL_ARB_transform_feedback3 GL_ARB_vertex_attrib_64bit GL_ARB_vertex_type_2_10_10_10_rev GL_ARB_viewport_array GL_EXT_debug_label GL_EXT_debug_marker GL_EXT_framebuffer_multisample_blit_scaled GL_EXT_texture_compression_s3tc GL_EXT_texture_filter_anisotropic GL_EXT_texture_sRGB_decode GL_APPLE_client_storage GL_APPLE_container_object_shareable GL_APPLE_flush_render GL_APPLE_object_purgeable GL_APPLE_rgb_422 GL_APPLE_row_bytes GL_APPLE_texture_range GL_ATI_texture_mirror_once GL_NV_texture_barrier","webgl1Extensions":"ANGLE_instanced_arrays EXT_blend_minmax EXT_color_buffer_half_float EXT_frag_depth EXT_sRGB EXT_shader_texture_lod EXT_texture_filter_anisotropic EXT_disjoint_timer_query OES_element_index_uint OES_standard_derivatives OES_texture_float OES_texture_float_linear OES_texture_half_float OES_texture_half_float_linear OES_vertex_array_object WEBGL_color_buffer_float WEBGL_compressed_texture_s3tc WEBGL_compressed_texture_s3tc_srgb WEBGL_debug_renderer_info WEBGL_debug_shaders WEBGL_depth_texture WEBGL_draw_buffers WEBGL_lose_context","webgl1WSIInfo":"CGL","webgl2Renderer":"Intel Inc. -- Intel Iris OpenGL Engine","webgl2Version":"4.1 INTEL-10.6.33","webgl2DriverExtensions":"GL_ARB_blend_func_extended GL_ARB_draw_buffers_blend GL_ARB_draw_indirect GL_ARB_ES2_compatibility GL_ARB_explicit_attrib_location GL_ARB_gpu_shader_fp64 GL_ARB_gpu_shader5 GL_ARB_instanced_arrays GL_ARB_internalformat_query GL_ARB_occlusion_query2 GL_ARB_sample_shading GL_ARB_sampler_objects GL_ARB_separate_shader_objects GL_ARB_shader_bit_encoding GL_ARB_shader_subroutine GL_ARB_shading_language_include GL_ARB_tessellation_shader GL_ARB_texture_buffer_object_rgb32 GL_ARB_texture_cube_map_array GL_ARB_texture_gather GL_ARB_texture_query_lod GL_ARB_texture_rgb10_a2ui GL_ARB_texture_storage GL_ARB_texture_swizzle GL_ARB_timer_query GL_ARB_transform_feedback2 GL_ARB_transform_feedback3 GL_ARB_vertex_attrib_64bit GL_ARB_vertex_type_2_10_10_10_rev GL_ARB_viewport_array GL_EXT_debug_label GL_EXT_debug_marker GL_EXT_framebuffer_multisample_blit_scaled GL_EXT_texture_compression_s3tc GL_EXT_texture_filter_anisotropic GL_EXT_texture_sRGB_decode GL_APPLE_client_storage GL_APPLE_container_object_shareable GL_APPLE_flush_render GL_APPLE_object_purgeable GL_APPLE_rgb_422 GL_APPLE_row_bytes GL_APPLE_texture_range GL_ATI_texture_mirror_once GL_NV_texture_barrier","webgl2Extensions":"EXT_color_buffer_float EXT_texture_filter_anisotropic EXT_disjoint_timer_query OES_texture_float_linear WEBGL_compressed_texture_s3tc WEBGL_compressed_texture_s3tc_srgb WEBGL_debug_renderer_info WEBGL_debug_shaders WEBGL_lose_context","webgl2WSIInfo":"CGL","info":{"AzureCanvasBackend":"skia","AzureFallbackCanvasBackend":"none","AzureContentBackend":"skia","AzureCanvasAccelerated":0,"ApzWheelInput":1,"ApzDragInput":1,"ApzKeyboardInput":1,"ApzAutoscrollInput":1,"TileHeight":512,"TileWidth":512},"featureLog":{"features":[{"name":"HW_COMPOSITING","description":"Compositing","status":"available","log":[{"type":"default","status":"available"}]},{"name":"OPENGL_COMPOSITING","description":"OpenGL Compositing","status":"available","log":[{"type":"default","status":"available"}]},{"name":"WEBRENDER","description":"WebRender","status":"opt-in","log":[{"type":"default","status":"opt-in","message":"WebRender is an opt-in feature"}]},{"name":"OMTP","description":"Off Main Thread Painting","status":"available","log":[{"type":"default","status":"available"}]}],"fallbacks":[]},"crashGuards":[]},"extensions":[{"name":"PageLoader extension","version":"1.0.32","isActive":true,"id":"[e-mail]"},{"name":"Talos Powers","version":"1.0.14","isActive":true,"id":"[e-mail]"},{"name":"TART - Tab Animation regression Test","version":"1.6.2","isActive":true,"id":"[e-mail]">[e-mail]"}],"features":[{"name":"Activity Stream","version":"2018.06.22.1337-8d599e17","id":"[e-mail]"},{"name":"Application Update Service Helper","version":"2.0","id":"[e-mail]"},{"name":"Firefox Screenshots","version":"33.0.0","id":"[e-mail]"},{"name":"Follow-on Search Telemetry","version":"0.9.7","id":"[e-mail]"},{"name":"Form Autofill","version":"1.0","id":"[e-mail]"},{"name":"Photon onboarding","version":"1.0","id":"[e-mail]"},{"name":"Pocket","version":"1.0.5","id":"[e-mail]"},{"name":"Web Compat","version":"2.0","id":"[e-mail]"},{"name":"WebCompat Reporter","version":"1.0.0","id":"[e-mail]"}]}2018-06-27 07:27:47.031 plugin-container[2464:10678] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x9f37, name = 'com.apple.tsm.portname'
07:27:47     INFO -  See /usr/include/servers/bootstrap_defs.h for the error codes.
07:27:48     INFO -  TEST-INFO | 2458: exit 0
07:27:48     INFO -  Clearing archive /Users/cltbld/tasks/task_1530109464/build/blobber_upload_dir/profile_tart.zip
07:27:48     INFO -  Activating Gecko Profiling. Temp. profile dir: /var/folders/3f/y5bycmyx5bqg0dqthr5vd_1r00000w/T/tmpZV2M1Z, interval: 10, entries: 1000000
07:27:48     INFO -  Browser initialized.
07:27:49     INFO -  Running cycle 1/1 for tart test...

07:28:00     INFO -  PID 2467 | newtab-open-preload-no.all.TART: 25.4
07:28:00     INFO -  PID 2467 | newtab-open-preload-no.error.TART: 36.0
07:28:00     INFO -  PID 2467 | newtab-open-preload-yes.raw.TART: 1035.6 3.1 10.6 3.2 8.0 4.7 7.5 6.5 5.9 5.9 5.7 5.4 6.2 5.5 6.0 5.6 5.5 5.9 6.0
07:28:00     INFO -  PID 2467 | newtab-open-preload-yes.half.TART: 5.8
07:28:00     INFO -  PID 2467 | newtab-open-preload-yes.all.TART: 60.1
07:28:00     INFO -  PID 2467 | newtab-open-preload-yes.error.TART: 45.0
07:28:05     INFO -  PID 2467 | 2018-06-27 07:28:05.017 firefox[2467:10808] Persistent UI failed to open file file:///Users/cltbld/Library/Saved%20Application%20State/org.mozilla.firefox.savedState/window_1.data: No such file or directory (2)
[taskcluster:error] Aborting task...
[taskcluster 2018-06-27T14:54:26.192Z] === Task Finished ===
[taskcluster 2018-06-27T14:54:26.192Z] Task Duration: 29m58.919690759s
[taskcluster 2018-06-27T14:54:26.833Z] Uploading artifact public/logs/localconfig.json from file logs/localconfig.json with content encoding "gzip", mime type "application/json" and expiry 2018-07-11T12:44:10.341Z
[taskcluster 2018-06-27T14:54:27.323Z] Uploading artifact public/logs/talos_critical.log from file logs/talos_critical.log with content encoding "gzip", mime type "text/plain" and expiry 2018-07-11T12:44:10.341Z
[taskcluster 2018-06-27T14:54:27.725Z] Uploading artifact public/logs/talos_error.log from file logs/talos_error.log with content encoding "gzip", mime type "text/plain" and expiry 2018-07-11T12:44:10.341Z
[taskcluster 2018-06-27T14:54:28.162Z] Uploading artifact public/logs/talos_fatal.log from file logs/talos_fatal.log with content encoding "gzip", mime type "text/plain" and expiry 2018-07-11T12:44:10.341Z
[taskcluster 2018-06-27T14:54:28.528Z] Uploading artifact public/logs/talos_info.log from file logs/talos_info.log with content encoding "gzip", mime type "text/plain" and expiry 2018-07-11T12:44:10.341Z
[taskcluster 2018-06-27T14:54:29.004Z] Uploading artifact public/logs/talos_raw.log from file logs/talos_raw.log with content encoding "gzip", mime type "text/plain" and expiry 2018-07-11T12:44:10.341Z
[taskcluster 2018-06-27T14:54:29.507Z] Uploading artifact public/logs/talos_warning.log from file logs/talos_warning.log with content encoding "gzip", mime type "text/plain" and expiry 2018-07-11T12:44:10.341Z
[taskcluster 2018-06-27T14:54:30.510Z] Uploading artifact public/test_info/missingsymbols.zip from file build/blobber_upload_dir/missingsymbols.zip with content encoding "", mime type "application/zip" and expiry 2018-07-11T12:44:10.341Z
[taskcluster 2018-06-27T14:54:31.958Z] Uploading artifact public/test_info/profile_tsvgr_opacity.zip from file build/blobber_upload_dir/profile_tsvgr_opacity.zip with content encoding "", mime type "application/zip" and expiry 2018-07-11T12:44:10.341Z
[taskcluster 2018-06-27T14:54:32.633Z] Uploading artifact public/test_info/profile_tsvgx.zip from file build/blobber_upload_dir/profile_tsvgx.zip with content encoding "", mime type "application/zip" and expiry 2018-07-11T12:44:10.341Z
[taskcluster 2018-06-27T14:54:33.441Z] Uploading artifact public/test_info/svgr-e10s_errorsummary.log from file build/blobber_upload_dir/svgr-e10s_errorsummary.log with content encoding "gzip", mime type "text/plain" and expiry 2018-07-11T12:44:10.341Z
[taskcluster 2018-06-27T14:54:33.873Z] Uploading artifact public/test_info/svgr-e10s_raw.log from file build/blobber_upload_dir/svgr-e10s_raw.log with content encoding "gzip", mime type "text/plain" and expiry 2018-07-11T12:44:10.341Z
[taskcluster:error] Task aborted - max run time exceeded

Comment 1

[Intermittent Failures Robot]

18 failures in 718 pushes (0.025 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 11
* mozilla-inbound: 3
* try: 2
* mozilla-central: 2

Platform breakdown:
* osx-10-10: 12
* macosx64-qr: 3
* macosx64-nightly: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1471803&startday=2018-06-25&endday=2018-07-01&tree=trunk

Comment 2

[Stefan Hindli [:stefan_hindli]]

Over the last 7 days there are 30 failures on this bug. These happen on windows7-32, windows2012-32, windows10-64, osx-10-10, macosx64-qr, macosx64-nightly

Here is the most recent log example: https://treeherder.mozilla.org/logviewer.html#?job_id=186195810&repo=mozilla-inbound&lineNumber=53171

[taskcluster:error] Aborting task...
[taskcluster 2018-07-03T16:02:50.097Z] === Task Finished ===
[taskcluster 2018-07-03T16:02:50.097Z] Task Duration: 1h59m58.690817991s
[taskcluster 2018-07-03T16:02:50.555Z] Uploading artifact public/logs/localconfig.json from file logs/localconfig.json with content encoding "gzip", mime type "application/json" and expiry 2019-07-03T13:35:30.369Z
[taskcluster 2018-07-03T16:02:51.486Z] Uploading artifact public/logs/log_critical.log from file logs/log_critical.log with content encoding "gzip", mime type "text/plain" and expiry 2019-07-03T13:35:30.369Z
[taskcluster 2018-07-03T16:02:52.527Z] Uploading artifact public/logs/log_error.log from file logs/log_error.log with content encoding "gzip", mime type "text/plain" and expiry 2019-07-03T13:35:30.369Z
[taskcluster 2018-07-03T16:02:52.967Z] Uploading artifact public/logs/log_fatal.log from file logs/log_fatal.log with content encoding "gzip", mime type "text/plain" and expiry 2019-07-03T13:35:30.369Z
[taskcluster 2018-07-03T16:02:53.341Z] Uploading artifact public/logs/log_info.log from file logs/log_info.log with content encoding "gzip", mime type "text/plain" and expiry 2019-07-03T13:35:30.369Z
[taskcluster 2018-07-03T16:02:55.578Z] Uploading artifact public/logs/log_raw.log from file logs/log_raw.log with content encoding "gzip", mime type "text/plain" and expiry 2019-07-03T13:35:30.369Z
[taskcluster 2018-07-03T16:02:57.463Z] Uploading artifact public/logs/log_warning.log from file logs/log_warning.log with content encoding "gzip", mime type "text/plain" and expiry 2019-07-03T13:35:30.369Z
[taskcluster 2018-07-03T16:02:57.869Z] Uploading artifact public/test_info/wpt_errorsummary.log from file build/blobber_upload_dir/wpt_errorsummary.log with content encoding "gzip", mime type "text/plain" and expiry 2019-07-03T13:35:30.369Z
[taskcluster 2018-07-03T16:02:58.265Z] Uploading artifact public/test_info/wpt_raw.log from file build/blobber_upload_dir/wpt_raw.log with content encoding "gzip", mime type "text/plain" and expiry 2019-07-03T13:35:30.369Z
[taskcluster 2018-07-03T16:03:02.443Z] Uploading artifact public/test_info/wptreport.json from file build/blobber_upload_dir/wptreport.json with content encoding "gzip", mime type "application/json" and expiry 2019-07-03T13:35:30.369Z
[taskcluster:error] Task aborted - max run time exceeded

Comment 3

[Robert Wood [:rwood]]

This is not a talos specific error. The log from comment 2 is web-platform-tests, for example. Looking at the intermittent errors, it is a mix of mochitest, reftest, wpt, talos, etc. This is probably an error message (max time exceeded) caused by all different test-specific issues that have been aggregated into this issue, which is not really valid.

Comment 4

[Andreea Pavel [:apavel]]

(In reply to Robert Wood [:rwood] from comment #3)
> This is not a talos specific error. The log from comment 2 is
> web-platform-tests, for example. Looking at the intermittent errors, it is a
> mix of mochitest, reftest, wpt, talos, etc. This is probably an error
> message (max time exceeded) caused by all different test-specific issues
> that have been aggregated into this issue, which is not really valid.

I looked in Orangefactor and some of the results there are missclasifications. 
The issue here is specifically tart (opt-talos-svgr-profiling-e10s T-P-e10s(s)):

https://treeherder.mozilla.org/logviewer.html#?job_id=185159322&repo=try&lineNumber=1165

07:28:00     INFO -  PID 2467 | TART_RESULTS_JSON=[{"name":"simple-open-DPI1.raw.TART","value":[533.1643676757812,15.97130298614502,2.9306368827819824,7.505402088165283,5.953417778015137,5.275247097015381,6.170005798339844,4.960402965545654,5.792179107666016,5.745396137237549,5.503996849060059,5.146557807922363,5.696459770202637,5.79405403137207,5.483041763305664,5.4984941482543945,4.286496162414551,4.14788293838501]},{"name":"simple-open-DPI1.half.TART","value":5.309455871582031},{"name":"simple-open-DPI1.all.TART","value":35.279185665978325},{"name":"simple-open-DPI1.error.TART","value":38.72025699999995},{"name":"simple-close-DPI1.raw.TART","value":[30.049596786499023]},{"name":"simple-close-DPI1.half.TART","value":30.049596786499023},{"name":"simple-close-DPI1.all.TART","value":30.049596786499023},{"name":"simple-close-DPI1.error.TART","value":59.42308400000002},{"name":"icon-open-DPI1.raw.TART","value":[51.53070068359375,6.700084209442139,17.06785011291504,17.706979751586914,8.491427421569824,8.093908309936523,2.7759909629821777,11.904196739196777,7.555787086486816,5.404076099395752,5.7501020431518555,5.401928901672363,5.715078830718994,4.824518203735352]},{"name":"icon-open-DPI1.half.TART","value":6.380620797475179},{"name":"icon-open-DPI1.all.TART","value":11.351616382598877},{"name":"icon-open-DPI1.error.TART","value":56.15362399999958},{"name":"icon-close-DPI1.raw.TART","value":[187.9969024658203,4.142271995544434,9.120185852050781,4.994456768035889,4.524115085601807,4.722390174865723,5.629619121551514,4.251025199890137,4.92379903793335,5.6516571044921875,4.371470928192139,5.174924850463867,5.076777935028076,4.975346088409424,5.439602851867676,5.282650947570801,5.422679901123047,4.069809913635254,4.530622959136963,4.867640018463135,5.116418838500977]},{"name":"icon-close-DPI1.half.TART","value":4.938904112035578},{"name":"icon-close-DPI1.all.TART","value":13.823065144675118},{"name":"icon-close-DPI1.error.TART","value":37.43231800000012},{"name":"icon-open-DPI2.raw.TART","value":[313.38983154296875,6.127394199371338,16.725223541259766,3.5349419116973877,5.973002910614014,7.974386215209961,6.199995994567871,7.701828956604004,5.331247806549072,5.248627185821533,5.363068103790283,5.058088779449463,5.899267196655273,5.2212958335876465,5.768485069274902,5.400116920471191,5.529908180236816,5.314184188842773]},{"name":"icon-open-DPI2.half.TART","value":5.413428926467896},{"name":"icon-open-DPI2.all.TART","value":23.431160807609558},{"name":"icon-open-DPI2.error.TART","value":42.01716699999906},{"name":"icon-close-DPI2.raw.TART","value":[62.55324935913086,2.7280569076538086,7.254948139190674,2.985029935836792,6.559605121612549,2.915497064590454,5.469498157501221,3.2927520275115967,4.643647193908691,4.175818920135498,3.94507098197937,4.231691837310791,3.9050281047821045,4.264665126800537,4.120984077453613,4.2580060958862305,4.416101932525635,4.182323932647705,4.236825942993164,6.330848217010498,4.424652099609375,4.53857421875,4.39371395111084]},{"name":"icon-close-DPI2.half.TART","value":4.441951294740041},{"name":"icon-close-DPI2.all.TART","value":6.948982145475305},{"name":"icon-close-DPI2.error.TART","value":22.064597000000504},{"name":"iconFade-close-DPI2.raw.TART","value":[62.05559539794922,3.118086099624634,4.12563419342041,2.565973997116089,3.750514030456543,4.043317794799805,2.434968948364258,4.566596984863281,2.517011880874634,4.507238864898682,2.545991897583008,4.457182884216309,2.422823905944824,4.468418121337891,2.6655220985412598,4.306665897369385,2.598994016647339,4.581949234008789,2.5662429332733154,4.432061195373535,2.5531721115112305,4.456103801727295,2.5996079444885254,4.318881988525391,2.5943820476531982,4.036859035491943,3.6602659225463867,2.5716230869293213,4.366576194763184,2.5295450687408447]},{"name":"iconFade-close-DPI2.half.TART","value":3.4781953652699786},{"name":"iconFade-close-DPI2.all.TART","value":5.413926919301351},{"name":"iconFade-close-DPI2.error.TART","value":7.374823000000106},{"name":"iconFade-open-DPI2.raw.TART","value":[105.69224548339844,2.759399890899658,4.585454940795898,3.2727460861206055,3.912964105606079,2.5551419258117676,5.131346225738525,3.6151280403137207,4.882513999938965,4.2999677658081055,4.378917217254639,4.097907066345215,4.463630199432373,4.1368279457092285,4.362059116363525,4.622821807861328,4.145079135894775,4.423579216003418,4.399136066436768,4.162452220916748,4.089424133300781,4.783265113830566,4.126026153564453,4.081096172332764,4.303922176361084]},{"name":"iconFade-open-DPI2.half.TART","value":4.302974104881287},{"name":"iconFade-open-DPI2.all.TART","value":8.211322088241577},{"name":"iconFade-open-DPI2.error.TART","value":5.2303579999997964},{"name":"newtab-open-preload-no.raw.TART","value":[300.125,25.917888641357422,5.905084133148193,8.256664276123047,6.358992099761963,10.255556106567383,3.805742025375366,4.517899990081787,5.1167402267456055,5.18474817276001,5.164629936218262,5.243534088134766,4.950644016265869,5.037303924560547,5.413041114807129,5.282787799835205]},{"name":"newtab-open-preload-no.half.TART","value":5.452057036486539},{"name":"newtab-open-preload-no.all.TART","value":25.40851603448391},{"name":"newtab-open-preload-no.error.TART","value":36.03568300000006},{"name":"newtab-open-preload-yes.raw.TART","value":[1035.555419921875,3.1178410053253174,10.564764022827148,3.2097349166870117,7.988626003265381,4.718986988067627,7.49436092376709,6.4566521644592285,5.857394218444824,5.941956996917725,5.712806224822998,5.381363868713379,6.2192158699035645,5.464425086975098,6.017754077911377,5.618928909301758,5.500864028930664,5.886987209320068,6.004055023193359]},{"name":"newtab-open-preload-yes.half.TART","value":5.756266699896918},{"name":"newtab-open-preload-yes.all.TART","value":60.1427440768794},{"name":"newtab-open-preload-yes.error.TART","value":44.96745800000099}]
07:28:00     INFO -  PID 2467 |
07:28:00     INFO -  PID 2467 | simple-open-DPI1.raw.TART: 533.2 16.0 2.9 7.5 6.0 5.3 6.2 5.0 5.8 5.7 5.5 5.1 5.7 5.8 5.5 5.5 4.3 4.1
07:28:00     INFO -  PID 2467 | simple-open-DPI1.half.TART: 5.3
07:28:00     INFO -  PID 2467 | simple-open-DPI1.all.TART: 35.3
07:28:00     INFO -  PID 2467 | simple-open-DPI1.error.TART: 38.7
07:28:00     INFO -  PID 2467 | simple-close-DPI1.raw.TART: 30.0
07:28:00     INFO -  PID 2467 | simple-close-DPI1.half.TART: 30.0
07:28:00     INFO -  PID 2467 | simple-close-DPI1.all.TART: 30.0
07:28:00     INFO -  PID 2467 | simple-close-DPI1.error.TART: 59.4
07:28:00     INFO -  PID 2467 | icon-open-DPI1.raw.TART: 51.5 6.7 17.1 17.7 8.5 8.1 2.8 11.9 7.6 5.4 5.8 5.4 5.7 4.8
07:28:00     INFO -  PID 2467 | icon-open-DPI1.half.TART: 6.4
07:28:00     INFO -  PID 2467 | icon-open-DPI1.all.TART: 11.4
07:28:00     INFO -  PID 2467 | icon-open-DPI1.error.TART: 56.2
07:28:00     INFO -  PID 2467 | icon-close-DPI1.raw.TART: 188.0 4.1 9.1 5.0 4.5 4.7 5.6 4.3 4.9 5.7 4.4 5.2 5.1 5.0 5.4 5.3 5.4 4.1 4.5 4.9 5.1
07:28:00     INFO -  PID 2467 | icon-close-DPI1.half.TART: 4.9
07:28:00     INFO -  PID 2467 | icon-close-DPI1.all.TART: 13.8
07:28:00     INFO -  PID 2467 | icon-close-DPI1.error.TART: 37.4
07:28:00     INFO -  PID 2467 | icon-open-DPI2.raw.TART: 313.4 6.1 16.7 3.5 6.0 8.0 6.2 7.7 5.3 5.2 5.4 5.1 5.9 5.2 5.8 5.4 5.5 5.3
07:28:00     INFO -  PID 2467 | icon-open-DPI2.half.TART: 5.4
07:28:00     INFO -  PID 2467 | icon-open-DPI2.all.TART: 23.4
07:28:00     INFO -  PID 2467 | icon-open-DPI2.error.TART: 42.0
07:28:00     INFO -  PID 2467 | icon-close-DPI2.raw.TART: 62.6 2.7 7.3 3.0 6.6 2.9 5.5 3.3 4.6 4.2 3.9 4.2 3.9 4.3 4.1 4.3 4.4 4.2 4.2 6.3 4.4 4.5 4.4
07:28:00     INFO -  PID 2467 | icon-close-DPI2.half.TART: 4.4
07:28:00     INFO -  PID 2467 | icon-close-DPI2.all.TART: 6.9
07:28:00     INFO -  PID 2467 | icon-close-DPI2.error.TART: 22.1
07:28:00     INFO -  PID 2467 | iconFade-close-DPI2.raw.TART: 62.1 3.1 4.1 2.6 3.8 4.0 2.4 4.6 2.5 4.5 2.5 4.5 2.4 4.5 2.7 4.3 2.6 4.6 2.6 4.4 2.6 4.5 2.6 4.3 2.6 4.0 3.7 2.6 4.4 2.5
07:28:00     INFO -  PID 2467 | iconFade-close-DPI2.half.TART: 3.5
07:28:00     INFO -  PID 2467 | iconFade-close-DPI2.all.TART: 5.4
07:28:00     INFO -  PID 2467 | iconFade-close-DPI2.error.TART: 7.4
07:28:00     INFO -  PID 2467 | iconFade-open-DPI2.raw.TART: 105.7 2.8 4.6 3.3 3.9 2.6 5.1 3.6 4.9 4.3 4.4 4.1 4.5 4.1 4.4 4.6 4.1 4.4 4.4 4.2 4.1 4.8 4.1 4.1 4.3
07:28:00     INFO -  PID 2467 | iconFade-open-DPI2.half.TART: 4.3
07:28:00     INFO -  PID 2467 | iconFade-open-DPI2.all.TART: 8.2
07:28:00     INFO -  PID 2467 | iconFade-open-DPI2.error.TART: 5.2
07:28:00     INFO -  PID 2467 | newtab-open-preload-no.raw.TART: 300.1 25.9 5.9 8.3 6.4 10.3 3.8 4.5 5.1 5.2 5.2 5.2 5.0 5.0 5.4 5.3
07:28:00     INFO -  PID 2467 | newtab-open-preload-no.half.TART: 5.5
07:28:00     INFO -  PID 2467 | newtab-open-preload-no.all.TART: 25.4
07:28:00     INFO -  PID 2467 | newtab-open-preload-no.error.TART: 36.0
07:28:00     INFO -  PID 2467 | newtab-open-preload-yes.raw.TART: 1035.6 3.1 10.6 3.2 8.0 4.7 7.5 6.5 5.9 5.9 5.7 5.4 6.2 5.5 6.0 5.6 5.5 5.9 6.0
07:28:00     INFO -  PID 2467 | newtab-open-preload-yes.half.TART: 5.8
07:28:00     INFO -  PID 2467 | newtab-open-preload-yes.all.TART: 60.1
07:28:00     INFO -  PID 2467 | newtab-open-preload-yes.error.TART: 45.0
07:28:05     INFO -  PID 2467 | 2018-06-27 07:28:05.017 firefox[2467:10808] Persistent UI failed to open file file:///Users/cltbld/Library/Saved%20Application%20State/org.mozilla.firefox.savedState/window_1.data: No such file or directory (2)
[taskcluster:error] Aborting task...
[taskcluster 2018-06-27T14:54:26.192Z] === Task Finished ===
[taskcluster 2018-06-27T14:54:26.192Z] Task Duration: 29m58.919690759s
[taskcluster 2018-06-27T14:54:26.833Z] Uploading artifact public/logs/localconfig.json from file logs/localconfig.json with content encoding "gzip", mime type "application/json" and expiry 2018-07-11T12:44:10.341Z
[taskcluster 2018-06-27T14:54:27.323Z] Uploading artifact public/logs/talos_critical.log from file logs/talos_critical.log with content encoding "gzip", mime type "text/plain" and expiry 2018-07-11T12:44:10.341Z
[taskcluster 2018-06-27T14:54:27.725Z] Uploading artifact public/logs/talos_error.log from file logs/talos_error.log with content encoding "gzip", mime type "text/plain" and expiry 2018-07-11T12:44:10.341Z
[taskcluster 2018-06-27T14:54:28.162Z] Uploading artifact public/logs/talos_fatal.log from file logs/talos_fatal.log with content encoding "gzip", mime type "text/plain" and expiry 2018-07-11T12:44:10.341Z
[taskcluster 2018-06-27T14:54:28.528Z] Uploading artifact public/logs/talos_info.log from file logs/talos_info.log with content encoding "gzip", mime type "text/plain" and expiry 2018-07-11T12:44:10.341Z
[taskcluster 2018-06-27T14:54:29.004Z] Uploading artifact public/logs/talos_raw.log from file logs/talos_raw.log with content encoding "gzip", mime type "text/plain" and expiry 2018-07-11T12:44:10.341Z
[taskcluster 2018-06-27T14:54:29.507Z] Uploading artifact public/logs/talos_warning.log from file logs/talos_warning.log with content encoding "gzip", mime type "text/plain" and expiry 2018-07-11T12:44:10.341Z
[taskcluster 2018-06-27T14:54:30.510Z] Uploading artifact public/test_info/missingsymbols.zip from file build/blobber_upload_dir/missingsymbols.zip with content encoding "", mime type "application/zip" and expiry 2018-07-11T12:44:10.341Z
[taskcluster 2018-06-27T14:54:31.958Z] Uploading artifact public/test_info/profile_tsvgr_opacity.zip from file build/blobber_upload_dir/profile_tsvgr_opacity.zip with content encoding "", mime type "application/zip" and expiry 2018-07-11T12:44:10.341Z
[taskcluster 2018-06-27T14:54:32.633Z] Uploading artifact public/test_info/profile_tsvgx.zip from file build/blobber_upload_dir/profile_tsvgx.zip with content encoding "", mime type "application/zip" and expiry 2018-07-11T12:44:10.341Z
[taskcluster 2018-06-27T14:54:33.441Z] Uploading artifact public/test_info/svgr-e10s_errorsummary.log from file build/blobber_upload_dir/svgr-e10s_errorsummary.log with content encoding "gzip", mime type "text/plain" and expiry 2018-07-11T12:44:10.341Z
[taskcluster 2018-06-27T14:54:33.873Z] Uploading artifact public/test_info/svgr-e10s_raw.log from file build/blobber_upload_dir/svgr-e10s_raw.log with content encoding "gzip", mime type "text/plain" and expiry 2018-07-11T12:44:10.341Z
[taskcluster:error] Task aborted - max run time exceeded

Hope this clarifies a bit the situation.

Comment 5

[Andreea Pavel [:apavel]]

Please disregard the orangefactor comment, those were misclassifications, corrected afterwards.

Comment 6

[Intermittent Failures Robot]

23 failures in 622 pushes (0.037 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 8
* mozilla-inbound: 6
* mozilla-central: 5
* try: 4

Platform breakdown:
* osx-10-10: 14
* windows7-32: 3
* macosx64-nightly: 2
* windows2012-32: 1
* windows10-64: 1
* macosx64-qr: 1
* macosx64-devedition: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1471803&startday=2018-07-02&endday=2018-07-08&tree=trunk

Comment 7

[Intermittent Failures Robot]

11 failures in 623 pushes (0.018 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* try: 3
* mozilla-inbound: 3
* autoland: 3
* mozilla-central: 2

Platform breakdown:
* osx-10-10: 7
* toolchains: 2
* windows7-32: 1
* macosx64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1471803&startday=2018-07-09&endday=2018-07-15&tree=trunk

Comment 8

[Intermittent Failures Robot]

2 failures in 760 pushes (0.003 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-central: 1
* autoland: 1

Platform breakdown:
* windows10-64-ccov: 1
* macosx64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1471803&startday=2018-07-23&endday=2018-07-29&tree=trunk

Comment 9

[Intermittent Failures Robot]

1 failures in 2672 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-central: 1

Platform breakdown:
* osx-10-10: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1471803&startday=2018-08-13&endday=2018-08-19&tree=all

Comment 10

[Intermittent Failures Robot]

3 failures in 2568 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-beta: 1
* try: 1
* autoland: 1

Platform breakdown:
* osx-10-10: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1471803&startday=2018-08-20&endday=2018-08-26&tree=all

Comment 11

[Intermittent Failures Robot]

3 failures in 2667 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 1
* autoland: 2

Platform breakdown:
* windows7-32: 1
* osx-10-10: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1471803&startday=2018-09-03&endday=2018-09-09&tree=all

Comment 12

[Intermittent Failures Robot]

7 failures in 2684 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-central: 3
* mozilla-inbound: 1
* try: 1
* mozilla-beta: 2

Platform breakdown:
* osx-10-10: 3
* windows10-64-ccov: 1
* macosx64-nightly: 2
* windows7-32: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1471803&startday=2018-09-10&endday=2018-09-16&tree=all

Comment 13

[Intermittent Failures Robot]

2 failures in 2902 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* autoland: 1
* mozilla-central: 1

Platform breakdown:
* windows7-32-msvc: 1
* osx-10-10: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1471803&startday=2018-09-17&endday=2018-09-23&tree=all

Comment 14

[Intermittent Failures Robot]

16 failures in 2708 pushes (0.006 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* autoland: 4
* mozilla-central: 1
* try: 8
* mozilla-inbound: 3

Platform breakdown:
* windows10-64: 2
* windows7-32: 3
* windows10-64-ccov: 1
* windows10-64-qr: 2
* osx-10-10: 7
* windows10-64-msvc: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1471803&startday=2018-10-01&endday=2018-10-07&tree=all

Comment 15

[Intermittent Failures Robot]

8 failures in 2740 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* try: 3
* autoland: 3
* mozilla-inbound: 2

Platform breakdown:
* osx-10-10: 8

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1471803&startday=2018-10-08&endday=2018-10-14&tree=all

Comment 17

[Intermittent Failures Robot]

3 failures in 2759 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* autoland: 2
* mozilla-central: 1

Platform breakdown:
* osx-10-10: 1
* linux64: 1
* windows10-64-ccov: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1471803&startday=2018-10-15&endday=2018-10-21&tree=all