1718621 - Intermittent [ccov] dom/media/test/reftest/color_quads/720p.png.bt709.bt709.tv.gbrp.vp9.webm == dom/media/test/reftest/color_quads/reftest_video.html?src=timeout | image comparison, max difference: 255, number of differing pixels: 505

Status: RESOLVED FIXED

(Core :: Audio/Video, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: apavel [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=344068818&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/JJxkr281TauBjPL11yJRSw/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/JJxkr281TauBjPL11yJRSw/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

---

[task 2021-06-29T11:06:47.311Z] 11:06:47     INFO - REFTEST TEST-START | dom/media/test/reftest/color_quads/720p.png.bt709.bt709.tv.gbrp.vp9.webm == dom/media/test/reftest/color_quads/reftest_video.html?src=timeout
[task 2021-06-29T11:06:47.313Z] 11:06:47     INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/dom/media/test/reftest/reftest_video.html?src=color_quads/720p.png.bt709.bt709.tv.gbrp.vp9.webm | 24 / 30 (80%)
[task 2021-06-29T11:06:47.686Z] 11:06:47     INFO - REFTEST TEST-UNEXPECTED-FAIL | dom/media/test/reftest/color_quads/720p.png.bt709.bt709.tv.gbrp.vp9.webm == dom/media/test/reftest/color_quads/reftest_video.html?src=timeout | image comparison, max difference: 255, number of differing pixels: 505

Comment 1

[Alexandru Michis [:malexandru]]

Probably permafailing the same way as Bug 1717078 and Bug 1718403.

Comment 2

[Intermittent Failures Robot]

16 failures in 723 pushes (0.022 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 16

Platform and build breakdown:

• linux1804-64-ccov-qr: 16
  • opt: 16

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1718621&startday=2021-06-28&endday=2021-07-04&tree=all

Comment 3

[Intermittent Failures Robot]

24 failures in 2387 pushes (0.01 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 24

Platform and build breakdown:

• linux1804-64-ccov-qr: 24
  • opt: 24

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1718621&startday=2021-07-05&endday=2021-07-11&tree=all

Comment 4

[Intermittent Failures Robot]

15 failures in 2224 pushes (0.007 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 15

Platform and build breakdown:

• linux1804-64-ccov-qr: 15
  • opt: 15

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1718621&startday=2021-07-12&endday=2021-07-18&tree=all

Comment 5

[Intermittent Failures Robot]

22 failures in 2940 pushes (0.007 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 22

Platform and build breakdown:

• linux1804-64-ccov-qr: 22
  • opt: 22

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1718621&startday=2021-07-19&endday=2021-07-25&tree=all

Comment 6

[Intermittent Failures Robot]

22 failures in 3131 pushes (0.007 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 22

Platform and build breakdown:

• linux1804-64-ccov-qr: 22
  • opt: 22

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1718621&startday=2021-07-26&endday=2021-08-01&tree=all

Comment 7

[Intermittent Failures Robot]

23 failures in 2829 pushes (0.008 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 23

Platform and build breakdown:

• linux1804-64-ccov-qr: 23
  • opt: 23

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1718621&startday=2021-08-02&endday=2021-08-08&tree=all

Comment 8

[Kelsey Gilbert [:jgilbert]]

I suspect this might be a permafail for linux1804-64-ccov-qr, but not other configurations, but I can't immediately think of a reason why this would be.

Comment 9

[Kelsey Gilbert [:jgilbert]]

I'm looking this example: https://treeherder.mozilla.org/jobs?repo=mozilla-central&revision=163caef701cb06f7a3636e48f0a9ceefe5f1cea2&selectedTaskRun=PwtOw-E2TSu8bZ3LzAIhJw.0
Here's the sandbox object from the log:

sandbox = {
  AddressSanitizer: false
  Android: false
  MinGW: false
  ThreadSanitizer: false
  advancedLayers: false
  appleSilicon: false
  asyncPan: true
  azureCairo: false
  azureSkia: true
  azureSkiaGL: false
  browserIsFission: false
  browserIsRemote: true
  cocoaWidget: false
  compareRetainedDisplayLists: false
  contentSameGfxBackendAsCanvas: true
  d2d: false
  d3d11: false
  d3d9: false
  dwrite: false
  embeddedInFirefoxReality: false
  fission: false
  geckoview: false
  gpuProcess: false
  gpuProcessForceEnabled: false
  gtkWidget: true
  http: Object { userAgent: "Mozilla/5.0 (X11; Linux x86_64; rv:93.0) Gecko/20100101 Firefox/93.0", appName: "Mozilla", appVersion: "5.0", … }
  is64Bit: true
  isCoverageBuild: true
  isDebugBuild: false
  jxl: true
  layerChecksEnabled: false
  layersGPUAccelerated: true
  layersOMTC: true
  layersOpenGL: false
  nativeThemePref: false
  prefs: Object {  }
  qtWidget: false
  release_or_beta: false
  remoteCanvas: false
  retainedDisplayList: true
  retainedDisplayLists: true
  serviceWorkerE10s: true
  skiaContent: true
  smallScreen: false
  swgl: false
  transparentScrollbars: true
  unsupportedWithDrawSnapshot: false
  useDrawSnapshot: false
  usesOverlayScrollbars: false
  usesRepeatResampling: false
  verify: false
  webrender: true
  webrtc: true
  winWidget: false
  windowsDefaultTheme: false
  xulRuntime: Object { widgetToolkit: "gtk", OS: "Linux", XPCOMABI: "x86_64-gcc3" }
}

Comparing against the non-ccov run on the same cset, isCoverageBuild: false was the only difference in the sandbox dict.

That means I can simply mark this failure expectation accordingly, but I'm curious what's different about it. Ideally we do definitely want to have ccov run on these tests!

I'm re-running the failing job to see if it's indeed perma.

Comment 10

[Kelsey Gilbert [:jgilbert]]

Yeah, looks like this is a perma-orange for (tier 2) ccov.

Comment 11

[Kelsey Gilbert [:jgilbert]]

Attachment: Bug 1718621 - Simplify color_quads gbrp expectations.

This seems to fix a perma-orange for linux-ccov R2 also.

Comment 12

[Kelsey Gilbert [:jgilbert]]

Well, simplifying it does look clean: https://treeherder.mozilla.org/jobs?repo=try&group_state=expanded&revision=6cd355bb8d93288634537b0353c3bdd51e697fe6

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7be3cdcf213a
Simplify color_quads gbrp expectations. r=gfx-reviewers,lsalzman

Comment 14

[Iulian Moraru]

Backed out for causing reftest failures in color_quads.

Push with failures

Failure log

Backout link

Comment 15

[Intermittent Failures Robot]

33 failures in 3025 pushes (0.011 failures/push) were associated with this bug in the last 7 days.

This is the #35 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 31
• autoland: 2

Platform and build breakdown:

• linux1804-64-ccov-qr: 31
  • opt: 31
• macosx1015-64-qr: 2
  • debug: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1718621&startday=2021-08-09&endday=2021-08-15&tree=all

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/dee2d3179b5f
Simplify color_quads gbrp expectations. r=gfx-reviewers,lsalzman

Comment 17

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/dee2d3179b5f

Comment 18

[Intermittent Failures Robot]

12 failures in 2899 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 12

Platform and build breakdown:

• linux1804-64-ccov-qr: 12
  • opt: 12

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1718621&startday=2021-08-16&endday=2021-08-22&tree=all