1239766 - Reftest-4 - layout/reftests/svg/smil/transform/additive-1.svg

Status: RESOLVED FIXED

(Testing :: General, defect)

Description

[Armen [:armenzg]]

I want take one single reftest failure and determine what is the root issue.
If you load [1][2][3], we're going to focus on the first test failure (there are no fonts involved just blue arrows).

While I took the steps of running the tests [4], I made sure I keep track of what errors appear in ~/.xsession-errors.
The errors which gets triggered when the test is run are similar to this [5]:
> unity-2d-shell: [WARNING] QDBusError("org.freedesktop.DBus.Error.UnknownMethod", "No such interface `org.ayatana.bamf.application' on object at path /org/ayatana/bamf/application1932146384")


NOTE: For my own experimenting, I increased my monitor's screen resolution sufficiently in order to stretch completely the VNC connection. I hope I can see if there was something happening out sight with xtightvncviewer




[1] http://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/SSV-uPofSx29EDlXCMGFDw/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1
[2] http://people.mozilla.org/~armenzg/sattap/bc136231.png
[3] https://treeherder.mozilla.org/#/jobs?repo=try&revision=3c9a54d68c95
[3]
STR:
# Add START_VNC, SKIP_MOZHARNESS_RUN and --device
docker run -ti \
  -e GECKO_HEAD_REPOSITORY='https://hg.mozilla.org/try/' \
  -e GECKO_HEAD_REV='3c9a54d68c9572369425e19a61360c16017da532' \
  -e MOZHARNESS_CONFIG='mozharness/configs/unittests/linux_unittest.py mozharness/configs/remove_executables.py' \
  -e MOZHARNESS_SCRIPT='mozharness/scripts/desktop_unittest.py' \
  -e MOZHARNESS_URL='https://queue.taskcluster.net/v1/task/I6xTPjPrQx6pDFhyXuA7Mw/artifacts/public/build/mozharness.zip' \
  -e MOZILLA_BUILD_URL='https://queue.taskcluster.net/v1/task/I6xTPjPrQx6pDFhyXuA7Mw/artifacts/public/build/target.tar.bz2' \
  -e NEED_PULSEAUDIO='true' \
  -e NEED_WINDOW_MANAGER='true' \
  -e START_VNC='true' \
  -e SKIP_MOZHARNESS_RUN='true' \
  --device /dev/video1:/dev/video1 \
  taskcluster/desktop-test:0.5.5

./bin/test.sh
# You can now VNC to it (xtightvncviewer recommended)
cd workspace
cp ~/.xsession-errors ~/xsession-errors_before_mozharness.tx
# Notice that I'm running with --no-run-tests
sudo -E -u worker python2.7 /home/worker/workspace/mozharness/scripts/desktop_unittest.py --config-file mozharness/configs/unittests/linux_unittest.py --config-file mozharness/configs/remove_executables.py --no-read-buildbot-config --installer-url=https://queue.taskcluster.net/v1/task/I6xTPjPrQx6pDFhyXuA7Mw/artifacts/public/build/target.tar.bz2 --test-packages-url=https://queue.taskcluster.net/v1/task/I6xTPjPrQx6pDFhyXuA7Mw/artifacts/public/build/test_packages.json --download-symbols=ondemand --reftest-suite=reftest --total-chunk=4 --this-chunk=4 --no-run-tests
cp ~/.xsession-errors ~/xsession-errors_before_tests.txt
vimdiff ~/xsession-errors_before_mozharness.txt ~/xsession-errors_before_tests.txt # No differences

# Run the individual test
source /home/worker/workspace/build/venv/bin/activate
cd build
/home/worker/workspace/build/venv/bin/python -u /home/worker/workspace/build/tests/reftest/runreftest.py --total-chunks 4 --this-chunk 4 --appname=/home/worker/workspace/build/application/firefox/firefox --utility-path=tests/bin --extra-profile-file=tests/bin/plugins --symbols-path=https://queue.taskcluster.net/v1/task/I6xTPjPrQx6pDFhyXuA7Mw/artifacts/public/build/target.crashreporter-symbols.zip --suite=reftest -- tests/reftest/tests/layout/reftests/svg/smil/transform/additive-1.svg

cp ~/.xsession-errors ~/xsession-errors_after_test.txt
vimdiff ~/xsession-errors_before_mozharness.txt ~/xsession-errors_after_test.txt

[5]
unity-2d-shell: [WARNING] QDBusError("org.freedesktop.DBus.Error.UnknownMethod", "No such interface `org.ayatana.bamf.application' on object at path /org/ayatana/bamf/application1932146384")
unity-2d-shell: [WARNING] QDBusError("org.freedesktop.DBus.Error.UnknownMethod", "No such interface `org.ayatana.bamf.view' on object at path /org/ayatana/bamf/application1932146384")
unity-2d-shell: [WARNING] QDBusError("org.freedesktop.DBus.Error.UnknownMethod", "No such interface `org.ayatana.bamf.application' on object at path /org/ayatana/bamf/application1932146384")
unity-2d-shell: [WARNING] QDBusError("org.freedesktop.DBus.Error.UnknownMethod", "No such interface `org.ayatana.bamf.view' on object at path /org/ayatana/bamf/application1932146384")
unity-2d-shell: [WARNING] QDBusError("org.freedesktop.DBus.Error.UnknownMethod", "No such interface `org.ayatana.bamf.view' on object at path /org/ayatana/bamf/application1932146384")
unity-2d-shell: [WARNING] QDBusError("org.freedesktop.DBus.Error.UnknownMethod", "No such interface `org.ayatana.bamf.view' on object at path /org/ayatana/bamf/application1932146384")
unity-2d-shell: [WARNING] QDBusError("org.freedesktop.DBus.Error.UnknownMethod", "No such interface `org.ayatana.bamf.application' on object at path /org/ayatana/bamf/application1932146384")
unity-2d-shell: [WARNING] QDBusError("org.freedesktop.DBus.Error.UnknownMethod", "No such interface `org.ayatana.bamf.view' on object at path /org/ayatana/bamf/application1932146384")
unity-2d-shell: [WARNING] QDBusError("org.freedesktop.DBus.Error.UnknownMethod", "No such interface `org.ayatana.bamf.application' on object at path /org/ayatana/bamf/application1932146384")
unity-2d-shell: [WARNING] QDBusError("org.freedesktop.DBus.Error.UnknownMethod", "No such interface `org.ayatana.bamf.view' on object at path /org/ayatana/bamf/application1932146384")
unity-2d-shell: [WARNING] QDBusError("org.freedesktop.DBus.Error.UnknownMethod", "No such interface `org.ayatana.bamf.view' on object at path /org/ayatana/bamf/application1932146384")
unity-2d-shell: [WARNING] QDBusError("org.freedesktop.DBus.Error.UnknownMethod", "No such interface `org.ayatana.bamf.view' on object at path /org/ayatana/bamf/application1932146384")
unity-2d-shell: [WARNING] QDBusError("org.freedesktop.DBus.Error.UnknownMethod", "No such interface `org.ayatana.bamf.view' on object at path /org/ayatana/bamf/application1932146384")

Comment 1

[Armen [:armenzg]]

The ayatana matter could be misleading since just opening the browser adds this to ~/.xsession-errors

unity-2d-shell: [WARNING] QDBusError("org.freedesktop.DBus.Error.UnknownMethod", "No such interface `org.ayatana.bamf.application' on object at path /org/ayatana/bamf/application829906520") 
unity-2d-shell: [WARNING] QDBusError("org.freedesktop.DBus.Error.UnknownMethod", "No such interface `org.ayatana.bamf.view' on object at path /org/ayatana/bamf/application829906520") 
unity-2d-shell: [WARNING] QDBusError("org.freedesktop.DBus.Error.UnknownMethod", "No such interface `org.ayatana.bamf.application' on object at path /org/ayatana/bamf/application829906520") 
unity-2d-shell: [WARNING] QDBusError("org.freedesktop.DBus.Error.UnknownMethod", "No such interface `org.ayatana.bamf.view' on object at path /org/ayatana/bamf/application829906520") 
unity-2d-shell: [WARNING] QDBusError("org.freedesktop.DBus.Error.UnknownMethod", "No such interface `org.ayatana.bamf.view' on object at path /org/ayatana/bamf/application829906520") 
unity-2d-shell: [WARNING] QDBusError("org.freedesktop.DBus.Error.UnknownMethod", "No such interface `org.ayatana.bamf.view' on object at path /org/ayatana/bamf/application829906520") 
unity-2d-shell: [WARNING] QDBusError("org.freedesktop.DBus.Error.UnknownMethod", "No such interface `org.ayatana.bamf.view' on object at path /org/ayatana/bamf/application829906520") 
Window manager warning: Buggy client sent a _NET_ACTIVE_WINDOW message with a timestamp of 0 for 0x2000002 (unity-2d-s)
Window manager warning: meta_window_activate called by a pager with a 0 timestamp; the pager needs to be fixed.

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

Ayatana is an Ubuntu UI toolkit, so probably not related to font rendering.

I'm glad we're not looking at fonts anymore, as I determined that the font config (fc-list and /etc/fonts) is identical between BB and TaskCluster jobs.

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

At a basic level, I can confirm that the RENDER extension is loaded in the Xvfb process.

Comment 4

[Armen [:armenzg]]

We believe that different version of mesa *could* be involved (we never tried the version currently installed by releng).
We have 8.0.4-0ubuntu0.6mozilla1 on releng instances while 8.0.4-0ubuntu0.7 on the docker image.


(venv)root@taskcluster-worker:~/workspace/build# dpkg -p libgl1-mesa-dri
Package: libgl1-mesa-dri
Multi-Arch: same
Priority: optional
Section: libs
Installed-Size: 15582
Maintainer: Ubuntu X-SWAT <[e-mail]>
Architecture: amd64
Source: mesa
Version: 8.0.4-0ubuntu0.7
Replaces: libgl1-mesa-dri-experimental (<< 7.11.1), xlibmesa-dri (<< 1:7.0.0)
Depends: libc6 (>= 2.14), libdrm-intel1 (>= 2.4.27), libdrm-nouveau1a (>= 2.4.23), libdrm-radeon1 (>= 2.4.17), libdrm2 (>= 2.4.3), libexpat1 (>= 1.95.8), libgcc1 (>= 1:4.1.1), libllvm3.0, libstdc++6 (>= 4.1.1)
Suggests: libglide3
Breaks: libgl1-mesa-dri-experimental (<< 7.11.1), libgl1-mesa-glx (<< 7.10.2-4), libgl1-mesa-glx-no-multiarch, xserver-xorg-core (<< 2:1.10.2-2), xserver-xorg-core-no-multiarch
Conflicts: xlibmesa-dri (<< 1:7.0.0)
Size: 3035414
Description: free implementation of the OpenGL API -- DRI modules
 This version of Mesa provides GLX and DRI capabilities: it is capable of
 both direct and indirect rendering.  For direct rendering, it can use DRI
 modules from the libgl1-mesa-dri package to accelerate drawing.
 .
 This package does not include the OpenGL library itself, only the DRI
 modules for accelerating direct rendering.
 .
 For a complete description of Mesa, please look at the
 libgl1-mesa-swx11 package.
 .
 The tdfx DRI module needs libglide3 to enable direct rendering.
Homepage: http://mesa3d.sourceforge.net/
Original-Maintainer: Debian X Strike Force <[e-mail]>

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

as a note for more data, I am unable to reproduce this specific error on my local 14.04 instance.  Many errors I can reproduce locally, not this one :)  This did reproduce in the gnome-shell world that I tried 3 weeks ago:
https://queue.taskcluster.net/v1/task/DJ-KvztOQMGzn4DQ3VjYYg/runs/0/artifacts/public/logs/live_backing.log

both the gnome-shell experiment and our current live system have the same failures and number of missed pixels:
http://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/DJ-KvztOQMGzn4DQ3VjYYg/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

Comment 6

[Dustin J. Mitchell [:dustin] (he/him)]

I reproduced this easily by following Armen's instructions in comment 0.  Nice work :)

Comment 7

[Dustin J. Mitchell [:dustin] (he/him)]

Worth noting that the latest mesa from upstream in precise is 8.0.4-0ubuntu0.7, which reads as newer than the 8.0.4-0ubuntu0.6mozilla1 that we have in puppet.  I'm forcing installation of the latter version and marking it to avoid unwanted updates.  We'll see what happens.  Doing this by hand in a docker container didn't work, but I don't take that as definitive.

Comment 8

[Dustin J. Mitchell [:dustin] (he/him)]

Sadly, same effect with 8.0.4-0ubuntu0.6mozilla1, using the reprop recipe in comment 0, so that hypothesis is out.  I'll keep the patch around just in case (the image is djmitche/desktop-test:0.5.6 if you want to try).

Comment 9

[Dustin J. Mitchell [:dustin] (he/him)]

Hi Brian -- hg log says you know something about this test :

Can you help provide some context for us?  The error we're seeing is

http://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/SSV-uPofSx29EDlXCMGFDw/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

and this is due to some difference in the test environments between Buildbot and TaskCluster.  We're just not sure what the difference might be.  Do you have any suggestions as to what parts of the system or its configuration might affect the anti-aliasing behavior of this test?  Any other ideas on how we might proceed to debug this, or who else we might ask for help with problems with anti-aliasing of graphics (this isn't the only failing reftest, just the one we chose to explore)?

Comment 10

[Dustin J. Mitchell [:dustin] (he/him)]

I'm missing something about Xsession startup here -- even adding the `-p` to `docker run` to get access to the Xvfb display via VNC, Xsession is failing to start up with errors about dbus, and thus I have no window manager.

Comment 11

[Brian Birtles (:birtles)]

(In reply to Dustin J. Mitchell [:dustin] from comment #9)
> Hi Brian -- hg log says you know something about this test :
> 
> Can you help provide some context for us?  The error we're seeing is
> 
> http://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/
> reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/SSV-
> uPofSx29EDlXCMGFDw/runs/0/artifacts/public/logs/live_backing.
> log&only_show_unexpected=1
> 
> and this is due to some difference in the test environments between Buildbot
> and TaskCluster.  We're just not sure what the difference might be.  Do you
> have any suggestions as to what parts of the system or its configuration
> might affect the anti-aliasing behavior of this test?  Any other ideas on
> how we might proceed to debug this, or who else we might ask for help with
> problems with anti-aliasing of graphics (this isn't the only failing
> reftest, just the one we chose to explore)?

The main difference between the test image and reference image there is that the test image uses animation. When we have a transform animation, we layerize the transformed element and then composite it on the compositor. When possible we composite using the GPU. When that's not available, we composite in software.

When there is no animation, we render it as a single layer on the main thread.

My guess is that you're seeing a difference in anti-aliasing due to the compositing being done in hardware for the animated case.

You can verify this by turning on layer borders in about:config, layers.draw-borders. If you load the test file in the browser and refresh it, you should see a green box around each of the arrows for about 1s.

(If you're trying to workaround it for this particular test, you can make the test wait until we delayerize the content. Basically, the test pauses the animation and seeks it. After a while we'll detect that the content is no longer animating and drop the layer and render it altogether on the main thread. That's not ideal, though, since it makes the test run longer and it means we're not really testing the the composited output. Marking the test as fuzzy might be appropriate.)

Comment 12

[Dustin J. Mitchell [:dustin] (he/him)]

Thanks -- good info!  In this case we're running in Xvfb on a non-GPU EC2 host, so it is using software in OMTC:

23:01:04     INFO -  REFTEST INFO | {"isDebugBuild":true,"xulRuntime":{"widgetToolkit":"gtk3","OS":"Linux","XPCOMABI":"x86_64-gcc3"},"smallScreen":false,"d2d":false,"azureCairo":true,"azureQuartz":false,"azureSkia":false,"skiaContent":false,"azureSkiaGL":0,"contentSameGfxBackendAsCanvas":true,"layersGPUAccelerated":false,"d3d11":false,"layersOpenGL":false,"layersOMTC":true,"B2G":false,"Android":false,"cocoaWidget":false,"gtkWidget":true,"qtWidget":false,"winWidget":false,"transparentScrollbars":true,"AddressSanitizer":false,"webrtc":true,"http":{"userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:46.0) Gecko/20100101 Firefox/46.0","appName":"Mozilla","appVersion":"5.0","platform":"X11","oscpu":"Linux x86_64","misc":"rv:46.0"},"haveTestPlugin":true,"windowsDefaultTheme":false,"nativeThemePref":true,"prefs":{},"browserIsRemote":false,"Mulet":false,"asyncPan":false,"asyncZoom":false} 

You're right that even on my own host, I can see the lower-rightmost arrow is fuzzy for a while and then snaps into focus 1s after the animation completes.  I assume the more subtle differences elsewhere in the image are doing the same -- I just can't see them that quickly.

So in this case we're looking at a composited test image vs. a single layer ref image, which could be a timing issue.  In that case, though, I wonder why it's so reproducible in one environment and not in another -- 1s (the animation duration) is a long time, and from my read of the test code the reftest snapshot should always occur very early in that 1s, thus capturing the compositied image.  I think that's the intent of the test anyway, otherwise you're just comparing an SVG to the same SVG.

Moreover, the other failures in that logfile, such as [1], don't seem to involve animation and yet still have errors around the element borders.  So there may be something else going on here, too -- differences in the compositor's behavior.

[1] https://hg.mozilla.org/mozilla-central/raw-file/9a358be6fa798f24deecac1b502742b2c37cd6bd/layout/reftests/writing-mode/tables/wm-row-progression-004.xht

Comment 13

[Dustin J. Mitchell [:dustin] (he/him)]

This seems to be one of the few reftests not fixed by bug 1240056.  Which makes sense since it's not font-related (to be honest, I'm surprised other tests in this suite were affected..)

I can confirm that this passes just fine if you add a one-second timeout with window.setTimeout.

Oh, hey, it IS already fuzzy -- maybe just not fuzzy enough?

  https://bugzilla.mozilla.org/show_bug.cgi?id=981344

Indeed, that allows 110 differences, and we're seeing 111.  I'll call that close enough.

Comment 14

[Dustin J. Mitchell [:dustin] (he/him)]

Attachment: MozReview Request: Bug 1239766: make additive-1.svg a tiny bit fuzzier; r?birtles

Review commit: https://reviewboard.mozilla.org/r/31085/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/31085/

Comment 15

[Brian Birtles (:birtles)]

Comment on attachment 8708524 [details]
MozReview Request: Bug 1239766: make additive-1.svg a tiny bit fuzzier; r?birtles

https://reviewboard.mozilla.org/r/31085/#review27971

Comment 16

[Joel Maher ( :jmaher ) (UTC -8)]

:dustin, can you push this to try server with:
try: -b do -p all -u reftest -t nonea

this will ensure we don't cause troubles on any platforms in the land of buildbot.

Comment 17

[Dustin J. Mitchell [:dustin] (he/him)]

 https://treeherder.mozilla.org/#/jobs?repo=try&revision=70625a550c05

Comment 18

[Dustin J. Mitchell [:dustin] (he/him)]

Everything looks good in BBland.  For some reason all of the TC builds failed, but that seems unrelated.

Comment 19

[Dustin J. Mitchell [:dustin] (he/him)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/afe3172bb06641345a5069f5c38e407175e436a8
Bug 1239766: make additive-1.svg a tiny bit fuzzier; r=birtles

Comment 20

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/afe3172bb066