Closed Bug 1383682 Opened 7 years ago Closed 6 years ago

Categories

(Core :: Graphics: ImageLib, defect, P3)

defect

Tracking

()

RESOLVED FIXED
mozilla60
Tracking Status
firefox-esr52 --- unaffected
firefox58 --- wontfix
firefox59 --- wontfix
firefox60 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: aosmond)

References

(Blocks 1 open bug)

Details

(Keywords: intermittent-failure, Whiteboard: [gfx-noted][stockwell unknown])

Attachments

(3 files, 5 obsolete files)

this has started failing a bit more this past week, specifically on linux32-stylo debug:
https://brasstacks.mozilla.com/orangefactor/index.html?display=Bug&bugid=1383682

:xidorn, could you look at this and help set a priority?
Flags: needinfo?(xidorn+moz)
Isn't that just because we didn't start testing linux32-stylo until this past week?

I don't believe this is related to stylo given it fails on normal builds as well.

As I mentioned in bug 1369957 comment 13, it may seem to happen more often on stylo builds simply because tests run more frequently on stylo builds than normal builds. We may want to set up SETA data for linux stylo builds for saving machine resources and reduce noises on intermittents.
Flags: needinfo?(xidorn+moz)
See Also: → 1383676
See Also: → 1382969
See Also: → 1383230
The see-also bugs fail the same way as this one: slight differences (I can't see them) around the "image" word. Collectively, these bugs account for about 100 failures / week.

:jmaher -- What's the best way to get some attention for these failures?
Flags: needinfo?(jmaher)
Whiteboard: [stockwell needswork]
starting with the right bugzilla component:
$ ./mach file-info bugzilla-component testing/web-platform/tests//css/css-backgrounds-3/background-size-031.html
Core :: CSS Parsing and Computation
  testing/web-platform/tests/css/css-backgrounds-3/background-size-031.html
Component: web-platform-tests → CSS Parsing and Computation
Flags: needinfo?(jmaher)
Product: Testing → Core
Version: Version 3 → unspecified
:jet, we have a series of failures here all seem to be related to the same root tests of background-size-xx.html- in total the failures are extremely high- can you help find someone to look at these failures, I assume there is a single root cause.
Flags: needinfo?(bugs)
Those are just images, and image showing something unstable (rather than, e.g. wrong position) sounds more like issue in ImageLib or Graphics, though it is also possible that Layout sends some subpixel position to them. Anyway I don't think it's related to style system.

Maybe the easiest way is to add some fuzzy, but we cannot do that for wpt reftest I suppose?
Component: CSS Parsing and Computation → ImageLib
:jgraham, in wpt reftests is there a way to add fuzzy conditions?
Flags: needinfo?(james)
No. It has been discussed, but there's no cross-vendor agreement that it's a good idea. But we will likely need something at some point if webrender makes exact references unworkable.
Flags: needinfo?(james)
Given the fact there is no immediate support for fuzzy in wpt-reftests, do you have suggestions of how to resolve this?
Flags: needinfo?(xidorn+moz)
I have no idea. Maybe ask people working on Graphics or ImageLib and see if they have any idea what may be happening here.
Flags: needinfo?(xidorn+moz)
See Also: → 1383061
:milan (when you get back) -- Do you have an idea of who could investigate these frequent intermittent test failures?
Flags: needinfo?(milan)
I'll leave needinfo on me, but chances are we won't get to the intermittents until 57 is out the door.
There have been 31 failures in the last week.

Number of failures per platform: 
 - 11 linux
 - 11 linux32-stylo-disabled
 - 5 linux64-ccov
 - 4 Linux x64
 
Number of failures per buid type: 
 - 26 debug
 - 5 opt

I see a peak in number of failures on November 24th and from that point on the number of failures is decreasing.

Here is an example of a recent log: https://treeherder.mozilla.org/logviewer.html#?repo=autoland&job_id=147635673&lineNumber=13605
and a snippet from it:

[task 2017-11-25T22:55:00.041Z] 22:55:00     INFO - PID 5511 | 1511650500033	Marionette	INFO	Testing http://web-platform.test:8000/css/css-backgrounds/background-size-031.html == http://web-platform.test:8000/css/css-backgrounds/reference/background-size-031-ref.xht
13602
[task 2017-11-25T22:55:00.117Z] 22:55:00     INFO - PID 5511 | ++DOMWINDOW == 83 (0xdf4c4000) [pid = 5618] [serial = 83] [outer = 0xe7997040]
13603
[task 2017-11-25T22:55:00.259Z] 22:55:00     INFO - PID 5511 | ++DOMWINDOW == 84 (0xdf4c4c00) [pid = 5618] [serial = 84] [outer = 0xe7997040]
13604
[task 2017-11-25T22:55:00.356Z] 22:55:00     INFO - PID 5511 | 1511650500351	Marionette	INFO	Found 2388 pixels different, maximum difference per channel 22
13605
[task 2017-11-25T22:55:00.400Z] 22:55:00     INFO - TEST-UNEXPECTED-FAIL | /css/css-backgrounds/background-size-031.html | Testing http://web-platform.test:8000/css/css-backgrounds/background-size-031.html == http://web-platform.test:8000/css/css-backgrounds/reference/background-size-031-ref.xht
13606
[task 2017-11-25T22:55:00.404Z] 22:55:00     INFO - REFTEST   IMAGE 1 (TEST): data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAlgAAAJYCAYAAAC+ZpjcAAAgAElEQVR4nO3dbaxkdX3A8f+9LLDbsourialN+kKMBhSfqm21b3ajqcE0Sompxm66iMgLNDERH1DTV6Y21Gr0Bbyw1bKEriy4y0IRUgJlEQUEFRcwKqHLg92KLS4Li3vdpzu/vph7997dvXPnnJnfnDn37ueTTMLOnDnnf87MnPly7rnnlgAAIFUZ9wAAAJYbgQUAkExgAQAkE1gAAMkEFgBAMoEFAJBMYAEAJBNYAADJBBYAQDKBBQCQTGABACQTWAAAyQQWAEAygQUAkExgAQAkE1gAAMkEFgBAMoG1TJTipZxlWwAw3zi+FyotsZTS9zaSwfmirKzXtqr6OvV7/lIyivGOcxssxddgUHXer4Psg0Y17VJ2sqwny0/dz/Ni8xjF56ByYNX5dxYf+uHUeZ0Gfexk0IYvoHEvvwlV36+9Xo+qr1OdbXkybPeIk2c9WX6Gee+OumUE1jKW9UVyMr8Os+s+7m0w7uU3Ydj/E606D4F1opNlPVl+MgMr20Bz7zeoKofc+h3ir/ujyPmPD/pjhkHGWXWazOX1es6g8+m3rbO3a9XXapBDt8O+RovNdxCZ74uFntNrXr3u6/X6Dvta9ZP1GaqynKrTjHq7H//YKLdbneUsNF2dcTX1+c5ctyrvuUHXv87r3uuzt9i61x131vavs66jHGe/eVb9zPdb/0H3OYsud6An9dkA/e6r+oYadGc06JgGGecw6zLodllI1fWu8/zFxjDMtl5snnXn02u8gyy/zjwHHccg74s69y02bdZ7vtd9vQy7PlXVGU+T273q8/tNV+fLuO46juK1rPP5bnrdBn1dht3fZ7+f6m63uvOuMt+FjPv1rTKmXvdlSg2sfl/SdZ9fd+WrLr/qNFXGOcw0g8yrynzqPK/KtBmv1zDTVn2s7rJGGViZ74te047ii37Y5dSZZtjP++xz6uxkM8Y0is9DVeMcTxOvZdVljmPfU3U8dec/7P6p7r6/ie+jqs9r8rN0UgXWsB/oYZZfdZoq46w6TZXH6n5R9FuGwJr797AfpMz3YpVpBFZ9w+xo64xJYA23rDrzb9O+Z/bxrP19v+nbEFiZATLuz9KyCKzFbotNW2c5dce10GNVxlhnnHXmMex2qbJ+ve6r8/x+j1Xdpr3Wc7HlVZ1P1fEOul2rjHfQ6QVWbmA1Nabsz0PV+xfbvwy6LlXHPsjzF9OmdRtk/zLImHs9VvdzPMx2W2we2d9H43p960yXsc9ZdJkDPWmIHe5i8xx2h1v1RRj2C6PqzqPOfIdZXq9l1P1A1H1smNcr4wMzzPMH2WGMYgxVH8vYMWfet9j9VacZ9vM+rjGN4vMw6DKyxtPkaznudcv+Uq2zvx/FOmZEXlVVv/+q3i+wFnrSADvcQQJn3IFVZbphphl0XFWnaVNgZX1gBtkWGf9DkPVeHHRHLLAGm+coxtTEl0Ibv6ROhsDKeF2WcmBlfB+N+/WtM92SCqxejy20sapOV3W5dec76H3Hz2/Qdak6rl7PrTLNMAEx6Adt2C/tYZ7Ta7phtusg0y42jirzrXPfMOvaRGBlLGeh6Zp6flPbPXsZdZbda77DznOxeYx73TJfl4WeW3cso/yOWsgov4/a8PoOM89Mtec+u0JVPkCLTXf8NFWmqzK2QZc//7664xxkXYbdLr2eU2dZdefTb9qqy646tqx1WGheg8yvzvPqrEOd7VXluVVew2Hu6zemYdex7nt+kHHUHVOV8WatW51lZH0+m34tm163Qdajl6rvvWHH0m/dq2y3xdZrkHUY5j0yzvfuQvMc9PWvY7T51rBRbSQAlqY2fS+0aSyM3rJ6tb15AU5OvY54tEnbxsNoLZtXe9SH+gBotyZ+7DOoto6L0fFKAwAkE1gAAMkEFgBAspMysPwMnEF43wBQVeVvjH7XyVhKltp426bq9VWGnX+d66NkWuwaLONYblu0fXwnI6+Jk8dpr6EDq99jLD+jDqzF5tHUjnRc7+m2f5baPr6TkddEaNJOAovaxn0USWCNT9vHdzLymggs2qnSO3Khy+Uff8n5haarOr9ej/ebdphL+FdZx6rzGnb96+wYspY56LyrvBeqLmOQo2CLLa/uWBYbU6/3Wr/nLva8qq9B1TH2enyx+6qOo9d0dT9ng7wfeq1Hr+dWnW+v51Z5/ep+TjPH0G+eVcaVtd+osw3qLLPOcqq8Z6AN0o5g1flCqnrfQvOsO59e4x1k+b3mVXf9q06bucwq9/Wbd9XxZo6jyv39vmgXU+e1H/R9k/FZyJpf9mdyIYN8nvsto87rO+xnsc68exnF/mCY12T+tG3ZVw2ynH5xBm2SFlhV7q8zj0GW1++xusuqE1hV7u8XAnV3lMPMp864606TsS4Z9w8SV4s9Nuj7Jut9X3d8x99XdRzZn6+MZQzzGa07z2EeyxpD1v7w+OnatK+qu5ysMUFTBFYM9+Fs206r3//hHX8bZByD7jQzxtG2wBomOHrNs4pBIqbJwBr1tq8znrrzHOaxrDEILIHF0td4YFX9kq3yoaz6ZV1lvHXncfzzqtw/jsDKCpg609RZ9+z7q4ZCv8erfhlWed8MG1h13uf9AqvK5ybzy3yx16vOMups26rba9T7mGHGUHd/WEXb9lXzH6/6vswYEzSlNUewMpaXMe/50wwzncBqNrCOf6zqa1fnsUHfD8MEVt1oW2z967y/Bnms3zyHibhh9zEZkVdVxhhOtsAadl4CizZqTWBl7VAG+WKt+yV2/DRt2mkJrOGOZC722KDvmyYDa/axYYIi+/OVsYw2BVaVz8ewYxBYc49njwmaMnBgVdlZVt2h1N35DfqcXtMN8oVU5bkZO7PFllllPk0FVr8dYROBVXX8vcZdZdmjnK7qc4fZBsN+Jof9TAy6jGHe74N+vjP3CXXHkP2a9BvDqPdVg46rzpiG3T6QrdY7b6E3cr8v+8Xe9IM+Xne6KuM6fl51dqSLbZMq61d3BzD/OVXms9DyBhn3sOs+7DjqrGsdC813kPd5vy+KOu+LuuNbaHl11rfqdIN+Nqq+Nxcbf9Xx1H09626DKrLeU4Nug6rrUec9mbENqk476Puy17aEcfDOW4LsNBbXtm3T9Hjatv51LfXxA0QIrCVJYB3r+P8jb5smxtTv6M9SshzWAcCebIkZ9HD9ctbmbdLUuNq6/nUtl/UAsBcDAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBI1jewSln6t3Xn7GhgUwIAdC37wFp3zo6IzQ7UAQDNWdaBdTSuBBYA0KBlG1jHxJXAAgAatCwD64S4ElgAQIOWXWAtGFcCCwBo0LIKrJ5xJbAAgAYtycA666xd8dnP/kNcccWnY926HbFmzfOLx5XAAgAatMQCqxOXXPL12Ldvdeza9cr4+c/Pjr17XxL33/O2iG8vElcCCwBo0BIKrE5s3HhNTE2tii1bPhAve9mzsXLlVFx77YbuajwksACAdlgygbVixaG4+uoLY/fuP4zTTjsQk5NH4jOfuqK7Ck/NRNQNAgsAGL8lE1innnowdu58Y2zatDFOOeVwfO7yL0YcKRFPlIjrS8R1M6F1m8ACAMar5YHViYmJztH/vv32v4jnn18Tn/jEV7pDf2IuoDpbS0SnRHxPYAEA49XqwPrwh78R11zzt7F69QtRSsTrX/9I/Pa3v9cNqSdnjlxtLt0T3HeXiP0lYqvAAgDGq7WBdcYZ++LLX74sOp0S27f/VUxOHol3nntHxI9Ld9j/XSLuKREPlohfzxy9uqdHXAksAKBBLQysTlx88TfiwQffGs8995LYt291TE9PxG3bz5s7YrVz5mhVZ+b2Yom4d5G4ElgAQINaF1gbNlwbU1OrYuvW98X69TvizW/+cXzzqxd1T2if/2PBbSXi7hJxV+l/DSyBBQA0qFWBddppB+LWW98djz326njpS/dEKfP+/M2PSne4T1eIKYEFAIxRywLrYOzadVZcddWlUUrn2D9/860S8ZuZyJp/JEtgAQAt06rAKqUTDz/8hnjooTfFeX9824mR9GTpnm81e5K7wAIAWqhlgRXx+c//fRw6tCLi4ZmjVrOB9O2ZuHqoRDxaIo6U6NwksACA9hlrYE1OHonTT/9dTE4eOXrfu99ya8RjJWK6RPyiRNxRupdf+J8ScbBEfKdE3F4iDpeI+wUWANA+YwusSy+9Ku699+3R6ZTYu/fM2LRpY7z3bTfNBdHjpTu86dK9FMNUmbsUw/dm7v++wAIA2mcMgdWJSy75ekxNrYpt2y6IjRs3xeWXXxFTUysjnilzV2K/rkTcXCLumwmpG2fu31q6003Nu09gAQAt0nBgdeK8826LiBI33fTeWLHicJTSiU9+8p+6Q+mUiD0l4oYFAulb8+LqYIn4bo24ElgAQIMaDayJiem48sqPxn33vT3WrHk+JiePxOcu/2L3IqJPzPwI8FA59kjW7O3fS8SzJWLvAHElsACABjV8BCviNa/5RXzkI/8c3T+J8y/dITwxL4QOlO59vynH/hbh5hJxS6l//SuBBQA0rPHAmv1R4brX7YjO7HWtZkPqztI9ef0npf/fFhRYAEBLjTiwOrF+/V1xwQXb4uyzfxYrV05FKTN//ua6mR/3PV7mrnO1u0Q8VyK2JMeVwAIAGjTSwPrYx66M/ftXRadTYv/+VXHzze+Jd5x7Z8TmEp3NJeJXpTuE/yrR+d/SPcm9zqUXBBYA0EIjC6xXvOJX8cwzfxDbt58fl132lfjSlz7VvRTDU2XuPKobSvf8q/8r3XOuRhVXAgsAaNAIAqsTpXTioou+Gfff/2dHL8Ww/rV3Rfxw5ojVL8uJJ7CP+gYA0JDkwOrEu971H3HhhVfHxz/+tdiy5f0xMTHdPedqNnR+VLqXZXiyDP4bgQILAGixxMCau0L79PREHDhwejzyyLnxN+/cfGLsPDDvSJbAAgCWmbTAOvXUg/H0038U11//13H++dvjgx/cHC+++Pvd3wrctkDw/LCM9pwrgQUAjElCYHVi48Zr4gtf+Lt49NHXxdq1z81diuE/S/fCoc+Wen83UGABAEtY7cB6+ct/HWeeuffov1evfiF+8IM/jenpiXjggT+JU045fOw5V22JLACAhtQKrFWr9sfdd6+LW275y1i7ds/R+1/1qsdj5843xJ49a+MzF//jiXFze4k4XCJeKM3/9qDAAgAaVvsI1oYN/xb79p2xYGT97KdnR/yuRNyxQODcWSLucQQLAFj+agdWr8had86OiJtL96T2XpE1zhsAQEMGCqxSIj70oX89GlkX/PmNcyHT1sgCAGjIwIE1/0hW7C7dP9Z8fGS9WCK2tiCuBBYA0KChAmvdOTu651V1ysKRdVcLwkpgAQANGziwjrkUw/dLxKEFIqtNNwCAhgwUWMfE1fGR9VQZ36UYBBYA0AK1A2vBuJq93VfGeykGgQUAtECtwFo0rtp+AwBoSOXAWtJxJbAAgAZVCqwlH1cCCwBoUN/yWBZxJbAAgAZVOoK11G/rztnRwKYEAOha9oF19AgcAEBDlnVgHfPjTQCAhizbwDrh3DEAgIYsy8Ba8MR8AICGLLvA6vlbjwAADVlWgbXoJSUAABqyJAPrrLN2xWc/+w9xxRWfjnXrdsSaNc/3v14XAEBDllhgdeKSS74e+/atjl27Xhk///nZsXfvS+L+e94W8e1F4kpgAQANWkKB1YmNG6+JqalVsWXLB+JlL3s2Vq6cimuv3dBdjYcEFgDQDksmsFasOBRXX31h7N79h3HaaQdicvJIfOZTV3RX4amZiLpBYAEA47dkAuvUUw/Gzp1vjE2bNsYppxyOz13+xYgjJeKJEnF9ibhuJrRuE1gAwHi1PLA6MTHROfrft9/+F/H882viE5/4SnfoT8wFVGdrieiUiO8JLABgvFodWB/+8Dfimmv+NlavfiFKiXj96x+J3/7297oh9eTMkavNpXuC++4Ssb9EbBVYAMB4tTawzjhjX3z5y5dFp1Ni+/a/isnJI/HOc++I+HHpDvu/S8Q9JeLBEvHrmaNX9/SIK4EFADSohYHViYsv/kY8+OBb47nnXhL79q2O6emJuG37eXNHrHbOHK3qzNxeLBH3LhJXAgsAaFDrAmvDhmtjampVbN36vli/fke8+c0/jm9+9aLuCe3zfyy4rUTcXSLuKv2vgSWwAIAGtSqwTjvtQNx667vjscdeHS996Z4oZd6fv/lR6Q736QoxJbAAgDFqWWAdjF27zoqrrro0Sukc++dvvlUifjMTWfOPZAksAKBlWhVYpXTi4YffEA899KY4749vOzGSnizd861mT3IXWABAC7UssCI+//m/j0OHVkQ8PHPUajaQvj0TVw+ViEdLxJESnZsEFgDQPmMNrMnJI3H66b+LyckjR+9791tujXisREyXiF+UiDtK9/IL/1MiDpaI75SI20vE4RJxv8ACANpnbIF16aVXxb33vj06nRJ7954ZmzZtjPe+7aa5IHq8dIc3XbqXYpgqc5di+N7M/d8XWABA+4whsDpxySVfj6mpVbFt2wWxceOmuPzyK2JqamXEM2XuSuzXlYibS8R9MyF148z9W0t3uql59wksAKBFGg6sTpx33m0RUeKmm94bK1YcjlI68clP/lN3KJ0SsadE3LBAIH1rXlwdLBHfrRFXAgsAaFCjgTUxMR1XXvnRuO++t8eaNc/H5OSR+NzlX+xeRPSJmR8BHirHHsmavf17iXi2ROwdIK4EFgDQoIaPYEW85jW/iI985J+j+ydx/qU7hCfmhdCB0r3vN+XY3yLcXCJuKfWvfyWwAICGNR5Ysz8qXPe6HdGZva7VbEjdWbonr/+k9P/bggILAGipEQdWJ9avvysuuGBbnH32z2LlyqkoZebP31w38+O+x8vcda52l4jnSsSW5LgSWABAg0YaWB/72JWxf/+q6HRK7N+/Km6++T3xjnPvjNhcorO5RPyqdIfwXyU6/1u6J7nXufSCwAIAWmhkgfWKV/wqnnnmD2L79vPjssu+El/60qe6l2J4qsydR3VD6Z5/9X+le87VqOJKYAEADRpBYHWilE5cdNE34/77/+zopRjWv/auiB/OHLH6ZTnxBPZR3wAAGpIcWJ1417v+Iy688Or4+Me/Flu2vD8mJqa751zNhs6PSveyDE+WwX8jUGABAC2WGFhzV2ifnp6IAwdOj0ceOTf+5p2bT4ydB+YdyRJYAMAykxZYp556MJ5++o/i+uv/Os4/f3t88IOb48UXf7/7W4HbFgieH5bRnnMlsACAMUkIrE5s3HhNfOELfxePPvq6WLv2ublLMfxn6V449NlS7+8GCiwAYAmrHVgvf/mv48wz9x799+rVL8QPfvCnMT09EQ888CdxyimHjz3nqi2RBQDQkFqBtWrV/rj77nVxyy1/GWvX7jl6/6te9Xjs3PmG2LNnbXzm4n88MW5uLxGHS8QLpfnfHhRYAEDDah/B2rDh32LfvjMWjKyf/fTsiN+ViDsWCJw7S8Q9jmABAMtf7cDqFVnrztkRcXPpntTeK7LGeQMAaMhAgVVKxIc+9K9HI+uCP79xLmTaGlkAAA0ZOLDmH8mK3aX7x5qPj6wXS8TWFsSVwAIAGjRUYK07Z0f3vKpOWTiy7mpBWAksAKBhAwfWMZdi+H6JOLRAZLXpBgDQkIEC65i4Oj6ynirjuxSDwAIAWqB2YC0YV7O3+8p4L8UgsACAFqgVWIvGVdtvAAANqRxYSzquBBYA0KBKgbXk40pgAQAN6lseyyKuBBYA0KBKR7CW+m3dOTsa2JQAAF3LPrCOHoEDAGjIsg6sY368CQDQkGUbWCecOwYA0JBlGVgLnpgPANCQZRdYPX/rEQCgIcsqsBa9pAQAQEOWZGCdddau+Oxn/yGuuOLTsW7djliz5vn+1+sCAGjIEgusTlxyyddj377VsWvXK+PnPz879u59Sdx/z9sivr1IXAksAKBBSyiwOrFx4zUxNbUqtmz5QLzsZc/GypVTce21G7qr8ZDAAgDaYckE1ooVh+Lqqy+M3bv/ME477UBMTh6Jz3zqiu4qPDUTUTcILABg/JZMYJ166sHYufONsWnTxjjllMPxucu/GHGkRDxRIq4vEdfNhNZtAgsAGK+WB1YnJiY6R//79tv/Ip5/fk184hNf6Q79ibmA6mwtEZ0S8T2BBQCMV6sD68Mf/kZcc83fxurVL0QpEa9//SPx29/+Xjeknpw5crW5dE9w310i9peIrQILABiv1gbWGWfsiy9/+bLodEps3/5XMTl5JN557h0RPy7dYf93ibinRDxYIn49c/Tqnh5xJbAAgAa1MLA6cfHF34gHH3xrPPfcS2LfvtUxPT0Rt20/b+6I1c6Zo1WdmduLJeLeReJKYAEADWpdYG3YcG1MTa2KrVvfF+vX74g3v/nH8c2vXtQ9oX3+jwW3lYi7S8Rdpf81sAQWANCgVgXWaacdiFtvfXc89tir46Uv3ROlzPvzNz8q3eE+XSGmBBYAMEYtC6yDsWvXWXHVVZdGKZ1j//zNt0rEb2Yia/6RLIEFALRMqwKrlE48/PAb4qGH3hTn/fFtJ0bSk6V7vtXsSe4CCwBooZYFVsTnP//3cejQioiHZ45azQbSt2fi6qES8WiJOFKic5PAAgDaZ6yBNTl5JE4//XcxOXnk6H3vfsutEY+ViOkS8YsScUfpXn7hf0rEwRLxnRJxe4k4XCLuF1gAQPuMLbAuvfSquPfet0enU2Lv3jNj06aN8d633TQXRI+X7vCmS/dSDFNl7lIM35u5//sCCwBonzEEVicuueTrMTW1KrZtuyA2btwUl19+RUxNrYx4psxdif26EnFzibhvJqRunLl/a+lONzXvPoEFALRIw4HVifPOuy0iStx003tjxYrDUUonPvnJf+oOpVMi9pSIGxYIpG/Ni6uDJeK7NeJKYAEADWo0sCYmpuPKKz8a99339liz5vmYnDwSn7v8i92LiD4x8yPAQ+XYI1mzt38vEc+WiL0DxJXAAgAa1PARrIjXvOYX8ZGP/HN0/yTOv3SH8MS8EDpQuvf9phz7W4SbS8Qtpf71rwQWANCwxgNr9keF6163Izqz17WaDak7S/fk9Z+U/n9bUGABAC014sDqxPr1d8UFF2yLs8/+WaxcORWlzPz5m+tmftz3eJm7ztXuEvFcidiSHFcCCwBo0EgD62MfuzL2718VnU6J/ftXxc03vyfece6dEZtLdDaXiF+V7hD+q0Tnf0v3JPc6l14QWABAC40ssF7xil/FM8/8QWzffn5cdtlX4ktf+lT3UgxPlbnzqG4o3fOv/q90z7kaVVwJLACgQSMIrE6U0omLLvpm3H//nx29FMP6194V8cOZI1a/LCeewD7qGwBAQ5IDqxPvetd/xIUXXh0f//jXYsuW98fExHT3nKvZ0PlR6V6W4cky+G8ECiwAoMUSA2vuCu3T0xNx4MDp8cgj58bfvHPzibHzwLwjWQILAFhm0gLr1FMPxtNP/1Fcf/1fx/nnb48PfnBzvPji73d/K3DbAsHzwzLac64EFgAwJgmB1YmNG6+JL3zh7+LRR18Xa9c+N3cphv8s3QuHPlvq/d1AgQUALGG1A+vlL/91nHnm3qP/Xr36hfjBD/40pqcn4oEH/iROOeXwsedctSWyAAAaUiuwVq3aH3ffvS5uueUvY+3aPUfvf9WrHo+dO98Qe/asjc9c/I8nxs3tJeJwiXihNP/bgwILAGhY7SNYGzb8W+zbd8aCkfWzn54d8bsScccCgXNnibjHESwAYPmrHVi9ImvdOTsibi7dk9p7RdY4bwAADRkosEqJ+NCH/vVoZF3w5zfOhUxbIwsAoCEDB9b8I1mxu3T/WPPxkfViidjagrgSWABAg4YKrHXn7OieV9UpC0fWXS0IK4EFADRs4MA65lIM3y8RhxaIrDbdAAAaMlBgHRNXx0fWU2V8l2IQWABAC9QOrAXjavZ2XxnvpRgEFgDQArUCa9G4avsNAKAhlQNrSceVwAIAGlQpsJZ8XAksAKBBfctjWcSVwAIAGqQ8AACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCXfUFR4AAASaSURBVCwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBI9v/NeAG0AATMwwAAAABJRU5ErkJggg==
13607
[task 2017-11-25T22:55:00.409Z] 22:55:00     INFO - REFTEST   IMAGE 2 (REFERENCE): data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAlgAAAJYCAYAAAC+ZpjcAAAgAElEQVR4nO3de4ylZWHH8eecmcUsoqQY29jYGGIMsVVJbU1sNfFWtN5rxVi0oCm6Ypv6V5MGNRqt8V8T65XKXtllh+UmKAreQQui0ZhYqxRFRKkrAq5Cd3dmzvvrH+/Z3bOzOzPnzDxz5uzy+SQnzM55z3s51y/P+857SgAAqKqs9woAAJxsBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBNZJohQP5SHuCwAGrcfnwlBLLKUse1mTlfNBObTF7qthH6flbn8iWYv1Xc/74ER8DFZqlOfrSt6D1mraE9kjZTs5+Yz6el5qHmvxOhg6sEb5dy1e9KszyuO00useCSbhA2i9lz8Owz5fF3s8hn2cRrkvHwn3e/LI2U5OPqt57q51ywisk1itD5JH8uNwaNvX+z5Y7+WPw2r/T3TYeQisYz1StpOTT83Aqm1Fc19upYYZcltuiH/UXZGD1690N8NK1nPYaWoub7HbrHQ+y93Xte/XYR+rlQzdrvYxWmq+K1HzeXG82yw2r8V+t9jju9rHajm1XkPDLGfYadb6fl943Vreb6Ms53jTjbJe43p919y2YZ5zK93+UR73xV57S237qOtd6/4fZVvXcj2Xm+ewr/nltn+l7zlLLndFN1rmDljud8M+oVb6ZrTSdVrJeq5mW1Z6vxzPsNs9yu2XWofV3NdLzXPU+Sy2vitZ/ijzXOl6rOR5Mcrvlpq21nN+sd8tZrXbM6xR1mec9/uwt19uulE+jEfdxrV4LEd5fY9721b6uKz2/b7282nU+23UeQ8z3+NZ78d3mHVa7Hc1VQ2s5T6kR739qBs/7PKHnWaY9VzNNCuZ1zDzGeV2w0xb4/FazbTDXjfqstYysGo+Lxabdi0+6Fe7nFGmWe3r/dBtRnmTrbFOa/F6GNZ6rs84Hsthl7ke7z3Drs+o81/t+9Oo7/3j+Dwa9nbjfC09ogJrtS/o1Sx/2GmGWc9hpxnmulE/KJZbhsA68u/VvpBqPheHmUZgjW41b7SjrJPAWt2yRpn/JL33HLq+1vv9ctNPQmDVDJD1fi2dFIG11GWpaUdZzqjrdbzrhlnHUdZzlHms9n4ZZvsW+90ot1/uumHv08W2c6nlDTufYdd3pffrMOu70ukFVt3AGtc61X49DPv7pd5fVrotw677Sm6/lEnatpW8v6xknRe7btTX8Wrut6XmUfvzaL0e31Gmq/Ges+QyV3SjVbzhLjXP1b7hDvsgrPYDY9g3j1Hmu5rlLbaMUV8Qo163mserxgtmNbdfyRvGWqzDsNfVeGOu+bulfj/sNKt9va/XOq3F62Gly6i1PuN8LNd722p/qI7yfr8W21gj8oY17OffsL8XWMe70QrecFcSOOsdWMNMt5ppVrpew04zSYFV6wWzkvuixv8Q1HourvSNWGCtbJ5rsU7j+FCYxA+pR0Jg1XhcTuTAqvF5tN6P7yjTnVCBtdh1x7uzhp1u2OWOOt+V/m7h/Fa6LcOu12K3HWaa1QTESl9oq/3QXs1tFptuNffrSqZdaj2Gme8ov1vNto4jsGos53jTjev247rfay9jlGUvNt/VznOpeaz3ttV8XI5321HXZS0/o45nLT+PJuHxXc08axp57oc2aJgX0FLTLZxmmOmGWbeVLn/wd6Ou50q2ZbX3y2K3GWVZo85nuWmHXfaw61ZrG443r5XMb5TbjbINo9xfw9x2mMdwNb9bbp1Wu42jPudXsh6jrtMw61tr20ZZRq3X57gfy3Fv20q2YzHDPvdWuy7Lbfsw99tS27WSbVjNc2Q9n7vHm+dKH/9RrG2+jdla3UkAnJgm6XNhktaFtXdSPdqevACPTIuNeEySSVsf1tZJ82iv9VAfAJNtHLt9VmpS14u145EGAKhMYAEAVCawAAAqe0QGln3grITnDQDDGvoTY7nzZJxITrT1nTTDnl9ltfMf5fwoNS11Dpb1WO6kmPT1eyTymDh4nMm16sBa7jpOPmsdWEvNY1xvpOv1nJ7019Kkr98jkcdEaDKZBBYjW+9RJIG1fiZ9/R6JPCYCi8k01DPyeKfLX3jK+eNNN+z8Frt+uWlXcwr/YbZx2HmtdvtHeWOotcyVznuY58Kwy1jJKNhSyxt1XZZap8Wea8vddqnbDfsYDLuOi12/1O+GXY/Fphv1dbaS58Ni27HYbYed72K3HebxG/V1WnMdlpvnMOtV631jlPtglGWOspxhnjMwCaqNYI3ygTTs7443z1Hns9j6rmT5i81r1O0fdtqayxzmd8vNe9j1rbkew/x+uQ/apYzy2K/0eVPjtVBrfrVfk8ezktfzcssY5fFd7WtxlHkvZi3eD1bzmAxOOynvVStZznJxBpOkWmAN8/tR5rGS5S133ajLGiWwhvn9ciEw6hvlauYzynqPOk2Nbanx+5XE1VLXrfR5U+t5P+r6LfzdsOtR+/VVYxmreY2OOs/VXFdrHWq9Hy6cbpLeq0ZdTq11gnERWFndi3PS3rSW+z+8hZeVrMdK3zRrrMekBdZqgmOxeQ5jJREzzsBa6/t+lPUZdZ6rua7WOggsgcWJb+yBNeyH7DAvymE/rIdZ31HnsfB2w/x+PQKrVsCMMs0o217798OGwnLXD/thOMzzZrWBNcrzfLnAGuZ1U/PDfKnHa5RljHLfDnt/rfV7zGrWYdT3w2FM2nvV4PXDPi9rrBOMy8SMYNVYXo15D06zmukE1ngDa+F1wz52o1y30ufDagJr1GhbavtHeX6t5Lrl5rmaiFvte0yNyBtWjXV4pAXWauclsJhEExNYtd5QVvLBOuqH2MJpJulNS2CtbiRzqetW+rwZZ2Adum41QVH79VVjGZMUWMO8Pla7DgLryPW11wnGZcWBNcyb5bBvKKO++a30NotNt5IPpGFuW+PNbKllDjOfcQXWcm+E4wisYdd/sfUeZtlrOd2wt13NfbDa1+RqXxMrXcZqnu8rfX3XfE8YdR1qPybLrcNav1etdL1GWafV3j9Q20jPvOM9kZf7sF/qSb/S60edbpj1WjivUd5Il7pPhtm+Ud8ABm8zzHyOt7yVrPdqt3216zHKto7iePNdyfN8uQ+KUZ4Xo67f8ZY3yvYOO91KXxvDPjeXWv9h12fUx3PU+2AYtZ5TK70Pht2OUZ6TNe6DYadd6fNysfsS1oNn3gnIm8bSJu2+Gff6TNr2j+pEX3+ARGCdkATW0Rb+H/mkGcc6LTf6cyI5GbYBwDvZCWalw/Uns0m+T8a1XpO6/aM6WbYDwLsYAEBlAgsAoDKBBQBQmcACAKhMYAEAVCawAAAqE1gAAJUJLACAygQWAEBlAgsAoDKBBQBQmcACAKhMYAEAVCawAAAqE1gAAJUJLACAygQWAEBlywZWKSfapelfkqnubEpJ/upPbhzHfQkAkOSkC6wjcTXdj6uXn31d9u86ZRz3JQBAkpMqsAbj6mBKSV76jM9kfqaT3uXLbiYAQDUnSWAdG1cve8b1mZvpprerk972zjjuSwCAJCdcYDULgmrwskhc7egkO41gAQDjc4IE1pGIKqVJp9PL9PRspqdnM9WdTynN4WOujhm52lnSXCawAIDxOQECqznq506nd8w0nTK/dFwZwQIAxugECKzBEaw2rs466wd597vflw996B159auuTuk2+eunfTZzM1PHjyuBBQCM0QkQWIfiqh2letvbPpb9+0/J3r2Pz0/v+qM8+JvH5pavPCf7d5+SXF6OH1cCCwAYowkNrKMPZu9251JK8prXXJmkZOfOv8vjHverlJLs2vn6JCXzt3fSu7SNqWYwrAQWADBmExhYzYKfm3S78+l05jMzc27uvfcPsmHDgZSS/Ou/fDBzvanM3jGV3mWdZHdJs6M/emUECwBYJxMYWAtHsHoppcn09IHcffcTs2fP36aU5N0Xvy9JSfPjflDtKsldJbmpJNuMYAEA62eCAmvwnFa9Y0aypqcP5nM3vDh33/3EvPPif0tSMn9nN73tJdla0lxd0qQk/1nSbBZYAMD6mZDAGoyp3uFjrjZt+ni2bjs/Gzc+nFKSpzz1h9m377QkJb0fdtJsLcnmkubykubnJc3+klxV0mwXWADA+pmQwBocwWpPxbBhw4FccsmFSUqumHltSklecfanc+B7GzLXdNP8oiRfKck3SnJfaTfl5oHdg47BAgDWyYQE1qGwauPqTW/anO9972m5554/zAP3n56k5IarXpLffurRyY6S3ndLcrDdJdikJA+XNN84smvwmFM0CCwAYIwmILDakatOp90teO65M2maks997sV5wfO/lKc89UfZ/bHXJSlp7urH07aS5pqSfK0/irWnHH1gu8ACANbRBARWDo9enXLKgXzpS8/PXXc9KY897cGUkrzwrC/k1584Iwe/OZ1eOsld7TFWzbbSHoO1tT01wzFBJbAAgHUyAYF15LirjRsfyi/3Pj5bNl+QUpJXPvPa9PaUZHdJb1tJ7i3tKv+ktKdlOHR6hqVGrgQWADBm6xBYzVGX9sube+l0epmens0Pvn9Wvn7rX+Q1z7oys5dPpbe7tCcR3d7fRfhwSZqS5u7+yNVli5xYVGABAOtkjIF1KKran7vd+aOun54+mG5nPm//53/PXLppvluS7SW9LZ1kS0kzU5IDJc23S/Ld/gHu15f2eKzldg8KLABgjNZlBKsdtWr/fcYZ9+XUU393+N+ve/buzP24m/l00vvvktxY0nylJL8syXxJri3JDf2fby/JlgW7CQUWALDOxhRYC3cJJhdcsCW33/7nmZ2dyn33nZEPffgdueBFW3Jw63SaHe2JRHPoNAxpR69ya0nzHyX5WknTlDRfL+1B7kawAIAJMuZdhG1cbdr08STtqRje+MbteefFH2hX5f6SZk//rwN3ljSfaaMqXy/JNSXZXJLdJdnbD66r292Iyx5/JbAAgDEa2wjWoa+/OeeczycpmZk5N1NT7e/+6R8/nIOzGzKXbnq/aiOq2d6Pp6393YCX9YNqb0l6JfnqCHElsACAMVrDwGoG/ttLpzOfUpps2/b3ueOOJ+fR/eOu3vuu9yQp6d1R0ru5tH8heG9JZkqyo7THV+0o7bFXvyzJvn5cHYquYeJKYAEAY7RGgXX0Xwt2u/OZmppNKU2e85ybc+GFl6SU5KJ/+GiSkrkfddPb2sZUs6+0q/WrfkANXj7bD6/lztousACAdbQGgXUkrgb/WnDh5eVPvy6zv5jO3G+76e0syaUlzRdLsr8k/1WS2/q7AAdDalt/NGvUuBJYAMAYVQ6swd2C7c/Pe96Xc955l+Xss7+Tqe5sSmnP0D53ZTe9/SW9O0vyqZJmV0nu7Y9gXbZgF+DxQmqUuBJYAMAYVQyswZGr9iSiF1300f4i2sul296cl//pdeldXtLbVTL/0/6pGP6npPl1O01zy8AuwMGIWuxngQUATJgKgXX0GdqnpmYzNTWXM8+8Mw89tDHXX/+yvG3Tx/KBD16c2WYquae0gbWlk8yUNHeW5MGS3F/a0zFsWWVICSwAYJ1VHcEaPObqrW/9RL7znbNzyvSBlJK84MlfzMO3b2xPxfCT0n634PaS7Gi/U7DZ0f/dWsSVwAIAxmgVgdUsuLRx9ZKX3JBNmz6R97//Xfnkx9+SUpJX/dk16e3ppNlW0ru9vyvwxwMhtfDLmmvHlcACAMZohMA69gD29tLL1NRsOp1eLrzwkgwec/WNbz47b3npJzO3fSrNTElvR7sLsLmtP83P2hGsNY8rgQUAjNGQgXUkqDqdXn9XYHNUbE1NzWbv3sfl2mtfmVe/6pq84fwd+c2+xyQPlTRXliPnudpZ2q+8+XZJvlWOPqfVWl4AAMZkxF2Eg6E13x+9mssb3rAjH/nI2/Otbz0zZ/zefSklee6TvpqHvnhqZg9OpXdfSfaUNIMxta20X4OzcPegwAIATnAjjGC1x1g94Qk/z+mnP3A4sk499Xe57bZnJSn5wo0vTCnJK5756TRXtQex924qyVz/NAxXlqNPHjrOCwDAmAwZWO0uwUc9an9uueUvc9NNL8ppp+07fN2ZZ96ZH3z/rDyw7/S856L35uDmDWl2l/T6JwxtbiztFzQ/WI49zmoco1cCCwAYo5FGsLrd+bz5zZcmKbn++pflMY/Zd3iX4e8/4X9zxw+fnKQNqmZzGzbNzjay8uWS3FqOPajdCBYAcJIZMrCO3k143nmX5VBkHdpdeM4ffz4PXHF6Dt4/nd6BTnJTSTP4dTfby5FjruwiBABOYiOMYLX/7XbnUkpy/vlbk5TMXHVuXv/cXeldXpJdJb2rSvKbkuZgSb7QD6uFUWUECwA4iY04gnV0ZP3Na6/K/x18VLK3tMdcbW+Dqrm6JPtK8nBJrujvFlzPuBJYAMAYjRhYbWRNTx1MKcmLnnJTHrp5Y+Yyld7PSnJ5/ytvtpXkupJ8tYzvNAwCCwCYEEMG1kBcddu4etkzrk/viv7X39xckqakuacku/tBdSi0JiGuBBYAMEZDBNbx42puppveroGvv7m5tLO7e8EuwfXcLSiwAIB1MGRgHS+uOult7yQ7B77+5taS3FaOPrB9EuJKYAEAYzREYCXT3dnF42owoLaVydotKLAAgHWwbHlM9ePqpc/4zOJxdbygmrTIAgAYk6FGsF5+9nWZPxRXO44zcrUwqCYtrgQWADBGy5bHOU//fA7u3JDe5eX4uwVPlAsAwJgMNYJ1Yl2O/NXjod2bf/UnN47jvgQASHLSBdbgKSVmD+/e3L/rlHHclwAASU6qwDr2fF0vfcZnMj/Tab8nEQBgTE6SwFrqZKj9v3oEABiTEyywmoH/LrwsElf9v3oEABiXEySwjkRUKU06nV6mp2czPT2bqe58SmmWPxkqAMCYnACB1Rz1c6fTO2aaTplfOq6MYAEAY3QCBNbgCFYbV2ed9YO8+93vy4c+9I68+lVXp3Sb/PXTPpu5manFv8YHAGBMToDAOhRX7SjV2972sezff0r27n18fnrXH+XB3zw2t3zlOdm/+5RkqZOhAgCMyYQG1tEHs3e7cyklec1rrkxSsnPn3+Vxj/tVSkl27Xx9kpL52zvpXdrGVONM7gDAOprAwGoW/Nyk251PpzOfmZlzc++9f5ANGw6klORf/+WDmetNZfaOqfQu6yS7S5od/dErI1gAwDqZwMBaOILVSylNpqcP5O67n5g9e/42pSTvvvh9SUqaH/eDaldJ7irJTSXZZgQLAFg/ExRYg+e06h0zkjU9fTCfu+HFufvuJ+adF/9bkpL5O7vpbS/J1pLm6pImJfnPkmazwAIA1s+EBNZgTPUOH3O1adPHs3Xb+dm48eGUkjzlqT/Mvn2nJSnp/bCTZmtJNpc0l5c0Py9p9pfkqpJmu8ACANbPhATW4AhWeyqGDRsO5JJLLkxScsXMa1NK8oqzP50D39uQuaab5hcl+UpJvlGS+0q7KTcP7B50DBYAsE4mJLAOhVUbV2960+Z873tPyz33/GEeuP/0JCU3XPWS/PZTj052lPS+W5KD7S7BJiV5uKT5xpFdg8ecokFgAQBjNAGB1Y5cdTrtbsFzz51J05R87nMvzgue/6U85ak/yu6PvS5JSXNXP562lTTXlORr/VGsPeXoA9sFFgCwjiYgsHJ49OqUUw7kS196fu6660l57GkPppTkhWd9Ib/+xBk5+M3p9NJJ7mqPsWq2lfYYrK3tqRmOCSqBBQCskwkIrCPHXW3c+FB+uffx2bL5gpSSvPKZ16a3pyS7S3rbSnJvaVf5J6U9LcOh0zMsNXIlsACAMVuHwGqOurRf3txLp9PL9PRsfvD9s/L1W/8ir3nWlZm9fCq93aU9iej2/i7Ch0vSlDR390euLlvkxKICCwBYJ2MMrENR1f7c7c4fdf309MF0O/N5+z//e+bSTfPdkmwv6W3pJFtKmpmSHChpvl2S7/YPcL++tMdjLbd7UGABAGO0LiNY7ahV++8zzrgvp576u8P/ft2zd2fux93Mp5Pef5fkxpLmKyX5ZUnmS3JtSW7o/3x7SbYs2E0osACAdTamwFq4SzC54IItuf32P8/s7FTuu++MfOjD78gFL9qSg1un0+xoTySaQ6dhSDt6lVtLmv8oyddKmqak+XppD3I3ggUATJAx7yJs42rTpo8naU/F8MY3bs87L/5Auyr3lzR7+n8duLOk+UwbVfl6Sa4pyeaS7C7J3n5wXd3uRlz2+CuBBQCM0dhGsA59/c0553w+ScnMzLmZmmp/90//+OEcnN2QuXTT+1UbUc32fjxt7e8GvKwfVHtL0ivJV0eIK4EFAIzRGgZWM/DfXjqd+ZTSZNu2v88ddzw5j+4fd/Xed70nSUnvjpLezaX9C8F7SzJTkh2lPb5qR2mPvfplSfb14+pQdA0TVwILABijNQqso/9asNudz9TUbEpp8pzn3JwLL7wkpSQX/cNHk5TM/aib3tY2ppp9pV2tX/UDavDy2X54LXfWdoEFAKyjNQisI3E1+NeCCy8vf/p1mf3FdOZ+201vZ0kuLWm+WJL9JfmvktzW3wU4GFLb+qNZo8aVwAIAxqhyYA3uFmx/ft7zvpzzzrssZ5/9nUx1Z1NKe4b2uSu76e0v6d1Zkk+VNLtKcm9/BOuyBbsAjxdSo8SVwAIAxqhiYA2OXLUnEb3ooo/2F9FeLt325rz8T69L7/KS3q6S+Z/2T8XwPyXNr9tpmlsGdgEORtRiPwssAGDCVAiso8/QPjU1m6mpuZx55p156KGNuf76l+Vtmz6WD3zw4sw2U8k9pQ2sLZ1kpqS5syQPluT+0p6OYcsqQ0pgAQDrrOoI1uAxV2996yfyne+cnVOmD6SU5AVP/mIevn1jeyqGn5T2uwW3l2RH+52CzY7+79YirgQWADBGqwisZsGljauXvOSGbNr0ibz//e/KJz/+lpSSvOrPrklvTyfNtpLe7f1dgT8eCKmFX9ZcO64EFgAwRiME1rEHsLeXXqamZtPp9HLhhZdk8Jirb3zz2XnLSz+Zue1TaWZKejvaXYDNbf1pftaOYK15XAksAGCMhgysI0HV6fT6uwKbo2Jramo2e/c+Ltde+8q8+lXX5A3n78hv9j0meaikubIcOc/VztJ+5c23S/KtcvQ5rdbyAgAwJiPuIhwMrfn+6NVc3vCGHfnIR96eb33rmTnj9+5LKclzn/TVPPTFUzN7cCq9+0qyp6QZjKltpf0anIW7BwUWAHCCG2EEqz3G6glP+HlOP/2Bw5F16qm/y223PStJyRdufGFKSV7xzE+nuao9iL13U0nm+qdhuLIcffLQcV4AAMZkyMBqdwk+6lH7c8stf5mbbnpRTjtt3+Hrzjzzzvzg+2flgX2n5z0XvTcHN29Is7uk1z9haHNjab+g+cFy7HFW4xi9ElgAwBiNNILV7c7nzW++NEnJ9de/LI95zL7Duwx//wn/mzt++OQkbVA1m9uwaXa2kZUvl+TWcuxB7UawAICTzJCBdfRuwvPOuyyHIuvQ7sJz/vjzeeCK03Pw/un0DnSSm0qawa+72V6OHHNlFyEAcBIbYQSr/W+3O5dSkvPP35qkZOaqc/P65+5K7/KS7CrpXVWS35Q0B0vyhX5YLYwqI1gAwElsxBGsoyPrb157Vf7v4KOSvaU95mp7G1TN1SXZV5KHS3JFf7fgesaVwAIAxmjEwGoja3rqYEpJXvSUm/LQzRszl6n0flaSy/tfebOtJNeV5KtlfKdhEFgAwIQYMrAG4qrbxtXLnnF9elf0v/7m5pI0Jc09JdndD6pDoTUJcSWwAIAxGiKwjh9XczPd9HYNfP3NzaWd3d0Ldgmu525BgQUArIMhA+t4cdVJb3sn2Tnw9Te3luS2cvSB7ZMQVwILABijIQIrme7OLh5XgwG1rUzWbkGBBQCsg2XLY6ofVy99xmcWj6vjBdWkRRYAwJgMNYL18rOvy/yhuNpxnJGrhUE1aXElsACAMVq2PM55+udzcOeG9C4vx98teKJcAADGZKgRrBPrcuSvHg/t3vyrP7lxHPclAECSky6wBk8pMXt49+b+XaeM474EAEhyUgXWsefreukzPpP5mU77PYkAAGNykgTWUidD7f/VIwDAmJxggdUM/HfhZZG46v/VIwDAuJwggXUkokpp0un0Mj09m+np2Ux151NKs/zJUAEAxuQECKzmqJ87nd4x03TK/NJxZQQLABijEyCwBkew2rg666wf5N3vfl8+9KF35NWvujql2+Svn/bZzM1MLf41PgAAY3ICBNahuGpHqd72to9l//5Tsnfv4/PTu/4oD/7msbnlK8/J/t2nJEudDBUAYEwmNLCOPpi9251LKclrXnNlkpKdO/8uj3vcr1JKsmvn65OUzN/eSe/SNqYaZ3IHANbRBAZWs+DnJt3ufDqd+czMnJt77/2DbNhwIKUk//ovH8xcbyqzd0yld1kn2V3S7OiPXhnBAgDWyQQG1sIRrF5KaTI9fSB33/3E7NnztykleffF70tS0vy4H1S7SnJXSW4qyTYjWADA+pmgwBo8p1XvmJGs6emD+dwNL87ddz8x77z435KUzN/ZTW97SbaWNFeXNCnJf5Y0mwUWALB+JiSwBmOqd/iYq02bPp6t287Pxo0Pp5TkKU/9YfbtOy1JSe+HnTRbS7K5pLm8pPl5SbO/JFeVNNsFFgCwfiYksAZHsNpTMWzYcCCXXHJhkpIrZl6bUpJXnP3pHPjehsw13TS/KMlXSvKNktxX2k25eWD3oGOwAIB1MiGBdSis2rh605s253vfe1ruuecP88D9pycpueGql+S3n3p0sqOk992SHGx3CTYpycMlzTeO7Bo85hQNAgsAGKMJCKx25KrTaXcLnnvuTJqm5HOfe3Fe8Pwv5SlP/VF2f+x1SUqau/rxtK2kuaYkX+uPYlXoaWwAAA4zSURBVO0pRx/YLrAAgHU0AYGVw6NXp5xyIF/60vNz111PymNPezClJC886wv59SfOyMFvTqeXTnJXe4xVs620x2BtbU/NcExQCSwAYJ1MQGAdOe5q48aH8su9j8+WzReklOSVz7w2vT0l2V3S21aSe0u7yj8p7WkZDp2eYamRK4EFAIzZOgRWc9Sl/fLmXjqdXqanZ/OD75+Vr9/6F3nNs67M7OVT6e0u7UlEt/d3ET5ckqakubs/cnXZIicWFVgAwDoZY2Adiqr25253/qjrp6cPptuZz9v/+d8zl26a75Zke0lvSyfZUtLMlORASfPtkny3f4D79aU9Hmu53YMCCwAYo3UZwWpHrdp/n3HGfTn11N8d/vfrnr07cz/uZj6d9P67JDeWNF8pyS9LMl+Sa0tyQ//n20uyZcFuQoEFAKyzMQXWwl2CyQUXbMntt/95Zmenct99Z+RDH35HLnjRlhzcOp1mR3si0Rw6DUPa0avcWtL8R0m+VtI0Jc3XS3uQuxEsAGCCjHkXYRtXmzZ9PEl7KoY3vnF73nnxB9pVub+k2dP/68CdJc1n2qjK10tyTUk2l2R3Sfb2g+vqdjfissdfCSwAYIzGNoJ16Otvzjnn80lKZmbOzdRU+7t/+scP5+Dshsylm96v2ohqtvfjaWt/N+Bl/aDaW5JeSb46QlwJLABgjNYwsJqB//bS6cynlCbbtv197rjjyXl0/7ir977rPUlKeneU9G4u7V8I3luSmZLsKO3xVTtKe+zVL0uyrx9Xh6JrmLgSWADAGK1RYB3914Ld7nympmZTSpPnPOfmXHjhJSkluegfPpqkZO5H3fS2tjHV7Cvtav2qH1CDl8/2w2u5s7YLLABgHa1BYB2Jq8G/Flx4efnTr8vsL6Yz99tuejtLcmlJ88WS7C/Jf5Xktv4uwMGQ2tYfzRo1rgQWADBGlQNrcLdg+/PznvflnHfeZTn77O9kqjubUtoztM9d2U1vf0nvzpJ8qqTZVZJ7+yNYly3YBXi8kBolrgQWADBGFQNrcOSqPYnoRRd9tL+I9nLptjfn5X96XXqXl/R2lcz/tH8qhv8paX7dTtPcMrALcDCiFvtZYAEAE6ZCYB19hvapqdlMTc3lzDPvzEMPbcz1178sb9v0sXzggxdntplK7iltYG3pJDMlzZ0lebAk95f2dAxbVhlSAgsAWGdVR7AGj7l661s/ke985+ycMn0gpSQvePIX8/DtG9tTMfyktN8tuL0kO9rvFGx29H+3FnElsACAMVpFYDULLm1cveQlN2TTpk/k/e9/Vz758beklORVf3ZNens6abaV9G7v7wr88UBILfyy5tpxJbAAgDEaIbCOPYC9vfQyNTWbTqeXCy+8JIPHXH3jm8/OW176ycxtn0ozU9Lb0e4CbG7rT/OzdgRrzeNKYAEAYzRkYB0Jqk6n198V2BwVW1NTs9m793G59tpX5tWvuiZvOH9HfrPvMclDJc2V5ch5rnaW9itvvl2Sb5Wjz2m1lhcAgDEZcRfhYGjN90ev5vKGN+zIRz7y9nzrW8/MGb93X0pJnvukr+ahL56a2YNT6d1Xkj0lzWBMbSvt1+As3D0osACAE9wII1jtMVZPeMLPc/rpDxyOrFNP/V1uu+1ZSUq+cOMLU0ryimd+Os1V7UHsvZtKMtc/DcOV5eiTh47zAgAwJkMGVrtL8FGP2p9bbvnL3HTTi3LaafsOX3fmmXfmB98/Kw/sOz3vuei9Obh5Q5rdJb3+CUObG0v7Bc0PlmOPsxrH6JXAAgDGaKQRrG53Pm9+86VJSq6//mV5zGP2Hd5l+PtP+N/c8cMnJ2mDqtnchk2zs42sfLkkt5ZjD2o3ggUAnGSGDKyjdxOed95lORRZh3YXnvPHn88DV5yeg/dPp3egk9xU0gx+3c32cuSYK7sIAYCT2AgjWO1/u925lJKcf/7WJCUzV52b1z93V3qXl2RXSe+qkvympDlYki/0w2phVBnBAgBOYiOOYB0dWX/z2qvyfwcflewt7TFX29ugaq4uyb6SPFySK/q7BdczrgQWADBGIwZWG1nTUwdTSvKip9yUh27emLlMpfezklze/8qbbSW5riRfLeM7DYPAAgAmxJCBNRBX3TauXvaM69O7ov/1NzeXpClp7inJ7n5QHQqtSYgrgQUAjNEQgXX8uJqb6aa3a+Drb24u7ezuXrBLcD13CwosAGAdDBlYx4urTnrbO8nOga+/ubUkt5WjD2yfhLgSWADAGA0RWMl0d3bxuBoMqG1lsnYLCiwAYB0sWx5T/bh66TM+s3hcHS+oJi2yAADGZKgRrJeffV3mD8XVjuOMXC0MqkmLK4EFAIzRsuVxztM/n4M7N6R3eTn+bsET5QIAMCbKAwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBU9v8G7vocVHkgKQAAAABJRU5ErkJggg==
13608
[task 2017-11-25T22:55:00.409Z] 22:55:00     INFO - TEST-INFO took 369ms
13609
[task 2017-11-25T22:55:00.553Z] 22:55:00     INFO - PID 5511 | --DOCSHELL 0xc72c1c00 == 3 [pid = 5511] [id = {cb9fef26-5b86-4dfc-86dc-514e6e4f09bd}]
13610
[task 2017-11-25T22:55:00.553Z] 22:55:00     INFO - PID 5511 | 
13611
[task 2017-11-25T22:55:00.554Z] 22:55:00     INFO - PID 5511 | ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
13612
[task 2017-11-25T22:55:00.554Z] 22:55:00     INFO - PID 5511 | 
13613
[task 2017-11-25T22:55:00.556Z] 22:55:00     INFO - PID 5511 | [Child 5618, Main Thread] WARNING: NS_ENSURE_TRUE(maybeContext) failed: file /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp, line 791
13614
[task 2017-11-25T22:55:00.657Z] 22:55:00     INFO - PID 5511 | [Parent 5511, Main Thread] WARNING: NS_ENSURE_TRUE(weak) failed: file /builds/worker/workspace/build/src/dom/base/nsFrameMessageManager.cpp, line 339
13615
[task 2017-11-25T22:55:00.941Z] 22:55:00     INFO - PID 5511 | [Parent 5511, Main Thread] WARNING: NS_ENSURE_TRUE(weak) failed: file /builds/worker/workspace/build/src/dom/base/nsFrameMessageManager.cpp, line 339
13616
[task 2017-11-25T22:55:00.979Z] 22:55:00     INFO - PID 5511 | [Child 5561, Main Thread] WARNING: NS_ENSURE_TRUE(maybeContext) failed: file /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp, line 791
13617
[task 2017-11-25T22:55:01.156Z] 22:55:01     INFO - PID 5511 | --DOMWINDOW == 3 (0xe7933800) [pid = 5561] [serial = 3] [outer = (nil)] [url = about:blank]
13618
[task 2017-11-25T22:55:01.158Z] 22:55:01     INFO - PID 5511 | --DOMWINDOW == 2 (0xe792b400) [pid = 5561] [serial = 2] [outer = (nil)] [url = about:blank]
13619
[task 2017-11-25T22:55:01.158Z] 22:55:01     INFO - PID 5511 | --DOCSHELL 0xe676a000 == 0 [pid = 5561] [id = {d6f15acc-d59d-4955-9e3f-47866903ea62}]
Whiteboard: [stockwell unknown] → [stockwell needswork]
this bug also appears to be blocking bug 1383233 (13 failures last week), bug 1383676 (17 failures last week), bug 1382969 (23 failures last week), bug 1383230 (9 failures last week), bug 1383061 (46 failures last week).

In total that is 141 failures that we believe are related to this bug.

:milan, can we get some traction on here now that 57 has been released?
See Also: → 1383233
Flags: needinfo?(milan) → needinfo?(aosmond)
I have been able to reproduce this locally with some effort. Will continue to investigate.
Assignee: nobody → aosmond
Flags: needinfo?(aosmond)
Priority: -- → P3
Whiteboard: [stockwell needswork:owner] → [gfx-noted][stockwell needswork:owner]
Flags: needinfo?(aosmond)
From my reproductions, it appears the imgRequest for the image under test is validated, and purged from the cache. However before it is purged, the layout code checks imgRequestProxy::GetImageStatus which says the image is fully decoded. This messes with the invalidation and causes us to exit nsDOMWindowUtils::UpdateLayerTree without drawing with the downscale-on-decoded size, and instead the native size.

imgRequestProxy::GetImageStatus should not return any state if it has a validator, similar to how imgRequestProxy::Clone defers notifications until the validator has completed.
Status: NEW → ASSIGNED
Flags: needinfo?(aosmond)
There is a second issue with imgRequestProxy::StartDecodingWithResult. Callers also use this as a way to indirectly get the state of the image, as it will return true if a Draw call will succeed with either DrawResult::SUCCESS or DrawResult::WRONG_SIZE. This implies LOAD_COMPLETE, SIZE_AVAILABLE, FRAME_COMPLETE, and DECODE_COMPLETE are set. As with imgRequestProxy::GetImageStatus, if we are validating, the result can be misleading if we throw out the cache. As such it should continue to return false, even if it has everything, until validation has completed.
Flags: needinfo?(bugs)
try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=08a094796acfbc2ccfb461712f4cddab359fdbb0

A consequence of this change is that it flashes red as it switches between the reference and the test displays in this test case. This is because when it switches to validating the cache, layout now believes the image to be incomplete and doesn't render any image at all. In theory this could be solved by keeping a previous imgIContainer reference around, so that if we switch from frame complete to frame incomplete, then we render what we used to have. In practice I'm not sure how important this is -- this would only affect new requests created via imgLoader::LoadImage which ended up causing a validate, or requests cloned from *that* request via imgRequestProxy::Clone. Existing imgRequestProxy objects will continue using the old RasterImage, even if the cache gets purged. As such, I believe this is much more likely to be observable when doing specific tests rather than general use.
I seem to be running into a similar root cause as to bug 1413378 by asserting notifications are deferred in GetImageStatus. Not all paths of validation will trip the assert in bug 1413378 so presumably I'm hitting the silent majority.

I'll take out the assert for now, and see about cornering the other problem as part of another bug.
Attachment #8934961 - Attachment is obsolete: true
Okay, this requires more thought given the sheer number of failures. I believe what we need is to distinguish between requests which were created before the validator, and requests created after. The former get grandfathered in and don't change state if and only if the validator evicts the cache. The latter don't get any state until the validator completes. The trouble with the attached patch (and why I tripped the assert in the first place) is that it does not distinguish between the two, and so all requests don't get any state anymore until validation completes.
The patch got a lot bigger. At this point, the only tests that should be failing are some intermittent accessiblity mochitests. I need to confirm whether or not I caused it to permafail due to timing, or an actual regression.

try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=9bf2752358cd3079d5d1cb0f11b69be5a3574dd0
Attachment #8934986 - Attachment is obsolete: true
accessible/tests/mochitest/elm/test_HTMLSpec.html and accessible/tests/mochitest/tree/test_txtcntr.html are failing very consistently and block landing of this patch.
Depends on: 1408636
(In reply to Andrew Osmond [:aosmond] from comment #42)
> accessible/tests/mochitest/elm/test_HTMLSpec.html and
> accessible/tests/mochitest/tree/test_txtcntr.html are failing very
> consistently and block landing of this patch.

It appears that if the image is not displayed at the time of onload, there is an extra text leaf containing a single space. This breaks both tests. If I remove the spaces in the <figure> block of test_HTMLSpec.html, it passes. This is why it failed intermittently before -- it usually (wrong) reused an unvalidated cache, which then expired but only after the test completed. With my changes, it consistently uses a fresh entry, but nothing is decoded, and we see this extra text leaf artifact breaking the test.
Comment on attachment 8936648 [details] [diff] [review]
0001-Bug-1383682-imgRequestProxy-should-avoid-leaking-sta.patch, v4

I'm still figuring out the (new) accessibility test failures, but I think they are all related to the whitespace optimizations, rather than a problem with this patch.
Attachment #8936648 - Flags: review?(tnikkel)
Comment on attachment 8936648 [details] [diff] [review]
0001-Bug-1383682-imgRequestProxy-should-avoid-leaking-sta.patch, v4

Review of attachment 8936648 [details] [diff] [review]:
-----------------------------------------------------------------

Would it make more sense to split this into two patches, one that splits the two notions of deferring notifications, and one that avoids leaking state via GetImageStatus etc?

::: image/imgRequestProxy.h
@@ +116,1 @@
>    virtual bool NotificationsDeferred() const override

I think it would be better if we worked on the naming here a bit more. It's confusing/surprising to have NotificationsDeferred() to return something other than mDeferNotifications.
There are 40 failures in the past 7 days on Linux/ Linux x64 debug, linux32-stylo-disabled debug, linux64-ccov opt, and linux64-stylo-disabled debug.
Most recent log: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=154938634
Relevant part of the log: 
[task 2018-01-09T05:14:41.488Z] 05:14:41     INFO - PID 5515 | 1515474881482	Marionette	INFO	Testing http://web-platform.test:8000/css/css-backgrounds/background-size-031.html == http://web-platform.test:8000/css/css-backgrounds/reference/background-size-031-ref.xht
13622
[task 2018-01-09T05:14:41.503Z] 05:14:41     INFO - PID 5515 | ++DOMWINDOW == 83 (0x7f28e6423c00) [pid = 5609] [serial = 83] [outer = 0x7f28f057a660]
13623
[task 2018-01-09T05:14:41.565Z] 05:14:41     INFO - PID 5515 | ++DOMWINDOW == 84 (0x7f28e6425400) [pid = 5609] [serial = 84] [outer = 0x7f28f057a660]
13624
[task 2018-01-09T05:14:41.621Z] 05:14:41     INFO - PID 5515 | 1515474881616	Marionette	INFO	Found 2388 pixels different, maximum difference per channel 22
13625
[task 2018-01-09T05:14:41.661Z] 05:14:41     INFO - TEST-UNEXPECTED-FAIL | /css/css-backgrounds/background-size-031.html | Testing http://web-platform.test:8000/css/css-backgrounds/background-size-031.html == http://web-platform.test:8000/css/css-backgrounds/reference/background-size-031-ref.xht
13626
[task 2018-01-09T05:14:41.667Z] 05:14:41     INFO - REFTEST   IMAGE 1 (TEST): data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAlgAAAJYCAYAAAC+ZpjcAAAgAElEQVR4nO3dbaxkdX3A8f+9LLDbsourialN+kKMBhSfqm21b3ajqcE0Sompxm66iMgLNDERH1DTV6Y21Gr0Bbyw1bKEriy4y0IRUgJlEQUEFRcwKqHLg92KLS4Li3vdpzu/vph7997dvXPnnJnfnDn37ueTTMLOnDnnf87MnPly7rnnlgAAIFUZ9wAAAJYbgQUAkExgAQAkE1gAAMkEFgBAMoEFAJBMYAEAJBNYAADJBBYAQDKBBQCQTGABACQTWAAAyQQWAEAygQUAkExgAQAkE1gAAMkEFgBAMoG1TJTipZxlWwAw3zi+FyotsZTS9zaSwfmirKzXtqr6OvV7/lIyivGOcxssxddgUHXer4Psg0Y17VJ2sqwny0/dz/Ni8xjF56ByYNX5dxYf+uHUeZ0Gfexk0IYvoHEvvwlV36+9Xo+qr1OdbXkybPeIk2c9WX6Gee+OumUE1jKW9UVyMr8Os+s+7m0w7uU3Ydj/E606D4F1opNlPVl+MgMr20Bz7zeoKofc+h3ir/ujyPmPD/pjhkHGWXWazOX1es6g8+m3rbO3a9XXapBDt8O+RovNdxCZ74uFntNrXr3u6/X6Dvta9ZP1GaqynKrTjHq7H//YKLdbneUsNF2dcTX1+c5ctyrvuUHXv87r3uuzt9i61x131vavs66jHGe/eVb9zPdb/0H3OYsud6An9dkA/e6r+oYadGc06JgGGecw6zLodllI1fWu8/zFxjDMtl5snnXn02u8gyy/zjwHHccg74s69y02bdZ7vtd9vQy7PlXVGU+T273q8/tNV+fLuO46juK1rPP5bnrdBn1dht3fZ7+f6m63uvOuMt+FjPv1rTKmXvdlSg2sfl/SdZ9fd+WrLr/qNFXGOcw0g8yrynzqPK/KtBmv1zDTVn2s7rJGGViZ74te047ii37Y5dSZZtjP++xz6uxkM8Y0is9DVeMcTxOvZdVljmPfU3U8dec/7P6p7r6/ie+jqs9r8rN0UgXWsB/oYZZfdZoq46w6TZXH6n5R9FuGwJr797AfpMz3YpVpBFZ9w+xo64xJYA23rDrzb9O+Z/bxrP19v+nbEFiZATLuz9KyCKzFbotNW2c5dce10GNVxlhnnHXmMex2qbJ+ve6r8/x+j1Xdpr3Wc7HlVZ1P1fEOul2rjHfQ6QVWbmA1Nabsz0PV+xfbvwy6LlXHPsjzF9OmdRtk/zLImHs9VvdzPMx2W2we2d9H43p960yXsc9ZdJkDPWmIHe5i8xx2h1v1RRj2C6PqzqPOfIdZXq9l1P1A1H1smNcr4wMzzPMH2WGMYgxVH8vYMWfet9j9VacZ9vM+rjGN4vMw6DKyxtPkaznudcv+Uq2zvx/FOmZEXlVVv/+q3i+wFnrSADvcQQJn3IFVZbphphl0XFWnaVNgZX1gBtkWGf9DkPVeHHRHLLAGm+coxtTEl0Ibv6ROhsDKeF2WcmBlfB+N+/WtM92SCqxejy20sapOV3W5dec76H3Hz2/Qdak6rl7PrTLNMAEx6Adt2C/tYZ7Ta7phtusg0y42jirzrXPfMOvaRGBlLGeh6Zp6flPbPXsZdZbda77DznOxeYx73TJfl4WeW3cso/yOWsgov4/a8PoOM89Mtec+u0JVPkCLTXf8NFWmqzK2QZc//7664xxkXYbdLr2eU2dZdefTb9qqy646tqx1WGheg8yvzvPqrEOd7VXluVVew2Hu6zemYdex7nt+kHHUHVOV8WatW51lZH0+m34tm163Qdajl6rvvWHH0m/dq2y3xdZrkHUY5j0yzvfuQvMc9PWvY7T51rBRbSQAlqY2fS+0aSyM3rJ6tb15AU5OvY54tEnbxsNoLZtXe9SH+gBotyZ+7DOoto6L0fFKAwAkE1gAAMkEFgBAspMysPwMnEF43wBQVeVvjH7XyVhKltp426bq9VWGnX+d66NkWuwaLONYblu0fXwnI6+Jk8dpr6EDq99jLD+jDqzF5tHUjnRc7+m2f5baPr6TkddEaNJOAovaxn0USWCNT9vHdzLymggs2qnSO3Khy+Uff8n5haarOr9ej/ebdphL+FdZx6rzGnb96+wYspY56LyrvBeqLmOQo2CLLa/uWBYbU6/3Wr/nLva8qq9B1TH2enyx+6qOo9d0dT9ng7wfeq1Hr+dWnW+v51Z5/ep+TjPH0G+eVcaVtd+osw3qLLPOcqq8Z6AN0o5g1flCqnrfQvOsO59e4x1k+b3mVXf9q06bucwq9/Wbd9XxZo6jyv39vmgXU+e1H/R9k/FZyJpf9mdyIYN8nvsto87rO+xnsc68exnF/mCY12T+tG3ZVw2ynH5xBm2SFlhV7q8zj0GW1++xusuqE1hV7u8XAnV3lMPMp864606TsS4Z9w8SV4s9Nuj7Jut9X3d8x99XdRzZn6+MZQzzGa07z2EeyxpD1v7w+OnatK+qu5ysMUFTBFYM9+Fs206r3//hHX8bZByD7jQzxtG2wBomOHrNs4pBIqbJwBr1tq8znrrzHOaxrDEILIHF0td4YFX9kq3yoaz6ZV1lvHXncfzzqtw/jsDKCpg609RZ9+z7q4ZCv8erfhlWed8MG1h13uf9AqvK5ybzy3yx16vOMups26rba9T7mGHGUHd/WEXb9lXzH6/6vswYEzSlNUewMpaXMe/50wwzncBqNrCOf6zqa1fnsUHfD8MEVt1oW2z967y/Bnms3zyHibhh9zEZkVdVxhhOtsAadl4CizZqTWBl7VAG+WKt+yV2/DRt2mkJrOGOZC722KDvmyYDa/axYYIi+/OVsYw2BVaVz8ewYxBYc49njwmaMnBgVdlZVt2h1N35DfqcXtMN8oVU5bkZO7PFllllPk0FVr8dYROBVXX8vcZdZdmjnK7qc4fZBsN+Jof9TAy6jGHe74N+vjP3CXXHkP2a9BvDqPdVg46rzpiG3T6QrdY7b6E3cr8v+8Xe9IM+Xne6KuM6fl51dqSLbZMq61d3BzD/OVXms9DyBhn3sOs+7DjqrGsdC813kPd5vy+KOu+LuuNbaHl11rfqdIN+Nqq+Nxcbf9Xx1H09626DKrLeU4Nug6rrUec9mbENqk476Puy17aEcfDOW4LsNBbXtm3T9Hjatv51LfXxA0QIrCVJYB3r+P8jb5smxtTv6M9SshzWAcCebIkZ9HD9ctbmbdLUuNq6/nUtl/UAsBcDAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBI1jewSln6t3Xn7GhgUwIAdC37wFp3zo6IzQ7UAQDNWdaBdTSuBBYA0KBlG1jHxJXAAgAatCwD64S4ElgAQIOWXWAtGFcCCwBo0LIKrJ5xJbAAgAYtycA666xd8dnP/kNcccWnY926HbFmzfOLx5XAAgAatMQCqxOXXPL12Ldvdeza9cr4+c/Pjr17XxL33/O2iG8vElcCCwBo0BIKrE5s3HhNTE2tii1bPhAve9mzsXLlVFx77YbuajwksACAdlgygbVixaG4+uoLY/fuP4zTTjsQk5NH4jOfuqK7Ck/NRNQNAgsAGL8lE1innnowdu58Y2zatDFOOeVwfO7yL0YcKRFPlIjrS8R1M6F1m8ACAMar5YHViYmJztH/vv32v4jnn18Tn/jEV7pDf2IuoDpbS0SnRHxPYAEA49XqwPrwh78R11zzt7F69QtRSsTrX/9I/Pa3v9cNqSdnjlxtLt0T3HeXiP0lYqvAAgDGq7WBdcYZ++LLX74sOp0S27f/VUxOHol3nntHxI9Ld9j/XSLuKREPlohfzxy9uqdHXAksAKBBLQysTlx88TfiwQffGs8995LYt291TE9PxG3bz5s7YrVz5mhVZ+b2Yom4d5G4ElgAQINaF1gbNlwbU1OrYuvW98X69TvizW/+cXzzqxd1T2if/2PBbSXi7hJxV+l/DSyBBQA0qFWBddppB+LWW98djz326njpS/dEKfP+/M2PSne4T1eIKYEFAIxRywLrYOzadVZcddWlUUrn2D9/860S8ZuZyJp/JEtgAQAt06rAKqUTDz/8hnjooTfFeX9824mR9GTpnm81e5K7wAIAWqhlgRXx+c//fRw6tCLi4ZmjVrOB9O2ZuHqoRDxaIo6U6NwksACA9hlrYE1OHonTT/9dTE4eOXrfu99ya8RjJWK6RPyiRNxRupdf+J8ScbBEfKdE3F4iDpeI+wUWANA+YwusSy+9Ku699+3R6ZTYu/fM2LRpY7z3bTfNBdHjpTu86dK9FMNUmbsUw/dm7v++wAIA2mcMgdWJSy75ekxNrYpt2y6IjRs3xeWXXxFTUysjnilzV2K/rkTcXCLumwmpG2fu31q6003Nu09gAQAt0nBgdeK8826LiBI33fTeWLHicJTSiU9+8p+6Q+mUiD0l4oYFAulb8+LqYIn4bo24ElgAQIMaDayJiem48sqPxn33vT3WrHk+JiePxOcu/2L3IqJPzPwI8FA59kjW7O3fS8SzJWLvAHElsACABjV8BCviNa/5RXzkI/8c3T+J8y/dITwxL4QOlO59vynH/hbh5hJxS6l//SuBBQA0rPHAmv1R4brX7YjO7HWtZkPqztI9ef0npf/fFhRYAEBLjTiwOrF+/V1xwQXb4uyzfxYrV05FKTN//ua6mR/3PV7mrnO1u0Q8VyK2JMeVwAIAGjTSwPrYx66M/ftXRadTYv/+VXHzze+Jd5x7Z8TmEp3NJeJXpTuE/yrR+d/SPcm9zqUXBBYA0EIjC6xXvOJX8cwzfxDbt58fl132lfjSlz7VvRTDU2XuPKobSvf8q/8r3XOuRhVXAgsAaNAIAqsTpXTioou+Gfff/2dHL8Ww/rV3Rfxw5ojVL8uJJ7CP+gYA0JDkwOrEu971H3HhhVfHxz/+tdiy5f0xMTHdPedqNnR+VLqXZXiyDP4bgQILAGixxMCau0L79PREHDhwejzyyLnxN+/cfGLsPDDvSJbAAgCWmbTAOvXUg/H0038U11//13H++dvjgx/cHC+++Pvd3wrctkDw/LCM9pwrgQUAjElCYHVi48Zr4gtf+Lt49NHXxdq1z81diuE/S/fCoc+Wen83UGABAEtY7cB6+ct/HWeeuffov1evfiF+8IM/jenpiXjggT+JU045fOw5V22JLACAhtQKrFWr9sfdd6+LW275y1i7ds/R+1/1qsdj5843xJ49a+MzF//jiXFze4k4XCJeKM3/9qDAAgAaVvsI1oYN/xb79p2xYGT97KdnR/yuRNyxQODcWSLucQQLAFj+agdWr8had86OiJtL96T2XpE1zhsAQEMGCqxSIj70oX89GlkX/PmNcyHT1sgCAGjIwIE1/0hW7C7dP9Z8fGS9WCK2tiCuBBYA0KChAmvdOTu651V1ysKRdVcLwkpgAQANGziwjrkUw/dLxKEFIqtNNwCAhgwUWMfE1fGR9VQZ36UYBBYA0AK1A2vBuJq93VfGeykGgQUAtECtwFo0rtp+AwBoSOXAWtJxJbAAgAZVCqwlH1cCCwBoUN/yWBZxJbAAgAZVOoK11G/rztnRwKYEAOha9oF19AgcAEBDlnVgHfPjTQCAhizbwDrh3DEAgIYsy8Ba8MR8AICGLLvA6vlbjwAADVlWgbXoJSUAABqyJAPrrLN2xWc/+w9xxRWfjnXrdsSaNc/3v14XAEBDllhgdeKSS74e+/atjl27Xhk///nZsXfvS+L+e94W8e1F4kpgAQANWkKB1YmNG6+JqalVsWXLB+JlL3s2Vq6cimuv3dBdjYcEFgDQDksmsFasOBRXX31h7N79h3HaaQdicvJIfOZTV3RX4amZiLpBYAEA47dkAuvUUw/Gzp1vjE2bNsYppxyOz13+xYgjJeKJEnF9ibhuJrRuE1gAwHi1PLA6MTHROfrft9/+F/H882viE5/4SnfoT8wFVGdrieiUiO8JLABgvFodWB/+8Dfimmv+NlavfiFKiXj96x+J3/7297oh9eTMkavNpXuC++4Ssb9EbBVYAMB4tTawzjhjX3z5y5dFp1Ni+/a/isnJI/HOc++I+HHpDvu/S8Q9JeLBEvHrmaNX9/SIK4EFADSohYHViYsv/kY8+OBb47nnXhL79q2O6emJuG37eXNHrHbOHK3qzNxeLBH3LhJXAgsAaFDrAmvDhmtjampVbN36vli/fke8+c0/jm9+9aLuCe3zfyy4rUTcXSLuKv2vgSWwAIAGtSqwTjvtQNx667vjscdeHS996Z4oZd6fv/lR6Q736QoxJbAAgDFqWWAdjF27zoqrrro0Sukc++dvvlUifjMTWfOPZAksAKBlWhVYpXTi4YffEA899KY4749vOzGSnizd861mT3IXWABAC7UssCI+//m/j0OHVkQ8PHPUajaQvj0TVw+ViEdLxJESnZsEFgDQPmMNrMnJI3H66b+LyckjR+9791tujXisREyXiF+UiDtK9/IL/1MiDpaI75SI20vE4RJxv8ACANpnbIF16aVXxb33vj06nRJ7954ZmzZtjPe+7aa5IHq8dIc3XbqXYpgqc5di+N7M/d8XWABA+4whsDpxySVfj6mpVbFt2wWxceOmuPzyK2JqamXEM2XuSuzXlYibS8R9MyF148z9W0t3uql59wksAKBFGg6sTpx33m0RUeKmm94bK1YcjlI68clP/lN3KJ0SsadE3LBAIH1rXlwdLBHfrRFXAgsAaFCjgTUxMR1XXvnRuO++t8eaNc/H5OSR+NzlX+xeRPSJmR8BHirHHsmavf17iXi2ROwdIK4EFgDQoIaPYEW85jW/iI985J+j+ydx/qU7hCfmhdCB0r3vN+XY3yLcXCJuKfWvfyWwAICGNR5Ysz8qXPe6HdGZva7VbEjdWbonr/+k9P/bggILAGipEQdWJ9avvysuuGBbnH32z2LlyqkoZebP31w38+O+x8vcda52l4jnSsSW5LgSWABAg0YaWB/72JWxf/+q6HRK7N+/Km6++T3xjnPvjNhcorO5RPyqdIfwXyU6/1u6J7nXufSCwAIAWmhkgfWKV/wqnnnmD2L79vPjssu+El/60qe6l2J4qsydR3VD6Z5/9X+le87VqOJKYAEADRpBYHWilE5cdNE34/77/+zopRjWv/auiB/OHLH6ZTnxBPZR3wAAGpIcWJ1417v+Iy688Or4+Me/Flu2vD8mJqa751zNhs6PSveyDE+WwX8jUGABAC2WGFhzV2ifnp6IAwdOj0ceOTf+5p2bT4ydB+YdyRJYAMAykxZYp556MJ5++o/i+uv/Os4/f3t88IOb48UXf7/7W4HbFgieH5bRnnMlsACAMUkIrE5s3HhNfOELfxePPvq6WLv2ublLMfxn6V449NlS7+8GCiwAYAmrHVgvf/mv48wz9x799+rVL8QPfvCnMT09EQ888CdxyimHjz3nqi2RBQDQkFqBtWrV/rj77nVxyy1/GWvX7jl6/6te9Xjs3PmG2LNnbXzm4n88MW5uLxGHS8QLpfnfHhRYAEDDah/B2rDh32LfvjMWjKyf/fTsiN+ViDsWCJw7S8Q9jmABAMtf7cDqFVnrztkRcXPpntTeK7LGeQMAaMhAgVVKxIc+9K9HI+uCP79xLmTaGlkAAA0ZOLDmH8mK3aX7x5qPj6wXS8TWFsSVwAIAGjRUYK07Z0f3vKpOWTiy7mpBWAksAKBhAwfWMZdi+H6JOLRAZLXpBgDQkIEC65i4Oj6ynirjuxSDwAIAWqB2YC0YV7O3+8p4L8UgsACAFqgVWIvGVdtvAAANqRxYSzquBBYA0KBKgbXk40pgAQAN6lseyyKuBBYA0KBKR7CW+m3dOTsa2JQAAF3LPrCOHoEDAGjIsg6sY368CQDQkGUbWCecOwYA0JBlGVgLnpgPANCQZRdYPX/rEQCgIcsqsBa9pAQAQEOWZGCdddau+Oxn/yGuuOLTsW7djliz5vn+1+sCAGjIEgusTlxyyddj377VsWvXK+PnPz879u59Sdx/z9sivr1IXAksAKBBSyiwOrFx4zUxNbUqtmz5QLzsZc/GypVTce21G7qr8ZDAAgDaYckE1ooVh+Lqqy+M3bv/ME477UBMTh6Jz3zqiu4qPDUTUTcILABg/JZMYJ166sHYufONsWnTxjjllMPxucu/GHGkRDxRIq4vEdfNhNZtAgsAGK+WB1YnJiY6R//79tv/Ip5/fk184hNf6Q79ibmA6mwtEZ0S8T2BBQCMV6sD68Mf/kZcc83fxurVL0QpEa9//SPx29/+Xjeknpw5crW5dE9w310i9peIrQILABiv1gbWGWfsiy9/+bLodEps3/5XMTl5JN557h0RPy7dYf93ibinRDxYIn49c/Tqnh5xJbAAgAa1MLA6cfHF34gHH3xrPPfcS2LfvtUxPT0Rt20/b+6I1c6Zo1WdmduLJeLeReJKYAEADWpdYG3YcG1MTa2KrVvfF+vX74g3v/nH8c2vXtQ9oX3+jwW3lYi7S8Rdpf81sAQWANCgVgXWaacdiFtvfXc89tir46Uv3ROlzPvzNz8q3eE+XSGmBBYAMEYtC6yDsWvXWXHVVZdGKZ1j//zNt0rEb2Yia/6RLIEFALRMqwKrlE48/PAb4qGH3hTn/fFtJ0bSk6V7vtXsSe4CCwBooZYFVsTnP//3cejQioiHZ45azQbSt2fi6qES8WiJOFKic5PAAgDaZ6yBNTl5JE4//XcxOXnk6H3vfsutEY+ViOkS8YsScUfpXn7hf0rEwRLxnRJxe4k4XCLuF1gAQPuMLbAuvfSquPfet0enU2Lv3jNj06aN8d633TQXRI+X7vCmS/dSDFNl7lIM35u5//sCCwBonzEEVicuueTrMTW1KrZtuyA2btwUl19+RUxNrYx4psxdif26EnFzibhvJqRunLl/a+lONzXvPoEFALRIw4HVifPOuy0iStx003tjxYrDUUonPvnJf+oOpVMi9pSIGxYIpG/Ni6uDJeK7NeJKYAEADWo0sCYmpuPKKz8a99339liz5vmYnDwSn7v8i92LiD4x8yPAQ+XYI1mzt38vEc+WiL0DxJXAAgAa1PARrIjXvOYX8ZGP/HN0/yTOv3SH8MS8EDpQuvf9phz7W4SbS8Qtpf71rwQWANCwxgNr9keF6163Izqz17WaDak7S/fk9Z+U/n9bUGABAC014sDqxPr1d8UFF2yLs8/+WaxcORWlzPz5m+tmftz3eJm7ztXuEvFcidiSHFcCCwBo0EgD62MfuzL2718VnU6J/ftXxc03vyfece6dEZtLdDaXiF+V7hD+q0Tnf0v3JPc6l14QWABAC40ssF7xil/FM8/8QWzffn5cdtlX4ktf+lT3UgxPlbnzqG4o3fOv/q90z7kaVVwJLACgQSMIrE6U0omLLvpm3H//nx29FMP6194V8cOZI1a/LCeewD7qGwBAQ5IDqxPvetd/xIUXXh0f//jXYsuW98fExHT3nKvZ0PlR6V6W4cky+G8ECiwAoMUSA2vuCu3T0xNx4MDp8cgj58bfvHPzibHzwLwjWQILAFhm0gLr1FMPxtNP/1Fcf/1fx/nnb48PfnBzvPji73d/K3DbAsHzwzLac64EFgAwJgmB1YmNG6+JL3zh7+LRR18Xa9c+N3cphv8s3QuHPlvq/d1AgQUALGG1A+vlL/91nHnm3qP/Xr36hfjBD/40pqcn4oEH/iROOeXwsedctSWyAAAaUiuwVq3aH3ffvS5uueUvY+3aPUfvf9WrHo+dO98Qe/asjc9c/I8nxs3tJeJwiXihNP/bgwILAGhY7SNYGzb8W+zbd8aCkfWzn54d8bsScccCgXNnibjHESwAYPmrHVi9ImvdOTsibi7dk9p7RdY4bwAADRkosEqJ+NCH/vVoZF3w5zfOhUxbIwsAoCEDB9b8I1mxu3T/WPPxkfViidjagrgSWABAg4YKrHXn7OieV9UpC0fWXS0IK4EFADRs4MA65lIM3y8RhxaIrDbdAAAaMlBgHRNXx0fWU2V8l2IQWABAC9QOrAXjavZ2XxnvpRgEFgDQArUCa9G4avsNAKAhlQNrSceVwAIAGlQpsJZ8XAksAKBBfctjWcSVwAIAGqQ8AACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCXfUFR4AAASaSURBVCwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBIJrAAAJIJLACAZAILACCZwAIASCawAACSCSwAgGQCCwAgmcACAEgmsAAAkgksAIBkAgsAIJnAAgBI9v/NeAG0AATMwwAAAABJRU5ErkJggg==
13627
[task 2018-01-09T05:14:41.674Z] 05:14:41     INFO - REFTEST   IMAGE 2 (REFERENCE): data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAlgAAAJYCAYAAAC+ZpjcAAAgAElEQVR4nO3de4ylZWHH8eecmcUsoqQY29jYGGIMsVVJbU1sNfFWtN5rxVi0oCm6Ypv6V5MGNRqt8V8T65XKXtllh+UmKAreQQui0ZhYqxRFRKkrAq5Cd3dmzvvrH+/Z3bOzOzPnzDxz5uzy+SQnzM55z3s51y/P+857SgAAqKqs9woAAJxsBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBNZJohQP5SHuCwAGrcfnwlBLLKUse1mTlfNBObTF7qthH6flbn8iWYv1Xc/74ER8DFZqlOfrSt6D1mraE9kjZTs5+Yz6el5qHmvxOhg6sEb5dy1e9KszyuO00useCSbhA2i9lz8Owz5fF3s8hn2cRrkvHwn3e/LI2U5OPqt57q51ywisk1itD5JH8uNwaNvX+z5Y7+WPw2r/T3TYeQisYz1StpOTT83Aqm1Fc19upYYZcltuiH/UXZGD1690N8NK1nPYaWoub7HbrHQ+y93Xte/XYR+rlQzdrvYxWmq+K1HzeXG82yw2r8V+t9jju9rHajm1XkPDLGfYadb6fl943Vreb6Ms53jTjbJe43p919y2YZ5zK93+UR73xV57S237qOtd6/4fZVvXcj2Xm+ewr/nltn+l7zlLLndFN1rmDljud8M+oVb6ZrTSdVrJeq5mW1Z6vxzPsNs9yu2XWofV3NdLzXPU+Sy2vitZ/ijzXOl6rOR5Mcrvlpq21nN+sd8tZrXbM6xR1mec9/uwt19uulE+jEfdxrV4LEd5fY9721b6uKz2/b7282nU+23UeQ8z3+NZ78d3mHVa7Hc1VQ2s5T6kR739qBs/7PKHnWaY9VzNNCuZ1zDzGeV2w0xb4/FazbTDXjfqstYysGo+Lxabdi0+6Fe7nFGmWe3r/dBtRnmTrbFOa/F6GNZ6rs84Hsthl7ke7z3Drs+o81/t+9Oo7/3j+Dwa9nbjfC09ogJrtS/o1Sx/2GmGWc9hpxnmulE/KJZbhsA68u/VvpBqPheHmUZgjW41b7SjrJPAWt2yRpn/JL33HLq+1vv9ctNPQmDVDJD1fi2dFIG11GWpaUdZzqjrdbzrhlnHUdZzlHms9n4ZZvsW+90ot1/uumHv08W2c6nlDTufYdd3pffrMOu70ukFVt3AGtc61X49DPv7pd5fVrotw677Sm6/lEnatpW8v6xknRe7btTX8Wrut6XmUfvzaL0e31Gmq/Ges+QyV3SjVbzhLjXP1b7hDvsgrPYDY9g3j1Hmu5rlLbaMUV8Qo163mserxgtmNbdfyRvGWqzDsNfVeGOu+bulfj/sNKt9va/XOq3F62Gly6i1PuN8LNd722p/qI7yfr8W21gj8oY17OffsL8XWMe70QrecFcSOOsdWMNMt5ppVrpew04zSYFV6wWzkvuixv8Q1HourvSNWGCtbJ5rsU7j+FCYxA+pR0Jg1XhcTuTAqvF5tN6P7yjTnVCBtdh1x7uzhp1u2OWOOt+V/m7h/Fa6LcOu12K3HWaa1QTESl9oq/3QXs1tFptuNffrSqZdaj2Gme8ov1vNto4jsGos53jTjev247rfay9jlGUvNt/VznOpeaz3ttV8XI5321HXZS0/o45nLT+PJuHxXc08axp57oc2aJgX0FLTLZxmmOmGWbeVLn/wd6Ou50q2ZbX3y2K3GWVZo85nuWmHXfaw61ZrG443r5XMb5TbjbINo9xfw9x2mMdwNb9bbp1Wu42jPudXsh6jrtMw61tr20ZZRq3X57gfy3Fv20q2YzHDPvdWuy7Lbfsw99tS27WSbVjNc2Q9n7vHm+dKH/9RrG2+jdla3UkAnJgm6XNhktaFtXdSPdqevACPTIuNeEySSVsf1tZJ82iv9VAfAJNtHLt9VmpS14u145EGAKhMYAEAVCawAAAqe0QGln3grITnDQDDGvoTY7nzZJxITrT1nTTDnl9ltfMf5fwoNS11Dpb1WO6kmPT1eyTymDh4nMm16sBa7jpOPmsdWEvNY1xvpOv1nJ7019Kkr98jkcdEaDKZBBYjW+9RJIG1fiZ9/R6JPCYCi8k01DPyeKfLX3jK+eNNN+z8Frt+uWlXcwr/YbZx2HmtdvtHeWOotcyVznuY58Kwy1jJKNhSyxt1XZZap8Wea8vddqnbDfsYDLuOi12/1O+GXY/Fphv1dbaS58Ni27HYbYed72K3HebxG/V1WnMdlpvnMOtV631jlPtglGWOspxhnjMwCaqNYI3ygTTs7443z1Hns9j6rmT5i81r1O0fdtqayxzmd8vNe9j1rbkew/x+uQ/apYzy2K/0eVPjtVBrfrVfk8ezktfzcssY5fFd7WtxlHkvZi3eD1bzmAxOOynvVStZznJxBpOkWmAN8/tR5rGS5S133ajLGiWwhvn9ciEw6hvlauYzynqPOk2Nbanx+5XE1VLXrfR5U+t5P+r6LfzdsOtR+/VVYxmreY2OOs/VXFdrHWq9Hy6cbpLeq0ZdTq11gnERWFndi3PS3rSW+z+8hZeVrMdK3zRrrMekBdZqgmOxeQ5jJREzzsBa6/t+lPUZdZ6rua7WOggsgcWJb+yBNeyH7DAvymE/rIdZ31HnsfB2w/x+PQKrVsCMMs0o217798OGwnLXD/thOMzzZrWBNcrzfLnAGuZ1U/PDfKnHa5RljHLfDnt/rfV7zGrWYdT3w2FM2nvV4PXDPi9rrBOMy8SMYNVYXo15D06zmukE1ngDa+F1wz52o1y30ufDagJr1GhbavtHeX6t5Lrl5rmaiFvte0yNyBtWjXV4pAXWauclsJhEExNYtd5QVvLBOuqH2MJpJulNS2CtbiRzqetW+rwZZ2Adum41QVH79VVjGZMUWMO8Pla7DgLryPW11wnGZcWBNcyb5bBvKKO++a30NotNt5IPpGFuW+PNbKllDjOfcQXWcm+E4wisYdd/sfUeZtlrOd2wt13NfbDa1+RqXxMrXcZqnu8rfX3XfE8YdR1qPybLrcNav1etdL1GWafV3j9Q20jPvOM9kZf7sF/qSb/S60edbpj1WjivUd5Il7pPhtm+Ud8ABm8zzHyOt7yVrPdqt3216zHKto7iePNdyfN8uQ+KUZ4Xo67f8ZY3yvYOO91KXxvDPjeXWv9h12fUx3PU+2AYtZ5TK70Pht2OUZ6TNe6DYadd6fNysfsS1oNn3gnIm8bSJu2+Gff6TNr2j+pEX3+ARGCdkATW0Rb+H/mkGcc6LTf6cyI5GbYBwDvZCWalw/Uns0m+T8a1XpO6/aM6WbYDwLsYAEBlAgsAoDKBBQBQmcACAKhMYAEAVCawAAAqE1gAAJUJLACAygQWAEBlAgsAoDKBBQBQmcACAKhMYAEAVCawAAAqE1gAAJUJLACAygQWAEBlywZWKSfapelfkqnubEpJ/upPbhzHfQkAkOSkC6wjcTXdj6uXn31d9u86ZRz3JQBAkpMqsAbj6mBKSV76jM9kfqaT3uXLbiYAQDUnSWAdG1cve8b1mZvpprerk972zjjuSwCAJCdcYDULgmrwskhc7egkO41gAQDjc4IE1pGIKqVJp9PL9PRspqdnM9WdTynN4WOujhm52lnSXCawAIDxOQECqznq506nd8w0nTK/dFwZwQIAxugECKzBEaw2rs466wd597vflw996B159auuTuk2+eunfTZzM1PHjyuBBQCM0QkQWIfiqh2letvbPpb9+0/J3r2Pz0/v+qM8+JvH5pavPCf7d5+SXF6OH1cCCwAYowkNrKMPZu9251JK8prXXJmkZOfOv8vjHverlJLs2vn6JCXzt3fSu7SNqWYwrAQWADBmExhYzYKfm3S78+l05jMzc27uvfcPsmHDgZSS/Ou/fDBzvanM3jGV3mWdZHdJs6M/emUECwBYJxMYWAtHsHoppcn09IHcffcTs2fP36aU5N0Xvy9JSfPjflDtKsldJbmpJNuMYAEA62eCAmvwnFa9Y0aypqcP5nM3vDh33/3EvPPif0tSMn9nN73tJdla0lxd0qQk/1nSbBZYAMD6mZDAGoyp3uFjrjZt+ni2bjs/Gzc+nFKSpzz1h9m377QkJb0fdtJsLcnmkubykubnJc3+klxV0mwXWADA+pmQwBocwWpPxbBhw4FccsmFSUqumHltSklecfanc+B7GzLXdNP8oiRfKck3SnJfaTfl5oHdg47BAgDWyYQE1qGwauPqTW/anO9972m5554/zAP3n56k5IarXpLffurRyY6S3ndLcrDdJdikJA+XNN84smvwmFM0CCwAYIwmILDakatOp90teO65M2maks997sV5wfO/lKc89UfZ/bHXJSlp7urH07aS5pqSfK0/irWnHH1gu8ACANbRBARWDo9enXLKgXzpS8/PXXc9KY897cGUkrzwrC/k1584Iwe/OZ1eOsld7TFWzbbSHoO1tT01wzFBJbAAgHUyAYF15LirjRsfyi/3Pj5bNl+QUpJXPvPa9PaUZHdJb1tJ7i3tKv+ktKdlOHR6hqVGrgQWADBm6xBYzVGX9sube+l0epmens0Pvn9Wvn7rX+Q1z7oys5dPpbe7tCcR3d7fRfhwSZqS5u7+yNVli5xYVGABAOtkjIF1KKran7vd+aOun54+mG5nPm//53/PXLppvluS7SW9LZ1kS0kzU5IDJc23S/Ld/gHu15f2eKzldg8KLABgjNZlBKsdtWr/fcYZ9+XUU393+N+ve/buzP24m/l00vvvktxY0nylJL8syXxJri3JDf2fby/JlgW7CQUWALDOxhRYC3cJJhdcsCW33/7nmZ2dyn33nZEPffgdueBFW3Jw63SaHe2JRHPoNAxpR69ya0nzHyX5WknTlDRfL+1B7kawAIAJMuZdhG1cbdr08STtqRje+MbteefFH2hX5f6SZk//rwN3ljSfaaMqXy/JNSXZXJLdJdnbD66r292Iyx5/JbAAgDEa2wjWoa+/OeeczycpmZk5N1NT7e/+6R8/nIOzGzKXbnq/aiOq2d6Pp6393YCX9YNqb0l6JfnqCHElsACAMVrDwGoG/ttLpzOfUpps2/b3ueOOJ+fR/eOu3vuu9yQp6d1R0ru5tH8heG9JZkqyo7THV+0o7bFXvyzJvn5cHYquYeJKYAEAY7RGgXX0Xwt2u/OZmppNKU2e85ybc+GFl6SU5KJ/+GiSkrkfddPb2sZUs6+0q/WrfkANXj7bD6/lztousACAdbQGgXUkrgb/WnDh5eVPvy6zv5jO3G+76e0syaUlzRdLsr8k/1WS2/q7AAdDalt/NGvUuBJYAMAYVQ6swd2C7c/Pe96Xc955l+Xss7+Tqe5sSmnP0D53ZTe9/SW9O0vyqZJmV0nu7Y9gXbZgF+DxQmqUuBJYAMAYVQyswZGr9iSiF1300f4i2sul296cl//pdeldXtLbVTL/0/6pGP6npPl1O01zy8AuwMGIWuxngQUATJgKgXX0GdqnpmYzNTWXM8+8Mw89tDHXX/+yvG3Tx/KBD16c2WYquae0gbWlk8yUNHeW5MGS3F/a0zFsWWVICSwAYJ1VHcEaPObqrW/9RL7znbNzyvSBlJK84MlfzMO3b2xPxfCT0n634PaS7Gi/U7DZ0f/dWsSVwAIAxmgVgdUsuLRx9ZKX3JBNmz6R97//Xfnkx9+SUpJX/dk16e3ppNlW0ru9vyvwxwMhtfDLmmvHlcACAMZohMA69gD29tLL1NRsOp1eLrzwkgwec/WNbz47b3npJzO3fSrNTElvR7sLsLmtP83P2hGsNY8rgQUAjNGQgXUkqDqdXn9XYHNUbE1NzWbv3sfl2mtfmVe/6pq84fwd+c2+xyQPlTRXliPnudpZ2q+8+XZJvlWOPqfVWl4AAMZkxF2Eg6E13x+9mssb3rAjH/nI2/Otbz0zZ/zefSklee6TvpqHvnhqZg9OpXdfSfaUNIMxta20X4OzcPegwAIATnAjjGC1x1g94Qk/z+mnP3A4sk499Xe57bZnJSn5wo0vTCnJK5756TRXtQex924qyVz/NAxXlqNPHjrOCwDAmAwZWO0uwUc9an9uueUvc9NNL8ppp+07fN2ZZ96ZH3z/rDyw7/S856L35uDmDWl2l/T6JwxtbiztFzQ/WI49zmoco1cCCwAYo5FGsLrd+bz5zZcmKbn++pflMY/Zd3iX4e8/4X9zxw+fnKQNqmZzGzbNzjay8uWS3FqOPajdCBYAcJIZMrCO3k143nmX5VBkHdpdeM4ffz4PXHF6Dt4/nd6BTnJTSTP4dTfby5FjruwiBABOYiOMYLX/7XbnUkpy/vlbk5TMXHVuXv/cXeldXpJdJb2rSvKbkuZgSb7QD6uFUWUECwA4iY04gnV0ZP3Na6/K/x18VLK3tMdcbW+Dqrm6JPtK8nBJrujvFlzPuBJYAMAYjRhYbWRNTx1MKcmLnnJTHrp5Y+Yyld7PSnJ5/ytvtpXkupJ8tYzvNAwCCwCYEEMG1kBcddu4etkzrk/viv7X39xckqakuacku/tBdSi0JiGuBBYAMEZDBNbx42puppveroGvv7m5tLO7e8EuwfXcLSiwAIB1MGRgHS+uOult7yQ7B77+5taS3FaOPrB9EuJKYAEAYzREYCXT3dnF42owoLaVydotKLAAgHWwbHlM9ePqpc/4zOJxdbygmrTIAgAYk6FGsF5+9nWZPxRXO44zcrUwqCYtrgQWADBGy5bHOU//fA7u3JDe5eX4uwVPlAsAwJgMNYJ1Yl2O/NXjod2bf/UnN47jvgQASHLSBdbgKSVmD+/e3L/rlHHclwAASU6qwDr2fF0vfcZnMj/Tab8nEQBgTE6SwFrqZKj9v3oEABiTEyywmoH/LrwsElf9v3oEABiXEySwjkRUKU06nV6mp2czPT2bqe58SmmWPxkqAMCYnACB1Rz1c6fTO2aaTplfOq6MYAEAY3QCBNbgCFYbV2ed9YO8+93vy4c+9I68+lVXp3Sb/PXTPpu5manFv8YHAGBMToDAOhRX7SjV2972sezff0r27n18fnrXH+XB3zw2t3zlOdm/+5RkqZOhAgCMyYQG1tEHs3e7cyklec1rrkxSsnPn3+Vxj/tVSkl27Xx9kpL52zvpXdrGVONM7gDAOprAwGoW/Nyk251PpzOfmZlzc++9f5ANGw6klORf/+WDmetNZfaOqfQu6yS7S5od/dErI1gAwDqZwMBaOILVSylNpqcP5O67n5g9e/42pSTvvvh9SUqaH/eDaldJ7irJTSXZZgQLAFg/ExRYg+e06h0zkjU9fTCfu+HFufvuJ+adF/9bkpL5O7vpbS/J1pLm6pImJfnPkmazwAIA1s+EBNZgTPUOH3O1adPHs3Xb+dm48eGUkjzlqT/Mvn2nJSnp/bCTZmtJNpc0l5c0Py9p9pfkqpJmu8ACANbPhATW4AhWeyqGDRsO5JJLLkxScsXMa1NK8oqzP50D39uQuaab5hcl+UpJvlGS+0q7KTcP7B50DBYAsE4mJLAOhVUbV2960+Z873tPyz33/GEeuP/0JCU3XPWS/PZTj052lPS+W5KD7S7BJiV5uKT5xpFdg8ecokFgAQBjNAGB1Y5cdTrtbsFzz51J05R87nMvzgue/6U85ak/yu6PvS5JSXNXP562lTTXlORr/VGsPeXoA9sFFgCwjiYgsHJ49OqUUw7kS196fu6660l57GkPppTkhWd9Ib/+xBk5+M3p9NJJ7mqPsWq2lfYYrK3tqRmOCSqBBQCskwkIrCPHXW3c+FB+uffx2bL5gpSSvPKZ16a3pyS7S3rbSnJvaVf5J6U9LcOh0zMsNXIlsACAMVuHwGqOurRf3txLp9PL9PRsfvD9s/L1W/8ir3nWlZm9fCq93aU9iej2/i7Ch0vSlDR390euLlvkxKICCwBYJ2MMrENR1f7c7c4fdf309MF0O/N5+z//e+bSTfPdkmwv6W3pJFtKmpmSHChpvl2S7/YPcL++tMdjLbd7UGABAGO0LiNY7ahV++8zzrgvp576u8P/ft2zd2fux93Mp5Pef5fkxpLmKyX5ZUnmS3JtSW7o/3x7SbYs2E0osACAdTamwFq4SzC54IItuf32P8/s7FTuu++MfOjD78gFL9qSg1un0+xoTySaQ6dhSDt6lVtLmv8oyddKmqak+XppD3I3ggUATJAx7yJs42rTpo8naU/F8MY3bs87L/5Auyr3lzR7+n8duLOk+UwbVfl6Sa4pyeaS7C7J3n5wXd3uRlz2+CuBBQCM0dhGsA59/c0553w+ScnMzLmZmmp/90//+OEcnN2QuXTT+1UbUc32fjxt7e8GvKwfVHtL0ivJV0eIK4EFAIzRGgZWM/DfXjqd+ZTSZNu2v88ddzw5j+4fd/Xed70nSUnvjpLezaX9C8F7SzJTkh2lPb5qR2mPvfplSfb14+pQdA0TVwILABijNQqso/9asNudz9TUbEpp8pzn3JwLL7wkpSQX/cNHk5TM/aib3tY2ppp9pV2tX/UDavDy2X54LXfWdoEFAKyjNQisI3E1+NeCCy8vf/p1mf3FdOZ+201vZ0kuLWm+WJL9JfmvktzW3wU4GFLb+qNZo8aVwAIAxqhyYA3uFmx/ft7zvpzzzrssZ5/9nUx1Z1NKe4b2uSu76e0v6d1Zkk+VNLtKcm9/BOuyBbsAjxdSo8SVwAIAxqhiYA2OXLUnEb3ooo/2F9FeLt325rz8T69L7/KS3q6S+Z/2T8XwPyXNr9tpmlsGdgEORtRiPwssAGDCVAiso8/QPjU1m6mpuZx55p156KGNuf76l+Vtmz6WD3zw4sw2U8k9pQ2sLZ1kpqS5syQPluT+0p6OYcsqQ0pgAQDrrOoI1uAxV2996yfyne+cnVOmD6SU5AVP/mIevn1jeyqGn5T2uwW3l2RH+52CzY7+79YirgQWADBGqwisZsGljauXvOSGbNr0ibz//e/KJz/+lpSSvOrPrklvTyfNtpLe7f1dgT8eCKmFX9ZcO64EFgAwRiME1rEHsLeXXqamZtPp9HLhhZdk8Jirb3zz2XnLSz+Zue1TaWZKejvaXYDNbf1pftaOYK15XAksAGCMhgysI0HV6fT6uwKbo2Jramo2e/c+Ltde+8q8+lXX5A3n78hv9j0meaikubIcOc/VztJ+5c23S/KtcvQ5rdbyAgAwJiPuIhwMrfn+6NVc3vCGHfnIR96eb33rmTnj9+5LKclzn/TVPPTFUzN7cCq9+0qyp6QZjKltpf0anIW7BwUWAHCCG2EEqz3G6glP+HlOP/2Bw5F16qm/y223PStJyRdufGFKSV7xzE+nuao9iL13U0nm+qdhuLIcffLQcV4AAMZkyMBqdwk+6lH7c8stf5mbbnpRTjtt3+Hrzjzzzvzg+2flgX2n5z0XvTcHN29Is7uk1z9haHNjab+g+cFy7HFW4xi9ElgAwBiNNILV7c7nzW++NEnJ9de/LI95zL7Duwx//wn/mzt++OQkbVA1m9uwaXa2kZUvl+TWcuxB7UawAICTzJCBdfRuwvPOuyyHIuvQ7sJz/vjzeeCK03Pw/un0DnSSm0qawa+72V6OHHNlFyEAcBIbYQSr/W+3O5dSkvPP35qkZOaqc/P65+5K7/KS7CrpXVWS35Q0B0vyhX5YLYwqI1gAwElsxBGsoyPrb157Vf7v4KOSvaU95mp7G1TN1SXZV5KHS3JFf7fgesaVwAIAxmjEwGoja3rqYEpJXvSUm/LQzRszl6n0flaSy/tfebOtJNeV5KtlfKdhEFgAwIQYMrAG4qrbxtXLnnF9elf0v/7m5pI0Jc09JdndD6pDoTUJcSWwAIAxGiKwjh9XczPd9HYNfP3NzaWd3d0Ldgmu525BgQUArIMhA+t4cdVJb3sn2Tnw9Te3luS2cvSB7ZMQVwILABijIQIrme7OLh5XgwG1rUzWbkGBBQCsg2XLY6ofVy99xmcWj6vjBdWkRRYAwJgMNYL18rOvy/yhuNpxnJGrhUE1aXElsACAMVq2PM55+udzcOeG9C4vx98teKJcAADGZKgRrBPrcuSvHg/t3vyrP7lxHPclAECSky6wBk8pMXt49+b+XaeM474EAEhyUgXWsefreukzPpP5mU77PYkAAGNykgTWUidD7f/VIwDAmJxggdUM/HfhZZG46v/VIwDAuJwggXUkokpp0un0Mj09m+np2Ux151NKs/zJUAEAxuQECKzmqJ87nd4x03TK/NJxZQQLABijEyCwBkew2rg666wf5N3vfl8+9KF35NWvujql2+Svn/bZzM1MLf41PgAAY3ICBNahuGpHqd72to9l//5Tsnfv4/PTu/4oD/7msbnlK8/J/t2nJEudDBUAYEwmNLCOPpi9251LKclrXnNlkpKdO/8uj3vcr1JKsmvn65OUzN/eSe/SNqYaZ3IHANbRBAZWs+DnJt3ufDqd+czMnJt77/2DbNhwIKUk//ovH8xcbyqzd0yld1kn2V3S7OiPXhnBAgDWyQQG1sIRrF5KaTI9fSB33/3E7NnztykleffF70tS0vy4H1S7SnJXSW4qyTYjWADA+pmgwBo8p1XvmJGs6emD+dwNL87ddz8x77z435KUzN/ZTW97SbaWNFeXNCnJf5Y0mwUWALB+JiSwBmOqd/iYq02bPp6t287Pxo0Pp5TkKU/9YfbtOy1JSe+HnTRbS7K5pLm8pPl5SbO/JFeVNNsFFgCwfiYksAZHsNpTMWzYcCCXXHJhkpIrZl6bUpJXnP3pHPjehsw13TS/KMlXSvKNktxX2k25eWD3oGOwAIB1MiGBdSis2rh605s253vfe1ruuecP88D9pycpueGql+S3n3p0sqOk992SHGx3CTYpycMlzTeO7Bo85hQNAgsAGKMJCKx25KrTaXcLnnvuTJqm5HOfe3Fe8Pwv5SlP/VF2f+x1SUqau/rxtK2kuaYkX+uPYlXoaWwAAA4zSURBVO0pRx/YLrAAgHU0AYGVw6NXp5xyIF/60vNz111PymNPezClJC886wv59SfOyMFvTqeXTnJXe4xVs620x2BtbU/NcExQCSwAYJ1MQGAdOe5q48aH8su9j8+WzReklOSVz7w2vT0l2V3S21aSe0u7yj8p7WkZDp2eYamRK4EFAIzZOgRWc9Sl/fLmXjqdXqanZ/OD75+Vr9/6F3nNs67M7OVT6e0u7UlEt/d3ET5ckqakubs/cnXZIicWFVgAwDoZY2Adiqr25253/qjrp6cPptuZz9v/+d8zl26a75Zke0lvSyfZUtLMlORASfPtkny3f4D79aU9Hmu53YMCCwAYo3UZwWpHrdp/n3HGfTn11N8d/vfrnr07cz/uZj6d9P67JDeWNF8pyS9LMl+Sa0tyQ//n20uyZcFuQoEFAKyzMQXWwl2CyQUXbMntt/95Zmenct99Z+RDH35HLnjRlhzcOp1mR3si0Rw6DUPa0avcWtL8R0m+VtI0Jc3XS3uQuxEsAGCCjHkXYRtXmzZ9PEl7KoY3vnF73nnxB9pVub+k2dP/68CdJc1n2qjK10tyTUk2l2R3Sfb2g+vqdjfissdfCSwAYIzGNoJ16Otvzjnn80lKZmbOzdRU+7t/+scP5+Dshsylm96v2ohqtvfjaWt/N+Bl/aDaW5JeSb46QlwJLABgjNYwsJqB//bS6cynlCbbtv197rjjyXl0/7ir977rPUlKeneU9G4u7V8I3luSmZLsKO3xVTtKe+zVL0uyrx9Xh6JrmLgSWADAGK1RYB3914Ld7nympmZTSpPnPOfmXHjhJSkluegfPpqkZO5H3fS2tjHV7Cvtav2qH1CDl8/2w2u5s7YLLABgHa1BYB2Jq8G/Flx4efnTr8vsL6Yz99tuejtLcmlJ88WS7C/Jf5Xktv4uwMGQ2tYfzRo1rgQWADBGlQNrcLdg+/PznvflnHfeZTn77O9kqjubUtoztM9d2U1vf0nvzpJ8qqTZVZJ7+yNYly3YBXi8kBolrgQWADBGFQNrcOSqPYnoRRd9tL+I9nLptjfn5X96XXqXl/R2lcz/tH8qhv8paX7dTtPcMrALcDCiFvtZYAEAE6ZCYB19hvapqdlMTc3lzDPvzEMPbcz1178sb9v0sXzggxdntplK7iltYG3pJDMlzZ0lebAk95f2dAxbVhlSAgsAWGdVR7AGj7l661s/ke985+ycMn0gpSQvePIX8/DtG9tTMfyktN8tuL0kO9rvFGx29H+3FnElsACAMVpFYDULLm1cveQlN2TTpk/k/e9/Vz758beklORVf3ZNens6abaV9G7v7wr88UBILfyy5tpxJbAAgDEaIbCOPYC9vfQyNTWbTqeXCy+8JIPHXH3jm8/OW176ycxtn0ozU9Lb0e4CbG7rT/OzdgRrzeNKYAEAYzRkYB0Jqk6n198V2BwVW1NTs9m793G59tpX5tWvuiZvOH9HfrPvMclDJc2V5ch5rnaW9itvvl2Sb5Wjz2m1lhcAgDEZcRfhYGjN90ev5vKGN+zIRz7y9nzrW8/MGb93X0pJnvukr+ahL56a2YNT6d1Xkj0lzWBMbSvt1+As3D0osACAE9wII1jtMVZPeMLPc/rpDxyOrFNP/V1uu+1ZSUq+cOMLU0ryimd+Os1V7UHsvZtKMtc/DcOV5eiTh47zAgAwJkMGVrtL8FGP2p9bbvnL3HTTi3LaafsOX3fmmXfmB98/Kw/sOz3vuei9Obh5Q5rdJb3+CUObG0v7Bc0PlmOPsxrH6JXAAgDGaKQRrG53Pm9+86VJSq6//mV5zGP2Hd5l+PtP+N/c8cMnJ2mDqtnchk2zs42sfLkkt5ZjD2o3ggUAnGSGDKyjdxOed95lORRZh3YXnvPHn88DV5yeg/dPp3egk9xU0gx+3c32cuSYK7sIAYCT2AgjWO1/u925lJKcf/7WJCUzV52b1z93V3qXl2RXSe+qkvympDlYki/0w2phVBnBAgBOYiOOYB0dWX/z2qvyfwcflewt7TFX29ugaq4uyb6SPFySK/q7BdczrgQWADBGIwZWG1nTUwdTSvKip9yUh27emLlMpfezklze/8qbbSW5riRfLeM7DYPAAgAmxJCBNRBX3TauXvaM69O7ov/1NzeXpClp7inJ7n5QHQqtSYgrgQUAjNEQgXX8uJqb6aa3a+Drb24u7ezuXrBLcD13CwosAGAdDBlYx4urTnrbO8nOga+/ubUkt5WjD2yfhLgSWADAGA0RWMl0d3bxuBoMqG1lsnYLCiwAYB0sWx5T/bh66TM+s3hcHS+oJi2yAADGZKgRrJeffV3mD8XVjuOMXC0MqkmLK4EFAIzRsuVxztM/n4M7N6R3eTn+bsET5QIAMCbKAwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBUJrAAACoTWAAAlQksAIDKBBYAQGUCCwCgMoEFAFCZwAIAqExgAQBU9v8G7vocVHkgKQAAAABJRU5ErkJggg==
13628
[task 2018-01-09T05:14:41.674Z] 05:14:41     INFO - TEST-INFO took 173ms
13629
[task 2018-01-09T05:14:41.752Z] 05:14:41     INFO - PID 5515 | 
13630
[task 2018-01-09T05:14:41.753Z] 05:14:41     INFO - PID 5515 | ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
13631
[task 2018-01-09T05:14:41.754Z] 05:14:41     INFO - PID 5515 | 
13632
[task 2018-01-09T05:14:41.755Z] 05:14:41     INFO - PID 5515 | [Child 5609, Main Thread] WARNING: NS_ENSURE_TRUE(maybeContext) failed: file /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp, line 791
13633
[task 2018-01-09T05:14:41.997Z] 05:14:41     INFO - PID 5515 | 
13634
[task 2018-01-09T05:14:41.998Z] 05:14:41     INFO - PID 5515 | ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
13635
[task 2018-01-09T05:14:41.999Z] 05:14:41     INFO - PID 5515 | 
13636
[task 2018-01-09T05:14:42.006Z] 05:14:42     INFO - PID 5515 | [Child 5570, Main Thread] WARNING: NS_ENSURE_TRUE(maybeContext) failed: file /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp, line 791
13637
[task 2018-01-09T05:14:42.007Z] 05:14:42     INFO - PID 5515 | [Child 5570, Main Thread] WARNING: NS_ENSURE_TRUE(maybeContext) failed: file /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp, line 791
13638
[task 2018-01-09T05:14:42.008Z] 05:14:42     INFO - PID 5515 | [Child 5570, Main Thread] WARNING: NS_ENSURE_TRUE(maybeContext) failed: file /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp, line 791
13639
[task 2018-01-09T05:14:42.197Z] 05:14:42     INFO - PID 5515 | --DOMWINDOW == 4 (0x7f96a0384400) [pid = 5570] [serial = 4] [outer = (nil)] [url = about:blank]
13640
[task 2018-01-09T05:14:42.198Z] 05:14:42     INFO - PID 5515 | --DOMWINDOW == 3 (0x7f96a0378800) [pid = 5570] [serial = 3] [outer = (nil)] [url = about:blank]
13641
[task 2018-01-09T05:14:42.198Z] 05:14:42     INFO - PID 5515 | --DOMWINDOW == 2 (0x7f96a00d4400) [pid = 5570] [serial = 2] [outer = (nil)] [url = about:blank]
13642
[task 2018-01-09T05:14:42.199Z] 05:14:42     INFO - PID 5515 | --DOCSHELL 0x7f96a024c000 == 0 [pid = 5570] [id = {1a452fd4-8ae2-4c98-beba-16212037e470}]
13643
[task 2018-01-09T05:14:42.481Z] 05:14:42     INFO - PID 5515 | --DOMWINDOW == 1 (0x7f96a1679660) [pid = 5570] [serial = 1] [outer = (nil)] [url = http://web-platform.test:8000/testharness_runner.html]
13644
[task 2018-01-09T05:14:42.481Z] 05:14:42     INFO - PID 5515 | [Child 5570, Main Thread] WARNING: '!mMainThread', file /builds/worker/workspace/build/src/xpcom/threads/nsThreadManager.cpp, line 403
13645
[task 2018-01-09T05:14:42.481Z] 05:14:42     INFO - PID 5515 | [Child 5570, Main Thread] WARNING: '!mMainThread', file /builds/worker/workspace/build/src/xpcom/threads/nsThreadManager.cpp, line 403
13646
[task 2018-01-09T05:14:42.481Z] 05:14:42     INFO - PID 5515 | [Child 5570, Main Thread] WARNING: '!mMainThread', file /builds/worker/workspace/build/src/xpcom/threads/nsThreadManager.cpp, line 403
13647
[task 2018-01-09T05:14:42.481Z] 05:14:42     INFO - PID 5515 | [Child 5570, Main Thread] WARNING: '!mMainThread', file /builds/worker/workspace/build/src/xpcom/threads/nsThreadManager.cpp, line 403
13648
[task 2018-01-09T05:14:42.481Z] 05:14:42     INFO - PID 5515 | [Child 5570, Main Thread] WARNING: '!mMainThread', file /builds/worker/workspace/build/src/xpcom/threads/nsThreadManager.cpp, line 403
13649
[task 2018-01-09T05:14:42.482Z] 05:14:42     INFO - PID 5515 | [Child 5570, Main Thread] WARNING: '!mMainThread', file /builds/worker/workspace/build/src/xpcom/threads/nsThreadManager.cpp, line 403
13650
[task 2018-01-09T05:14:42.482Z] 05:14:42     INFO - PID 5515 | [Child 5570, Main Thread] WARNING: '!mMainThread', file /builds/worker/workspace/build/src/xpcom/threads/nsThreadManager.cpp, line 403
13651
[task 2018-01-09T05:14:42.482Z] 05:14:42     INFO - PID 5515 | [Child 5570, Main Thread] WARNING: '!mMainThread', file /builds/worker/workspace/build/src/xpcom/threads/nsThreadManager.cpp, line 403
13652
[task 2018-01-09T05:14:42.482Z] 05:14:42     INFO - PID 5515 | --DOMWINDOW == 0 (0x7f96a16eb400) [pid = 5570] [serial = 5] [outer = (nil)] [url = http://web-platform.test:8000/testharness_runner.html]
13653
[task 2018-01-09T05:14:42.489Z] 05:14:42     INFO - PID 5515 | --DOCSHELL 0x7fca6abbd800 == 3 [pid = 5515] [id = {c2e31cf2-1aba-4842-9463-5baa761268a5}]
13654
[task 2018-01-09T05:14:42.506Z] 05:14:42     INFO - PID 5515 | --DOMWINDOW == 10 (0x7fca92bd8800) [pid = 5515] [serial = 2] [outer = (nil)] [url = about:blank]
13655
[task 2018-01-09T05:14:42.506Z] 05:14:42     INFO - PID 5515 | --DOCSHELL 0x7fcab037a000 == 2 [pid = 5515] [id = {ac05895b-a43c-48d7-be75-be7a5bd04d7a}]
13656
[task 2018-01-09T05:14:42.528Z] 05:14:42     INFO - PID 5515 | nsStringStats
13657
[task 2018-01-09T05:14:42.528Z] 05:14:42     INFO - PID 5515 |  => mAllocCount:          17761
13658
[task 2018-01-09T05:14:42.528Z] 05:14:42     INFO - PID 5515 |  => mReallocCount:          112
13659
[task 2018-01-09T05:14:42.528Z] 05:14:42     INFO - PID 5515 |  => mFreeCount:           17761
13660
[task 2018-01-09T05:14:42.529Z] 05:14:42     INFO - PID 5515 |  => mShareCount:           8823
13661
[task 2018-01-09T05:14:42.530Z] 05:14:42     INFO - PID 5515 |  => mAdoptCount:           1526
13662
[task 2018-01-09T05:14:42.530Z] 05:14:42     INFO - PID 5515 |  => mAdoptFreeCount:       1526
13663
[task 2018-01-09T05:14:42.530Z] 05:14:42     INFO - PID 5515 |  => Process ID: 5570, Thread ID: 140285391853376
13664
[task 2018-01-09T05:14:42.530Z] 05:14:42     INFO - PID 5515 | --DOCSHELL 0x7fca877a4000 == 1 [pid = 5515] [id = {3207091e-0a5a-4a6d-ac69-5d8b3f6a949e}]
13665
[task 2018-01-09T05:14:42.530Z] 05:14:42     INFO - PID 5515 | --DOCSHELL 0x7fca9203c800 == 0 [pid = 5515] [id = {8546835e-dccd-4073-ab9a-fdd0ee82567b}]
13666
[task 2018-01-09T05:14:42.545Z] 05:14:42     INFO - PID 5515 | 
13667
[task 2018-01-09T05:14:42.546Z] 05:14:42     INFO - PID 5515 | ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
13668
[task 2018-01-09T05:14:42.546Z] 05:14:42     INFO - PID 5515 | 
13669
[task 2018-01-09T05:14:42.547Z] 05:14:42     INFO - PID 5515 | [Child 5667, Main Thread] WARNING: NS_ENSURE_TRUE(maybeContext) failed: file /builds/worker/workspace/build/src/xpcom/threads/nsThread.cpp, line 791
13670
[task 2018-01-09T05:14:42.547Z] 05:14:42     INFO - PID 5515 | [Child 5667, Main Thread] WARNING: '!gThread', file /builds/worker/workspace/build/src/xpcom/threads/nsTimerImpl.cpp, line 399
13671
[task 2018-01-09T05:14:42.568Z] 05:14:42     INFO - PID 5515 | --DOCSHELL 0x7f28ef2df000 == 0 [pid = 5609] [id = {754b921c-17e1-40e4-ae24-59fad7e2a669}]
13672
[task 2018-01-09T05:14:42.670Z] 05:14:42     INFO - PID 5515 | [Child 5667, Main Thread] WARNING: '!gThread', file /builds/worker/workspace/build/src/xpcom/threads/nsTimerImpl.cpp, line 399
13673
[task 2018-01-09T05:14:42.671Z] 05:14:42     INFO - PID 5515 | [Child 5667, Main Thread] WARNING: '!gThread', file /builds/worker/workspace/build/src/xpcom/threads/nsTimerImpl.cpp, line 399
13674
[task 2018-01-09T05:14:42.692Z] 05:14:42     INFO - PID 5515 | [Child 5667, Main Thread] WARNING: '!gThread', file /builds/worker/workspace/build/src/xpcom/threads/nsTimerImpl.cpp, line 399
13675
[task 2018-01-09T05:14:42.728Z] 05:14:42     INFO - PID 5515 | nsStringStats
:milan Could you please take a look?
Flags: needinfo?(milan)
This bug is assigned and there are patches just waiting for review comments to be addressed.
Flags: needinfo?(milan)
:tnikkel, there is a patch here for review from 2+ months ago- we continue to have a very high rate of failures- can you please take a look at this patch soon so we can land it or the patch can be updated to pass review and then land?
Flags: needinfo?(tnikkel)
I'm waiting for review comments to be addressed, see comment 48, comment 54.
Flags: needinfo?(tnikkel)
Comment on attachment 8936648 [details] [diff] [review]
0001-Bug-1383682-imgRequestProxy-should-avoid-leaking-sta.patch, v4

:aosmond, can you address the feedback from comment 48?
Flags: needinfo?(aosmond)
Attachment #8936648 - Flags: review?(tnikkel)
This patch mostly splits off the validation state tracking from regular notification deferral.
Attachment #8936648 - Attachment is obsolete: true
Flags: needinfo?(aosmond)
Attachment #8947568 - Flags: review?(tnikkel)
This part renames SetNotificationsDeferred to Mark/ClearPendingNotify to make things more clear.
Attachment #8947569 - Flags: review?(tnikkel)
This part prevents StartDecoding/StartDecodingWithResult/GetImageStatus from leaking state information when validating a request.
Attachment #8947570 - Flags: review?(tnikkel)
Comment on attachment 8947568 [details] [diff] [review]
0001-Bug-1383682-Part-1.-Split-off-imgRequestProxy-notifi.patch, v1

Review of attachment 8947568 [details] [diff] [review]:
-----------------------------------------------------------------

::: image/imgLoader.cpp
@@ +2947,5 @@
> +
> +void
> +imgCacheValidator::UpdateProxies()
> +{
> +  for (auto& proxy : mProxies) {

ChangeOwner can SyncNotify too (if mCancelled) so shouldn't we make a copy of the mProxies array in case it is changed? I guess the old code didn't do this, so is that a bug in the old code?

::: image/imgRequestProxy.cpp
@@ +270,2 @@
>  
>    // If we'd previously requested a synchronous decode, request a decode on the

Why move the decode requesting into ClearValidating? It adds an ordering requirement for the ChangeOwner/ClearValidating call for what benefit?
Attachment #8947569 - Flags: review?(tnikkel) → review+
Attachment #8947570 - Flags: review?(tnikkel) → review+
(In reply to Timothy Nikkel (:tnikkel) from comment #65)
> Comment on attachment 8947568 [details] [diff] [review]
> 0001-Bug-1383682-Part-1.-Split-off-imgRequestProxy-notifi.patch, v1
> 
> Review of attachment 8947568 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: image/imgLoader.cpp
> @@ +2947,5 @@
> > +
> > +void
> > +imgCacheValidator::UpdateProxies()
> > +{
> > +  for (auto& proxy : mProxies) {
> 
> ChangeOwner can SyncNotify too (if mCancelled) so shouldn't we make a copy
> of the mProxies array in case it is changed? I guess the old code didn't do
> this, so is that a bug in the old code?
> 

Grrr, you are right. I missed this. This is a bug in the old code. Maybe I should make imgCacheValidator::RemoveProxy dispatch. It should only be called rarely.

> ::: image/imgRequestProxy.cpp
> @@ +270,2 @@
> >  
> >    // If we'd previously requested a synchronous decode, request a decode on the
> 
> Why move the decode requesting into ClearValidating? It adds an ordering
> requirement for the ChangeOwner/ClearValidating call for what benefit?

I wanted to call StartDecoding if deferred even if ChangeOwner was not called.
(In reply to Andrew Osmond [:aosmond] from comment #67)
> (In reply to Timothy Nikkel (:tnikkel) from comment #65)
> > Comment on attachment 8947568 [details] [diff] [review]
> > 0001-Bug-1383682-Part-1.-Split-off-imgRequestProxy-notifi.patch, v1
> > 
> > Review of attachment 8947568 [details] [diff] [review]:
> > -----------------------------------------------------------------
> > 
> > ::: image/imgLoader.cpp
> > @@ +2947,5 @@
> > > +
> > > +void
> > > +imgCacheValidator::UpdateProxies()
> > > +{
> > > +  for (auto& proxy : mProxies) {
> > 
> > ChangeOwner can SyncNotify too (if mCancelled) so shouldn't we make a copy
> > of the mProxies array in case it is changed? I guess the old code didn't do
> > this, so is that a bug in the old code?
> > 
> 
> Grrr, you are right. I missed this. This is a bug in the old code. Maybe I
> should make imgCacheValidator::RemoveProxy dispatch. It should only be
> called rarely.
> 

Actually while ChangeOwner can call SyncNotify, it will be deferred as I haven't cleared the validation flag yet:

https://searchfox.org/mozilla-central/rev/f80722d4f3bfb722c5ec53880c4a7efb71285676/image/ProgressTracker.cpp#305

This strikes me as sort of strange, but it is *not* a change in behaviour as the original code does the same thing:

https://searchfox.org/mozilla-central/rev/f80722d4f3bfb722c5ec53880c4a7efb71285676/image/imgLoader.cpp#3065

I did however move the array onto the stack, just in case. I think the SyncNotifyListener in the second loop is still unsafe for similar reasons...

try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=35ba549bbf86515591d0bb2dd9034eae2498a376
Attachment #8947568 - Attachment is obsolete: true
Attachment #8947568 - Flags: review?(tnikkel)
Attachment #8948670 - Flags: review?(tnikkel)
Attachment #8948670 - Flags: review?(tnikkel) → review+
Blocks: 1436323
Pushed by aosmond@gmail.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/c657d87a0011
Part 1. Split off imgRequestProxy notification deferrals for validation. r=tnikkel
https://hg.mozilla.org/integration/mozilla-inbound/rev/62497967f092
Part 2. Rename IProgressObserver::SetNotificationsDeferred to make purpose clear. r=tnikkel
https://hg.mozilla.org/integration/mozilla-inbound/rev/30ee5a21a6a1
Part 3. Prevent imgRequestProxy from leaking the current state when validating. r=tnikkel
https://hg.mozilla.org/mozilla-central/rev/c657d87a0011
https://hg.mozilla.org/mozilla-central/rev/62497967f092
https://hg.mozilla.org/mozilla-central/rev/30ee5a21a6a1
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla60
Depends on: 1439285
Depends on: 1445479
Depends on: 1523555
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: