Closed Bug 1605283 Opened 2 years ago Closed 1 year ago

Improve support for invalidation debugging and testing

Categories

(Core :: Graphics: WebRender, enhancement, P3)

enhancement

Tracking

()

RESOLVED FIXED

People

(Reporter: gw, Assigned: bpeers)

References

(Blocks 3 open bugs)

Details

Attachments

(10 files)

No description provided.

Background:

There are many reasons that a picture cache tile may be invalidated and re-rasterized. For example, opacity animations, primitive content changes, texture cache evictions, relative transform changes.

When WR decides to invalidate a tile, it also tries to calculate a minimal update rectangle. This update region is set as a scissor rect when rasterizing the new tile content, and can also be supplied to the embedder for use by various operating system level features (e.g. partial present, or native compositor update regions).

If we fail to invalidate a tile when something has changed, we get a correctness issue (stale content). If we conservatively invalidate a tile when the content hasn't actually changed, we do redundant work, wasting power and performance.

The second case, doing redundant work, is something that happens quite often, which we'd like to improve.

These redundant invalidations can be caused by a number of factors - for example, Gecko may supply a display list with slightly different values that trips up the WR primitive interning / caching logic, or WR may have a bug in how it checks primitive dependencies, or WR may simply be missing an optimization opportunity to recognize that some change can't affect a specific tile or update region.

We currently have some support in WR for dumping the state of tile invalidation calculations, but debugging such issues is error prone and time consuming.

Goal:

Add functionality in WR that:
(1) Makes it simpler and more efficient to diagnose why tiles are being invalidated and find the root cause.
(2) Can be used in wrench reftests to regression test and verify various invalidation scenarios.

There is already support for dumping the state of a picture cache dependency state to stdout [1]. By dumping these, it's possible via a visual diff tool to identify the differences in the tile dependencies. If the invalidation is due to interning differences, these can be debugged by logging information via [2].

For (1) we can probably build on the functionality in [1] and [2]. Perhaps we could extend that and add support for logging each frame in a more structured way to disk that can be analyzed by a small external command line tool that identifies relevant differences? Invalidations often happen over a number of frames, during an animation or scroll, so there is often a lot of data produced that is time consuming to analyze manually.

For (2) we could perhaps have a small set of information that can be specified along with the reftest identifying what parts of the scene get invalidated, and what the reasons are. This could then be compared with the output discussed in (1) to ensure that expected invalidations are occurring?

Those are just a couple of ideas, there might be better ways to achieve the overall goal. I think it's going to take some prototyping and iteration to find the ideal solution.

References:

[1] https://searchfox.org/mozilla-central/rev/b243debf6235b050b42fd2eb615fdc729636ca6b/gfx/wr/webrender/src/picture.rs#2232
[2] https://searchfox.org/mozilla-central/rev/b243debf6235b050b42fd2eb615fdc729636ca6b/gfx/wr/webrender/src/intern.rs#278

Priority: -- → P3
Assignee: nobody → bpeers

I've spent a day or two testing out ETL/WPA for item (1). WPA is high-bandwidth & low-overhead so it's great for "always on" logging. We could add markers with detailed info about what we're invalidating and why. Plus we already use it in some cases like debugging/visualizing the WDM compositor, so it could be nice to have the info about cache invalidation and the WDM all in one place (plus all the other WPA bonus stuff).
It's doable, just a bit messy so it will take more time to promote from "quick hack" to "shippable".

Still, it's probably not enough to solve (1), just a nice supplement to it. Recording and debugging all tile activity could use its own log and tool. I'll think about that a bit more.

For (2) we need to be careful IMHO that we're not spending too many resources testing an implementation detail. If we augment reftests with precise invalidation expectations, that could be a lot of work to keep up to date as we optimize and rewrite.
I agree that it'd be great to catch a regression in the number of unnecessary invalidations, we just need to find the right granularity to test at.

Just a few thoughts for now. Thanks :)

Optionally serialize N frames into a circular memory buffer, and save
them as part of wr-capture into tilecache/.

The new tile_view utility reads the folder and converts the frames to
svg for easier visualization, with a few extra features:

  • color code each primitive based on which slice it is on;
  • highlight new or moving primitives in red (brute force diff);
  • print all invalidated tiles at the top and the invalidation reason;
  • overlay the tile quadtree to visualize splitting & merging;
  • HTML and JS wrappers for animation playback, timeline scrubbing;

Positioning of the tiles on the screen is a bit broken still; especially
during scrolling and "special" pages (like about:config).

Interning info is not used yet.

Pushed by bpeers@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/53cce741bb46
Improve support for invalidation debugging and testing r=gw
Keywords: leave-open

Backed out for bustage related to cargo-lock

backout: https://hg.mozilla.org/integration/autoland/rev/9f1672510747b88720ce08bf1aec726b17d2a544

push: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=53cce741bb469480c121b023c94b58618e6a3355&group_state=expanded&selectedJob=285287355

failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=285287355&repo=autoland&lineNumber=213

[task 2020-01-17T00:38:15.114Z] export NDK_HOME="${MOZ_FETCHES_DIR}/android-ndk"
[task 2020-01-17T00:38:15.114Z] + export NDK_HOME=/builds/worker/fetches/android-ndk
[task 2020-01-17T00:38:15.114Z] + NDK_HOME=/builds/worker/fetches/android-ndk
[task 2020-01-17T00:38:15.114Z]
[task 2020-01-17T00:38:15.114Z] # cargo apk build fails whilst attempting to sign the output apk,
[task 2020-01-17T00:38:15.114Z] # unless the .android directory exists.
[task 2020-01-17T00:38:15.114Z] # See https://github.com/rust-windowing/android-rs-glue/issues/252
[task 2020-01-17T00:38:15.114Z] mkdir /builds/worker/.android
[task 2020-01-17T00:38:15.114Z] + mkdir /builds/worker/.android
[task 2020-01-17T00:38:15.115Z] if [ "$MODE" == "debug" ]; then
[task 2020-01-17T00:38:15.115Z] ../cargo-apk/bin/cargo-apk build --frozen --verbose
[task 2020-01-17T00:38:15.115Z] elif [ "$MODE" == "release" ]; then
[task 2020-01-17T00:38:15.115Z] ../cargo-apk/bin/cargo-apk build --frozen --verbose --release
[task 2020-01-17T00:38:15.115Z] else
[task 2020-01-17T00:38:15.115Z] echo "Unknown mode '${MODE}'; must be 'debug' or 'release'"
[task 2020-01-17T00:38:15.115Z] exit 1
[task 2020-01-17T00:38:15.115Z] fi
[task 2020-01-17T00:38:15.115Z] + '[' debug == debug ']'
[task 2020-01-17T00:38:15.115Z] + ../cargo-apk/bin/cargo-apk build --frozen --verbose
[task 2020-01-17T00:38:15.136Z] Compiling injected-glue for armv7-linux-androideabi
[task 2020-01-17T00:38:15.785Z] error: the lock file /builds/worker/checkouts/gecko/gfx/wr/Cargo.lock needs to be updated but --frozen was passed to prevent this
[fetches 2020-01-17T00:38:15.787Z] removing /builds/worker/fetches
[fetches 2020-01-17T00:38:17.954Z] finished
[taskcluster 2020-01-17 00:38:18.470Z] === Task Finished ===
[taskcluster 2020-01-17 00:38:18.598Z] Artifact "public/build/wrench-debug.apk" not found at "/builds/worker/checkouts/gecko/gfx/wr/target/android-artifacts/debug/apk/wrench.apk"
[taskcluster 2020-01-17 00:38:18.698Z] Artifact "public/build/reftests.tar.gz" not found at "/builds/worker/checkouts/gecko/gfx/wr/wrench/reftests.tar.gz"
[taskcluster 2020-01-17 00:38:18.822Z] Unsuccessful task run with exit code: 101 completed in 162.637 seconds

Flags: needinfo?(bpeers)

Hi, i don't know what any of this means; what exactly do you need me to do here to fix this?

Flags: needinfo?(bpeers) → needinfo?(ncsoregi)

Hi Glenn, can you help out with this one? Thank you.

Flags: needinfo?(ncsoregi) → needinfo?(gwatson)

I think if you run ./mach vendor rust locally, and then add the updated files to your commit, that should fix the issue.

You'll also want to check what that command does - in particular, if we're adding new third party rust crates, we need to double check the license on them to make sure they're fine to use.

Flags: needinfo?(gwatson)

Hm, I haven't added any new crates, or at least not intentionally.
I tried the command, alas no luck:

$ ./mach vendor rust                                                                                               
    Updating crates.io index                                                                                       
    Updating git repository `https://github.com/ChunMinChang/cubeb-coreaudio-rs`                                   
    Updating git repository `https://github.com/djg/cubeb-pulse-rs`                                                
    Updating git repository `https://github.com/jfkthame/mapped_hyph.git`                                          
    Updating git repository `https://github.com/mozilla/rkv`                                                       
    Updating git repository `https://github.com/mozilla/neqo`                                                      
    Updating git repository `https://github.com/PLSysSec/lucet_sandbox_compiler`                                   
    Updating git repository `https://github.com/alexcrichton/mio-named-pipes`                                      
    Updating git repository `https://github.com/NikVolf/tokio-named-pipes`                                         
Error running mach:                                                                                                
                                                                                                                   
    ['vendor', 'rust']                                                                                             
                                                                                                                   
The error occurred in code that was called by the mach command. This is either                                     
a bug in the called code itself or in the way that mach is calling it.                                             
You can invoke |./mach busted| to check if this issue is already on file. If it                                    
isn't, please use |./mach busted file| to report it. If |./mach busted| is                                         
misbehaving, you can also inspect the dependencies of bug 1543241.                                                 
                                                                                                                   
If filing a bug, please include the full output of mach, including this error                                      
message.                                                                                                           
                                                                                                                   
The details of the failure are as follows:                                                                         
                                                                                                                   
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 634: character maps to <undefined>          
                                                                                                                   
  File "c:\code\1605283_tilecache\python/mozbuild/mozbuild/mach_commands.py", line 1326, in vendor_rust            
    vendor_command.vendor(**kwargs)                                                                                
  File "c:\code\1605283_tilecache\python/mozbuild\mozbuild\vendor_rust.py", line 431, in vendor                    
    if not self._check_licenses(vendor_dir):                                                                       
  File "c:\code\1605283_tilecache\python/mozbuild\mozbuild\vendor_rust.py", line 342, in _check_licenses           
    results = [check_package(p) for p in os.listdir(vendor_dir)                                                    
  File "c:\code\1605283_tilecache\python/mozbuild\mozbuild\vendor_rust.py", line 343, in <listcomp>                
    if os.path.isdir(os.path.join(vendor_dir, p))]                                                                 
  File "c:\code\1605283_tilecache\python/mozbuild\mozbuild\vendor_rust.py", line 285, in check_package             
    license_lines = [l for l in f if l.strip().startswith('license')]                                              
  File "c:\code\1605283_tilecache\python/mozbuild\mozbuild\vendor_rust.py", line 285, in <listcomp>                
    license_lines = [l for l in f if l.strip().startswith('license')]                                              
  File "c:\mozilla-build\python3\lib\encodings\cp1252.py", line 23, in decode                                      
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]                                              

The error sounds vaguely similar to Bug 1606970.

Any suggestions Natalia on how to proceed?

Flags: needinfo?(ncsoregi)

Any ideas?

Flags: needinfo?(jmuizelaar)
Flags: needinfo?(a.beingessner)
Flags: needinfo?(ncsoregi)

Need to update the lockfile in wrench and co.

Hi Emilio, thanks for the suggestion.

There are only three Cargo.lock files outside of third_party\. One is gfx\wr, which I've updated as part of the commit. The other two are in the root and in gfx\wgpu and they haven't changed. So I'm not clear which file needs to be updated and how.

I haven't added new crates, haven't bumped any versions and haven't made changes to wrench.
Thanks.

From a glance it looks like you're requiring a higher version of ron than is in the gfx/wr/Cargo.lock. So it won't build unless the Cargo.lock file is updated, which is prohibited in automation.

Edit: whoops, misread 5 as 15. Ignore this!

Pushed by bpeers@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/05c1456103ea
Improve support for invalidation debugging and testing r=gw
Pushed by bpeers@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/edd05995832f
Improve support for invalidation debugging and testing r=gw
Flags: needinfo?(bpeers)
Attachment #9120949 - Attachment description: Bug 1605283 - Improve support for invalidation debugging and testing → Bug 1605283 - Improve support for invalidation debugging and testing r=gw

Second part: trace the updates that are sent to the DataStore, and save
at least the Insert/Remove and ItemUID as part of the wr-capture.
(We could expand this with more info, eg. the actual Keys, later).

TileView then reads them back and generates a color coded report to
overlay with the page view. This helps to see the types and amounts of
interned primitives that lead to cache invalidations.

Flags: needinfo?(a.beingessner)
Flags: needinfo?(jmuizelaar)
Pushed by bpeers@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/2e04c320ad23
Improve support for invalidation debugging and testing r=gw
Pushed by bpeers@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/7ed018d13880
Improve support for invalidation debugging and testing r=gw

Backed out changeset 7ed018d13880 (Bug 1605283) for causing build bustage with could not compile webrender error.

Push with failure: https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&selectedJob=286105591&resultStatus=testfailed%2Cbusted%2Cexception&classifiedState=unclassified&revision=7ed018d13880bee244338bf6e1c65ab17d58a870

Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=286105591&repo=autoland&lineNumber=815

Backout link: https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&resultStatus=testfailed%2Cbusted%2Cexception&classifiedState=unclassified&revision=693c1fe229bc143776742206a9c4338aa01ae5db

[task 2020-01-23T06:34:09.284Z]      Running `rustc --edition=2018 --crate-name webrender webrender/src/lib.rs --color never --crate-type lib --emit=dep-info,metadata,link -C panic=abort -C debuginfo=2 --cfg 'feature="default"' --cfg 'feature="freetype-lib"' --cfg 'feature="replay"' --cfg 'feature="ron"' --cfg 'feature="serde"' -C metadata=d91f485b8b4dea1f -C extra-filename=-d91f485b8b4dea1f --out-dir /builds/worker/checkouts/gecko/gfx/wr/target/debug/deps -C incremental=/builds/worker/checkouts/gecko/gfx/wr/target/debug/incremental -L dependency=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps --extern bincode=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libbincode-0ebacb661636853f.rmeta --extern bitflags=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libbitflags-39c91276b9e49ec0.rmeta --extern byteorder=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libbyteorder-265225cc1967c054.rmeta --extern cfg_if=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libcfg_if-56e2758a34eab492.rmeta --extern cstr=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libcstr-3c3755b2451850eb.rmeta --extern euclid=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libeuclid-335ac6e250463045.rmeta --extern freetype=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libfreetype-2f909aced5160f6d.rmeta --extern fxhash=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libfxhash-f30b1c84a4c598d4.rmeta --extern gleam=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libgleam-1c733d03f1dd8577.rmeta --extern lazy_static=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/liblazy_static-4a3546ac48157593.rmeta --extern libc=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/liblibc-178199f1ef6882e2.rmeta --extern log=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/liblog-a24216476d4cecda.rmeta --extern malloc_size_of_derive=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libmalloc_size_of_derive-1c43e3974b1e21fe.so --extern num_traits=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libnum_traits-0767eff3834a02b1.rmeta --extern plane_split=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libplane_split-a9770f2f9064aa0a.rmeta --extern rayon=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/librayon-98334ffd87a0a946.rmeta --extern ron=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libron-45fa5041e8b5d234.rmeta --extern serde=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libserde-7b96fe1751e06b6d.rmeta --extern sha2=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libsha2-fd0fddccf6f95ac5.rmeta --extern smallvec=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libsmallvec-9bfcf6470c9d051f.rmeta --extern svg_fmt=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libsvg_fmt-ddaaa69186944f22.rmeta --extern thread_profiler=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libthread_profiler-503a4a76edb7d760.rmeta --extern time=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libtime-4101e2091d9166be.rmeta --extern api=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libwebrender_api-b823a085ad666c23.rmeta --extern webrender_build=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libwebrender_build-855566756be6c7ee.rmeta --extern malloc_size_of=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libwr_malloc_size_of-bc8b8cd784d195a9.rmeta --deny warnings -L native=/usr/lib/x86_64-linux-gnu`
[task 2020-01-23T06:34:10.754Z] error[E0428]: the name `TileCacheLoggerUpdateLists` is defined multiple times
[task 2020-01-23T06:34:10.754Z]     --> webrender/src/picture.rs:1591:1
[task 2020-01-23T06:34:10.755Z]      |
[task 2020-01-23T06:34:10.755Z] 1510 |         pub struct TileCacheLoggerUpdateLists {
[task 2020-01-23T06:34:10.755Z]      |         ------------------------------------- previous definition of the type `TileCacheLoggerUpdateLists` here
[task 2020-01-23T06:34:10.755Z] ...
[task 2020-01-23T06:34:10.755Z] 1591 | pub struct TileCacheLoggerUpdateLists {
[task 2020-01-23T06:34:10.755Z]      | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `TileCacheLoggerUpdateLists` redefined here
[task 2020-01-23T06:34:10.755Z]      |
[task 2020-01-23T06:34:10.755Z]      = note: `TileCacheLoggerUpdateLists` must be defined only once in the type namespace of this module
[task 2020-01-23T06:34:10.755Z] 
[task 2020-01-23T06:34:13.164Z] error[E0412]: cannot find type `Update` in this scope
[task 2020-01-23T06:34:13.164Z]     --> webrender/src/picture.rs:1519:41
[task 2020-01-23T06:34:13.164Z]      |
[task 2020-01-23T06:34:13.164Z] 1519 |                 pub $name: (String, Vec<Update>),
[task 2020-01-23T06:34:13.164Z]      |                                         ^^^^^^ not found in this scope
[task 2020-01-23T06:34:13.164Z] ...
[task 2020-01-23T06:34:13.164Z] 1587 | enumerate_interners!(declare_tile_cache_logger_updatelists);
[task 2020-01-23T06:34:13.164Z]      | ------------------------------------------------------------ in this macro invocation
[task 2020-01-23T06:34:13.164Z]      |
[task 2020-01-23T06:34:13.164Z]      = note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)
[task 2020-01-23T06:34:13.164Z] help: possible candidates are found in other modules, you can import them into scope
[task 2020-01-23T06:34:13.164Z]      |
[task 2020-01-23T06:34:13.164Z] 72   | use crate::intern::Update;
[task 2020-01-23T06:34:13.164Z]      |
[task 2020-01-23T06:34:13.164Z] 72   | use rayon::iter::Update;
[task 2020-01-23T06:34:13.164Z]      |
[task 2020-01-23T06:34:13.164Z] 
[task 2020-01-23T06:34:13.229Z] error[E0412]: cannot find type `Update` in this scope
[task 2020-01-23T06:34:13.229Z]     --> webrender/src/picture.rs:1527:56
[task 2020-01-23T06:34:13.229Z]      |
[task 2020-01-23T06:34:13.229Z] 1527 |                         $name : ( String::new(), Vec::<Update>::new() ),
[task 2020-01-23T06:34:13.229Z]      |                                                        ^^^^^^ not found in this scope
[task 2020-01-23T06:34:13.229Z] ...
[task 2020-01-23T06:34:13.229Z] 1587 | enumerate_interners!(declare_tile_cache_logger_updatelists);
[task 2020-01-23T06:34:13.229Z]      | ------------------------------------------------------------ in this macro invocation
[task 2020-01-23T06:34:13.229Z]      |
[task 2020-01-23T06:34:13.229Z]      = note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)
[task 2020-01-23T06:34:13.229Z] help: possible candidates are found in other modules, you can import them into scope
[task 2020-01-23T06:34:13.229Z]      |
[task 2020-01-23T06:34:13.229Z] 72   | use crate::intern::Update;
[task 2020-01-23T06:34:13.229Z]      |
[task 2020-01-23T06:34:13.229Z] 72   | use rayon::iter::Update;
[task 2020-01-23T06:34:13.229Z]      |
[task 2020-01-23T06:34:13.229Z] 
[task 2020-01-23T06:34:13.284Z] error[E0412]: cannot find type `InternerUpdates` in this scope
[task 2020-01-23T06:34:13.284Z]     --> webrender/src/picture.rs:1535:27
[task 2020-01-23T06:34:13.284Z]      |
[task 2020-01-23T06:34:13.284Z] 1535 |                 updates: &InternerUpdates
[task 2020-01-23T06:34:13.284Z]      |                           ^^^^^^^^^^^^^^^ not found in this scope
[task 2020-01-23T06:34:13.284Z] ...
[task 2020-01-23T06:34:13.284Z] 1587 | enumerate_interners!(declare_tile_cache_logger_updatelists);
[task 2020-01-23T06:34:13.284Z]      | ------------------------------------------------------------ in this macro invocation
[task 2020-01-23T06:34:13.284Z]      |
[task 2020-01-23T06:34:13.284Z]      = note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)
[task 2020-01-23T06:34:13.284Z] help: possible candidate is found in another module, you can import it into scope
[task 2020-01-23T06:34:13.284Z]      |
[task 2020-01-23T06:34:13.284Z] 72   | use crate::scene_builder_thread::InternerUpdates;
[task 2020-01-23T06:34:13.284Z]      |
[task 2020-01-23T06:34:13.284Z] 
[task 2020-01-23T06:34:14.906Z] error[E0592]: duplicate definitions with name `new`
[task 2020-01-23T06:34:14.906Z]     --> webrender/src/picture.rs:1596:5
[task 2020-01-23T06:34:14.906Z]      |
[task 2020-01-23T06:34:14.906Z] 1524 | /             pub fn new() -> Self {
[task 2020-01-23T06:34:14.906Z] 1525 | |                 TileCacheLoggerUpdateLists {
[task 2020-01-23T06:34:14.906Z] 1526 | |                     $(
[task 2020-01-23T06:34:14.907Z] 1527 | |                         $name : ( String::new(), Vec::<Update>::new() ),
[task 2020-01-23T06:34:14.907Z] 1528 | |                     )+
[task 2020-01-23T06:34:14.907Z] 1529 | |                 }
[task 2020-01-23T06:34:14.907Z] 1530 | |             }
[task 2020-01-23T06:34:14.907Z]      | |_____________- other definition for `new`
[task 2020-01-23T06:34:14.907Z] ...
[task 2020-01-23T06:34:14.907Z] 1596 |       pub fn new() -> Self { TileCacheLoggerUpdateLists {} }
[task 2020-01-23T06:34:14.907Z]      |       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ duplicate definitions for `new`
[task 2020-01-23T06:34:14.907Z] 
[task 2020-01-23T06:34:14.907Z] error[E0592]: duplicate definitions with name `is_empty`
[task 2020-01-23T06:34:14.907Z]     --> webrender/src/picture.rs:1597:5
[task 2020-01-23T06:34:14.907Z]      |
[task 2020-01-23T06:34:14.907Z] 1542 | /             fn is_empty(&self) -> bool {
[task 2020-01-23T06:34:14.907Z] 1543 | |                 $(
[task 2020-01-23T06:34:14.907Z] 1544 | |                     if !self.$name.0.is_empty() { return false; }
[task 2020-01-23T06:34:14.907Z] 1545 | |                 )+
[task 2020-01-23T06:34:14.907Z] 1546 | |                 true
[task 2020-01-23T06:34:14.907Z] 1547 | |             }
[task 2020-01-23T06:34:14.907Z]      | |_____________- other definition for `is_empty`
[task 2020-01-23T06:34:14.907Z] ...
[task 2020-01-23T06:34:14.907Z] 1597 |       fn is_empty(&self) -> bool { true }
[task 2020-01-23T06:34:14.907Z]      |       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ duplicate definitions for `is_empty`
[task 2020-01-23T06:34:14.907Z] 
[task 2020-01-23T06:34:14.908Z] error: aborting due to 6 previous errors
[task 2020-01-23T06:34:14.908Z] 
[task 2020-01-23T06:34:14.908Z] Some errors have detailed explanations: E0412, E0428, E0592.
[task 2020-01-23T06:34:14.908Z] For more information about an error, try `rustc --explain E0412`.
[task 2020-01-23T06:34:15.002Z] error: could not compile `webrender`.
[task 2020-01-23T06:34:15.002Z] 
[task 2020-01-23T06:34:15.002Z] Caused by:
[task 2020-01-23T06:34:15.002Z]   process didn't exit successfully: `rustc --edition=2018 --crate-name webrender webrender/src/lib.rs --color never --crate-type lib --emit=dep-info,metadata,link -C panic=abort -C debuginfo=2 --cfg 'feature="default"' --cfg 'feature="freetype-lib"' --cfg 'feature="replay"' --cfg 'feature="ron"' --cfg 'feature="serde"' -C metadata=d91f485b8b4dea1f -C extra-filename=-d91f485b8b4dea1f --out-dir /builds/worker/checkouts/gecko/gfx/wr/target/debug/deps -C incremental=/builds/worker/checkouts/gecko/gfx/wr/target/debug/incremental -L dependency=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps --extern bincode=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libbincode-0ebacb661636853f.rmeta --extern bitflags=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libbitflags-39c91276b9e49ec0.rmeta --extern byteorder=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libbyteorder-265225cc1967c054.rmeta --extern cfg_if=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libcfg_if-56e2758a34eab492.rmeta --extern cstr=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libcstr-3c3755b2451850eb.rmeta --extern euclid=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libeuclid-335ac6e250463045.rmeta --extern freetype=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libfreetype-2f909aced5160f6d.rmeta --extern fxhash=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libfxhash-f30b1c84a4c598d4.rmeta --extern gleam=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libgleam-1c733d03f1dd8577.rmeta --extern lazy_static=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/liblazy_static-4a3546ac48157593.rmeta --extern libc=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/liblibc-178199f1ef6882e2.rmeta --extern log=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/liblog-a24216476d4cecda.rmeta --extern malloc_size_of_derive=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libmalloc_size_of_derive-1c43e3974b1e21fe.so --extern num_traits=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libnum_traits-0767eff3834a02b1.rmeta --extern plane_split=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libplane_split-a9770f2f9064aa0a.rmeta --extern rayon=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/librayon-98334ffd87a0a946.rmeta --extern ron=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libron-45fa5041e8b5d234.rmeta --extern serde=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libserde-7b96fe1751e06b6d.rmeta --extern sha2=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libsha2-fd0fddccf6f95ac5.rmeta --extern smallvec=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libsmallvec-9bfcf6470c9d051f.rmeta --extern svg_fmt=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libsvg_fmt-ddaaa69186944f22.rmeta --extern thread_profiler=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libthread_profiler-503a4a76edb7d760.rmeta --extern time=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libtime-4101e2091d9166be.rmeta --extern api=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libwebrender_api-b823a085ad666c23.rmeta --extern webrender_build=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libwebrender_build-855566756be6c7ee.rmeta --extern malloc_size_of=/builds/worker/checkouts/gecko/gfx/wr/target/debug/deps/libwr_malloc_size_of-bc8b8cd784d195a9.rmeta --deny warnings -L native=/usr/lib/x86_64-linux-gnu` (exit code: 1)
[fetches 2020-01-23T06:34:15.004Z] removing /builds/worker/fetches
[fetches 2020-01-23T06:34:15.212Z] finished
[taskcluster 2020-01-23 06:34:15.721Z] === Task Finished ===
[taskcluster 2020-01-23 06:34:15.722Z] Unsuccessful task run with exit code: 101 completed in 435.972 seconds
Pushed by bpeers@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/77d7f22c81c4
Improve support for invalidation debugging and testing r=gw
Flags: needinfo?(bpeers)
Pushed by bpeers@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/39db0be47ff6
Improve support for invalidation debugging and testing r=gw

Third iteration:

Fix broken scrolling (and incorrect positioning of quad tree lines) by
serializing the SpaceMapper(-transform) from take_context, and using it
to transform the primitive rects (instead of the previous translation
based on unclipped.origin);
Note: this is done at visualization time and not at export time to
distinguish actually moving elements from merely-scrolling ones.

Serialize the entire UpdateList, so we get the data (Keys) that's being
added; add it to the overview;

Move the static CSS code into tilecache_base.css; add this and the .js
file to the binary, write them as part of output (instead of manual
copy); clean up CSS a bit;

Landed, clearing NMI.

...

Flags: needinfo?(bpeers)
Pushed by bpeers@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/535ae1d4818a
Improve support for invalidation debugging and testing r=gw

Relanding the patch as it seems the tests have not run until this push.
Marking above comment as obsolete.

Pushed by ncsoregi@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/748cc6e57cd0
Improve support for invalidation debugging and testing r=gw a=reland CLOSED TREE

Yeay.

Flags: needinfo?(bpeers)
Pushed by ncsoregi@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/388a4d04e911
Improve support for invalidation debugging and testing r=nataliaCs
Attached image blinking_cursor.png

WIP to record changed ItemUids for InvalidationReason == PrimCount, and converting them back to readable text.

Attached image urlbar_expanding.png

Special cased Content==Descriptor. This should help detect invalidations due to minor-but-bigger-than-EPSILON==0.001 changes.

Clip, Transform, OpacityBinding and Image would also be interesting candidates to provide more detail, I just need to be careful to not hurt performance when logging is disabled. To be continued...

Edit: example output when a tile invalidates due to !self.clip_comparer.is_same

Content: Clip ItemUids changed from 64 to 55:
old:

    ClipItemKey { kind: BoxShadow(PointKey { x: 0.0, y: 0.0 }, SizeKey { w: 839.0, h: 44.0 }, BorderRadiusAu { top_left: 2px×2px, top_right: 2px×2px, bottom_left: 2px×2px, bottom_right: 2px×2px }, RectangleKey { x: -54.0, y: -49.0, w: 947.0, h: 152.0 }, 18px, Outset), spatial_node_index: SpatialNodeIndex(5) }

new:

    ClipItemKey { kind: BoxShadow(PointKey { x: 0.0, y: 0.0 }, SizeKey { w: 839.0, h: 44.0 }, BorderRadiusAu { top_left: 2px×2px, top_right: 2px×2px, bottom_left: 2px×2px, bottom_right: 2px×2px }, RectangleKey { x: 165.0, y: -17.0, w: 947.0, h: 152.0 }, 18px, Outset), spatial_node_index: SpatialNodeIndex(1) }
Attached image zoom_demo.png

Support to zoom and shift the generated SVG to visualize out-of-viewport activity.

Fourth iteration: improve the detail in reported tile invalidations.

The invalidation enum stores the old and new values for lightweight
types. For a change in PrimCount, the old and new list of ItemUids is
stored (if logging is enabled); the tool can then diff the two to see
what was added and removed. To convert that back into friendly strings,
the interning activity is used to build up a map of ItemUid to a string.

A similar special-casing of Content Descriptor will print the item
that's invalidating the tile, plus the origin and/or rectangle.

Also adding zoom and pan command line options both to fix high-DPI
issues and also to allow zooming out far enough to see out-of-viewport
cache lifetime and prefetching due to scrolling.

Pushed by bpeers@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/813768d074e3
Improve support for invalidation debugging and testing r=gw

Glenn, how would you feel about closing this bug as fixed for now and we can open a new one if/when we are ready to iterate more on this?

Flags: needinfo?(gwatson)
Status: NEW → RESOLVED
Closed: 1 year ago
Flags: needinfo?(gwatson)
Resolution: --- → FIXED

Support hiding slices to better understand what's on which layer,
and to hide UI when not relevant.
Requires using a HTTP server due to cross-scripting.

Pushed by bpeers@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/c034bb22ce03
Improve support for invalidation debugging and testing r=gw
You need to log in before you can comment on or make changes to this bug.