2001151 - [Spike] Prototype Speculation Rules prefetch (proof-of-concept)

Status: RESOLVED FIXED

(Core :: Networking, task, P3)

Description

[Andrew Creskey [:acreskey]]

It could be useful to understand scope and potential bottlenecks to build a minimal, non-compliant, proof-of-concept of Speculation Rules, prefetch.

Comment 1

[Andrew Creskey [:acreskey]]

Attachment: WIP: Bug 2001151 - [Spike] Prototype Speculation Rules prefetch (proof-of-concept)

• Not production code, just for investigating implications *

Comment 2

[Andrew Creskey [:acreskey]]

Attachment: WIP: Bug 2001151 - Add manual tests for Speculation Rules prefetch

Comment 3

[Andrew Creskey [:acreskey]]

The attached patch implements a proof-of-concept for Speculation Rules prefetch functionality.

Disclaimer: This code was almost entirely generated by Claude from my instructions and is for investigative proof-of-concept purposes only; it has not been fully reviewed or tested and should not be used in production.

Notes:
• Add SpeculationRulesParser to parse speculation rules JSON
• Parses speculation rules from: inline scripts, dynamically added scripts, external JSON files, and HTTP Speculation-Rules response headers
• Implement prefetch action with eagerness levels:
immediate: Prefetch as soon as rules are parsed
eager: (not yet implemented)
moderate: Prefetch on mouse hover over matching links
conservative: Prefetch on mousedown on matching links
• Prerender action is mapped to prefetch
• Add prefetchURIForSpeculationRules() to bypass must-revalidate directives (not working for all requests)
• Use cache metadata to mark speculation-rules prefetches, cleared after first use
• Cache entries valid for 5 minutes via ForceValidFor(300)
• Add prefs: dom.speculation_rules.enabled and dom.speculation_rules.prefetch_on_mousedown

It's easiest to observe the behavior through the web console (currently quite verbose)

Comment 4

[Andrew Creskey [:acreskey]]

I'm testing this on a series of simple tests that I'm hosting here: https://acreskeymoz.github.io/speculation_rules/prefetch/

This is working quite well on some production sites like scalemates.com
https://www.scalemates.com/

And it's partially working on the deployed-at-scale Shopify speculation rules.
e.g. this rule: https://jeffreestarcosmetics.com/cdn/shopifycloud/storefront/assets/storefront/storefronts.specrules-dd5621a1.json
seen here: https://jeffreestarcosmetics.com/
The rules (provided in the header response) will be loaded and the runtime will preload the link target (e.g. a collection) based on the conservative mouse down action. However on the subsequent load the document is still fetched over the wire. To be investigated.

Comment 5

[Andrew Creskey [:acreskey]]

Attachment: WIP: Bug 2001151 - [Spike] Prototype Speculation Rules prefetch (proof-of-concept)

Comment 6

[Andrew Creskey [:acreskey]]

Attachment: WIP: Bug 2001151 - Add manual tests for Speculation Rules prefetch

Comment 7

[Andrew Creskey [:acreskey]]

It's just a simple test case, but bug 2007155 is showing that the POC and the HTTP cache is working well so far.

Comment 8

[Andrew Creskey [:acreskey]]

Was prototyped, portions of which were landed, but this bug is no longer needed as we are landing the full impl in bug 1969396