1373376 - The webpack/Neutrino build uses the same hash for the manifest regardless of content

Status: RESOLVED FIXED

(Tree Management :: Treeherder, enhancement, P1)

Description

[Ed Morley [:emorley]]

After today's prod deploy, Treeherder users were seeing errors JS exceptions until they force reloaded the page, of form:

    TypeError: a[f] is undefined[Learn More] manifest.d41d8cd98f00b204e980.bundle.js:1:96
       (see https://pastebin.mozilla.org/9024757)

The cache-control header on index.html (and other html pages) was confirmed to be fine (60 seconds), so the issue instead lay with the filename hashes.

By looking at the Heroku build log output (which includes the filenames output from webpack), I found that the `manifest.<HASH>.bundle.js` filename never changed, even if the other filenames did (and the manifest references those other files, so should have changed too):
https://emorley.pastebin.mozilla.org/9024760

This appears to be related to:
https://github.com/erm0l0v/webpack-md5-hash/issues/9

Neutrino v4/v5 don't have native support for fully-deterministic caching, so as part of bug 1336556 a custom implementation was used, I'm guessing based on the docs here:
https://webpack.js.org/guides/caching/

However those docs are very unclear (there's a plan to overhaul them: https://github.com/webpack/webpack.js.org/issues/652), so I believe the current custom config is just incorrect/incomplete.

Neutrino v6 (soon to be released) supports caching out of the box [1], so short term we should just remove the custom implementation Treeherder uses and wait until it's released. In the meantime we'll still have long-lived/hashed files, they'll just be a number of edge-cases where they are needlessly invalidated and re-downloaded, but that's preferred to having the browser use a broken manifest.


[1] It follows https://medium.com/webpack/predictable-long-term-caching-with-webpack-d3eee1d3fa31 which is more correct than the webpack docs at present.

Comment 1

[GitHub Autolander Bot]

Attachment: [treeherder] mozilla:fix-webpack-hashes > mozilla:master

Comment 2

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/23e7c973874509ec3265f083aabf0241e4b04161
Bug 1373376 - Remove broken webpack-md5-hash configuration

The webpack-md5-hash plugin attempts to make the generated hashes
less susceptible to churn when chunks haven't actually changed.
However it requires careful use and clearly wasn't set up correctly,
since the manifest file hashes were always the same regardless of
content, causing stale files and runtime errors in production.

Once Neutrino v6 is released, we can use its new in-built support
for caching to save having to implement this ourselves.

Comment 3

[Ed Morley [:emorley]]

Deployed to prod.

A normal (non-force) reload of any Treeherder page will pull in the new manifest, that now has a hash in the filename that actually changes as it's content does.