[meta] Switch to django-pipeline

RESOLVED FIXED

Status

Mozilla Developer Network
Code Cleanup
RESOLVED FIXED
3 years ago
2 years ago

People

(Reporter: openjck, Unassigned)

Tracking

({meta})

Details

(Reporter)

Description

3 years ago
Switch from jingo-minify to django-pipeline for building and cache-busting assets.
(Reporter)

Updated

3 years ago
Blocks: 1119397
(Reporter)

Comment 1

3 years ago
The compressed assets should be stored in a STATICFILES_DIRS directory.

Comment 2

2 years ago
Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/59c1332cba1d154cb3463886b8c80e6fbdf4aa19
Bug 1139490: Rename $media-url-dir variable

When we switch to django-pipeline, these files will no longer live in
the media/ directory. The name $path-to-images is more generic.

https://github.com/mozilla/kuma/commit/a0731758fdc8f9414fa9019d8c580c34d90c4dbb
Bug 1139490: Replace /media/img with variable

Direct references to /media/img are updated to use the $path-to-images
variable instead. The path will be updated when we switch to
django-pipeline, so using a variable now will make the switch a little
easier.

https://github.com/mozilla/kuma/commit/82056cd2678ec60d95f1cc22f4d889b11ced2357
Bug 1139490: Replace /media/ with variable

This will make updating the path a little bit easier in the future,
which will help us switch to django-pipeline.

https://github.com/mozilla/kuma/commit/7942730fb1f4ce1525628f3f77a16f1c80fa0995
Bug 1139490: Remove asset URL from comment

The URL will change when we switch to django-pipeline. To be as generic
as possible, let's just mention the filename.

https://github.com/mozilla/kuma/commit/58dae9a95be2f5fc1bfe4ed377baff0fbdc51c2a
Merge pull request #3389 from openjck/bug-1139490-update-stylus-paths

Bug 1139490: Improve path-related code in Stylus

Comment 3

2 years ago
Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/fbed67c0778a7167b6d6fd44568b7caffa2ce3a3
Bug 1139490: Remove unused events JavaScript

These files were formerly included in the "events" bundle, which was
removed along with the events app.

https://github.com/mozilla/kuma/commit/cdab893388776fefda5a8f6d0edceba6d481a58e#diff-bdf3ecebd8379ca98cc89e545fc90899

https://github.com/mozilla/kuma/commit/2f40bd485df9172fa12cfe122a8f752578d291d5
Merge pull request #3409 from openjck/bug-1139490-remove-calendar.js

Bug 1139490: Remove unused events JavaScript

Comment 4

2 years ago
@openjck -- what are the next steps on this? What help will you need from IT?
(Reporter)

Comment 5

2 years ago
At a high level, my task list looks something like this:

1. Fix hardcoded paths in source (in progress)
2. Host assets at /static/assets
3. Install and use django-pipeline

We could use some help from IT on steps 2 and 3. For step 2, IT will need to manually set up production redirects from /media to /static/assets (see bug 1145793 for more on why we can't do this ourselves). For part 3, we may need to switch from clean-css to CSSMin. If we do, IT will need to install CSSMin on stage and production.

Does that sound about right :jezdez? I wonder if IT would also need to do something special to get /static/assets hosted. In the last pull request [1], there was an issue hosting /static/assets/compressed/ locally.

[1] https://github.com/mozilla/kuma/pull/3219
Flags: needinfo?(jezdez)
(Reporter)

Updated

2 years ago
Depends on: 1197443
(Reporter)

Updated

2 years ago
Keywords: meta
Summary: Switch to django-pipeline → [meta] Switch to django-pipeline
(Reporter)

Updated

2 years ago
Depends on: 1197444
(Reporter)

Updated

2 years ago
No longer depends on: 1197443
(Reporter)

Updated

2 years ago
Depends on: 1197443
(Reporter)

Updated

2 years ago
Depends on: 1197447
(Reporter)

Updated

2 years ago
No longer depends on: 1197447
(Reporter)

Updated

2 years ago
Depends on: 1197452
(Reporter)

Updated

2 years ago
Blocks: 1197452
No longer depends on: 1197452

Comment 6

2 years ago
Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/92d81c6ea81bd17e4f976c5c509769ec910928d1
Bug 1139490: Switch to django-pipeline

https://github.com/mozilla/kuma/commit/90ec06e86266b4aa33734025c9b24e58a095393c
bug 1139490 - Install pipeline helpers in TravisCI

* Add node, npm, stylus, etc. to ansible build
* Move ./manage.py collectstatic to tox

https://github.com/mozilla/kuma/commit/d4ecf9d6dff9be78d6bf7f852a552db7187639ae
bug 1139490 - Limit memory usage in TravisCI

* Limit elasticsearch / java to 256MB (was 1 GB)
* Set elasticsearch replicas=0, shards=1 (thanks @robhudson)
* Switch mysql innodb_log_file_size back to default 64MB (was 1GB)
* Pass DJANGO_SETTINGS_MODULE to tox environment

https://github.com/mozilla/kuma/commit/6618d108a6313875e8ae11353b327df16d4bc46b
Merge pull request #3580 from openjck/bug-1139490-django-pipeline-working-travis

Fix bug 1139490: Switch to django-pipeline

Updated

2 years ago
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → FIXED

Comment 7

2 years ago
Commit pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/baf976193b6d699f2efd93450ab8045448ea3e7c
bug 1139490 - Add /usr/local/bin to PATH on admin

The stylus and cleancss node commands are symlinked to /usr/local/bin on
admin.  This allows ./manage.py collectstatic to compile stylesheets on
deploys.

Comment 8

2 years ago
Hello
Maybe I missed something but to get the site running again I had to manually install requirements in the vm through vagrant ssh.
Was there a better way? I naively tried to provision but it didn't do the trick.

Other question: is there any chance this commit resolved bug 1175618 too? (trying to find why the bug is not here anymore)

Thanks!
(Reporter)

Comment 9

2 years ago
Hi Florian,

We'd love to help you get MDN up and running. Please send us a note at:

dev-mdn@lists.mozilla.org
(Reporter)

Updated

2 years ago
Flags: needinfo?(jezdez)

Comment 10

2 years ago
Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/0868577f66c6d9c1e4a30377fe8137b96041f0e8
Bug 1139490: Remove cache-busting params in styles

collectstatic automatically updates stylesheets to cache-bust images and
other assets when their contents change, so these manual cache-busting
parameters are no longer needed.

https://github.com/mozilla/kuma/commit/2f34e5ef12c1f62c834d759a3747278a90a13ca4
Merge pull request #3586 from openjck/bug-1139490-remove-version-params

Bug 1139490: Remove cache-busting params in styles

Comment 11

2 years ago
Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/2abbaa9fd7523b3fe4316877723c1dab1716b891
Bug 1139490: Use sourcemaps in DEBUG mode

https://github.com/mozilla/kuma/commit/8023909b2fc5f02d679b62cbb14ede27626f6551
Merge pull request #3596 from openjck/1139490-django-pipeline-sourcemaps

Bug 1139490: Use sourcemaps in DEBUG mode

Comment 12

2 years ago
Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/2fd6dd1c1d797a317688dea75079896298b0598c
Bug 1139490: Don't load tech icon on Derby pages

This addresses the following error:
https://errormill.mozilla.org/mdn/mdn/group/401204/

https://github.com/mozilla/kuma/commit/d88411c41056913bbf341453df36f0bdf84415a6
Merge pull request #3585 from openjck/bug-1139490-devderby-contest-500

Bug 1139490: Don't load tech icon on Derby pages

Comment 13

2 years ago
Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/aed9fd0ccc31fd6878c4eb2c970997829e682074
Bug 1139490: Manage callout datauri with Pipeline

https://github.com/mozilla/kuma/commit/6a511ef54e46be4016ab50a30c8949f0c15974a4
Merge pull request #3605 from openjck/bug-1139490-datauri-home

Bug 1139490: Manage callout datauri with Pipeline

Comment 14

2 years ago
Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/a2e95ca371cd32e4f6a5a0eefdef31c614387e29
Bug 1139490: Manage all data URIs with Pipeline

https://github.com/mozilla/kuma/commit/d185f09c51748a423defdbc7d0140cd7b831ab4e
Merge pull request #3610 from openjck/bug-1139490-all-data-uris

Bug 1139490: Manage all data URIs with Pipeline

Comment 15

2 years ago
Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/23d3430eb4541e9d1e59858d7a944ed73c4fbfed
Bug 1139490: Improve local Pipeline performance

Kuma pages load very slowly locally when Pipeline kicks off compilation.
This change restores the old workflow, where the `stylus` executable
compiles individual stylesheets as needed.

More info:
https://github.com/mozilla/kuma/pull/3597#issuecomment-151984236

https://github.com/mozilla/kuma/commit/1f517b902e9d02d0b685ac4dd8eb6c5bb4b9a8c0
Merge pull request #3614 from openjck/bug-1139490-no-pipeline-compilation

Bug 1139490: Improve local Pipeline performance

Comment 16

2 years ago
Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/1545b902ae01f5daa0b20cbe4fafea0bb793f498
bug 1139490 - remove GitPython

Now that we use django-pipeline instead of jingo-minify.

https://github.com/mozilla/kuma/commit/22f1a58de47e9a05779c83894def88729279bb3e
Merge pull request #3648 from mozilla/remove-gitpython-1139490

bug 1139490 - remove GitPython

Comment 17

2 years ago
Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/a5d5fb6c4136bb1cedc058ea0bdd7951fe1863e7
Bug 1139490: Don't use Pipeline to wrap JavaScript

When PIPELINE_DISABLE_WRAPPER is False (the default), Pipeline wraps all
JavaScript in an anonymous function:

    (function(){
      //JS output...
    })();

We already wrap JavaScript manually, so this extra wrap is not needed.

https://github.com/mozilla/kuma/commit/54cf77bf3f4f161ec9fd572e2801e4bf2a1deaeb
Merge pull request #3657 from openjck/bug-1139490-pipeline-disable-wrapper

Bug 1139490: Don't use Pipeline to wrap JavaScript
You need to log in before you can comment on or make changes to this bug.