Closed Bug 1139490 Opened 9 years ago Closed 9 years ago

[meta] Switch to django-pipeline

Categories

(developer.mozilla.org Graveyard :: Code Cleanup, defect)

Product:
developer.mozilla.org Graveyard
Component:
Code Cleanup
Platform:
x86
macOS
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: openjck, Unassigned)

References

Details

(Keywords: meta) 

Switch from jingo-minify to django-pipeline for building and cache-busting assets.
Blocks: 1119397
The compressed assets should be stored in a STATICFILES_DIRS directory.
Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/59c1332cba1d154cb3463886b8c80e6fbdf4aa19
Bug 1139490: Rename $media-url-dir variable

When we switch to django-pipeline, these files will no longer live in
the media/ directory. The name $path-to-images is more generic.

https://github.com/mozilla/kuma/commit/a0731758fdc8f9414fa9019d8c580c34d90c4dbb
Bug 1139490: Replace /media/img with variable

Direct references to /media/img are updated to use the $path-to-images
variable instead. The path will be updated when we switch to
django-pipeline, so using a variable now will make the switch a little
easier.

https://github.com/mozilla/kuma/commit/82056cd2678ec60d95f1cc22f4d889b11ced2357
Bug 1139490: Replace /media/ with variable

This will make updating the path a little bit easier in the future,
which will help us switch to django-pipeline.

https://github.com/mozilla/kuma/commit/7942730fb1f4ce1525628f3f77a16f1c80fa0995
Bug 1139490: Remove asset URL from comment

The URL will change when we switch to django-pipeline. To be as generic
as possible, let's just mention the filename.

https://github.com/mozilla/kuma/commit/58dae9a95be2f5fc1bfe4ed377baff0fbdc51c2a
Merge pull request #3389 from openjck/bug-1139490-update-stylus-paths

Bug 1139490: Improve path-related code in Stylus
Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/fbed67c0778a7167b6d6fd44568b7caffa2ce3a3
Bug 1139490: Remove unused events JavaScript

These files were formerly included in the "events" bundle, which was
removed along with the events app.

https://github.com/mozilla/kuma/commit/cdab893388776fefda5a8f6d0edceba6d481a58e#diff-bdf3ecebd8379ca98cc89e545fc90899

https://github.com/mozilla/kuma/commit/2f40bd485df9172fa12cfe122a8f752578d291d5
Merge pull request #3409 from openjck/bug-1139490-remove-calendar.js

Bug 1139490: Remove unused events JavaScript
@openjck -- what are the next steps on this? What help will you need from IT?
At a high level, my task list looks something like this:

1. Fix hardcoded paths in source (in progress)
2. Host assets at /static/assets
3. Install and use django-pipeline

We could use some help from IT on steps 2 and 3. For step 2, IT will need to manually set up production redirects from /media to /static/assets (see bug 1145793 for more on why we can't do this ourselves). For part 3, we may need to switch from clean-css to CSSMin. If we do, IT will need to install CSSMin on stage and production.

Does that sound about right :jezdez? I wonder if IT would also need to do something special to get /static/assets hosted. In the last pull request [1], there was an issue hosting /static/assets/compressed/ locally.

[1] https://github.com/mozilla/kuma/pull/3219
Flags: needinfo?(jezdez)
Depends on: 1197443
Keywords: meta
Summary: Switch to django-pipeline → [meta] Switch to django-pipeline
Depends on: 1197444
No longer depends on: 1197443
Depends on: 1197443
Depends on: 1197447
No longer depends on: 1197447
Depends on: 1197452
Blocks: 1197452
No longer depends on: 1197452
Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/92d81c6ea81bd17e4f976c5c509769ec910928d1
Bug 1139490: Switch to django-pipeline

https://github.com/mozilla/kuma/commit/90ec06e86266b4aa33734025c9b24e58a095393c
bug 1139490 - Install pipeline helpers in TravisCI

* Add node, npm, stylus, etc. to ansible build
* Move ./manage.py collectstatic to tox

https://github.com/mozilla/kuma/commit/d4ecf9d6dff9be78d6bf7f852a552db7187639ae
bug 1139490 - Limit memory usage in TravisCI

* Limit elasticsearch / java to 256MB (was 1 GB)
* Set elasticsearch replicas=0, shards=1 (thanks @robhudson)
* Switch mysql innodb_log_file_size back to default 64MB (was 1GB)
* Pass DJANGO_SETTINGS_MODULE to tox environment

https://github.com/mozilla/kuma/commit/6618d108a6313875e8ae11353b327df16d4bc46b
Merge pull request #3580 from openjck/bug-1139490-django-pipeline-working-travis

Fix bug 1139490: Switch to django-pipeline
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Commit pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/baf976193b6d699f2efd93450ab8045448ea3e7c
bug 1139490 - Add /usr/local/bin to PATH on admin

The stylus and cleancss node commands are symlinked to /usr/local/bin on
admin.  This allows ./manage.py collectstatic to compile stylesheets on
deploys.
Hello
Maybe I missed something but to get the site running again I had to manually install requirements in the vm through vagrant ssh.
Was there a better way? I naively tried to provision but it didn't do the trick.

Other question: is there any chance this commit resolved bug 1175618 too? (trying to find why the bug is not here anymore)

Thanks!
Hi Florian,

We'd love to help you get MDN up and running. Please send us a note at:

dev-mdn@lists.mozilla.org
Flags: needinfo?(jezdez)
Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/0868577f66c6d9c1e4a30377fe8137b96041f0e8
Bug 1139490: Remove cache-busting params in styles

collectstatic automatically updates stylesheets to cache-bust images and
other assets when their contents change, so these manual cache-busting
parameters are no longer needed.

https://github.com/mozilla/kuma/commit/2f34e5ef12c1f62c834d759a3747278a90a13ca4
Merge pull request #3586 from openjck/bug-1139490-remove-version-params

Bug 1139490: Remove cache-busting params in styles
Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/2abbaa9fd7523b3fe4316877723c1dab1716b891
Bug 1139490: Use sourcemaps in DEBUG mode

https://github.com/mozilla/kuma/commit/8023909b2fc5f02d679b62cbb14ede27626f6551
Merge pull request #3596 from openjck/1139490-django-pipeline-sourcemaps

Bug 1139490: Use sourcemaps in DEBUG mode
Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/2fd6dd1c1d797a317688dea75079896298b0598c
Bug 1139490: Don't load tech icon on Derby pages

This addresses the following error:
https://errormill.mozilla.org/mdn/mdn/group/401204/

https://github.com/mozilla/kuma/commit/d88411c41056913bbf341453df36f0bdf84415a6
Merge pull request #3585 from openjck/bug-1139490-devderby-contest-500

Bug 1139490: Don't load tech icon on Derby pages
Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/aed9fd0ccc31fd6878c4eb2c970997829e682074
Bug 1139490: Manage callout datauri with Pipeline

https://github.com/mozilla/kuma/commit/6a511ef54e46be4016ab50a30c8949f0c15974a4
Merge pull request #3605 from openjck/bug-1139490-datauri-home

Bug 1139490: Manage callout datauri with Pipeline
Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/a2e95ca371cd32e4f6a5a0eefdef31c614387e29
Bug 1139490: Manage all data URIs with Pipeline

https://github.com/mozilla/kuma/commit/d185f09c51748a423defdbc7d0140cd7b831ab4e
Merge pull request #3610 from openjck/bug-1139490-all-data-uris

Bug 1139490: Manage all data URIs with Pipeline
Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/23d3430eb4541e9d1e59858d7a944ed73c4fbfed
Bug 1139490: Improve local Pipeline performance

Kuma pages load very slowly locally when Pipeline kicks off compilation.
This change restores the old workflow, where the `stylus` executable
compiles individual stylesheets as needed.

More info:
https://github.com/mozilla/kuma/pull/3597#issuecomment-151984236

https://github.com/mozilla/kuma/commit/1f517b902e9d02d0b685ac4dd8eb6c5bb4b9a8c0
Merge pull request #3614 from openjck/bug-1139490-no-pipeline-compilation

Bug 1139490: Improve local Pipeline performance
Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/1545b902ae01f5daa0b20cbe4fafea0bb793f498
bug 1139490 - remove GitPython

Now that we use django-pipeline instead of jingo-minify.

https://github.com/mozilla/kuma/commit/22f1a58de47e9a05779c83894def88729279bb3e
Merge pull request #3648 from mozilla/remove-gitpython-1139490

bug 1139490 - remove GitPython
Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/a5d5fb6c4136bb1cedc058ea0bdd7951fe1863e7
Bug 1139490: Don't use Pipeline to wrap JavaScript

When PIPELINE_DISABLE_WRAPPER is False (the default), Pipeline wraps all
JavaScript in an anonymous function:

    (function(){
      //JS output...
    })();

We already wrap JavaScript manually, so this extra wrap is not needed.

https://github.com/mozilla/kuma/commit/54cf77bf3f4f161ec9fd572e2801e4bf2a1deaeb
Merge pull request #3657 from openjck/bug-1139490-pipeline-disable-wrapper

Bug 1139490: Don't use Pipeline to wrap JavaScript
Product: developer.mozilla.org → developer.mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.