1091421 - New Commonplace Stylus watcher/compiler is too slow

Status: VERIFIED FIXED

(Marketplace Graveyard :: Code Quality, defect, P2)

Description

[Christopher Van Wiemeersch [:cvan]]

I love gulp, but something in the proces is causing it to take an average of 4-10 seconds to compile a `.styl` file after saving the file. Damper takes 0.2-2 seconds.

Comment 1

[Kevin Ngo [:ngoke]]

It compiles immediately for me. The watch/compile code is as simple it can be in Gulp. Can you try something for me?

I read that Gulp's watcher is based on Gaze, which may be affected by your OS's limit on concurrent open files. Perhaps we can try increasing your file limit as described in http://docs.basho.com/riak/latest/ops/tuning/open-files-limit/

I'm going to poll other developers and see how many people have this issue.

Comment 2

[Mathieu Pillard [:mat]]

It's super fast for me on 2 different linux machines.

Comment 3

[Christopher Van Wiemeersch [:cvan]]

I'm on a beefy machine:

Mac OS X 10.9.5 
2.6 GHz Intel Core i7
16 GB 1600 MHz DDR3

Try this:

1. Make sure `make serve` is running
2. Run in terminal: `find . -name '*.styl.css' -delete`
3. In your text editor open `navbar.styl` and `navbar.styl.css` in two different tabs/windows
4. Save `navbar.styl` one and notice how long it takes for `navbar.styl.css`'s content to appear

vs.

1. Make sure `damper` is running
2. (repeat step #2 above)
3. (repeat step #3 above)
4. (repeat step #4 above)

Comment 4

[Kevin Ngo [:ngoke]]

Mac OS X 10.8.4
2.7 GHz Intel Core i7
16 GB 1600 MHz DDR3

Tried the steps. Watcher kicks off instantly. Compiles every CSS file in 1.91s (could probably change it to compile only the modified CSS file).

I'd try my link above. Might be something with the number of open files on your machine? Because damper uses polling whereas Gulp's watcher uses native filesystem events.

Comment 5

[Christopher Van Wiemeersch [:cvan]]

(In reply to Kevin Ngo [:ngoke] from comment #4)
> Mac OS X 10.8.4
> 2.7 GHz Intel Core i7
> 16 GB 1600 MHz DDR3
> 
> Tried the steps. Watcher kicks off instantly. Compiles every CSS file in
> 1.91s (could probably change it to compile only the modified CSS file).

How'd you time that?

> I'd try my link above. Might be something with the number of open files on
> your machine? Because damper uses polling whereas Gulp's watcher uses native
> filesystem events.

Mine says this:

	maxfiles    256            unlimited

Comment 6

[Kevin Ngo [:ngoke]]

Gulp told me in the console.

Maybe you have a lot of files open? Try bumping it up a bit.

Comment 7

[Christopher Van Wiemeersch [:cvan]]

Okay, so it's taking just as long for you.

With `make serve`, I individually saved a couple Stylus files. And it's always taking about 2.1-2.2 seconds. That's quite a long time.

> [12:46:03] Starting 'css_compile'...
> [12:46:05] Finished 'css_compile' after 2.16 s
> [12:46:38] Starting 'css_compile'...
> [12:46:40] Finished 'css_compile' after 2.14 s
> [12:46:46] Starting 'css_compile'...
> [12:46:48] Finished 'css_compile' after 2.1 s
> [12:46:50] Starting 'css_compile'...
> [12:46:53] Finished 'css_compile' after 2.17 s
> [12:46:56] Starting 'css_compile'...
> [12:46:58] Finished 'css_compile' after 2.13 s
> [12:47:05] Starting 'css_compile'...
> [12:47:07] Finished 'css_compile' after 2.15 s
> [12:47:16] Starting 'css_compile'...
> [12:47:18] Finished 'css_compile' after 2.2 s
> [12:47:25] Starting 'css_compile'...
> [12:47:28] Finished 'css_compile' after 2.16 s

Compare that to Damper's watcher (I just edited my `damper` to add the timing):

> > /opt/fireplace_/src/media/css/navbar.styl changed.
> > /opt/fireplace_/src/media/css/navbar.styl recompiled after 61 ms.
> > /opt/fireplace_/src/media/css/category-icons.styl changed.
> > /opt/fireplace_/src/media/css/category-icons.styl recompiled after 71 ms.
> > /opt/fireplace_/src/media/css/site.styl changed.
> > /opt/fireplace_/src/media/css/site.styl recompiled after 103 ms.
> > /opt/fireplace_/src/media/css/feed.styl changed.
> > /opt/fireplace_/src/media/css/feed.styl recompiled after 110 ms.
> > /opt/fireplace_/src/media/css/site-banner.styl changed.
> > /opt/fireplace_/src/media/css/site-banner.styl recompiled after 50 ms.
> > /opt/fireplace_/src/media/css/tiles.styl changed.
> > /opt/fireplace_/src/media/css/tiles.styl recompiled after 49 ms.
> > /opt/fireplace_/src/media/css/categories.styl changed.
> > /opt/fireplace_/src/media/css/categories.styl recompiled after 59 ms.
> > /opt/fireplace_/src/media/css/category-dropdown-desktop.styl changed.
> > /opt/fireplace_/src/media/css/category-dropdown-desktop.styl recompiled after 64 ms.

We're talking 2100-2200 ms vs. 49-110 ms. By my math, that means Damper was 20x/42.9x faster.

It becomes quite noticeable when you have to keep refreshing your browser because the CSS/templates haven't updated yet.

Comment 9

[Kevin Ngo [:ngoke]]

Two steps to this bug:

1. Only recompile modified CSS files on the watcher.
2. Look at replacing our gulp-stylus plugin to call stylus directly rather than using Accord.

Comment 10

[Kevin Ngo [:ngoke]]

Only recompile modified CSS files rather than all using gulp-watch. Very instant.

https://github.com/mozilla/marketplace-gulp/commit/d07b0c507ba992503889ba1c413e28b8f9691732

Comment 11

[Christopher Van Wiemeersch [:cvan]]

Awesome. Stylus compilation is blazing fast now. We're back to damper speeds.

Thanks for the quick turnaround, Kevin. You just made my week. Now front-end Fireplace dev is such a breeze.

Comment 12

[Christopher Van Wiemeersch [:cvan]]

Attachment: post-fix screenshot