981206 - Marketplace assets are broken

Status: RESOLVED FIXED

(Marketplace Graveyard :: General, defect, P1)

Description

[Christopher Van Wiemeersch [:cvan]]

So people were wondering why my spinner styles were gone. And on top of that, I pushed code to Fireplace and it's not appearing in our minified CSS+JS; I've cachebusted the URL and everything.

So, looks like our Marketplace assets are broken; marketplace-assets has been barking up ...

Latest console output: https://ci-addons.allizom.org/job/marketplace-assets/5671/console

Console output from the last breakage: https://ci-addons.allizom.org/job/marketplace-assets/5707/console

The trends: https://ci-addons.allizom.org/job/marketplace-assets/buildTimeTrend

Comment 1

[Christopher Van Wiemeersch [:cvan]]

Amy, this is why the /nominate page isn't showing up yet, FYI.

Comment 2

[Mathieu Pillard [:mat]]

The real error is a bit hard to find in jenkins builds between pypi-related failures (we should have jenkins use pyrepo) and mis-configured environment (missing git submodule update --recursive for some reason)


To reproduce locally:
- clone/checkout zamboni, make sure you have no built files in media (I just rm -r media/ and then run git reset --hard HEAD and git submodule update --recursive) to go back to a clean state)
- run manage.py compress_assets with mkt settings

Somewhere in the output, you'll get the error that matters here:
/bin/sh: 1: cannot create /xxx/yyy/zzz/media/js/mkt/devreg-all.js.tmp: Directory nonexistent

The concatenation fails because of this, and therefore the whole command fails too with a traceback saying it can't find the concatenated file.

Comment 3

[Mathieu Pillard [:mat]]

https://github.com/mozilla/zamboni/commit/2beabea1ab70d475b16021865de5e765e11fdd0b fixed jenkins: https://ci-addons.allizom.org/job/marketplace-assets/5711/console

However, that's just a workaround, a proper fix would be to ensure the directories exist before concatenating files, in jingo-minify's compress_assets command.

Comment 4

[Mathieu Pillard [:mat]]

FYI I think this started happening in December because of the cleanups done in https://github.com/mozilla/zamboni/commit/d74791dbbc224b91c270d7299e69643fa5357f7f (or one of those in that period) but was hidden till we had to mess around with stage/dev/jenkins/etc when we merged django 1.6 and reverted it a few weeks ago. I didn't notice it locally because the directory already existed thanks to the old concatenated files.

Comment 5

[Mathieu Pillard [:mat]]

https://marketplace-dev-cdn.allizom.org/media/fireplace/js/include.js?b=omgamazing now contains cvan's changes from bug 960338

Comment 6

[Christopher Van Wiemeersch [:cvan]]

You're the best :)