888168 - CSS hasn't rebuilt on production mozillaignite.org

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Description

[Ross Bruniges [:boozeniges]]

It looks like the final push I did with chief up to production didn't rebuild the CSS files correctly (or possibly at all) as we've got a fair few layout inconsistencies on the homepage.

Compare with stage (https://mozillaignite-dev.allizom.org/) and you can see what I mean.

The files are definitely merged in as we have the new templates and layouts (this is what is new - https://github.com/mozilla/mozilla-ignite/commit/1afc175a3063e66d9f60d22f1d12f82002ff9897) but the CSS hasn't made it.

Could we compare staging with production to see what's been missed and have it run?

Comment 1

[Jason Crowe [:jd]]

I have some confusion here. The URL you posted for stage is actually the dev site. I ran a recursive diff between the dev site (URL you posted) and the production site but the output was far to large for me to reasonably look through, however...

The dev site is configured for:
  Local branch configured for 'git pull':
    master merges with remote master

While the prod site is configured for:
  Local branch configured for 'git pull':
    stage merges with remote stage

This really has nothing to do with us since you control all of this through Chief now. I suspect that you have a checkout on prod that is not what you desire. Currently I see

For dev:
  commit 03b3f15aac529f21646f956cef31f8683e8b301f

For prod:
  commit 31461c2aacb514b1189cb9d25a8daeba20bbff55

Let me know if this information is helpful or if there is something I am not understanding about the issue.

Regards

Comment 2

[Ross Bruniges [:boozeniges]]

Sorry :jd - errant typing but the staging URL is https://mozillaignite.allizom.org/ but it's still showing the same results.

Looking at the site the only differences that I can see if that staging is that it's not using the compiled CSS where as production is; indeed if I compare the two production CSS files (compressed - https://mozillaignite.org/media/ignite/css/ignite_desktop-min.css and uncompressed - https://mozillaignite.org/media/ignite/css/desktop.css) and do a search for 'splash-winners' it's indeed not found in the compressed version and this (and other things) are what are causing the visual discrepancies.

I can't remember if I actually did a push using chief that required re-building the CSS, it could be that it never actually ever did it but I don't work for Mozilla anymore so don't have an LDAP account and therefore access to the VPN so that I can run a chief myself, would it be possible to rebuild the CSS manually?

I know it's a Friday so no need to do it today but I worked hard to try and make things look good so would be nice if we can get things looking as expected on production...

Comment 3

[Jason Crowe [:jd]]

Ross,

I am looking over the old update scripts and do not see any commands that would have done this. Also looking over the Chief scripts I do not see this either. I am not sure how it was ever accomplished TBH. I am happy to run the commands manually for them but will need you to provide me with the exact commands you would like me to run and what output I should expect. We can coordinate via IRC if that is simpler for you.

Comment 4

[Ross Bruniges [:boozeniges]]

I think that you'll have to consult with :cturra about this - he did most of our deploys before we had Cheif...

Comment 5

[Jason Crowe [:jd]]

Umm, okay but he did not build the site so I imagine he did not choose the library that would be used for minification. That is generally the choice of the builder / developer.

Comment 6

[Jason Crowe [:jd]]

also FYI I tried the usual 'python2.6 manage.py collectstatic' but got:

Unknown command: 'collectstatic'

Comment 7

[Ross Bruniges [:boozeniges]]

No - but he ran the commands that pulled in and synced the files which must have compressed the CSS somewhere along the line.

Comment 8

[Ross Bruniges [:boozeniges]]

Collect static won't work as this is using an older django build (1.3.1), we didn't have collect static back in those days :(

Comment 9

[Jason Crowe [:jd]]

Ross,

I spoke with cturra and after quite a bit of digging we found the problem. It looks as tough when the site was converted to use Chief the compress_assets command was not included in the update.py script. For completeness I will inform you that this command was never existent on the stage site (I did not check dev).

For this immediate issue I ran the command manually on stage and then on prod and pushed both to their respective web servers.

Long term you will need to add the command to the Chief script in your git repository. This will be as simple as adding the command:

python manage.py compress_assets

to the file:

scripts/update/update.py

This should fix you up now and in the future. Let me know if you have further issues with this.

Cheers

Comment 10

[Will Barkis]

Hey, just wanted to say a big thanks to everyone for their work on this. Really appreciate it. 

-Will

Comment 11

[Ross Bruniges [:boozeniges]]

Yeah thanks :jd!

Looking good :)