434603 - SAMO should not pull images from AMO

Status: RESOLVED FIXED

(addons.mozilla.org Graveyard :: API, defect, P2)

Description

[Michael Morgan [:morgamic]]

We need to update SAMO's config so SITE_URL points to SAMO instead of AMO (which is the default).

We'd be looking at adding this to site/app/config/config.php:
define('SITE_URL', 'https://services.addons.mozilla.org');

Right now SAMO pulls images from AMO because SITE_URL is defaulting to AMO.

Comment 1

[Michael Morgan [:morgamic]]

oremj - hold on for a sec, making sure this is all that is needed.

Comment 2

[Justin Fitzhugh]

even more than just images - the sites should be 100% self sufficient with no reliance between the two...hoping that's covered in this bug as well.

Comment 3

[Laura Thomson :laura]

Easy enough.  The old urls are a hangover from when they weren't separate.

Comment 4

[Justin Fitzhugh]

not sure when 3.4.3 but this must happen before ff3 launch...what's the eta on 3.4.3?

Comment 5

[Laura Thomson :laura]

Code freeze is Monday, will push next Thursday.

Comment 6

[Laura Thomson :laura]

Fixed in r14442.

Comment 7

[Stephen Donner [:stephend] Not actively reading bugmail]

Hey Laura - is there a good testcase I can use to verify this?  Something along the lines of changing the Firefox 3 in-client |extensions.getAddons.*| URLs to point to preview.*, and then using livehttpheaders to verify from where the images are being pulled?

Comment 8

[Laura Thomson :laura]

Yep, look at e.g.
https://services.addons.mozilla.org/en-US/firefox/api/1.1/addon/6534
compare to same page on preview (when preview is up again)
The icon, thumbnail, and install urls should be on SAMO not on AMO.

Comment 9

[Stephen Donner [:stephend] Not actively reading bugmail]

Verified FIXED, comparing https://services.addons.mozilla.org/en-US/firefox/api/1.1/addon/6534 against https://preview.addons.mozilla.org/en-US/firefox/api/1.1/addon/6534.

preview has:

<icon>
https://services.addons.mozilla.org/img/addon-icn.png
</icon>

<thumbnail>
https://services.addons.mozilla.org/img/no-preview.png
</thumbnail>

<install hash="sha1:c2bf3313a1533a36dc94b4902e391addfa3290e3" os="Linux">
https://services.addons.mozilla.org/downloads/file/23772/_find_bar_-1.0-fx+tb-linux.xpi
</install>

Thanks for the testcase, Laura!

Comment 10

[Michael Morgan [:morgamic]]

Hey, because of this preview images and installation buttons are all completely broken in the add-ons manager.

Comment 11

[Dave Townsend [:mossop]]

(In reply to comment #8)
> Yep, look at e.g.
> https://services.addons.mozilla.org/en-US/firefox/api/1.1/addon/6534
> compare to same page on preview (when preview is up again)
> The icon, thumbnail, and install urls should be on SAMO not on AMO.

Verifying by inspection is probably necessary in this case. But we should also be verifying that the Add-ons manager is still working with the new data. I'd be happy to make some notes on how to do so if there is somewhere appropriate to put them.

Comment 12

[Michael Morgan [:morgamic]]

Dave - if you can poke something in http://wiki.mozilla.org/Update:Remora_Testing that would be awesome.

In the meantime, I've filed a blocker IT bug to revert the config change for SITE_URL so images/files will point to AMO until we sort out the right solution.

Comment 13

[Michael Morgan [:morgamic]]

SAMO fixed in AMO prod, the images point to AMO as well as downloads.

We need to make sure to QA the add-ons manager with the new XML anytime we flip the domains next time.

So if we want to achieve 100% separation we'll probably need to allow preview images and downloads to go through SAMO.  Right now their denied via apache.

Comment 14

[Reed Loden [:reed]]

(In reply to comment #13)
> So if we want to achieve 100% separation we'll probably need to allow preview
> images and downloads to go through SAMO.  Right now their denied via apache.

This is easy to do...

Comment 15

[Michael Morgan [:morgamic]]

That's great, Reed.  We'll take a look at it next week and QA it appropriately.  Go have a weekend.

Comment 16

[Reed Loden [:reed]]

(In reply to comment #15)
> That's great, Reed.  We'll take a look at it next week and QA it appropriately.
>  Go have a weekend.

Weekend? What's that? I have things to fix! :)

Comment 17

[Michael Morgan [:morgamic]]

We should also consider how this affects download counts before we switch it back.  So we shouldn't rush and change it back to SAMO at this point until we've had more discussion about metrics and QA.

Comment 18

[Michael Morgan [:morgamic]]

IT - how important is it that images and downloads come only from SAMO?

Comment 19

[Reed Loden [:reed]]

(In reply to comment #18)
> IT - how important is it that images and downloads come only from SAMO?

I won't speak for Justin, but comment #2 does say they must be 100% self-sufficient, and that requirement hasn't changed, afaik. SAMO and AMO are not both on the same cluster machines, so it's important to have them separated so if AMO goes down, it doesn't take SAMO down with it.

Comment 20

[Michael Morgan [:morgamic]]

I'd like to see images on a CDN at some point, but downloads should be easily doable on SAMO.  Laura - does it make sense to add controller actions to the API that serve images and files?

Comment 21

[Laura Thomson :laura]

morgamic, I think that will add load we don't need.

I would vote for just re-enabling images and file serving from here for now and moving them onto the CDN as soon as IT is done building it.  When is that due, Reed?

Comment 22

[Justin Fitzhugh]

It'd really be great to separate these out 100%, for cdn and just general load tracking and scaling.  CDN is a ways out - is this a huge amount of work?  I'd rate it as a really-nice-to-have, but not worth months of dev effort.

Comment 23

[Fred Wenzel [:wenzel]]

(In reply to comment #22)
> It'd really be great to separate these out 100%, for cdn and just general load
> tracking and scaling.

Still separately for SAMO and AMO itself then, or having both point to the same place (like images.addons.mozilla.org or so)?

Comment 24

[Justin Fitzhugh]

samo and amo completely self contained - i.e. no shared hostnames, so we could totally disable one or see the load from one vs the other.  if there is some architecture reason this can't happen, I am all ears.

Comment 25

[Fred Wenzel [:wenzel]]

Nah, I don't see a problem with that. It just means we'd need to keep two separate copies of the very same image files, one for AMO to access, the other for SAMO.

Comment 26

[Justin Scott [:fligtar]]

3.4.5 was pushed on Friday.

Comment 27

[Dave Dash [:davedash, :dd] (assign all bugs to mbrandt)]

Is this still necessary?  If so it can be fixed in Zamboni by IT in settings_local.py