665620 - videos-cdn.mozilla.net doesn't have a valid SSL cert

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Description

[Wil Clouser [:clouserw]]

We're trying to use some videos on AMO but the SSL cert on videos-cdn.mozilla.net is invalid for the domain.  Please get a valid cert.  Thanks

Comment 1

[Wil Clouser [:clouserw]]

These videos are going to be on https://addons.mozilla.org/en-US/developers/builder

Until this is fixed, the EV notice is killed when people play videos

Comment 2

[matthew zeier [:mrz]]

We aren't setup to do SSL with EdgeCast on videos-cdn and doing so would require us to move to their ADN platform.  

What do you expect the video load to be?  May be better to serve this ourselves and not off videos-cdn.

Comment 3

[matthew zeier [:mrz]]

We aren't setup to do SSL with EdgeCast on videos-cdn and doing so would require us to move to their ADN platform.  

What do you expect the video load to be?  May be better to serve this ourselves and not off videos-cdn.

Comment 4

[Wil Clouser [:clouserw]]

I don't know expected traffic.  Tara can tell us.

Comment 5

[Tara (musingt)]

Hi there.  Not sure why I'm the main point of contact here :)  it's not my project. I just managed the design of the site and related assets.  Cc-ing Dan and Daniel to help offer more insight...

Comment 6

[Tara (musingt)]

and Grace, who is actually doing the videos.

Comment 7

[Grace Jimenez :grace]

for firefox 4, the video page was visited about about 50K the day of launch and 400K the week of launch. 

also from firefox 4, the video with the most views the day of launch was 50K (what's new video desktop) 

my assumption is that it would be  less than those numbers.

Comment 8

[matthew zeier [:mrz]]

Fx4's whatsnew videos were the single largest source of traffic during the release.  And by a lot.  We peaked at 5Gbps through EdgeCast just with video traffic.  We average 5Gbps during a non-release period globally through all web properties.

Wil - how/where is the AMO video stuff?  It's hit rate can't be anything like the whatsnew content can it?  

Also, Grace tells me this is in the launch path?  I don't know how to spin up capacity in < 18 hours for something like this -over- SSL.

About the only short-term hack would be to run this through static-cdn.addons.mozilla.net and that'd probably involve some magic (my wand is busted but I hear Jake and Oremj have working ones).

Comment 9

[Wil Clouser [:clouserw]]

(In reply to comment #8)
> Wil - how/where is the AMO video stuff?  It's hit rate can't be anything
> like the whatsnew content can it?

The videos are linked in the URL in comment 1.  The expected traffic is comment 7 - Grace's estimate will be better than mine.
  
> Also, Grace tells me this is in the launch path?  I don't know how to spin
> up capacity in < 18 hours for something like this -over- SSL.
> 
> About the only short-term hack would be to run this through
> static-cdn.addons.mozilla.net and that'd probably involve some magic (my
> wand is busted but I hear Jake and Oremj have working ones).

This bug isn't marked a blocker - this is a conversation for how can we fix this.  Don't panic. :)

Comment 10

[matthew zeier [:mrz]]

> Don't panic. :)

If I could, I'd totally give you a bro hug (http://tinyurl.com/3tl8wdn).

Comment 11

[matthew zeier [:mrz]]

(In reply to comment #1)
> These videos are going to be on
> https://addons.mozilla.org/en-US/developers/builder
> 
> Until this is fixed, the EV notice is killed when people play videos

Any reason to not serve this from the addons.mozilla.org host name or even that static-cdn hostname?  

Tara/Grace - I suspect the rest of this bug will be noise.  Feel free to take yourselves off.

Comment 12

[Wil Clouser [:clouserw]]

Nope, that's just where they put the videos. If IT puts them somewhere else we can link there. They are to big for the repo.  AMO has another push on Thurs if you want to do it then. Next week works too

Comment 13

[Jake Maul [:jakem]]

Old bug... can I WONTFIX and leave videos-cdn as a non-SSL CDN, or is there some action for IT here?

Comment 14

[Wil Clouser [:clouserw]]

AMO is about to get a video on it's home page which shouldn't break SSL.  I don't remember if we moved the videos originally in this bug to a different CDN or not (comment 11 mentions this).  If we did to avoid this problem, we'll have to put the new video there.  If not, we should fix this bug.

Comment 15

[Jake Maul [:jakem]]

The videos at this link are from videos-cdn.mozilla.net:

https://addons.mozilla.org/en-US/developers/builder

This does not break SSL when *visiting the page*, but it does break once the videos are clicked.

I do not see any videos on the AMO homepage... either that hasn't happened, or already happened and they're removed again.


These 2 videos are actually 6 files (3 different encodings each) totaling about 150MB.

If AMO is the only site using videos, we should probably just move these videos to AMO's NetApp share, and serve them up through AMO's already-SSL-capable CDN.


If other sites are intending to start serving videos more, it might be worthwhile to upgrade videos-cdn to an SSL CDN. Some web devs might find this more convenient than to import these big files into their repos, or to deal with how to get data onto their shared storage. This seems to me the more likely scenario... for instance, SUMO uses their CDN for videos, and it is non-SSL-capable. This works most of the time (they don't force SSL), but can cause mixed-mode situations at times.


I will look into getting videos-cdn SSL-capable. This seems like the better long-term solution.

Comment 16

[Jake Maul [:jakem]]

We've rolled getting a cert for this into the PO for static-ssl-cdn.addons.mozilla.net. No ETA on when it will be live yet, but I'm hoping sometime in the next week or so.

Comment 18

[Jake Maul [:jakem]]

This is now resolved (pending a few minutes of propagation and TTL). videos-cdn.mozilla.net has been migrated from the Edgecast "HTTP Large" platform to the ADN platform, which is where we have our SSL certs. Last month they added videos-cdn.mozilla.net to the cert on that, and it appears to be working properly for me.

Please let me know if anything seems incorrect. Thanks!

Comment 19

[Jake Maul [:jakem]]

This is temporarily reverted, but only for an hour or so. This broke videos.mozilla.org and I had to revert DNS until Edgecast propagates the new/proper settings on their end, at which point I can... un-revert.

Comment 20

[Jake Maul [:jakem]]

This is un-reverted.