1309942 - Kick off jenkins post-deploy

Status: RESOLVED WONTFIX

(Socorro :: Infra, task)

Description

[Peter Bengtsson [:peterbe]]

After both stage and prod deploys can we add a thing that kicks off jenkins jobs (different jenkins) that runs the WebQA suite. 

At the moment the webqa test suite in jenkins is kicked off on an hourly cron job (or someone asking it to start in #breakpad). This means that you might not get notified in channel about a breakage until ~59min after you've upgraded stage/prod.

Also, when it runs in a cron job, if it's unlucky and fails because of network IO timeouts, it might blather that the tests are broken even though nothing has changed.

Comment 1

[Peter Bengtsson [:peterbe]]

Matt, 
Can you provide the URL/instructions for something that JP can add to the deployment hooks?

Comment 2

[Matt Brandt [:mbrandt]]

I'll setup a meeting that includes rpap, jp, and myself. Updates to the bug coming shortly.

Comment 3

[Miles Crabill [:miles]]

Matt and I talked about this and it sounded as though configuring authentication to the web QA Jenkins would be a little tricky.

If the web QA Jenkins is:
  1. behind the VPN
  2. LDAP authenticated

This will be pretty unresolvable. Otherwise, we could create some kind of authentication token and hit an API on the web QA Jenkins.

Either way, it sounds like the the tests take <5 minutes and run once an hour. We could also just decrease the interval of the test runs, and consider that a sufficient resolution.

Needinfoing Matt for a plan of action.

Comment 4

[Peter Bengtsson [:peterbe]]

Matt, 
Another course of action would be, if possible, to point Jenkins to build on content-change of https://crash-stats.allizom.org/status/revision/ 
Then it could execute on a really short cron interval but not execute unless it's different. Is there such a plugin/config?

Comment 5

[Matt Brandt [:mbrandt]]

Let's wait until the new year to put some thought into once we have all returned from the holidays. I don't have the bandwidth to work on this atm.

Comment 6

[Lonnen :lonnen]

open anew when and if we are ready to tackle this