1087397 - DNS Update: Increase thottle for Hello to 20%

Status: VERIFIED FIXED

(Cloud Services :: Operations: Miscellaneous, task)

Description

[Adam Roach [:abr]]

tldr: Please change the DNS record for soft-start.loop.services.mozilla.com to "127.51.51.51"

Now that Bug 1084097 has been fixed and verified, we should start turning the throttle up on the Hello service. The overall scheme is described at https://wiki.mozilla.org/Loop/Load_Handling#Recommended_Handling

According to that plan, the first increase is from 10% to 20% (which should have taken place on Monday, if not for Bug 1084097). In our throttling scheme, the IP value corresponding to 20% is 127.51.51.51.

Comment 1

[Bob Micheletto [:bobm]]

Taking this ticket.  Per discussion in IRC, waiting for input from QA.

Comment 2

[Adam Roach [:abr]]

Proposed testing:

1) Start Beta 34 using a new profile.
2) Open about:config
  * Ensure that "loop.throttled" is true.
  * Set "loop.soft_start_ticket_number" to 3355442
3) Restart the browser with the same profile
4) Check for Hello in the customization palette. It should _not_ be there.
5) Open about:config, and set "loop.soft_start_ticket_number" to 3355444
6) Restart the browser with the same profile
7) Check for Hello in the customization palette. It _should_ be there.

Comment 3

[James Bonacci [:jbonacci]]

Adding appropriate QA to the discussion: Anthony and Tony
I can help with the OPs, server-side stuff...

Comment 4

[u279076]

Comment 2 should be simple enough to verify once this is live.

Comment 5

[Adam Roach [:abr]]

(In reply to Anthony Hughes, QA Mentor (:ashughes) from comment #4)
> Comment 2 should be simple enough to verify once this is live.

Gah! I just realized that I got the numbering backwards in the test steps.

Step 2 should say "3355444", and step 5 should say "3355442".

Tagging Anthony for needinfo here, since overlooking this will cause us pain.

Comment 6

[Bob Micheletto [:bobm]]

The DNS has been changed.

Comment 7

[u279076]

I just tested this with Firefox 34b2 on Ubuntu 14.04:

1) Start Beta 34 using a new profile.
2) Open about:config
  * Ensure that "loop.throttled" is true.
  * Set "loop.soft_start_ticket_number" to 3355444
3) Restart the browser with the same profile
4) Check for Hello in the customization palette. It should _not_ be there.
> Success: Hello button is not in the customization palette

5) Open about:config, and set "loop.soft_start_ticket_number" to 3355442
6) Restart the browser with the same profile
7) Check for Hello in the customization palette. It _should_ be there.
> Fail: Hello button is still not in the customization palette

Comment 8

[u279076]

Adam, this isn't working. See comment 7.

Comment 9

[u279076]

(In reply to Anthony Hughes, QA Mentor (:ashughes) from comment #8)
> Adam, this isn't working. See comment 7.

I chatted with Adam about this on IRC and he suggested including some wait time in the steps since it could take several minutes for the DNS information to propagate. Also, steps should include installing https://dl.dropboxusercontent.com/u/53717247/loopthrottle-0.0.1.xpi and verifying the percentage.

Comment 10

[Adam Roach [:abr]]

So, I'm theorizing that the value may have not propagated through DNS all the way before the test. At Anthony's suggestion, I'm amending the test to include the following steps at the beginning:

a) Install the add-on at https://dl.dropboxusercontent.com/u/53717247/loopthrottle-0.0.1.xpi
b) If the blue "0%" does not automatically show up somewhere visible, move it from the customization palette onto the navbar.
c) Restart the browser. The indicator should not read 0% after this restart. If it does, contact :abr on IRC.
d) Wait until the indicator reflects the expected value (20%, in this case). This may take several minutes. If it takes longer than an hour, then something has gone wrong with the DNS update, and the test should be considered to fail.
e) Once the indicator correctly reflects the value we're expecting, run the test steps in comment 2, as amended by comment 5.

Comment 11

[u279076]

Retested this in Firefox 34.0b2 on Ubuntu 14.04 and it passed this time.

1. Start Firefox with a new profile
2. Install https://dl.dropboxusercontent.com/u/53717247/loopthrottle-0.0.1.xpi and restart Firefox
3. Enter customize mode and add the 0% blue button to your toolbar then restart Firefox
> The 0% blue button reads 20% and has a tooltip of "soft-start.loop.services.mozilla.com = 127.51.51.51"

4. Open about:config and set the following:
* loop.throttled = true
* loop.soft_start_ticket_number = 3355444

5. Restart Firefox and enter customize mode
> There is no Hello button in the customization palette

6. Open about:config and set the following:
* loop.soft_start_ticket_number = 3355442

7. Restart Firefox and enter customize mode
> The Hello button appeared in the customization palette after waiting several seconds

8. Drag the Hello button to the toolbar, exit customize mode, and try to start a call
> I was able to make a call using the Hello button

Comment 12

[u279076]

I've now tested these steps on Mac OS X 10.9 and Windows 8.1 64-bit without any problems. Calling this bug verified fixed.

Comment 13

[James Bonacci [:jbonacci]]

:ashughes thanks for picking this up...

Comment 14

[u279076]

(In reply to James Bonacci [:jbonacci] from comment #13)
> :ashughes thanks for picking this up...

No problem. Please keep qe-verify+ on this bug to count towards our workload metrics.