Status

RESOLVED FIXED
2 years ago
2 years ago

People

(Reporter: mythmon, Assigned: relud)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

2 years ago
Please deploy Normandy v26.


# Major changes

* Start enforcing storage durability checks
* Add global rate limit of once-per-day for heartbeat prompts
* Don't allow a recipe's heartbeat prompts to repeat for the same user twice (previously it could repeat every 7 days)
* Make database connection re-use configurable (previously it would always be disabled)
* Enable application request logging

# Migrations

N/A

# Operations considerations

Request logging will add a significant amount of logging, since we will now log the classify_client endpoint requests on the app. It should still be less than nginx, but it may run into scaling limits of systemd logging. We should watch out for this.

# Commit log

$ git log v25..v26 --pretty=format:"%h %s"

751e515 Merge pull request #372 from mythmon/request-summary-logging
6d1a4ab Merge pull request #373 from mythmon/max-con-age-units
6e3f4fc docs: Note units for max-conn-age
1efb2e5 Enable request.summary logging
f4d819a Merge pull request #371 from mythmon/connection-max-age
36d9c5c Merge pull request #368 from andymikulski/348-htbt-thrttle-again
4dbbeac Simplify timeSince calculation
37c70b0 settings: Add DJANGO_MAX_CONN_AGE envvar
ccab837 Move heartbeat throttle comparison into own fn
9421604 Simplify lastShown logic
4124cb8 Merge pull request #370 from mythmon/give-durability-checks-teeth
85d89fa self-repair: Enable storage durability checking
cb6150c Fixes #347, #348


Thanks!
Version check verifies v26 is installed

(venv) ~/m/normandy ((v26)) $ curl https://normandy.stage.mozaws.net/__version__
{"commit":"751e515e09d79cbd21c72963b572ee87040fc427","commit_link":"https://github.com/mozilla/normandy/commit/751e515e09d79cbd21c72963b572ee87040fc427","configuration":"ProductionReadOnly","source":"https://github.com/mozilla/normandy","version":"v26"}

Contract tests of regular nodes passed and results posted to Testrail

(venv) ~/m/normandy ((v26)) $ py.test --no-ssl-cert-check --testrail=contract-tests/testrail.cfg --server=https://normandy.stage.mozaws.net -v --tr_name='Regular Node Stage Deploy Test' contract-tests/
============================================================ test session starts =============================================================
platform darwin -- Python 3.5.2, pytest-3.0.2, py-1.4.31, pluggy-0.3.1 -- /Users/chartjes/mozilla/normandy/venv/bin/python3.5
cachedir: .cache
django settings: normandy.settings (from ini file)
rootdir: /Users/chartjes/mozilla/normandy, inifile: pytest.ini
plugins: django-2.9.1, mock-0.11.0, testrail-0.0.10
collected 16 items

contract-tests/test_api.py::test_expected_action_types PASSED
contract-tests/test_api.py::test_console_log PASSED
contract-tests/test_api.py::test_show_heartbeat PASSED
contract-tests/test_api.py::test_recipe_signatures PASSED
contract-tests/test_api.py::testrail <- venv/lib/python3.5/site-packages/pytest_testrail/plugin.py PASSED
contract-tests/test_heartbeat.py::test_heartbeat_is_ok PASSED
contract-tests/test_heartbeat.py::testrail <- venv/lib/python3.5/site-packages/pytest_testrail/plugin.py PASSED
contract-tests/test_performance.py::TestHotPaths::test_no_redirects[/en-US/repair] PASSED
contract-tests/test_performance.py::TestHotPaths::test_no_redirects[/en-US/repair/] PASSED
contract-tests/test_performance.py::TestHotPaths::test_no_redirects[/api/v1/recipe/] PASSED
contract-tests/test_performance.py::TestHotPaths::test_no_redirects[/api/v1/action/] PASSED
contract-tests/test_performance.py::TestHotPaths::test_no_vary_cookie[/en-US/repair] PASSED
contract-tests/test_performance.py::TestHotPaths::test_no_vary_cookie[/en-US/repair/] PASSED
contract-tests/test_performance.py::TestHotPaths::test_no_vary_cookie[/api/v1/recipe/] PASSED
contract-tests/test_performance.py::TestHotPaths::test_no_vary_cookie[/api/v1/action/] PASSED
contract-tests/test_performance.py::testrail <- venv/lib/python3.5/site-packages/pytest_testrail/plugin.py PASSED

Contract tests of admin nodes passed and results posted to Testrail

(venv) ~/m/normandy ((v26)) $ py.test --no-ssl-cert-check --testrail=contract-tests/testrail.cfg --server=https://normandy-admin.stage.mozaws.net -v --tr_name='Admin Node Stage Deploy Test' contract-tests/
============================================================ test session starts =============================================================
platform darwin -- Python 3.5.2, pytest-3.0.2, py-1.4.31, pluggy-0.3.1 -- /Users/chartjes/mozilla/normandy/venv/bin/python3.5
cachedir: .cache
django settings: normandy.settings (from ini file)
rootdir: /Users/chartjes/mozilla/normandy, inifile: pytest.ini
plugins: django-2.9.1, mock-0.11.0, testrail-0.0.10
collected 16 items

contract-tests/test_api.py::test_expected_action_types PASSED
contract-tests/test_api.py::test_console_log PASSED
contract-tests/test_api.py::test_show_heartbeat PASSED
contract-tests/test_api.py::test_recipe_signatures PASSED
contract-tests/test_api.py::testrail <- venv/lib/python3.5/site-packages/pytest_testrail/plugin.py PASSED
contract-tests/test_heartbeat.py::test_heartbeat_is_ok PASSED
contract-tests/test_heartbeat.py::testrail <- venv/lib/python3.5/site-packages/pytest_testrail/plugin.py PASSED
contract-tests/test_performance.py::TestHotPaths::test_no_redirects[/en-US/repair] PASSED
contract-tests/test_performance.py::TestHotPaths::test_no_redirects[/en-US/repair/] PASSED
contract-tests/test_performance.py::TestHotPaths::test_no_redirects[/api/v1/recipe/] PASSED
contract-tests/test_performance.py::TestHotPaths::test_no_redirects[/api/v1/action/] PASSED
contract-tests/test_performance.py::TestHotPaths::test_no_vary_cookie[/en-US/repair] SKIPPED
contract-tests/test_performance.py::TestHotPaths::test_no_vary_cookie[/en-US/repair/] SKIPPED
contract-tests/test_performance.py::TestHotPaths::test_no_vary_cookie[/api/v1/recipe/] SKIPPED
contract-tests/test_performance.py::TestHotPaths::test_no_vary_cookie[/api/v1/action/] SKIPPED
contract-tests/test_performance.py::testrail <- venv/lib/python3.5/site-packages/pytest_testrail/plugin.py PASSED

Preview of testing heartbeat recipe using SHIELD Control Panel failed with the message:

ERROR RUNNING RECIPE: TYPEERROR: N IS UNDEFINED

Preview of testing console-log recipe using SHIELD Control Panel passed
Flags: needinfo?(mcooper)
Upon testing, there were some issues with the client code with regard to some optional function parameters. We've corrected the error, as well as written some tests to prevent this case from slipping past dev again.

Additionally, some old caching code/documentation was removed!

Please deploy v27.

# Commit Log

$ git log v26..v27 --pretty=format:"%h %s"

0069969 Merge pull request #376 from Osmose/fix-storage-options
6f7716d Fix #375: Make LocalStorage options actually optional.
9e63b32 Merge pull request #374 from mythmon/we-dont-need-no-stinking-caches
738297d Remove unused in-memory caching infrastructure and docs

Thank you!
Summary: Normandy Deploy v26 → Normandy Deploy v26/v27
Version check confirms v27 deployed in staging:

~/mozilla/normandy (0069969 ✘)✭ ᐅ curl https://normandy.stage.mozaws.net/__version__
{"commit":"006996913fd9cd563881da611112e2992ace4935","commit_link":"https://github.com/mozilla/normandy/commit/006996913fd9cd563881da611112e2992ace4935","configuration":"ProductionReadOnly","source":"https://github.com/mozilla/normandy","version":"v27"}

Contract tests for regular nodes passed

(venv) ~/mozilla/normandy (0069969 ✘)✭ ᐅ py.test --no-ssl-cert-check --testrail=contract-tests/testrail.cfg --server=https://normandy.stage.mozaws.net -v --tr_name='Regular Node Stage Deploy Test' contract-tests/
============================================================ test session starts =============================================================
platform darwin -- Python 3.5.2, pytest-3.0.2, py-1.4.31, pluggy-0.3.1 -- /Users/chartjes/mozilla/normandy/venv/bin/python3.5
cachedir: .cache
django settings: normandy.settings (from ini file)
rootdir: /Users/chartjes/mozilla/normandy, inifile: pytest.ini
plugins: django-2.9.1, mock-0.11.0, testrail-0.0.10
collected 16 items

contract-tests/test_api.py::test_expected_action_types PASSED
contract-tests/test_api.py::test_console_log PASSED
contract-tests/test_api.py::test_show_heartbeat PASSED
contract-tests/test_api.py::test_recipe_signatures PASSED
contract-tests/test_api.py::testrail <- venv/lib/python3.5/site-packages/pytest_testrail/plugin.py PASSED
contract-tests/test_heartbeat.py::test_heartbeat_is_ok PASSED
contract-tests/test_heartbeat.py::testrail <- venv/lib/python3.5/site-packages/pytest_testrail/plugin.py PASSED
contract-tests/test_performance.py::TestHotPaths::test_no_redirects[/en-US/repair] PASSED
contract-tests/test_performance.py::TestHotPaths::test_no_redirects[/en-US/repair/] PASSED
contract-tests/test_performance.py::TestHotPaths::test_no_redirects[/api/v1/recipe/] PASSED
contract-tests/test_performance.py::TestHotPaths::test_no_redirects[/api/v1/action/] PASSED
contract-tests/test_performance.py::TestHotPaths::test_no_vary_cookie[/en-US/repair] PASSED
contract-tests/test_performance.py::TestHotPaths::test_no_vary_cookie[/en-US/repair/] PASSED
contract-tests/test_performance.py::TestHotPaths::test_no_vary_cookie[/api/v1/recipe/] PASSED
contract-tests/test_performance.py::TestHotPaths::test_no_vary_cookie[/api/v1/action/] PASSED
contract-tests/test_performance.py::testrail <- venv/lib/python3.5/site-packages/pytest_testrail/plugin.py PASSED

Contract tests for admin nodes passed

(venv) ~/mozilla/normandy (0069969 ✘)✭ ᐅ py.test --no-ssl-cert-check --testrail=contract-tests/testrail.cfg --server=https://normandy-admin.stage.mozaws.net -v --tr_name='Admin Node Stage Deploy Test' contract-tests/
==================================================================== test session starts =====================================================================
platform darwin -- Python 3.5.2, pytest-3.0.2, py-1.4.31, pluggy-0.3.1 -- /Users/chartjes/mozilla/normandy/venv/bin/python3.5
cachedir: .cache
django settings: normandy.settings (from ini file)
rootdir: /Users/chartjes/mozilla/normandy, inifile: pytest.ini
plugins: django-2.9.1, mock-0.11.0, testrail-0.0.10
collected 16 items

contract-tests/test_api.py::test_expected_action_types PASSED
contract-tests/test_api.py::test_console_log PASSED
contract-tests/test_api.py::test_show_heartbeat PASSED
contract-tests/test_api.py::test_recipe_signatures PASSED
contract-tests/test_api.py::testrail <- venv/lib/python3.5/site-packages/pytest_testrail/plugin.py PASSED
contract-tests/test_heartbeat.py::test_heartbeat_is_ok PASSED
contract-tests/test_heartbeat.py::testrail <- venv/lib/python3.5/site-packages/pytest_testrail/plugin.py PASSED
contract-tests/test_performance.py::TestHotPaths::test_no_redirects[/en-US/repair] PASSED
contract-tests/test_performance.py::TestHotPaths::test_no_redirects[/en-US/repair/] PASSED
contract-tests/test_performance.py::TestHotPaths::test_no_redirects[/api/v1/recipe/] PASSED
contract-tests/test_performance.py::TestHotPaths::test_no_redirects[/api/v1/action/] PASSED
contract-tests/test_performance.py::TestHotPaths::test_no_vary_cookie[/en-US/repair] SKIPPED
contract-tests/test_performance.py::TestHotPaths::test_no_vary_cookie[/en-US/repair/] SKIPPED
contract-tests/test_performance.py::TestHotPaths::test_no_vary_cookie[/api/v1/recipe/] SKIPPED
contract-tests/test_performance.py::TestHotPaths::test_no_vary_cookie[/api/v1/action/] SKIPPED
contract-tests/test_performance.py::testrail <- venv/lib/python3.5/site-packages/pytest_testrail/plugin.py PASSED

Preview of heartbeat recipe worked in admin control UI
Preview of console-log recipe worked in admin control UI

QA approves deployment to production
(Assignee)

Updated

2 years ago
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Flags: needinfo?(mcooper)
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.