Closed Bug 623600 Opened 13 years ago Closed 13 years ago

503 on all backend failures

Categories

(Cloud Services Graveyard :: Server: Sync, defect)

x86_64
Linux
defect
Not set
major

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: tarek, Assigned: tarek)

Details

Attachments

(2 files)

Make sure :

* the auth server
** returns 503 on ldap failures
** get back on track when the ldap server is back

* the storage server
** returns 503 on ldap, sql and memcached failures
** get back on track when the ldap or sql server is back
Severity: normal → major
This patch makes sure that:
- add_s and modified_s are also re-tried when a SERVER_DOWN error occured. python-ldap's ReconnectLDAP class did not cover those for some reason. The test is not easy to reproduce in the unittest: it consists or relaunching the ldap server and make sure existing connectors are still running correctly.

- SERVER_DOWN and OTHER are now caught on every failure like TIMEOUT

- a new config was added: global.retry_after. Defaults to 1800.
Attachment #503766 - Flags: review?(telliott)
Goes with patch 503766
Attachment #503768 - Flags: review?(rsoderberg)
Attachment #503768 - Flags: review?(rsoderberg) → review+
Comment on attachment 503766 [details] [diff] [review]
retry-after+503 for LDAP

Looks good. I worry a little that 10 retries is too many - you can clog up a lot of pipes if you're waiting 10s for auth - but that's a separate issue.
Attachment #503766 - Flags: review?(telliott) → review+
(In reply to comment #4)
> Comment on attachment 503766 [details] [diff] [review]
> retry-after+503 for LDAP
> 
> Looks good. I worry a little that 10 retries is too many - you can clog up a
> lot of pipes if you're waiting 10s for auth - but that's a separate issue.

Right, those are aggressive defaults, I'll set the retries to 3 attempts, and the delay between two retries to 100 ms.
Status: ASSIGNED → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
Product: Cloud Services → Cloud Services Graveyard
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: