1886018 - Implement PubSub queue class for GCP in Socorro

Status: RESOLVED FIXED

(Socorro :: Backend, task)

Description

[Daniel Thorn [:relud]]

like bug 1878423 but the socorro class must read and write to pubsub.

make sure to test the integration in both socorro and webapp.

testing a non-default queue in webapp will require a separate pytest call which should follow the practice set forth in bug 1885442

pubsub no longer supports return_immediately for pull operations, so CI tests should make use of the "test" crash id to avoid slow tests because the server is waiting for more messages that aren't coming. This aspect also needs to be tested carefully in stage to determine the impact of pulling from normally-empty priority and reprocessing queues.

Comment 1

[Daniel Thorn [:relud]]

Attachment: [mozilla-services/socorro] bug-1886018: support pubsub crashqueue (#6554)

Comment 2

[Daniel Thorn [:relud]]

relud merged PR [mozilla-services/socorro]: bug-1886018: support pubsub crashqueue (#6554) in e03e1fe.

Comment 3

[Daniel Thorn [:relud]]

resolved before merging to prod because this change shouldn't impact AWS prod

Comment 4

[Daniel Thorn [:relud]]

reopening due to test failures: https://app.circleci.com/pipelines/github/mozilla-services/socorro/3054/workflows/667411a2-852c-426e-b546-bd29ed61dafa/jobs/9142

Comment 5

[Daniel Thorn [:relud]]

Attachment: [mozilla-services/socorro] bug-1886018: Fix flaky tests due to pubsub duplicate delivery (#6571)

Comment 6

[Daniel Thorn [:relud]]

relud merged PR [mozilla-services/socorro]: bug-1886018: Fix flaky tests due to pubsub duplicate delivery (#6571) in 1a17a08.

Comment 7

[Will Kahn-Greene [:willkg] ET needinfo? me]

This is another CI failure from what looks to be another flaky test:

https://app.circleci.com/pipelines/github/mozilla-services/socorro/3069/workflows/3cd2a80f-b5c4-4bd1-bb26-1f360ac88e36/jobs/9158

https://app.circleci.com/pipelines/github/mozilla-services/socorro/3071/workflows/ea38fc3c-8dc8-49b4-a683-302b64cb53ac/jobs/9161

Comment 8

[Daniel Thorn [:relud]]

Attachment: [mozilla-services/socorro] bug-1886018: add delay in pubsub tests between publish and pull (#6575)

Comment 9

[Daniel Thorn [:relud]]

relud merged PR [mozilla-services/socorro]: bug-1886018: add delay in pubsub tests between publish and pull (#6575) in bbcd80f.

Comment 10

[Daniel Thorn [:relud]]

Attachment: [mozilla-services/socorro] bug-1886018: update dev.rst to mention pubsub (#6578)

Comment 11

[Daniel Thorn [:relud]]

relud merged PR [mozilla-services/socorro]: bug-1886018: update dev.rst to mention pubsub (#6578) in f83f2d3.