No pulse notifications for mulet builds using the normalized exchange

RESOLVED FIXED

Status

Webtools
Pulse
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: ahal, Assigned: jgriffin)

Tracking

Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments)

(Reporter)

Description

3 years ago
I created a NormalizedBuildConsumer that listens on 'build.#', prints the platform and exits. After a short while I saw all platforms except for b2g ones.

Specifically, emulator*, mulet and device builds are not triggering pulse notifications. Though for some reason b2g_desktop builds *are* triggering notifications. I don't know what the common denominator here is.

This is blocking me from collecting test manifest data from emulator and mulet builds.
(Assignee)

Updated

3 years ago
Assignee: nobody → jgriffin
(Reporter)

Updated

3 years ago
OS: Linux → All
Hardware: x86_64 → All
Summary: No pulse notifications for B2G builds → No pulse notifications for B2G builds using the normalized exchange
(Reporter)

Updated

3 years ago
Blocks: 996183
No longer blocks: 1066279
(Assignee)

Comment 1

3 years ago
So, this won't be a trivial fix.  The reason we're missing these is because releng isn't publishing a build_url for emulator builds, probably because they're uploaded differently than regular builds (to an LDAP-protected url under https://pvtbuilds.mozilla.org/pub/mozilla.org/b2g/tinderbox-builds).

The normalizer skips notifications for builds that don't have a build_url, because it queries the build_url to determine when to publish the normalized message, and because presumably most consumers can't use a notification that isn't linked to a build location, which is true in this case as well.

The fix would require two things:

1 - Either releng to publish the build_url for builds like these, or (more fragile) we could guess the url based on other properties of the pulse message.
2 - We'd have to have a way of fetching files at https://pvtbuilds.mozilla.org/pub/mozilla.org/b2g/tinderbox-builds from the VM that the translator is running on; I'm not sure if we could ask IT to open this up, or if we'd have to provide some set of credentials.

I'll ping releng for #1 and file a bug for #2, but I don't know how long fixing this might take.
(Assignee)

Updated

3 years ago
Depends on: 650989
(Assignee)

Updated

3 years ago
Depends on: 1071945
(Assignee)

Comment 2

3 years ago
Created attachment 8496120 [details] [review]
Link to Github pull-request: https://github.com/mozilla/pulsetranslator/pull/3

This allows the pulse translator to publish normalized messages for B2G emulator builds.  Unfortunately, such messages will have an empty 'testsurl' field, because the testsurl is not published to the pulse stream, and we're unable to guess it correctly without scraping https://pvtbuilds.mozilla.org/pub/mozilla.org/b2g/tinderbox-builds/.
Attachment #8496120 - Flags: review?(mcote)

Comment 3

3 years ago
Comment on attachment 8496120 [details] [review]
Link to Github pull-request: https://github.com/mozilla/pulsetranslator/pull/3

I admit I didn't test it, but the code looks sane.
Attachment #8496120 - Flags: review?(mcote) → review+
(Assignee)

Comment 4

3 years ago
https://github.com/mozilla/pulsetranslator/commit/376cb5dad6c4f802e924201bdd2d8c93afa11068
(Assignee)

Comment 5

3 years ago
(In reply to Jonathan Griffin (:jgriffin) from comment #4)
> https://github.com/mozilla/pulsetranslator/commit/
> 376cb5dad6c4f802e924201bdd2d8c93afa11068

Deployed.  We should start notifying about B2G emulator builds, however those notifications won't contain a valid testsurl field.  :(  You could figure it out by scraping the FTP directory that the build is located in, but that's yucky too.  I'll see if I can figure out the buildbot patch to fix this in bug 650989.
(Assignee)

Comment 6

3 years ago
Messages for emulator builds are now coming in.  I'll have to see what's going on wrt Mulet, I have a feeling it's a misconfiguration in the pulse translator.
(Reporter)

Updated

3 years ago
Summary: No pulse notifications for B2G builds using the normalized exchange → No pulse notifications for mulet builds using the normalized exchange
(Assignee)

Comment 7

3 years ago
Created attachment 8517123 [details] [review]
Link to Github pull-request: https://github.com/mozilla/pulsetranslator/pull/4
Attachment #8517123 - Flags: review?(mcote)
(Assignee)

Comment 8

3 years ago
(In reply to Jonathan Griffin (:jgriffin) from comment #7)
> Created attachment 8517123 [details] [review]
> Link to Github pull-request:
> https://github.com/mozilla/pulsetranslator/pull/4

We were just missing the mulet platforms; this fixes that.

Comment 9

3 years ago
Comment on attachment 8517123 [details] [review]
Link to Github pull-request: https://github.com/mozilla/pulsetranslator/pull/4

Honestly I think this kinds of minor config patches can be landed without review... particularly since I'm not 100% sure how to verify them :) (I looked at build logs and saw the new strings, so I presume that's sufficient...)
Attachment #8517123 - Flags: review?(mcote) → review+
(Assignee)

Comment 10

3 years ago
https://github.com/mozilla/pulsetranslator/commit/0022d6755f9ebd06f30a8fc6cef57ef0ed70debb
(Assignee)

Comment 11

3 years ago
(In reply to Jonathan Griffin (:jgriffin) from comment #10)
> https://github.com/mozilla/pulsetranslator/commit/
> 0022d6755f9ebd06f30a8fc6cef57ef0ed70debb

deployed.  Re-open if you see any more problems.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.