1071642 - No pulse notifications for mulet builds using the normalized exchange

Status: RESOLVED FIXED

(Webtools :: Pulse, defect)

Description

[Andrew Halberstadt [:ahal]]

I created a NormalizedBuildConsumer that listens on 'build.#', prints the platform and exits. After a short while I saw all platforms except for b2g ones.

Specifically, emulator*, mulet and device builds are not triggering pulse notifications. Though for some reason b2g_desktop builds *are* triggering notifications. I don't know what the common denominator here is.

This is blocking me from collecting test manifest data from emulator and mulet builds.

Comment 1

[Jonathan Griffin (:jgriffin)]

So, this won't be a trivial fix.  The reason we're missing these is because releng isn't publishing a build_url for emulator builds, probably because they're uploaded differently than regular builds (to an LDAP-protected url under https://pvtbuilds.mozilla.org/pub/mozilla.org/b2g/tinderbox-builds).

The normalizer skips notifications for builds that don't have a build_url, because it queries the build_url to determine when to publish the normalized message, and because presumably most consumers can't use a notification that isn't linked to a build location, which is true in this case as well.

The fix would require two things:

1 - Either releng to publish the build_url for builds like these, or (more fragile) we could guess the url based on other properties of the pulse message.
2 - We'd have to have a way of fetching files at https://pvtbuilds.mozilla.org/pub/mozilla.org/b2g/tinderbox-builds from the VM that the translator is running on; I'm not sure if we could ask IT to open this up, or if we'd have to provide some set of credentials.

I'll ping releng for #1 and file a bug for #2, but I don't know how long fixing this might take.

Comment 2

[Jonathan Griffin (:jgriffin)]

Attachment: Link to Github pull-request: https://github.com/mozilla/pulsetranslator/pull/3

This allows the pulse translator to publish normalized messages for B2G emulator builds.  Unfortunately, such messages will have an empty 'testsurl' field, because the testsurl is not published to the pulse stream, and we're unable to guess it correctly without scraping https://pvtbuilds.mozilla.org/pub/mozilla.org/b2g/tinderbox-builds/.

Comment 3

[Mark Côté [:mcote]]

Comment on attachment 8496120 [details] [review]
Link to Github pull-request: https://github.com/mozilla/pulsetranslator/pull/3

I admit I didn't test it, but the code looks sane.

Comment 4

[Jonathan Griffin (:jgriffin)]

https://github.com/mozilla/pulsetranslator/commit/376cb5dad6c4f802e924201bdd2d8c93afa11068

Comment 5

[Jonathan Griffin (:jgriffin)]

(In reply to Jonathan Griffin (:jgriffin) from comment #4)
> https://github.com/mozilla/pulsetranslator/commit/
> 376cb5dad6c4f802e924201bdd2d8c93afa11068

Deployed.  We should start notifying about B2G emulator builds, however those notifications won't contain a valid testsurl field.  :(  You could figure it out by scraping the FTP directory that the build is located in, but that's yucky too.  I'll see if I can figure out the buildbot patch to fix this in bug 650989.

Comment 6

[Jonathan Griffin (:jgriffin)]

Messages for emulator builds are now coming in.  I'll have to see what's going on wrt Mulet, I have a feeling it's a misconfiguration in the pulse translator.

Comment 7

[Jonathan Griffin (:jgriffin)]

Attachment: Link to Github pull-request: https://github.com/mozilla/pulsetranslator/pull/4

Comment 8

[Jonathan Griffin (:jgriffin)]

(In reply to Jonathan Griffin (:jgriffin) from comment #7)
> Created attachment 8517123 [details] [review]
> Link to Github pull-request:
> https://github.com/mozilla/pulsetranslator/pull/4

We were just missing the mulet platforms; this fixes that.

Comment 9

[Mark Côté [:mcote]]

Comment on attachment 8517123 [details] [review]
Link to Github pull-request: https://github.com/mozilla/pulsetranslator/pull/4

Honestly I think this kinds of minor config patches can be landed without review... particularly since I'm not 100% sure how to verify them :) (I looked at build logs and saw the new strings, so I presume that's sufficient...)

Comment 10

[Jonathan Griffin (:jgriffin)]

https://github.com/mozilla/pulsetranslator/commit/0022d6755f9ebd06f30a8fc6cef57ef0ed70debb

Comment 11

[Jonathan Griffin (:jgriffin)]

(In reply to Jonathan Griffin (:jgriffin) from comment #10)
> https://github.com/mozilla/pulsetranslator/commit/
> 0022d6755f9ebd06f30a8fc6cef57ef0ed70debb

deployed.  Re-open if you see any more problems.