Last Comment Bug 1071642 - No pulse notifications for mulet builds using the normalized exchange
: No pulse notifications for mulet builds using the normalized exchange
Status: RESOLVED FIXED
:
Product: Webtools
Classification: Server Software
Component: Pulse (show other bugs)
: Trunk
: All All
-- normal (vote)
: ---
Assigned To: Jonathan Griffin (:jgriffin)
:
:
Mentors:
Depends on: 650989 1071945
Blocks: 996183
  Show dependency treegraph
 
Reported: 2014-09-23 08:57 PDT by Andrew Halberstadt [:ahal]
Modified: 2014-11-06 11:15 PST (History)
3 users (show)
See Also:
QA Whiteboard:
Iteration: ---
Points: ---


Attachments
Link to Github pull-request: https://github.com/mozilla/pulsetranslator/pull/3 (49 bytes, text/x-github-pull-request)
2014-09-26 11:32 PDT, Jonathan Griffin (:jgriffin)
mcote: review+
Details | Review | Splinter Review
Link to Github pull-request: https://github.com/mozilla/pulsetranslator/pull/4 (49 bytes, text/x-github-pull-request)
2014-11-04 17:36 PST, Jonathan Griffin (:jgriffin)
mcote: review+
Details | Review | Splinter Review

Description User image Andrew Halberstadt [:ahal] 2014-09-23 08:57:49 PDT
I created a NormalizedBuildConsumer that listens on 'build.#', prints the platform and exits. After a short while I saw all platforms except for b2g ones.

Specifically, emulator*, mulet and device builds are not triggering pulse notifications. Though for some reason b2g_desktop builds *are* triggering notifications. I don't know what the common denominator here is.

This is blocking me from collecting test manifest data from emulator and mulet builds.
Comment 1 User image Jonathan Griffin (:jgriffin) 2014-09-23 14:48:05 PDT
So, this won't be a trivial fix.  The reason we're missing these is because releng isn't publishing a build_url for emulator builds, probably because they're uploaded differently than regular builds (to an LDAP-protected url under https://pvtbuilds.mozilla.org/pub/mozilla.org/b2g/tinderbox-builds).

The normalizer skips notifications for builds that don't have a build_url, because it queries the build_url to determine when to publish the normalized message, and because presumably most consumers can't use a notification that isn't linked to a build location, which is true in this case as well.

The fix would require two things:

1 - Either releng to publish the build_url for builds like these, or (more fragile) we could guess the url based on other properties of the pulse message.
2 - We'd have to have a way of fetching files at https://pvtbuilds.mozilla.org/pub/mozilla.org/b2g/tinderbox-builds from the VM that the translator is running on; I'm not sure if we could ask IT to open this up, or if we'd have to provide some set of credentials.

I'll ping releng for #1 and file a bug for #2, but I don't know how long fixing this might take.
Comment 2 User image Jonathan Griffin (:jgriffin) 2014-09-26 11:32:28 PDT
Created attachment 8496120 [details] [review]
Link to Github pull-request: https://github.com/mozilla/pulsetranslator/pull/3

This allows the pulse translator to publish normalized messages for B2G emulator builds.  Unfortunately, such messages will have an empty 'testsurl' field, because the testsurl is not published to the pulse stream, and we're unable to guess it correctly without scraping https://pvtbuilds.mozilla.org/pub/mozilla.org/b2g/tinderbox-builds/.
Comment 3 User image Mark Côté [:mcote] 2014-09-29 13:30:51 PDT
Comment on attachment 8496120 [details] [review]
Link to Github pull-request: https://github.com/mozilla/pulsetranslator/pull/3

I admit I didn't test it, but the code looks sane.
Comment 5 User image Jonathan Griffin (:jgriffin) 2014-10-01 10:26:46 PDT
(In reply to Jonathan Griffin (:jgriffin) from comment #4)
> https://github.com/mozilla/pulsetranslator/commit/
> 376cb5dad6c4f802e924201bdd2d8c93afa11068

Deployed.  We should start notifying about B2G emulator builds, however those notifications won't contain a valid testsurl field.  :(  You could figure it out by scraping the FTP directory that the build is located in, but that's yucky too.  I'll see if I can figure out the buildbot patch to fix this in bug 650989.
Comment 6 User image Jonathan Griffin (:jgriffin) 2014-10-13 13:18:40 PDT
Messages for emulator builds are now coming in.  I'll have to see what's going on wrt Mulet, I have a feeling it's a misconfiguration in the pulse translator.
Comment 7 User image Jonathan Griffin (:jgriffin) 2014-11-04 17:36:00 PST
Created attachment 8517123 [details] [review]
Link to Github pull-request: https://github.com/mozilla/pulsetranslator/pull/4
Comment 8 User image Jonathan Griffin (:jgriffin) 2014-11-04 17:36:25 PST
(In reply to Jonathan Griffin (:jgriffin) from comment #7)
> Created attachment 8517123 [details] [review]
> Link to Github pull-request:
> https://github.com/mozilla/pulsetranslator/pull/4

We were just missing the mulet platforms; this fixes that.
Comment 9 User image Mark Côté [:mcote] 2014-11-04 17:59:46 PST
Comment on attachment 8517123 [details] [review]
Link to Github pull-request: https://github.com/mozilla/pulsetranslator/pull/4

Honestly I think this kinds of minor config patches can be landed without review... particularly since I'm not 100% sure how to verify them :) (I looked at build logs and saw the new strings, so I presume that's sufficient...)
Comment 11 User image Jonathan Griffin (:jgriffin) 2014-11-06 11:15:01 PST
(In reply to Jonathan Griffin (:jgriffin) from comment #10)
> https://github.com/mozilla/pulsetranslator/commit/
> 0022d6755f9ebd06f30a8fc6cef57ef0ed70debb

deployed.  Re-open if you see any more problems.

Note You need to log in before you can comment on or make changes to this bug.