1131240 - Enable New Relic Browser metrics

Status: RESOLVED WONTFIX

(Tree Management :: Treeherder: Infrastructure, defect, P4)

Description

[Ed Morley [:emorley]]

Client side metrics, looks like it might be useful.
https://docs.newrelic.com/docs/browser/new-relic-browser

I've tried enabling using the auto-injection new relic agent feature, but the JS snippet isn't appearing on production.

Need to figure out what's happening, eg:
https://docs.newrelic.com/docs/browser/new-relic-browser/installation-configuration/troubleshooting-browser-monitoring-installation

Comment 1

[Ed Morley [:emorley]]

Mauro, do you know what the advantages of our current new relic integration are:
https://github.com/mozilla/treeherder-service/blob/master/treeherder/webapp/wsgi.py#L29

vs this approach?
https://docs.newrelic.com/docs/agents/python-agent/hosting-mechanisms/python-agent-gunicorn

Wondering if using the latter might allow the JS injection to work?

Comment 2

[Ed Morley [:emorley]]

Interestingly, we're getting data from URLs that were 404s, just not those that hit read endpoints.

eg: https://treeherder.mozilla.org/api/foo has the JS injected.

Comment 3

[Ed Morley [:emorley]]

So the 404s are being served by gunicorn, which is injecting the JS, but that's no use, since the client side assets are served with httpd. I thought the new relic httpd plugin would take care of that, but clearly not.

Comment 4

[Mauro Doglio [:mdoglio]]

:edmorley I guess that was the suggested activation method when we set it up a few months ago. I'm not aware of any difference between those two.
As far as I understand this browser agent is a js client for new relic; the wsgi server doesn't know anything about the angular app, that is served directly by apache. You are only seeing the script injected in the 404 responses because those are the only ones with Content-Type text/html. A 200 api response has Content-Type application/json so the js snippet injection doesn't apply.
The best method to instrument the ui is probably https://docs.newrelic.com/docs/browser/new-relic-browser/installation-configuration/adding-apps-new-relic-browser#copy-paste-app

Comment 5

[Ed Morley [:emorley]]

Yeah agree I think that's the only/best way forwards - I was hoping the new relic httpd plugin would also be able to inject, but obviously not. We could stick the copy/paste snippet version in local.conf.js (since we'll need different versions for stage vs prod).

Comment 6

[Ed Morley [:emorley]]

This is actually already working for /embed/ since it's generated by the service - eg view source on:
https://treeherder.mozilla.org/embed/resultset-status/mozilla-central/cf1060d8ce9f/

However we'll need to manually add the snippet for our UI.

Comment 8

[Ed Morley [:emorley]]

So I was looking into this now that we're using WhiteNoise on stage/prod/Heroku.
It appears that the New Relic Python agent is unable to inject the JS when we serve the gzipped static content.
I'm not sure if there is any way around that (given that iirc it uses sendfile to serve the file on disk directly, for performance reasons).

However the simple solution would just to be to exclude the html files from the gzip deploy step. They are very small in size anyway (eg 2-7KB for the main files, and 29KB for help), so it makes little difference (compared to the 140-180KB css and 650-800KB js asserts in dist/).

Comment 9

[Ed Morley [:emorley]]

Wontfix for now, given:
https://groups.google.com/forum/#!topic/mozilla.dev.webdev/ragGTzhyY2w

Will file a bug to disable it explicitly in newrelic.ini (it's not enabled at the moment, but only because the web UI option is preffed off).