1473571 - tree status info shown with ? instead of open/closed circle

Status: RESOLVED FIXED

(Tree Management :: Treeherder: Frontend, defect, P2)

Description

[Andreea Pavel [:apavel]]

I sometimes see the tree status info in treeherder shown with ? instead of a red or green circle. I think this happens when you leave a page.. idle for couple of minutes (not sure how long.

Please see https://irccloud.mozilla.com/file/asRfLcPj/image.png 

Also, the in the inspector there are the following errors:

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://treestatus.mozilla-releng.net/trees/try. (Reason: CORS request did not succeed).
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://treestatus.mozilla-releng.net/trees/mozilla-inbound. (Reason: CORS request did not succeed).
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://treestatus.mozilla-releng.net/trees/autoland. (Reason: CORS request did not succeed).

Comment 1

[Ed Morley [:emorley]]

Hi! I can't reproduce.

Could you paste the full HTTP request details for the request that then triggers those CORS errors? (from the networking tab)
(Include full URL, request headers, response HTTP code, response headers and response body.)

Comment 2

[Andreea Pavel [:apavel]]

There is also: https://irccloud.mozilla.com/file/gm4iYubj/image.png

Yes:

URL: https://treeherder.mozilla.org/api/project/try/jobs/?result_set_id__in=353645&count=2000&last_modified__gt=2018-07-05T13:55:06.968&return_type=list

Status code: 200

Response headers (470 B)	
Access-Control-Allow-Origin: *
Allow:	GET, POST, HEAD, OPTIONS
Connection: keep-alive
Content-Encoding:	gzip
Content-Length:909
Content-Type: application/json
Date: Thu, 05 Jul 2018 13:56:08 GMT
Server: gunicorn/19.8.1
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Vary: Accept, Cookie, Accept-Encoding
Via: 1.1 vegur
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 1; mode=block


Request headers (998 B)	
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.5
Connection: keep-alive
Cookie: optimizelyEndUserId=oeu1499850…gid=GA1.2.61053086.1530767109
Host: treeherder.mozilla.org
origin: https://treeherder.mozilla.org
Referer: https://treeherder.mozilla.org/
User-Agent: Mozilla/5.0 (Windows NT 10.0; …) Gecko/20100101 Firefox/61.0

Response body:

meta	{…}
repository	try
offset	0
count	2000
results	[…]
0	[…]
0	-
1	-
2	windows10-64
3	201
4	taskcluster
5	1530798901
6	1
7	186605361
8	
9	445
10	Web platform tests with e10s
11	W-e10s
12	2c34afe9-9830-4a44-a40d-5a00f58b7253/0
13	
14	62232
15	test-windows10-64/debug-web-platform-tests-e10s-8
16	wpt8
17	2018-07-05T13:55:06.968468
18	i-029a4ec0169488901
19	-
20	-
21	32faaecac742100f7753f0c1d0aa0add01b4046b
22	windows10-64
23	353645
24	scheduled
25	42557bbc81370b6da636bd3af5241dc128c9bb62
26	success
27	353645
28	42557bbc81370b6da636bd3af5241dc128c9bb62
29	1530795212
30	completed
31	1530790036
32	1
33	[e-mail]
34	debug
1	[…]
0	-

This is for try. do you need the info for the 2 other trees too?

Comment 3

[Ed Morley [:emorley]]

That's a request to Treeherder's API. The tree status info comes from the request to the domain shown in comment 0. Just the one repo request will be fine :-)

Comment 4

[Andreea Pavel [:apavel]]

(In reply to Ed Morley [:emorley] from comment #3)
> That's a request to Treeherder's API. The tree status info comes from the
> request to the domain shown in comment 0. Just the one repo request will be
> fine :-)

That is the only info from networking i have, there is nothing else :(

Will look with Aryx and get back to you.

Comment 5

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

We can't provide that data because TH initially shows the status fine but later that tab (or plural) stops to show the treestatus. And the web console doesn't show anymore any requests before the CORS messages for Andreea. For my stale beta tab, it doesn't even list those. And it's in Nightly which doesn't have the GCLI anymore, so |security csp| also isn't available.

For Andreea, her tabs' treestatus became stale again an hour ago when she had a meeting and didn't interact with treeherder. Here there are TH which aren't used for hours but are not affected by the issue (but my stale mozilla-beta tab might be the one which was unused for the longest time).

Comment 6

[Ed Morley [:emorley]]

Yeah I can see how it's hard to get hold of those.

Unfortunately without additional information it's hard to debug this any further. The API being used is not under our control, and requests are made client-side to it. One possibility is that N% of the time treestatus.mozilla-releng.net is returning a non-HTTP 200 or other invalid response, which causes this stuck stage (Treeherder's UI should be made to handle this though, if it doesn't already).

Comment 7

[Ed Morley [:emorley]]

s/stage/state/

Comment 8

[Cameron Dawson [:camd]]

It's possible this is an issue with the model being non-angular, but that navbar still being in angular.  The model may be "losing state" once it's gathered the status once.  Hard to say for sure right now, so this is just a SWAG.  I am currently working to convert that navbar to React.  This may fix the issue.  While working on that PR, I will keep an eye on this to see if I can reproduce or determine the cause.

Comment 9

[Cameron Dawson [:camd]]

This may well be related to Bug 1473777.  I have a PR in for that, so after it lands, let's see if it helps with this problem.

Comment 10

[Ed Morley [:emorley]]

Now that bug 1473777 is fixed, have you seen this since?

Comment 11

[Andreea Pavel [:apavel]]

(In reply to Ed Morley [:emorley] from comment #10)
> Now that bug 1473777 is fixed, have you seen this since?

Hi, no, i haven't. I'll keep an eye on this tonight.

Comment 12

[Cameron Dawson [:camd]]

It's been a couple days, so I'll go ahead close this as fixed for now.  Please reopen if you see this again.