1650190 - Fxr + C#: ThreadAbortException when doing HttpClient send.

Status: RESOLVED DUPLICATE of bug 1649925

(Data Platform and Tools :: Glean: SDK, defect, P1)

Description

[Daosheng Mu[:daoshengmu]]

Setting up 4 worker threads for Enlighten.
  Thread -> id: 64c4 -> priority: 1 
  Thread -> id: 2af8 -> priority: 1 
  Thread -> id: 4e88 -> priority: 1 
  Thread -> id: 1a98 -> priority: 1 
[12:00:57 ERR] glean/Dispatchers: Exception thrown by task and swallowed.
System.Threading.ThreadAbortException
  at System.Net.Http.HttpClient+<SendAsyncWorker>d__47.MoveNext () [0x001b0] in <7ebf3529ba0e4558a5fa1bc982aa8605>:0 
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[TResult].Start[TStateMachine] (TStateMachine& stateMachine) [0x0002c] in <437ba245d8404784b9fbab9b439ac908>:0 
  at System.Net.Http.HttpClient.SendAsyncWorker (System.Net.Http.HttpRequestMessage request, System.Net.Http.HttpCompletionOption completionOption, System.Threading.CancellationToken cancellationToken) [0x0003b] in <7ebf3529ba0e4558a5fa1bc982aa8605>:0 
  at System.Net.Http.HttpClient.SendAsync (System.Net.Http.HttpRequestMessage request, System.Net.Http.HttpCompletionOption completionOption, System.Threading.CancellationToken cancellationToken) [0x000c9] in <7ebf3529ba0e4558a5fa1bc982aa8605>:0 
  at System.Net.Http.HttpClient.SendAsync (System.Net.Http.HttpRequestMessage request) [0x00008] in <7ebf3529ba0e4558a5fa1bc982aa8605>:0 
  at Mozilla.Glean.Net.HttpClientUploader.Upload (System.String url, System.Byte[] data, System.ValueTuple`2[System.String,System.String][] headers) [0x00090] in <5fbc21e475034c498b50d61896774b2c>:0 
  at Mozilla.Glean.Net.BaseUploader.Upload (System.String url, System.Byte[] data, System.ValueTuple`2[System.String,System.String][] headers, Mozilla.Glean.Configuration config) [0x0003a] in <5fbc21e475034c498b50d61896774b2c>:0 
  at Mozilla.Glean.Net.BaseUploader.TriggerUpload (Mozilla.Glean.Configuration config) [0x000b4] in <5fbc21e475034c498b50d61896774b2c>:0 
  at Mozilla.Glean.Glean.SubmitPingByNameSync (System.String name, System.String reason) [0x00054] in <5fbc21e475034c498b50d61896774b2c>:0 
  at Mozilla.Glean.Glean+<>c__DisplayClass22_0.<SubmitPingByName>b__0 () [0x00001] in <5fbc21e475034c498b50d61896774b2c>:0 
  at Mozilla.Glean.Dispatchers+<>c__DisplayClass16_0.<LaunchAPI>b__0 () [0x00002] in <5fbc21e475034c498b50d61896774b2c>:0 

This is only happened in the application that is built from FxR PC Unity. If I run it in Unity editor, this will be good. GleanUnity doesn't have this problem though.

Comment 1

[Alessio Placitelli [:Dexter]]

Hey Daosheng,

I investigated this a bit and was unable to reproduce. Would you kindly give me a few more details?

1. Can you point me to the FxR PC Unity repo that's triggering this?
2. Does this happen right after startup?
3. Does this always happen?
4. Is this able to upload pings at all?

Comment 2

[Daosheng Mu[:daoshengmu]]

It's always happened and never upload pings successfully. Remember to use the latest build of Glean_ffi and Glean. It sometimes can't be reproduced in debugger, but the logs are always the same. If using the debugger, the debugger will detach the process at some places do not make sense.

[17:56:44 ERR] glean/Dispatchers: Exception thrown by task and swallowed.
System.Threading.ThreadAbortException
  at System.Net.Http.HttpClient+<SendAsyncWorker>d__47.MoveNext () [0x001b0] in <7ebf3529ba0e4558a5fa1bc982aa8605>:0 
  at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[TResult].Start[TStateMachine] (TStateMachine& stateMachine) [0x0002c] in <437ba245d8404784b9fbab9b439ac908>:0 
  at System.Net.Http.HttpClient.SendAsyncWorker (System.Net.Http.HttpRequestMessage request, System.Net.Http.HttpCompletionOption completionOption, System.Threading.CancellationToken cancellationToken) [0x0003b] in <7ebf3529ba0e4558a5fa1bc982aa8605>:0 
  at System.Net.Http.HttpClient.SendAsync (System.Net.Http.HttpRequestMessage request, System.Net.Http.HttpCompletionOption completionOption, System.Threading.CancellationToken cancellationToken) [0x000c9] in <7ebf3529ba0e4558a5fa1bc982aa8605>:0 
  at System.Net.Http.HttpClient.SendAsync (System.Net.Http.HttpRequestMessage request) [0x00008] in <7ebf3529ba0e4558a5fa1bc982aa8605>:0 
  at Mozilla.Glean.Net.HttpClientUploader.Upload (System.String url, System.Byte[] data, System.ValueTuple`2[System.String,System.String][] headers) [0x000a2] in C:\Projects\mozilla\FirefoxRealityPC\Source\FirefoxRealityUnity\Assets\Scripts\glean\Net\HttpClientUploader.cs:71 
  at Mozilla.Glean.Net.BaseUploader.Upload (System.String url, System.Byte[] data, System.ValueTuple`2[System.String,System.String][] headers, Mozilla.Glean.Configuration config) [0x00027] in C:\Projects\mozilla\FirefoxRealityPC\Source\FirefoxRealityUnity\Assets\Scripts\glean\Net\BaseUploader.cs:65 
  at Mozilla.Glean.Net.BaseUploader.TriggerUpload (Mozilla.Glean.Configuration config) [0x000b0] in C:\Projects\mozilla\FirefoxRealityPC\Source\FirefoxRealityUnity\Assets\Scripts\glean\Net\BaseUploader.cs:127 
  at Mozilla.Glean.Glean.SubmitPingByNameSync (System.String name, System.String reason) [0x00054] in C:\Projects\mozilla\FirefoxRealityPC\Source\FirefoxRealityUnity\Assets\Scripts\glean\Glean.cs:507 
  at Mozilla.Glean.Glean+<>c__DisplayClass23_0.<SubmitPingByName>b__0 () [0x00001] in C:\Projects\mozilla\FirefoxRealityPC\Source\FirefoxRealityUnity\Assets\Scripts\glean\Glean.cs:470 
  at Mozilla.Glean.Dispatchers+<>c__DisplayClass16_0.<LaunchAPI>b__0 () [0x00002] in C:\Projects\mozilla\FirefoxRealityPC\Source\FirefoxRealityUnity\Assets\Scripts\glean\Dispatchers.cs:103 
##utp:{"type":"MemoryLeaks","version":2,"phase":"Immediate","time":1594083404653,"processId":26148,"allocatedMemory":16539,"memoryLabels":[{"Permanent":40},{"NewDelete":472},{"Manager":2448},{"GfxDevice":7236},{"Physics":32},{"Serialization":40},{"String":246},{"PoolAlloc":56},{"GI":296},{"VR":2016},{"Secure":4313},{"Subsystems":-656}]}

I think we have a memory leak somewhere, I will continue to investigate this part. I was thinking that our JSON parser or HTTP upload has a bug.

Comment 3

[Daosheng Mu[:daoshengmu]]

I think it is resolved. There are two things are related to Unity environment.

1. Using Application.identifier can't get a bundled app id from a Windows standalone application. It would be an empty string. In Unity Editor, it is able to get a correct id. Using an empty app id makes our pings can't submit.

2. Unity allows users calling Application.Quit() to quit its application. However, our ping uploader hasn't finished its tasks before the application unloads Glean.dll. So, it will cause the application crash. I am using Thread.Sleep(2000) to sleep the main thread before finishing our upload tasks in Glean threads. The better solution should consider to use off-main process approach to serve Glean. But, now is workable, we can make it lower priority.

Comment 4

[Alessio Placitelli [:Dexter]]

(In reply to Daosheng Mu[:daoshengmu] from comment #3)

I think it is resolved. There are two things are related to Unity environment.

1. Using Application.identifier can't get a bundled app id from a Windows standalone application. It would be an empty string. In Unity Editor, it is able to get a correct id. Using an empty app id makes our pings can't submit.

We just changed Glean in bug 1649925 to make it raise an error if that ever happens again.

2. Unity allows users calling Application.Quit() to quit its application. However, our ping uploader hasn't finished its tasks before the application unloads Glean.dll. So, it will cause the application crash. I am using Thread.Sleep(2000) to sleep the main thread before finishing our upload tasks in Glean threads. The better solution should consider to use off-main process approach to serve Glean. But, now is workable, we can make it lower priority.

Yes, I believe for the current plans we should stick to delaying killing the process. If you want to make this "prettier", you could hide the current Window, then wait 2-4 seconds, and then quit. This way the window will not clutter the desktop anymore.

A bigger discussion about (2) and off-the-main-process upload is happening in bug 1651340. Also see bug 1649925 comment 4 for more details.