1649925 - Fxr + C#: x86 Glean doesn't upload pings to debug viewer

Status: RESOLVED FIXED

(Data Platform and Tools :: Glean: SDK, defect, P1)

Description

[Daosheng Mu[:daoshengmu]]

I have tried it couple of times. But, x86 glean_ffi always doesn't help show pings in debug viewer when using x86 arch build with GleanUnity

Comment 1

[Alessio Placitelli [:Dexter]]

Did you clean or make sure to use a different glean data directory in the configuration? The x86 database is not compatible with the x64 database.

Comment 2

[Daosheng Mu[:daoshengmu]]

I tried to create a new folder for x86 build. It is only able to reproduce when building GleanUnity as a x86 application instead of running in Editor. Because running in Editor is always treated as a x86_64 application.

Here is its logs

Begin Glean test.
 
(Filename: C:\buildslave\unity\build\Runtime/Export/Debug/Debug.bindings.h Line: 35)

[09:27:39 INF] glean/Glean: Registering pings
 
(Filename: C:\buildslave\unity\build\Runtime/Export/Debug/Debug.bindings.h Line: 35)

[09:27:39 INF] glean/Glean: Registering pings
[09:27:39 INF] glean/Dispatchers: Task queued for execution and delayed until flushed
 
(Filename: C:\buildslave\unity\build\Runtime/Export/Debug/Debug.bindings.h Line: 35)

 
(Filename: C:\buildslave\unity\build\Runtime/Export/Debug/Debug.bindings.h Line: 35)

 
(Filename: C:\buildslave\unity\build\Runtime/Export/Debug/Debug.bindings.h Line: 35)

[09:27:39 INF] glean/Dispatchers: Task queued for execution and delayed until flushed
 
(Filename: C:\buildslave\unity\build\Runtime/Export/Debug/Debug.bindings.h Line: 35)

 
(Filename: C:\buildslave\unity\build\Runtime/Export/Debug/Debug.bindings.h Line: 35)

End of Glean test.
 
(Filename: C:\buildslave\unity\build\Runtime/Export/Debug/Debug.bindings.h Line: 35)

And it doesn't generate any pending_pings in the data folder.

Comment 3

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/glean/pull/1043

Comment 4

[Alessio Placitelli [:Dexter]]

Doasheng found the culprit of this. The embedding application was calling Application.Quit which was rightfully killing all the threads to prepare for exiting the application.

In addition to that, Unity was feeding an empty string to Glean using Application.identifier. Glean was silently accepting the empty string and failing to upload the pings. This last problem is fixed in the latest Glean master by the PR linked in this bug.

Unfortunately, to fix the problem of ping upload being aborted when quitting the application, we have two paths:

1. Wait a reasonable time before quitting the product, as a temporary workaround (what's currently being done in FxR PC).
2. Allow the Glean SDK to upload pings off the main process, when the application is shutting down. This is currently being discussed in bug 1651340, and it's a bit involved, so it won't probably happen in the short term.

I'm closing this a FIXED because the app id bug was indeed fixed. All the discussion with respect off-the-main-process upload can happen as part of bug 1651340.

Comment 6

[Daosheng Mu[:daoshengmu]]

(In reply to Alessio Placitelli [:Dexter] from comment #4)

Doasheng found the culprit of this. The embedding application was calling Application.Quit which was rightfully killing all the threads to prepare for exiting the application.

In addition to that, Unity was feeding an empty string to Glean using Application.identifier. Glean was silently accepting the empty string and failing to upload the pings. This last problem is fixed in the latest Glean master by the PR linked in this bug.

Unfortunately, to fix the problem of ping upload being aborted when quitting the application, we have two paths:

1. Wait a reasonable time before quitting the product, as a temporary workaround (what's currently being done in FxR PC).
2. Allow the Glean SDK to upload pings off the main process, when the application is shutting down. This is currently being discussed in bug 1651340, and it's a bit involved, so it won't probably happen in the short term.

I'm closing this a FIXED because the app id bug was indeed fixed. All the discussion with respect off-the-main-process upload can happen as part of bug 1651340.

It might not be related to x86 Glean can't upload pings. In GleanUnity, we actually hard-coded the app id. So, I guess the x86 issue would still be around.

Comment 7

[Alessio Placitelli [:Dexter]]

(In reply to Daosheng Mu[:daoshengmu] from comment #6)

(In reply to Alessio Placitelli [:Dexter] from comment #4)
It might not be related to x86 Glean can't upload pings. In GleanUnity, we actually hard-coded the app id. So, I guess the x86 issue would still be around.

Ok, let me reopen this for further investigation!

Comment 8

[Daosheng Mu[:daoshengmu]]

Checked. It is still unable to upload pings in x86 standalone application. The log is the same with Comment 2.

Comment 9

[Jan-Erik Rediger [:janerik]]

Can you set the environt variable RUST_LOG=debug, then run the programm? This should give additional logs from glean_core that might help us here.

Comment 10

[Alessio Placitelli [:Dexter]]

Ok looks like I'm able to reproduce this in the glean csharp sample app:

[[13:59:4513:59:45  INFINF] ] glean/Dispatchersglean/Glean: : Task queued for execution and delayed until flushedRegistering pings

[2020-07-09T11:59:45Z INFO  glean_core] Creating new Glean v31.2.3
[[2020-07-09T11:59:45Z DEBUG glean_core::database] Database path: "D:\\Mozilla\\glean.rs\\samples\\csharp\\bin\\x86\\Debug\\netcoreapp3.1\\glean_data\\db"
13:59:45 INF] glean/Dispatchers: Task queued for execution and delayed until flushed
Press any key to exit the sample...
[2020-07-09T11:59:45Z INFO  glean_core::database] Database initialized
[2020-07-09T11:59:45Z INFO  glean_core::upload::directory] Processing persisted pings.
[2020-07-09T11:59:45Z INFO  glean_ffi] Glean initialized
[2020-07-09T11:59:45Z WARN  ffi_support::handle_map] Illegal handle! f9e8ef1300000002
thread '<unnamed>' panicked at 'assertion failed: error.get_code().is_success()', glean-core\ffi\src\handlemap_ext.rs:125:9
stack backtrace:
   0: core::fmt::write
             at /rustc/b8cedc00407a4c56a3bda1ed605c6fc166655447\/src\libcore\fmt\mod.rs:1052
   1: std::io::Write::write_fmt<std::sys::windows::stdio::Stderr>
             at /rustc/b8cedc00407a4c56a3bda1ed605c6fc166655447\/src\libstd\io\mod.rs:1426
   2: std::sys_common::backtrace::_print
             at /rustc/b8cedc00407a4c56a3bda1ed605c6fc166655447\/src\libstd\sys_common\backtrace.rs:62
   3: std::sys_common::backtrace::print
             at /rustc/b8cedc00407a4c56a3bda1ed605c6fc166655447\/src\libstd\sys_common\backtrace.rs:49
   4: std::panicking::default_hook::{{closure}}
             at /rustc/b8cedc00407a4c56a3bda1ed605c6fc166655447\/src\libstd\panicking.rs:204
   5: std::panicking::default_hook
             at /rustc/b8cedc00407a4c56a3bda1ed605c6fc166655447\/src\libstd\panicking.rs:224
   6: std::panicking::rust_panic_with_hook
             at /rustc/b8cedc00407a4c56a3bda1ed605c6fc166655447\/src\libstd\panicking.rs:472
   7: std::panicking::begin_panic<str*>
             at /rustc/b8cedc00407a4c56a3bda1ed605c6fc166655447\src\libstd\panicking.rs:399
   8: glean_ffi::handlemap_ext::{{impl}}::call_infallible<glean_core::metrics::ping::PingType,(),closure-0>
             at D:\Mozilla\glean.rs\glean-core\ffi\src\handlemap_ext.rs:125
   9: glean_ffi::ping_type::glean_register_ping_type
             at D:\Mozilla\glean.rs\glean-core\ffi\src\ping_type.rs:45

Comment 11

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/glean/pull/1050

Comment 12

[Daosheng Mu[:daoshengmu]]

I checked that GleanUnity x86 is able to upload pings. However, when running glean-core/csharp.sln with x86 platform, the tests can't pass and get a few failures. The logs are as below:

Message: 
    System.BadImageFormatException : An attempt was made to load a program with an incorrect format. (0x8007000B)
  Stack Trace: 
    LibGleanFFI.glean_enable_logging()
    Glean.ctor() line 68
    <.cctor>b__27_0() line 27
    Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
    --- End of stack trace from previous location where exception was thrown ---
    Lazy`1.CreateValue()
    Lazy`1.get_Value()
    Glean.get_GleanInstance() line 28
    GleanTests.ctor() line 20

I can help investigate it tomorrow.

Comment 13

[Alessio Placitelli [:Dexter]]

(In reply to Daosheng Mu[:daoshengmu] from comment #12)

I can help investigate it tomorrow.

Thanks for reporting this. This is due to how the project file is configured: it currently needs to copy the "Glean.dll" file to the tests directory, but it copies the wrong architecture version. I'm fixing this in bug 1648030