1548958 - MemoryInfoDumper fails to open fifo on Android

Status: RESOLVED WORKSFORME

(Core :: XPCOM, defect)

Description

[Eric Rahm [:erahm]]

On android, if we enable the memory info dumper fifo that we used in the b2g days via the following prefs we fail to actually open the fifo:

pref("memory_info_dumper.watch_fifo.enabled", true);
pref("memory_info_dumper.watch_fifo.directory", "/data/local");

from logcat:

I/Gecko:DumpUtils( 2820): FifoWatcher::OpenFifo unlink failed; errno=2.  Continuing despite error.
I/Gecko:DumpUtils( 2820): FifoWatcher::OpenFifo mkfifo failed; errno=13
I/Gecko:DumpUtils( 2820): FdWatcher: OpenFd failed.

It's possible we're supposed to write to a different directory now and that's all that's going on.

Comment 1

[Eric Rahm [:erahm]]

I've gotten a little further. If I don't specify a directory we end up using the app temp dir for our fifo, ie:

/data/user/0/org.mozilla.fennec_erahm/app_tmpdir/

But then when we want to create a temp file we do some slightly weird logic and try to use the Downloads directory, ie:

/storage/emulated/0/Download/memory-reports

But at least on my emulator we can't write to that directory and things fall apart. If I remove the downloads directory logic things work okay:

I/Gecko:DumpUtils( 4291): FifoWatcher::OpenFifo working with path: /data/user/0/org.mozilla.fennec_erahm/app_tmpdir/debug_info_trigger
I/Gecko:DumpUtils( 4291): FifoWatcher(command:memory report) dispatching memory report runnable.
I/Gecko:DumpUtils( 4291): FifoWatcher closing and re-opening fifo.
I/Gecko:DumpUtils( 4291): FifoWatcher::OpenFifo working with path: /data/user/0/org.mozilla.fennec_erahm/app_tmpdir/debug_info_trigger
I/Gecko:DumpUtils( 4291): nsDumpUtils::OpenTempFile dirPath = /data/user/0/org.mozilla.fennec_erahm/app_tmpdir/memory-reports
I/Gecko:DumpUtils( 4291): nsDumpUtils::OpenTempFile using /data/user/0/org.mozilla.fennec_erahm/app_tmpdir/memory-reports/incomplete-unified-memory-report-1556931108-4291.json.gz

...

$ adb shell ls -l /data/user/0/org.mozilla.fennec_erahm/app_tmpdir/memory-reports/
total 56
-rw-rw-rw- 1 u0_a62 u0_a62 26697 2019-05-04 01:51 unified-memory-report-1556931108-4291.json.gz

Comment 2

[Eric Rahm [:erahm]]

kats, this logic seems to have been added back in bug 845342. Any thoughts on just giving up on writing to the download dir and if not do you know who might be more up to speed on writing files on android?

Comment 3

[Kartikaya Gupta (email:[e-mail])]

As far as I'm aware nobody is using this feature on Android at the moment so if you want to change the directory it's writing to I doubt you'll run into any objections. That being said I'm a little surprised you're not able to write to the Downloads dir; that dir comes from the android APIs here and it should be writable.

Passing needinfo to snorp who's probably more up to speed on all this and might have other thoughts.

Comment 4

[James Willcox (:snorp) ([e-mail]) (he/him)]

I'm sure this is just due to lacking the external files permission in whatever app you're using. Make sure you have the "Storage" permission in the app's permission settings. We could have Gecko prompt for this, but appears we don't actually have a good way to do this currently if it's not tied to a web permission.

Comment 5

[Eric Rahm [:erahm]]

(In reply to James Willcox (:snorp) ([e-mail]) (he/him) from comment #4)

I'm sure this is just due to lacking the external files permission in whatever app you're using. Make sure you have the "Storage" permission in the app's permission settings. We could have Gecko prompt for this, but appears we don't actually have a good way to do this currently if it's not tied to a web permission.

I've confirmed that if I manually download a file via fennec I get prompted for storage permissions, and then a subsequent attempt to write memory reports to the downloads dir works (even after installing a new build). I'm not sure this is going to really work for my purposes in automated testing unless our test frameworks take care of the permissions.

Comment 6

[Eric Rahm [:erahm]]

Jesup can you try something like the following (modify it to whatever not-fennec thing you might be working on)?

1. Package fennec with the following change and install it:

diff --git a/mobile/android/app/mobile.js b/mobile/android/app/mobile.js
--- a/mobile/android/app/mobile.js
+++ b/mobile/android/app/mobile.js
@@ -855,8 +855,12 @@ pref("media.navigator.permission.device"
 
 // Allow system add-on updates
 pref("extensions.systemAddon.update.url", "https://aus5.mozilla.org/update/3/SystemAddons/%VERSION%/%BUILD_ID%/%BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/update.xml");
 
 // E10s stuff. We don't support 'file' or 'priveleged' process types.
 pref("browser.tabs.remote.separateFileUriProcess", false);
 pref("browser.tabs.remote.allowLinkedWebInFileUriProcess", true);
 pref("browser.tabs.remote.separatePrivilegedContentProcess", false);
+
+// Allow nsMemoryInfoDumper to create a fifo in the temp directory.  We use
+// this fifo to trigger about:memory dumps, among other things.
+pref("memory_info_dumper.watch_fifo.enabled", true);

2. Run fennec and download a random file, approve the permissions prompt you should only need to do this once
3. Run the following to generate a memory report:

$ adb shell "echo -n 'memory report' > /data/data/org.mozilla.fennec_$(whoami)/app_tmpdir/debug_info_trigger"

4. Determine where your Download directory is and check for a memory report, on my emulator this works:

$ adb shell ls -l /storage/emulated/0/Download/memory-reports
total 80
-rw-rw---- 1 root sdcard_rw 19307 2019-05-06 16:53 unified-memory-report-1557157998-7267.json.gz

5. Now you can pull that file and then remove it

$ adb pull /storage/emulated/0/Download/memory-reports/unified-memory-report-1557157998-7267.json.gz
$ adb shell rm /storage/emulated/0/Download/memory-reports/unified-memory-report-1557157998-7267.json.gz

Comment 7

[James Willcox (:snorp) ([e-mail]) (he/him)]

(In reply to Eric Rahm [:erahm] from comment #5)

(In reply to James Willcox (:snorp) ([e-mail]) (he/him) from comment #4)

I'm sure this is just due to lacking the external files permission in whatever app you're using. Make sure you have the "Storage" permission in the app's permission settings. We could have Gecko prompt for this, but appears we don't actually have a good way to do this currently if it's not tied to a web permission.

I've confirmed that if I manually download a file via fennec I get prompted for storage permissions, and then a subsequent attempt to write memory reports to the downloads dir works (even after installing a new build). I'm not sure this is going to really work for my purposes in automated testing unless our test frameworks take care of the permissions.

For automation we can (and do[0]) grant the permissions ahead of time, so I'm not sure what's going wrong there.

[0] https://searchfox.org/mozilla-central/source/testing/mozbase/mozrunner/mozrunner/devices/android_device.py#418

Comment 8

[Eric Rahm [:erahm]]

(In reply to James Willcox (:snorp) ([e-mail]) (he/him) from comment #7)

For automation we can (and do[0]) grant the permissions ahead of time, so I'm not sure what's going wrong there.

That's perfect! Nothing is going wrong automation-wise (yet), I was just trying to confirm things manually and stumbled on this issue.

Comment 9

[Randell Jesup [:jesup] (needinfo me)]

But it is important to understand/deal-with for local use.
Checking FireTV today

Comment 10

[Randell Jesup [:jesup] (needinfo me)]

I think we can mark this as resolved given comment 6 and comment 8