Closed Bug 1626581 Opened 5 years ago Closed 4 years ago

Use mozfile.remove in test_refresh_firefox.py and add the "\\?\" prefix for file path on Windows for mozfile.remove

Categories

(Core :: Storage: Quota Manager, task, P3)

task

Tracking

()

RESOLVED FIXED
mozilla80
Tracking Status
firefox80 --- fixed

People

(Reporter: tt, Assigned: tt)

References

(Blocks 1 open bug)

Details

Attachments

(1 file, 2 obsolete files)

No description provided.

try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=11dac849d269ad2864aa0d7b54116ce408028352&selectedJob=295731818

I think I am close to do the job but I'm not familir with Python and the code around. Therefore, move patches here and fix remaining issues for safty.

I printed more log: https://treeherder.mozilla.org/#/jobs?repo=try&revision=2a737f353a1ec37c87ec774eee4f4b476378a31f&selectedJob=295751969
I quote part of them below:

[task 2020-04-01T13:05:13.851Z] 13:05:13     INFO -  [TT]3 \\?\Z:\task_1585742307\build\application
[task 2020-04-01T13:05:13.851Z] 13:05:13    ERROR - Errors received:
[task 2020-04-01T13:05:13.851Z] 13:05:13     INFO - Reading from file tmpfile_stderr
[task 2020-04-01T13:05:13.851Z] 13:05:13    ERROR -  Traceback (most recent call last):
[task 2020-04-01T13:05:13.851Z] 13:05:13    ERROR -    File "Z:\task_1585742307\build\venv\Scripts\mozuninstall-script.py", line 9, in <module>
[task 2020-04-01T13:05:13.851Z] 13:05:13    ERROR -      load_entry_point('mozInstall==2.0.0', 'console_scripts', 'mozuninstall')()
[task 2020-04-01T13:05:13.851Z] 13:05:13    ERROR -    File "z:\task_1585742307\build\venv\lib\site-packages\mozinstall\mozinstall.py", line 374, in uninstall_cli
[task 2020-04-01T13:05:13.852Z] 13:05:13    ERROR -      uninstall(argv[0])
[task 2020-04-01T13:05:13.852Z] 13:05:13    ERROR -    File "z:\task_1585742307\build\venv\lib\site-packages\mozinstall\mozinstall.py", line 252, in uninstall
[task 2020-04-01T13:05:13.852Z] 13:05:13    ERROR -      mozfile.remove(install_folder)
[task 2020-04-01T13:05:13.852Z] 13:05:13    ERROR -    File "z:\task_1585742307\build\venv\lib\site-packages\mozfile\mozfile.py", line 250, in remove
[task 2020-04-01T13:05:13.852Z] 13:05:13    ERROR -      _call_with_windows_retry(shutil.rmtree, (tmp_path,))
[task 2020-04-01T13:05:13.852Z] 13:05:13    ERROR -    File "z:\task_1585742307\build\venv\lib\site-packages\mozfile\mozfile.py", line 191, in _call_with_windows_retry
[task 2020-04-01T13:05:13.852Z] 13:05:13    ERROR -      _call_windows_retry(*args, **kwargs)
[task 2020-04-01T13:05:13.852Z] 13:05:13    ERROR -    File "z:\task_1585742307\build\venv\lib\site-packages\mozfile\mozfile.py", line 149, in _call_windows_retry
[task 2020-04-01T13:05:13.852Z] 13:05:13    ERROR -      func(*args)
[task 2020-04-01T13:05:13.852Z] 13:05:13    ERROR -    File "c:\mozilla-build\python\Lib\shutil.py", line 239, in rmtree
[task 2020-04-01T13:05:13.852Z] 13:05:13    ERROR -      onerror(os.listdir, path, sys.exc_info())
[task 2020-04-01T13:05:13.852Z] 13:05:13    ERROR -    File "c:\mozilla-build\python\Lib\shutil.py", line 237, in rmtree
[task 2020-04-01T13:05:13.852Z] 13:05:13    ERROR -      names = os.listdir(path)
[task 2020-04-01T13:05:13.852Z] 13:05:13    ERROR -  WindowsError: [Error 123] The filename, directory name, or volume label syntax is incorrect: '\\\\?\\Z:\\task_1585742307\\build\\application/*.*'
[task 2020-04-01T13:05:13.852Z] 13:05:13     INFO - Using _rmtree_windows ...

I will need to find where /* was appended to the path.

Hey Tom, will you be able to continue on this bug?

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+2] from comment #5)

Hey Tom, will you be able to continue on this bug?

Hi Henrik,

It's still in my queue, but I might not be able to deal with that really soon. If you want to fix that soon, please feel free to take it.

The prefix was disabled in bug 1626846 and we are working on turning it on and then I will fix the follow-up bugs for bug 1536796 one by one (including this bug). And, currently, I am in bug 1626514.

Priority: -- → P2

Thanks for your feedback. No, it's not pressing. It can wait for a bit.

Priority: P2 → P3
Attachment #9137416 - Attachment is obsolete: true
Attachment #9137417 - Attachment is obsolete: true
Blocks: 1650374
Pushed by ttung@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/64456c63779a Use mozfile.remove in test_refresh_firefox.py and add the "\\?\" prefix for file path on Windows for mozfile.remove; r=whimboo,gbrown

Sorry, I will check xpcshell tests (I only checked the marrionette tests https://treeherder.mozilla.org/#/jobs?repo=try&revision=fb018398a2caf95a38f52dac0b71b70360681be1)

Flags: needinfo?(ttung)

Hmm, I saw file paths like \\\\?\\c:\\users\\task_1593784908\\appdata\\local\\temp\\tmpsa4vpz/*.* after prepending the \\\\?\\.
The file path in this case should be \\\\?\\c:\\users\\task_1593784908\\appdata\\local\\temp\\tmpsa4vpz\\*.*.

I will check if the file path is like c:\\users\\task_1593784908\\appdata\\local\\temp\\tmpsa4vpz/*.* before the prepending or not first.

I found https://stackoverflow.com/questions/29557760/long-paths-in-python-on-windows.

It seems that it's related to shutil.rmtree.
If the file path for a directory doesn't end with \\ for non-unicode version, it will append /*.*. For unicode version, append \*.*.

Status: NEW → ASSIGNED
Pushed by ttung@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/caa6d07ab019 Use mozfile.remove in test_refresh_firefox.py and add the "\\?\" prefix for file path on Windows for mozfile.remove; r=whimboo,gbrown
Status: ASSIGNED → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla80
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: