1064154 - "Invalid cross-device link" in metadata.py write_changes with web-platform-tests-update

Status: RESOLVED WORKSFORME

(Testing :: web-platform-tests, defect)

Description

[Karl Tomlinson (:karlt)]

Following instructions at 
https://github.com/mozilla/gecko-dev/blob/master/testing/web-platform/README.md#autogenerating-expectation-data

and running the command
  ./mach web-platform-tests-update --no-check-clean wpt.with.mse

with attachment 8485604 [details] [diff] [review] and attachment 8485605 [details] [diff] [review] applied yields

./mach web-platform-tests-update --no-check-clean wpt.with.mse
git rev-parse --show-cdup
fatal: Not a git repository (or any parent up to mount point /mnt/ssd1)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
{'cwd': u'/mnt/ssd1/karl/moz/dev/testing/web-platform/sync'}

git rev-parse --show-cdup
fatal: Not a git repository (or any parent up to mount point /mnt/ssd1)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
{'cwd': u'/mnt/ssd1/karl/moz/dev/testing/web-platform/sync'}

Traceback (most recent call last):
  File "/mnt/ssd1/karl/moz/dev/testing/web-platform/harness/wptrunner/update.py", line 306, in update_metadata
    ignore_existing=ignore_existing)
  File "/mnt/ssd1/karl/moz/dev/testing/web-platform/harness/wptrunner/metadata.py", line 63, in update_expected
    write_changes(metadata_root, expected_map)
  File "/mnt/ssd1/karl/moz/dev/testing/web-platform/harness/wptrunner/metadata.py", line 153, in write_changes
    os.rename(temp_path, metadata_path)
OSError: [Errno 18] Invalid cross-device link
Error running mach:

    ['web-platform-tests-update', '--no-check-clean', 'wpt.with.mse']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

OSError: [Errno 18] Invalid cross-device link

  File "/mnt/ssd1/karl/moz/dev/testing/web-platform/mach_commands.py", line 120, in update_web_platform_tests
    return wpt_updater.run_update(**params)
  File "/mnt/ssd1/karl/moz/dev/testing/web-platform/mach_commands.py", line 78, in run_update
    update.run_update(**kwargs)
  File "/mnt/ssd1/karl/moz/dev/testing/web-platform/harness/wptrunner/update.py", line 368, in run_update
    kwargs["run_log"], kwargs["ignore_existing"])
  File "/mnt/ssd1/karl/moz/dev/testing/web-platform/harness/wptrunner/update.py", line 306, in update_metadata
    ignore_existing=ignore_existing)
  File "/mnt/ssd1/karl/moz/dev/testing/web-platform/harness/wptrunner/metadata.py", line 63, in update_expected
    write_changes(metadata_root, expected_map)
  File "/mnt/ssd1/karl/moz/dev/testing/web-platform/harness/wptrunner/metadata.py", line 153, in write_changes
    os.rename(temp_path, metadata_path)

testing/web-platform/meta/ is gone and there is now testing/web-platform/meta39ef105c-853f-4f14-adc1-0b5f95024119/

Comment 1

[Karl Tomlinson (:karlt)]

Attachment: create tempdir for metadata changes in same parent dir

so that it will usually be on the same filesystem.

This is enough to make things work for me, and more efficient than
shutil.move().

tempfile.mkdtemp() is still making the new directory with permissions that
will likely differ from the old directory.  I don't know how much that matters
or how best to address that - perhaps uuid4() again.

Comment 2

[James Graham [:jgraham]]

Merged this in https://github.com/w3c/wptrunner/pull/25 It will get pulled in from upstream soon.