Closed Bug 1754125 Opened 2 years ago Closed 1 year ago

Address mozilla-mobile git warnings

Categories

(Webtools :: Searchfox, task)

task

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: asuth, Assigned: asuth)

References

Details

I landed changes on Friday in https://github.com/mozsearch/mozsearch-mozilla/pull/158 to address the following problem:

+ pushd /mnt/index-scratch/mozilla-mobile/git
/mnt/index-scratch/mozilla-mobile/git ~
+ git submodule update --remote
From https://github.com/mozilla-mobile/android-components
   1f0e0c8b2e..2d152a477f  main       -> origin/main
Submodule path 'android-components': checked out '2d152a477f3c1ca162304db466a504753c0c9f24'
From https://github.com/mozilla-mobile/firefox-ios
 * [new branch]          lm/#9422-Epic-branch-bottom-searchbar -> origin/lm/#9422-Epic-branch-bottom-searchbar
   a62962279..df6c2db93  update-br-new-xcode-version -> origin/update-br-new-xcode-version
From https://github.com/mozilla-mobile/focus-android
   3887ed02..0d874021  main                    -> origin/main
error: cannot lock ref 'refs/remotes/origin/mergify': 'refs/remotes/origin/mergify/bp/main/pr-5560' exists; cannot create 'refs/remotes/origin/mergify'
 ! [new branch]        mergify                 -> origin/mergify  (unable to update local ref)
 * [new branch]        oana-androidxDepUpgrade -> origin/oana-androidxDepUpgrade
fatal: Unable to fetch in submodule path 'focus-android'
+ git submodule update --remote
error: cannot lock ref 'refs/remotes/origin/mergify': 'refs/remotes/origin/mergify/bp/main/pr-5560' exists; cannot create 'refs/remotes/origin/mergify'
From https://github.com/mozilla-mobile/focus-android
 ! [new branch]        mergify    -> origin/mergify  (unable to update local ref)
fatal: Unable to fetch in submodule path 'focus-android'

This has now resulted in some additional warnings for the most recent UTC22 run:

+ wget -nv https://s3-us-west-2.amazonaws.com/searchfox.repositories/mozilla-mobile.tar
2022-02-07 04:22:18 URL:https://s3-us-west-2.amazonaws.com/searchfox.repositories/mozilla-mobile.tar [2412216320/2412216320] -> "mozilla-mobile.tar" [1]
+ tar xf mozilla-mobile.tar
+ rm mozilla-mobile.tar
+ popd
~
+ date
Mon Feb  7 04:22:21 UTC 2022
+ echo Updating git
Updating git
+ pushd /mnt/index-scratch/mozilla-mobile/git
/mnt/index-scratch/mozilla-mobile/git ~
+ git submodule update --remote
From https://github.com/mozilla-mobile/android-components
   40d136f23a..a78871809b  main       -> origin/main
Auto packing the repository in background for optimum performance.
See "git help gc" for manual housekeeping.
warning: The last gc run reported the following. Please correct the root cause
and remove /mnt/index-scratch/mozilla-mobile/git/.git/modules/android-components/gc.log
Automatic cleanup will not be performed until the file is removed.

warning: There are too many unreachable loose objects; run 'git prune' to remove them.

Submodule path 'android-components': checked out 'a78871809bfc080a60244c64a54427f1917c1dd4'
Auto packing the repository in background for optimum performance.
See "git help gc" for manual housekeeping.
warning: The last gc run reported the following. Please correct the root cause
and remove /mnt/index-scratch/mozilla-mobile/git/.git/modules/application-services/gc.log
Automatic cleanup will not be performed until the file is removed.

warning: There are too many unreachable loose objects; run 'git prune' to remove them.

From https://github.com/mozilla-mobile/fenix
   6473381df..363d0b193  main       -> origin/main
Submodule path 'fenix': checked out '363d0b1937063ec1ebbba369993d6684f805282c'
Auto packing the repository in background for optimum performance.
See "git help gc" for manual housekeeping.
warning: The last gc run reported the following. Please correct the root cause
and remove /mnt/index-scratch/mozilla-mobile/git/.git/modules/firefox-ios/gc.log
Automatic cleanup will not be performed until the file is removed.

warning: There are too many unreachable loose objects; run 'git prune' to remove them.

From https://github.com/mozilla-mobile/focus-android
   a170acb8..ace6c1cf  main       -> origin/main
 * [new branch]        relbot/AC-Nightly-98.0.20220206190121 -> origin/relbot/AC-Nightly-98.0.20220206190121
Submodule path 'focus-android': checked out 'ace6c1cfb6a46d37fb17104d193242ccc84c0b7b'
Auto packing the repository in background for optimum performance.
See "git help gc" for manual housekeeping.
warning: The last gc run reported the following. Please correct the root cause
and remove /mnt/index-scratch/mozilla-mobile/git/.git/modules/lockbox-android/gc.log
Automatic cleanup will not be performed until the file is removed.

warning: There are too many unreachable loose objects; run 'git prune' to remove them.

However, the subsequent UTC10 run didn't share the same errors, saying:

+ wget -nv https://s3-us-west-2.amazonaws.com/searchfox.repositories/mozilla-mobile.tar
2022-02-07 16:28:52 URL:https://s3-us-west-2.amazonaws.com/searchfox.repositories/mozilla-mobile.tar [2412728320/2412728320] -> "mozilla-mobile.tar" [1]
+ tar xf mozilla-mobile.tar
+ rm mozilla-mobile.tar
+ popd
~
+ date
Mon Feb  7 16:28:56 UTC 2022
+ echo Updating git
Updating git
+ pushd /mnt/index-scratch/mozilla-mobile/git
/mnt/index-scratch/mozilla-mobile/git ~
+ git submodule update --remote
From https://github.com/mozilla-mobile/android-components
   a78871809b..ec0201897a  main       -> origin/main
 + 6a102e36fc...2b74419978 relbot/upgrade-geckoview-ac-main -> origin/relbot/upgrade-geckoview-ac-main  (forced update)
Auto packing the repository in background for optimum performance.
See "git help gc" for manual housekeeping.
Submodule path 'android-components': checked out 'ec0201897ae2b2d91ec36be41986dff1c3192501'
From https://github.com/mozilla/application-services
 * [new branch]        jhugman/preprep-nimbus-for-custom-types -> origin/jhugman/preprep-nimbus-for-custom-types
Auto packing the repository in background for optimum performance.
See "git help gc" for manual housekeeping.
From https://github.com/mozilla-mobile/fenix
 + 5832ed95f...5428f3580 jhugman/23492-perf-regression-on-isFirefoxDefault -> origin/jhugman/23492-perf-regression-on-isFirefoxDefault  (forced update)
   363d0b193..bae20d650  main             -> origin/main
   c97eed4da..069113de2  releases_v97.0.0 -> origin/releases_v97.0.0
 * [new tag]             v97.1.1          -> v97.1.1
Submodule path 'fenix': checked out 'bae20d6505b7c5a4b8028ac5987d2db861dbf4aa'
From https://github.com/mozilla-mobile/firefox-ios
   45bea8cba..17174ac12  main       -> origin/main
 * [new branch]          sbasca/FixAutocompleteTests -> origin/sbasca/FixAutocompleteTests
   93b5ac983..87ac00b44  update-br-new-xcode-version -> origin/update-br-new-xcode-version
Auto packing the repository in background for optimum performance.
See "git help gc" for manual housekeeping.
Submodule path 'firefox-ios': checked out '17174ac12c460013f172b8f4c35217f555bea54b'
From https://github.com/mozilla-mobile/focus-android
   ace6c1cf..70c0b95b  main                    -> origin/main
 + 61d0406a...b2418996 oana-androidxDepUpgrade -> origin/oana-androidxDepUpgrade  (forced update)
 * [new branch]        relbot/AC-Nightly-98.0.20220207143129 -> origin/relbot/AC-Nightly-98.0.20220207143129
Submodule path 'focus-android': checked out '70c0b95ba117c55605e610f99eb6a04984705c10'
From https://github.com/mozilla-mobile/focus-ios
 * [new branch]        2988-Onboarding-Refresh -> origin/2988-Onboarding-Refresh
   a5347aab..e348251e  rlitianu/urlbar-refactor -> origin/rlitianu/urlbar-refactor
 * [new tag]           v97.0.0                 -> v97.0.0
Auto packing the repository in background for optimum performance.
See "git help gc" for manual housekeeping.
From https://github.com/mozilla-mobile/reference-browser
   d1f3742..79bfa64  master     -> origin/master
Submodule path 'reference-browser': checked out '79bfa64a7c0225de1a2a21cb3b691a36cf8ea1e5'

but for example mozilla-mobile/git/.git/modules/lockbox-android/gc.log does continue to exist, so presumably it will need to be cleaned up.

The utc22 run is running now, so I'll see tomorrow if it's still upset.

The utc22 and utc10 run were both angry about git; I presume it's a question of whether changes have landed or not or something like that.

I've landed some additional speculative fixes at https://github.com/mozsearch/mozsearch-mozilla/pull/159 which is just:

  • clean up the gc logs that prevent gc
  • do a top-level gc

Having done this manually on an obsolete web server seemed to not result in complaints about needing to prune so git gc's internal pruning behavior may already be sufficient. Or maybe it's not and we'll find out!

I've assigned this bug to me because I'll continue to try and address this, but I am very happy to hand this over to anyone who wants to address this more from a first principles approach as opposed to my current YOLO-based approach.

Assignee: nobody → bugmail
Status: NEW → ASSIGNED

Release1 emitted some warnings on Feb 11th again (noting that these are grepped and so there is missing git output in between the lines):

Searchfox produced warnings during indexing! The first 50 warnings:

warning: The last gc run reported the following. Please correct the root cause
warning: There are too many unreachable loose objects; run 'git prune' to remove them.
warning: The last gc run reported the following. Please correct the root cause
warning: There are too many unreachable loose objects; run 'git prune' to remove them.
warning: The last gc run reported the following. Please correct the root cause
warning: There are too many unreachable loose objects; run 'git prune' to remove them.
warning: The last gc run reported the following. Please correct the root cause
warning: There are too many unreachable loose objects; run 'git prune' to remove them.
warning: The last gc run reported the following. Please correct the root cause
warning: There are too many unreachable loose objects; run 'git prune' to remove them.
warning: The last gc run reported the following. Please correct the root cause
warning: There are too many unreachable loose objects; run 'git prune' to remove them.

Feb 11th was a Friday and on Feb 12th which was a Saturday we would have run https://github.com/mozsearch/mozsearch-mozilla/blob/master/shared/git-maintenance.sh but that just does a ~git gc which is something I added in comment 1.

I haven't received any email warnings about this in a long time so I think we fixed it.

Status: ASSIGNED → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
See Also: → 1869837
You need to log in before you can comment on or make changes to this bug.