Closed Bug 1220641 Opened 9 years ago Closed 9 years ago

Push to mirrors prefix is too greedy

Categories

(Release Engineering :: Release Automation: Other, defect)

defect
Not set
normal

Tracking

(firefox43 fixed, firefox44 fixed, firefox45 fixed, firefox-esr38 fixed, b2g-v2.5 fixed, b2g-master fixed)

RESOLVED FIXED
Tracking Status
firefox43 --- fixed
firefox44 --- fixed
firefox45 --- fixed
firefox-esr38 --- fixed
b2g-v2.5 --- fixed
b2g-master --- fixed

People

(Reporter: rail, Assigned: rail)

References

Details

Attachments

(2 files, 1 obsolete file)

Attached patch slash.diff (obsolete) — Splinter Review
05:37:30 INFO - ##### Running push-to-releases step. 05:37:30 INFO - ##### 05:37:30 INFO - Running main action method: push_to_releases 05:37:30 INFO - Connecting to S3 05:37:30 INFO - Getting bucket net-mozaws-prod-delivery-firefox 05:37:30 INFO - Checking destination pub/firefox/releases/42.0 is empty 05:37:41 FATAL - Destination already exists with 26331 keys, aborting 05:37:41 FATAL - Running post_fatal callback... 05:37:41 FATAL - Exiting -1 program finished with exit code 255 pub/firefox/releases/42.0 matches pub/firefox/releases/42.0b1, pub/firefox/releases/42.0b2, etc, etc Adding a trailing slash resolves the issue. The batch should also address a possible build2 vs build29 issue. :P
Attachment #8681947 - Flags: review?(nthomas)
Assignee: nobody → rail
Summary: Push to mirrors prefix is too gready → Push to mirrors prefix is too greedy
Comment on attachment 8681947 [details] [diff] [review] slash.diff Review of attachment 8681947 [details] [diff] [review]: ----------------------------------------------------------------- Please also patch antivirus: https://dxr.mozilla.org/mozilla-central/search?q=_get_candidates_prefix&redirect=false&case=true (I'm stealing from nick to avoid the timezone delay in this patch)
Attachment #8681947 - Flags: review?(nthomas) → review+
I tested this patch by running it twice to push 42.0.
(In reply to Justin Wood (:Callek) from comment #1) > Comment on attachment 8681947 [details] [diff] [review] > slash.diff > > Review of attachment 8681947 [details] [diff] [review]: > ----------------------------------------------------------------- > > Please also patch antivirus: > https://dxr.mozilla.org/mozilla-central/ > search?q=_get_candidates_prefix&redirect=false&case=true > > (I'm stealing from nick to avoid the timezone delay in this patch) ooooh, good point. Let me combine the patches.
Attached patch slash2.diffSplinter Review
Attachment #8681947 - Attachment is obsolete: true
Attachment #8681960 - Flags: review?(bugspam.Callek)
Attachment #8681960 - Flags: review?(bugspam.Callek) → review+
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Comment on attachment 8681960 [details] [diff] [review] slash2.diff This change is a part of release automation and doesn't affect the product.
Attachment #8681960 - Flags: approval-mozilla-release?
Attachment #8681960 - Flags: approval-mozilla-esr38?
Attachment #8681960 - Flags: approval-mozilla-beta?
Attachment #8681960 - Flags: approval-mozilla-aurora?
reopening to keep it on my radar
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Comment on attachment 8681960 [details] [diff] [review] slash2.diff Since this is a releng fix, approving for uplift to all branches.
Attachment #8681960 - Flags: approval-mozilla-release?
Attachment #8681960 - Flags: approval-mozilla-esr38?
Attachment #8681960 - Flags: approval-mozilla-esr38+
Attachment #8681960 - Flags: approval-mozilla-beta?
Attachment #8681960 - Flags: approval-mozilla-beta+
Attachment #8681960 - Flags: approval-mozilla-aurora?
Attachment #8681960 - Flags: approval-mozilla-aurora+
Tomcat, Wes: I wasn't sure if this one would show up in your to-uplift queries as status-ffXX is not set. N-I'ing you just in case it does not.
Flags: needinfo?(wkocher)
Flags: needinfo?(cbook)
I was going to land the fixes myself.
(In reply to Rail Aliiev [:rail] from comment #12) > I was going to land the fixes myself. Rail, feel free to ping us when we should land this, otherwise you can push this :)
Flags: needinfo?(wkocher)
Flags: needinfo?(cbook)
rail, could you also land on the thunderbird verbranch in esr38. Unfortunately it's a kind of fork so we have to double land for automation fixes.
Reopening for followup fix for antivirus, which broke for 43.0b2 build1 like so: 22:05:50 INFO - ##### 22:05:50 INFO - ##### Running get-files step. 22:05:50 INFO - ##### 22:05:50 INFO - Running main action method: get_files 22:05:50 INFO - Connecting to S3 22:05:50 INFO - Getting bucket net-mozaws-prod-delivery-firefox 22:05:51 INFO - Getting key names from candidates 22:05:52 FATAL - Uncaught exception: Traceback (most recent call last): 22:05:52 FATAL - File "/builds/slave/rel-m-beta-av-0000000000000000/scripts/mozharness/base/script.py", line 1698, in run 22:05:52 FATAL - self.run_action(action) 22:05:52 FATAL - File "/builds/slave/rel-m-beta-av-0000000000000000/scripts/mozharness/base/script.py", line 1640, in run_action 22:05:52 FATAL - self._possibly_run_method(method_name, error_if_missing=True) 22:05:52 FATAL - File "/builds/slave/rel-m-beta-av-0000000000000000/scripts/mozharness/base/script.py", line 1581, in _possibly_run_method 22:05:52 FATAL - return getattr(self, method_name)() 22:05:52 FATAL - File "scripts/scripts/release/antivirus.py", line 177, in get_files 22:05:52 FATAL - pool.map(worker, find_release_files()) 22:05:52 FATAL - File "/tools/python27/lib/python2.7/multiprocessing/pool.py", line 227, in map 22:05:52 FATAL - return self.map_async(func, iterable, chunksize).get() 22:05:52 FATAL - File "/tools/python27/lib/python2.7/multiprocessing/pool.py", line 280, in map_async 22:05:52 FATAL - iterable = list(iterable) 22:05:52 FATAL - File "scripts/scripts/release/antivirus.py", line 173, in find_release_files 22:05:52 FATAL - os.makedirs(dest_dir) 22:05:52 FATAL - File "/tools/python27/lib/python2.7/os.py", line 157, in makedirs 22:05:52 FATAL - mkdir(name, mode) 22:05:52 FATAL - OSError: [Errno 2] No such file or directory: ''
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Maybe this will teach me to use os.path.join() for everything (except urls).
Attachment #8685305 - Flags: review?(jlund)
Comment on attachment 8685305 [details] [diff] [review] [gecko] Use os.path.join Stealing this
Attachment #8685305 - Flags: review?(jlund) → review+
Keywords: leave-open
Comment on attachment 8685305 [details] [diff] [review] [gecko] Use os.path.join This change is a part of release automation and doesn't affect the product.
Attachment #8685305 - Flags: approval-mozilla-release?
Attachment #8685305 - Flags: approval-mozilla-esr38?
Attachment #8685305 - Flags: approval-mozilla-beta?
Attachment #8685305 - Flags: approval-mozilla-aurora?
in addition to(In reply to Nick Thomas [:nthomas] from comment #18) > Reopening for followup fix for antivirus, which broke for 43.0b2 build1 like in addition to rail reviewing this, he also got a loaner and manually ran av.
Comment on attachment 8685305 [details] [diff] [review] [gecko] Use os.path.join Doesn't affect the product, you can land without approval.
Attachment #8685305 - Flags: approval-mozilla-release?
Attachment #8685305 - Flags: approval-mozilla-release+
Attachment #8685305 - Flags: approval-mozilla-esr38?
Attachment #8685305 - Flags: approval-mozilla-esr38+
Attachment #8685305 - Flags: approval-mozilla-beta?
Attachment #8685305 - Flags: approval-mozilla-beta+
Attachment #8685305 - Flags: approval-mozilla-aurora?
Attachment #8685305 - Flags: approval-mozilla-aurora+
(In reply to Liz Henry (:lizzard) (needinfo? me) from comment #25) > Comment on attachment 8685305 [details] [diff] [review] > [gecko] Use os.path.join > > Doesn't affect the product, you can land without approval. rail: do you want to push fix or do you need sheriff assistance ?
I'll land them, don't worry.
Status: REOPENED → RESOLVED
Closed: 9 years ago9 years ago
Resolution: --- → FIXED
Landed on THUNDERBIRD_38_VERBRANCH because Thunderbird 38.4.0 build3 a/v failed: https://hg.mozilla.org/releases/mozilla-esr38/rev/c19844a38295
Removing leave-open keyword from resolved bugs, per :sylvestre.
Keywords: leave-open
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: