Push to mirrors prefix is too greedy

RESOLVED FIXED

Status

RESOLVED FIXED
3 years ago
8 months ago

People

(Reporter: rail, Assigned: rail)

Tracking

unspecified

Firefox Tracking Flags

(firefox43 fixed, firefox44 fixed, firefox45 fixed, firefox-esr38 fixed, b2g-v2.5 fixed, b2g-master fixed)

Details

Attachments

(2 attachments, 1 obsolete attachment)

Created attachment 8681947 [details] [diff] [review]
slash.diff

05:37:30     INFO - ##### Running push-to-releases step.
05:37:30     INFO - #####
05:37:30     INFO - Running main action method: push_to_releases
05:37:30     INFO - Connecting to S3
05:37:30     INFO - Getting bucket net-mozaws-prod-delivery-firefox
05:37:30     INFO - Checking destination pub/firefox/releases/42.0 is empty
05:37:41    FATAL - Destination already exists with 26331 keys, aborting
05:37:41    FATAL - Running post_fatal callback...
05:37:41    FATAL - Exiting -1
program finished with exit code 255

pub/firefox/releases/42.0 matches pub/firefox/releases/42.0b1, pub/firefox/releases/42.0b2, etc, etc

Adding a trailing slash resolves the issue.

The batch should also address a possible build2 vs build29 issue. :P
Attachment #8681947 - Flags: review?(nthomas)
Assignee: nobody → rail
Summary: Push to mirrors prefix is too gready → Push to mirrors prefix is too greedy
Comment on attachment 8681947 [details] [diff] [review]
slash.diff

Review of attachment 8681947 [details] [diff] [review]:
-----------------------------------------------------------------

Please also patch antivirus: 
https://dxr.mozilla.org/mozilla-central/search?q=_get_candidates_prefix&redirect=false&case=true

(I'm stealing from nick to avoid the timezone delay in this patch)
Attachment #8681947 - Flags: review?(nthomas) → review+
I tested this patch by running it twice to push 42.0.
(In reply to Justin Wood (:Callek) from comment #1)
> Comment on attachment 8681947 [details] [diff] [review]
> slash.diff
> 
> Review of attachment 8681947 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> Please also patch antivirus: 
> https://dxr.mozilla.org/mozilla-central/
> search?q=_get_candidates_prefix&redirect=false&case=true
> 
> (I'm stealing from nick to avoid the timezone delay in this patch)

ooooh, good point. Let me combine the patches.
Created attachment 8681960 [details] [diff] [review]
slash2.diff
Attachment #8681947 - Attachment is obsolete: true
Attachment #8681960 - Flags: review?(bugspam.Callek)

Updated

3 years ago
Attachment #8681960 - Flags: review?(bugspam.Callek) → review+

Comment 6

3 years ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/f7c9752aeaf6
Status: NEW → RESOLVED
Last Resolved: 3 years ago
status-firefox45: --- → fixed
Resolution: --- → FIXED
Comment on attachment 8681960 [details] [diff] [review]
slash2.diff

This change is a part of release automation and doesn't affect the product.
Attachment #8681960 - Flags: approval-mozilla-release?
Attachment #8681960 - Flags: approval-mozilla-esr38?
Attachment #8681960 - Flags: approval-mozilla-beta?
Attachment #8681960 - Flags: approval-mozilla-aurora?
reopening  to keep it on my radar
Status: RESOLVED → REOPENED
Resolution: FIXED → ---

Comment 9

3 years ago
bugherderuplift
https://hg.mozilla.org/releases/mozilla-b2g44_v2_5/rev/f7c9752aeaf6
status-b2g-v2.5: --- → fixed
Comment on attachment 8681960 [details] [diff] [review]
slash2.diff

Since this is a releng fix, approving for uplift to all branches.
Attachment #8681960 - Flags: approval-mozilla-release?
Attachment #8681960 - Flags: approval-mozilla-esr38?
Attachment #8681960 - Flags: approval-mozilla-esr38+
Attachment #8681960 - Flags: approval-mozilla-beta?
Attachment #8681960 - Flags: approval-mozilla-beta+
Attachment #8681960 - Flags: approval-mozilla-aurora?
Attachment #8681960 - Flags: approval-mozilla-aurora+
Tomcat, Wes: I wasn't sure if this one would show up in your to-uplift queries as status-ffXX is not set. N-I'ing you just in case it does not.
Flags: needinfo?(wkocher)
Flags: needinfo?(cbook)
I was going to land the fixes myself.
(In reply to Rail Aliiev [:rail] from comment #12)
> I was going to land the fixes myself.

Rail, feel free to ping us when we should land this, otherwise you can push this :)
Flags: needinfo?(wkocher)
Flags: needinfo?(cbook)
rail, could you also land on the thunderbird verbranch in esr38. Unfortunately it's a kind of fork so we have to double land for automation fixes.
Reopening for followup fix for antivirus, which broke for 43.0b2 build1 like so:

22:05:50     INFO - #####
22:05:50     INFO - ##### Running get-files step.
22:05:50     INFO - #####
22:05:50     INFO - Running main action method: get_files
22:05:50     INFO - Connecting to S3
22:05:50     INFO - Getting bucket net-mozaws-prod-delivery-firefox
22:05:51     INFO - Getting key names from candidates
22:05:52    FATAL - Uncaught exception: Traceback (most recent call last):
22:05:52    FATAL -   File "/builds/slave/rel-m-beta-av-0000000000000000/scripts/mozharness/base/script.py", line 1698, in run
22:05:52    FATAL -     self.run_action(action)
22:05:52    FATAL -   File "/builds/slave/rel-m-beta-av-0000000000000000/scripts/mozharness/base/script.py", line 1640, in run_action
22:05:52    FATAL -     self._possibly_run_method(method_name, error_if_missing=True)
22:05:52    FATAL -   File "/builds/slave/rel-m-beta-av-0000000000000000/scripts/mozharness/base/script.py", line 1581, in _possibly_run_method
22:05:52    FATAL -     return getattr(self, method_name)()
22:05:52    FATAL -   File "scripts/scripts/release/antivirus.py", line 177, in get_files
22:05:52    FATAL -     pool.map(worker, find_release_files())
22:05:52    FATAL -   File "/tools/python27/lib/python2.7/multiprocessing/pool.py", line 227, in map
22:05:52    FATAL -     return self.map_async(func, iterable, chunksize).get()
22:05:52    FATAL -   File "/tools/python27/lib/python2.7/multiprocessing/pool.py", line 280, in map_async
22:05:52    FATAL -     iterable = list(iterable)
22:05:52    FATAL -   File "scripts/scripts/release/antivirus.py", line 173, in find_release_files
22:05:52    FATAL -     os.makedirs(dest_dir)
22:05:52    FATAL -   File "/tools/python27/lib/python2.7/os.py", line 157, in makedirs
22:05:52    FATAL -     mkdir(name, mode)
22:05:52    FATAL - OSError: [Errno 2] No such file or directory: ''
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Created attachment 8685305 [details] [diff] [review]
[gecko] Use os.path.join

Maybe this will teach me to use os.path.join() for everything (except urls).
Attachment #8685305 - Flags: review?(jlund)
Comment on attachment 8685305 [details] [diff] [review]
[gecko] Use os.path.join

Stealing this
Attachment #8685305 - Flags: review?(jlund) → review+
Keywords: leave-open
Comment on attachment 8685305 [details] [diff] [review]
[gecko] Use os.path.join

This change is a part of release automation and doesn't affect the product.
Attachment #8685305 - Flags: approval-mozilla-release?
Attachment #8685305 - Flags: approval-mozilla-esr38?
Attachment #8685305 - Flags: approval-mozilla-beta?
Attachment #8685305 - Flags: approval-mozilla-aurora?
in addition to(In reply to Nick Thomas [:nthomas] from comment #18)
> Reopening for followup fix for antivirus, which broke for 43.0b2 build1 like

in addition to rail reviewing this, he also got a loaner and manually ran av.
Comment on attachment 8685305 [details] [diff] [review]
[gecko] Use os.path.join

Doesn't affect the product, you can land without approval.
Attachment #8685305 - Flags: approval-mozilla-release?
Attachment #8685305 - Flags: approval-mozilla-release+
Attachment #8685305 - Flags: approval-mozilla-esr38?
Attachment #8685305 - Flags: approval-mozilla-esr38+
Attachment #8685305 - Flags: approval-mozilla-beta?
Attachment #8685305 - Flags: approval-mozilla-beta+
Attachment #8685305 - Flags: approval-mozilla-aurora?
Attachment #8685305 - Flags: approval-mozilla-aurora+
(In reply to Liz Henry (:lizzard) (needinfo? me) from comment #25)
> Comment on attachment 8685305 [details] [diff] [review]
> [gecko] Use os.path.join
> 
> Doesn't affect the product, you can land without approval.

rail: do you want to push fix or do you need sheriff assistance ?
I'll land them, don't worry.
and release:
https://hg.mozilla.org/releases/mozilla-release/rev/fa829a9c6898
Status: REOPENED → RESOLVED
Last Resolved: 3 years ago3 years ago
Resolution: --- → FIXED
Landed on THUNDERBIRD_38_VERBRANCH because Thunderbird 38.4.0 build3 a/v failed:
https://hg.mozilla.org/releases/mozilla-esr38/rev/c19844a38295

Updated

3 years ago
status-b2g-master: --- → fixed
status-firefox43: --- → fixed
status-firefox44: --- → fixed

Updated

3 years ago
status-firefox-esr38: --- → fixed
Removing leave-open keyword from resolved bugs, per :sylvestre.
Keywords: leave-open
You need to log in before you can comment on or make changes to this bug.