Closed Bug 1453026 Opened 6 years ago Closed 5 years ago

Snap partial updates: Delta service failed to apply delta successfully

Categories

(Release Engineering Graveyard :: Release Automation: Snap, enhancement)

Product:
Release Engineering Graveyard
Component:
Release Automation: Snap
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: jlorenzo, Unassigned)

References

(
URL
)

Details

We could generate delta updates for snap, if we had xdelta3 installed on the machine[1][2]:

> 2018-04-09 22:35:51,286 - snapcraft._store - INFO - Found cached source snap /home/cltbld/.cache/snapcraft/projects/firefox/snap_hashes/amd64/1549949fa22a73acbeace379745171095cfd8c122f9109851bf76c326a0119fa560db41b0a785b390009936ae861fc3c.
> 2018-04-09 22:35:51,288 - snapcraft._store - WARNING - Error generating delta: delta_tool_path must be set in subclass!

Sadly, we have to make the package ourselves. There is no rpm available per [3]

[1] https://tools.taskcluster.net/groups/BH0QdNzbQ8iLXlM28cQ4jw/tasks/Vm7C-ovDSMOLlEuT1dQEVA/runs/0/logs/public%2Flogs%2Flive_backing.log#L12
[2] https://github.com/snapcore/snapcraft/blob/c59977709262f8c960fd0ebbafc9e7799093109e/snapcraft/internal/deltas/_xdelta3.py#L29-L30
[3] https://pkgs.org/download/xdelta3
Blocks: 1451694
Component: Release Automation: Other → Release Automation: Pushapk
Component: Release Automation: Pushapk → Release Automation: Snap
See Also: → 1493053
Bug 1493053 made me install xdelta3 on the worker. However, something went wrong on the Store side[1]. The logs don't say much more than:

> 2018-09-21T13:03:53 INFO - 2018-09-21 13:03:53,855 - snapcraft._store - INFO - Pushing delta /builds/scriptworker/work/cot/BtwB6HZZTCSL2KbfPCTRzg/public/build/target.snap.xdelta3.
> Pushing target.snap.xdelta3 [============================================] 100%
Processing...|
> 2018-09-21T13:04:13 INFO - 2018-09-21 13:04:13,786 - snapcraft._store - WARNING - Error generating delta: There has been a problem while processing a snap delta.
> 2018-09-21T13:04:13 INFO - - Delta service failed to apply delta successfully
> 2018-09-21T13:04:13 INFO - Falling back to pushing full snap...

I filed https://bugs.launchpad.net/snapstore/+bug/1793752 to get more input.

[1] https://tools.taskcluster.net/groups/CTU6CA-PTGyDBu4IExIgoA/tasks/USJ2lRdtQ16waOO0kzT-zw/runs/2/logs/public%2Flogs%2Flive_backing.log#L198
URL: https://bugs.launchpad.net/snapstore/...
Depends on: 1493053, 1488396
See Also: 1493053 →
Summary: pushsnapworker: Missing xdelta3 → Snap partial updates: Delta service failed to apply delta successfully

I realized I've never came back to this bug. The Canonical folks fixed it a while ago 😃 Like I said in [1], our most recent push looks great[2]:

2019-09-03 08:58:19,518 - snapcraft.internal.deltas._xdelta3 - DEBUG - xdelta3 delta diff generation:
VCDIFF version: 0
VCDIFF header size: 117
VCDIFF header indicator: VCD_SECONDARY VCD_APPHEADER
VCDIFF secondary compressor: lzma
VCDIFF application header: target.snap//c553f2f91fffe9cd02110d31620942db7babee02bac5d73dbb303f55cd43b02fa30becd888d4e5887c3949177bfa45ca/
XDELTA filename (output): target.snap
XDELTA filename (source): c553f2f91fffe9cd02110d31620942db7babee02bac5d73dbb303f55cd43b02fa30becd888d4e5887c3949177bfa45ca
VCDIFF window number: 0
VCDIFF window indicator: VCD_SOURCE VCD_ADLER32
VCDIFF adler32 checksum: B13BFA3D
VCDIFF delta indicator: VCD_DATACOMP VCD_INSTCOMP VCD_ADDRCOMP
VCDIFF copy window length: 66951984
VCDIFF copy window offset: 0
VCDIFF delta encoding length: 7431080
VCDIFF target window length: 8388608
VCDIFF data section length: 7427894
VCDIFF inst section length: 1794
VCDIFF addr section length: 1375

2019-09-03 08:58:27,988 - snapcraft._store - INFO - Pushing delta /builds/scriptworker/work/cot/M3vBcSoISlKtTAOitQ9tmQ/public/build/target.snap.xdelta3.
Pushing target.snap.xdelta3 [ ] 0%
Pushing target.snap.xdelta3 [ ] 1%
Pushing target.snap.xdelta3 [ ] 2%
Pushing target.snap.xdelta3 [= ] 3%
Pushing target.snap.xdelta3 [= ] 4%
Pushing target.snap.xdelta3 [== ] 5%
Pushing target.snap.xdelta3 [== ] 6%
Pushing target.snap.xdelta3 [=== ] 7%
Pushing target.snap.xdelta3 [=== ] 8%
Pushing target.snap.xdelta3 [=== ] 9%
Pushing target.snap.xdelta3 [==== ] 10%
Pushing target.snap.xdelta3 [==== ] 11%
Pushing target.snap.xdelta3 [===== ] 12%
Pushing target.snap.xdelta3 [===== ] 13%
Pushing target.snap.xdelta3 [====== ] 14%
Pushing target.snap.xdelta3 [====== ] 15%
Pushing target.snap.xdelta3 [======= ] 16%
Pushing target.snap.xdelta3 [======= ] 17%
Pushing target.snap.xdelta3 [======= ] 18%
Pushing target.snap.xdelta3 [======== ] 19%
Pushing target.snap.xdelta3 [======== ] 20%
Pushing target.snap.xdelta3 [========= ] 21%
Pushing target.snap.xdelta3 [========= ] 22%
Pushing target.snap.xdelta3 [========== ] 23%
Pushing target.snap.xdelta3 [========== ] 24%
Pushing target.snap.xdelta3 [=========== ] 25%
Pushing target.snap.xdelta3 [=========== ] 26%
Pushing target.snap.xdelta3 [=========== ] 27%
Pushing target.snap.xdelta3 [============ ] 28%
Pushing target.snap.xdelta3 [============ ] 29%
Pushing target.snap.xdelta3 [============= ] 30%
Pushing target.snap.xdelta3 [============= ] 31%
Pushing target.snap.xdelta3 [============== ] 32%
Pushing target.snap.xdelta3 [============== ] 33%
Pushing target.snap.xdelta3 [=============== ] 34%
Pushing target.snap.xdelta3 [=============== ] 35%
Pushing target.snap.xdelta3 [=============== ] 36%
Pushing target.snap.xdelta3 [================ ] 37%
Pushing target.snap.xdelta3 [================ ] 38%
Pushing target.snap.xdelta3 [================= ] 39%
Pushing target.snap.xdelta3 [================= ] 40%
Pushing target.snap.xdelta3 [================== ] 41%
Pushing target.snap.xdelta3 [================== ] 42%
Pushing target.snap.xdelta3 [=================== ] 43%
Pushing target.snap.xdelta3 [=================== ] 44%
Pushing target.snap.xdelta3 [=================== ] 45%
Pushing target.snap.xdelta3 [==================== ] 46%
Pushing target.snap.xdelta3 [==================== ] 47%
Pushing target.snap.xdelta3 [===================== ] 48%
Pushing target.snap.xdelta3 [===================== ] 49%
Pushing target.snap.xdelta3 [====================== ] 50%
Pushing target.snap.xdelta3 [====================== ] 51%
Pushing target.snap.xdelta3 [====================== ] 52%
Pushing target.snap.xdelta3 [======================= ] 53%
Pushing target.snap.xdelta3 [======================= ] 54%
Pushing target.snap.xdelta3 [======================== ] 55%
Pushing target.snap.xdelta3 [======================== ] 56%
Pushing target.snap.xdelta3 [========================= ] 57%
Pushing target.snap.xdelta3 [========================= ] 58%
Pushing target.snap.xdelta3 [========================== ] 59%
Pushing target.snap.xdelta3 [========================== ] 60%
Pushing target.snap.xdelta3 [========================== ] 61%
Pushing target.snap.xdelta3 [=========================== ] 62%
Pushing target.snap.xdelta3 [=========================== ] 63%
Pushing target.snap.xdelta3 [============================ ] 64%
Pushing target.snap.xdelta3 [============================ ] 65%
Pushing target.snap.xdelta3 [============================= ] 66%
Pushing target.snap.xdelta3 [============================= ] 67%
Pushing target.snap.xdelta3 [============================== ] 68%
Pushing target.snap.xdelta3 [============================== ] 69%
Pushing target.snap.xdelta3 [============================== ] 70%
Pushing target.snap.xdelta3 [=============================== ] 71%
Pushing target.snap.xdelta3 [=============================== ] 72%
Pushing target.snap.xdelta3 [================================ ] 73%
Pushing target.snap.xdelta3 [================================ ] 74%
Pushing target.snap.xdelta3 [================================= ] 75%
Pushing target.snap.xdelta3 [================================= ] 76%
Pushing target.snap.xdelta3 [================================== ] 77%
Pushing target.snap.xdelta3 [================================== ] 78%
Pushing target.snap.xdelta3 [================================== ] 79%
Pushing target.snap.xdelta3 [=================================== ] 80%
Pushing target.snap.xdelta3 [=================================== ] 81%
Pushing target.snap.xdelta3 [==================================== ] 82%
Pushing target.snap.xdelta3 [==================================== ] 83%
Pushing target.snap.xdelta3 [===================================== ] 84%
Pushing target.snap.xdelta3 [===================================== ] 85%
Pushing target.snap.xdelta3 [====================================== ] 86%
Pushing target.snap.xdelta3 [====================================== ] 87%
Pushing target.snap.xdelta3 [====================================== ] 88%
Pushing target.snap.xdelta3 [======================================= ] 89%
Pushing target.snap.xdelta3 [======================================= ] 90%
Pushing target.snap.xdelta3 [======================================== ] 91%
Pushing target.snap.xdelta3 [======================================== ] 92%
Pushing target.snap.xdelta3 [========================================= ] 93%
Pushing target.snap.xdelta3 [========================================= ] 94%
Pushing target.snap.xdelta3 [========================================== ] 95%
Pushing target.snap.xdelta3 [========================================== ] 96%
Pushing target.snap.xdelta3 [========================================== ] 97%
Pushing target.snap.xdelta3 [=========================================== ] 98%
Pushing target.snap.xdelta3 [=========================================== ] 99%
Pushing target.snap.xdelta3 [============================================] 100%
Ready to release!

[1] https://bugs.launchpad.net/snapstore/+bug/1793752/comments/8
[2] https://taskcluster-artifacts.net/e554EXG2RFiR7fljZfOsHA/0/public/logs/live_backing.log

Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Blocks: 1590267
Product: Release Engineering → Release Engineering Graveyard
You need to log in before you can comment on or make changes to this bug.