Closed Bug 1073399 Opened 7 years ago Closed 7 years ago

[flame] v180 leaves phone with not enough space on /system, which means OTA will fail or might render phone unusable (/system/b2g/libmozglue.so not found)

Categories

(Firefox OS Graveyard :: General, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED WORKSFORME

People

(Reporter: freddy, Unassigned)

References

Details

Attachments

(4 files)

Attached file adblogcat.txt
I am on the v180 base image and shallow-flashed gecko+gaia yesterday (Thu Sep 25, noonish CEST).

Accepting an OTA update this morning (also CEST) had the phone stuck at the ThunderSoft boot screen after reboot.

Log file attached.
Another gaia/gecko shallow flash on top gives me no different results.
A shallow flash of your own build? From a flame-kk config?
I flashed a pvt build with the flash_pvt script from <https://github.com/Mozilla-TWQA/B2G-flash-tool/>.
Summary: [flame[ v180 OTA renders phone unusable (/system/b2g/libmozglue.so not found) → [flame] v180 OTA renders phone unusable (/system/b2g/libmozglue.so not found)
it has been discussed in dev-b2g as well as dev-gaia that this is likely caused by the device running out of space in /system.

Workaround:
Make room in /system by removing things that aren't strictly necessary.
E.g. fonts except Fira* in /system/fonts, or media in /system/media/audio
Adding my own experience: I tried to install OTA, but after decompressing the file I didn't get the usual dialog to install the update. Instead the message "Update available" was displayed all over again.

At that point I tried to install with B2G-flash-tool, display froze on this error

failed to copy '/var/folders/5h/1kskt9vd0bs43q0_hw4b3qym0000gn/T/shallowflashgecko.XXXXXXXXXXXX.w3XBSLCR/b2g/b2g' to '/system/b2g/b2g': No space left on device

Removed the battery, ended up with the frozen ThunderSoft splash screen.

In my case I was able to reinstall v180 and flash gecko+gaia with B2G-flash-tool (20140929). 
Too bad that now I can't flash my own Gaia on the device anymore, phone gets stuck on the fox, so my only option is to go back to the last B2G-flash-tool build.
Same here, I found that I had a very large directory called 'updated' under /system/b2g, wiping it from the device allowed me to shallow flash and go on. It seems like the OTA update is getting decompressed under /system and this is causing the issue.
The updated directory contains a patched version of what is otherwise in /system/b2g

IIRC, it will get deleted if the update is successful.

So this sounds like an aborted/incomplete/failed update.

I think that it makes sense for shallow flash to arbitrarily remove that diectory tree, if it exists.
(In reply to Gabriele Svelto [:gsvelto] from comment #6)
> Same here, I found that I had a very large directory called 'updated' under
> /system/b2g, wiping it from the device allowed me to shallow flash and go
> on. It seems like the OTA update is getting decompressed under /system and
> this is causing the issue.

I can´t delete the folder updated, it says "rm for updated failed, read-only file system"

Well, I am in adb shell so can you tell me what I´m doing wrong?
(In reply to michaelcollatz from comment #8)
> I can´t delete the folder updated, it says "rm for updated failed, read-only
> file system"

You need to remount:

adb remount

If it complains about adb not running as root just execute:

adb root

Once you've got the filesystem mounted in r/w mode you can use this command to wipe leftover updates as well as Android bits we don't use. This should free quite a bit of space on /system and make updates work correctly:

adb shell "rm -r /system/fonts/Android* /system/fonts/Charis* \
                 /system/fonts/Clockopia.ttf /system/fonts/DDC_Uchen.ttf \
                 /system/fonts/Droid* /system/fonts/Lohit* \
                 /system/fonts/MTL* /system/fonts/NanumGothic.ttf \
                 /system/fonts/Noto* /system/fonts/Padauk* \
                 /system/fonts/Roboto* /system/media/audio \
                 /system/b2g/updated"
Thank you very much, deleting the fonts and the updated folder resolved the problem, the OTA updates no work again.
Duplicate of this bug: 1077944
FWIW:
 (1) After clearing out b2g/updated and most fonts & media from /system, I was able to do one or two OTA updates, and then was again unable. It seems like I need 80+ MB for a successful OTA update, and I'm back down to ~50 somehow.  (Deleting the partial "b2g/updated" folder again [from the failed update] doesn't help; that doesn't free up any space at all, it seems.)

 (2) I was able to do a shallow_flash update yesterday (when I couldn't do an OTA update), so that seems to be doable with less free space than an OTA update requires. However, this isn't a feasible solution for keeping up to date on a day-to-day basis, because it requires more manual work (including backing up & restoring my userdata).

So basically, my Flame can't reliably apply OTA (aurora) updates, which means I'm lagging behind on updates.  Anything I can do (on top of comment 9) to address this?  And are there any plans to fix this for users like me? (beyond apparently-temporary fixes like comment 9)
After following the instructions of comment 9, I was able to update again (although due to comment 12, I'm not sure if this will last), but now some fonts don't display in Firefox OS and the keyboard (Lots of non English languages and the ℃ and ℉ symbol).
Attached image 2014-10-06-23-28-03.png
(In reply to Rika Pi from comment #13)
> After following the instructions of comment 9, I was able to update again
> (although due to comment 12, I'm not sure if this will last), but now some
> fonts don't display in Firefox OS and the keyboard (Lots of non English
> languages and the ℃ and ℉ symbol).

Have you run the font fix script?

https://developer.mozilla.org/en-US/Firefox_OS/Developer_phone_guide/Flame#Font_fix
I ran it before I did comment 9, should I run it again?
I had also noticed that. It seems like some of these fonts are used for non-ASCII characters (the most obvious in the language selector of the first run app).
I ran the font fix script again but I still don't get the fonts for non-ASCII characters back
We need to sort this out quickly. Tony or Francis, is this something on us or partner to solve. It wasn't a problem with v123 and it is with v180.
Hi Mike:

Looks like system left space would become smaller&smaller each time user perform OTA, then error occur once the space is not enough?

Would you help try if we can repro this in both 

1) v180 + shallow flash KK gaia/gecko, then perform 1 or more times OTA
2) v184 + shallow flash KK gaia/gecko, then perform 1 or more times OTA

Since there is a main difference in v184 that is QCT-CS release. (which we should pay more attention to)

If both are reproducible I suggest we follow this with v184.

Thanks!
Flags: needinfo?(mlien)
(In reply to Rika Pi from comment #18)
> I ran the font fix script again but I still don't get the fonts for
> non-ASCII characters back

I've verified the problem is present on my device too. I selectively deleted all fonts until the issue appeared and found out that we're still relying on DroidSansFallback.ttf for those characters so following my advice in comment 9 will cause the issue you've been experiencing.
Filed bug 1079269 to track the issue with the °C and °F symbols.
BTW I've run multiple updates on my device after wiping the fonts/media/updated folders and I haven't seen anymore issues with free space. My /system partition has ~100MiB of free space and it stays the same after an OTA update.
(In reply to Gabriele Svelto [:gsvelto] from comment #9)
> adb shell "rm -r /system/fonts/Android* /system/fonts/Charis* \
>                  /system/fonts/Clockopia.ttf /system/fonts/DDC_Uchen.ttf \
>                  /system/fonts/Droid* /system/fonts/Lohit* \
>                  /system/fonts/MTL* /system/fonts/NanumGothic.ttf \
>                  /system/fonts/Noto* /system/fonts/Padauk* \
>                  /system/fonts/Roboto* /system/media/audio \
>                  /system/b2g/updated"

Charis*, Droid*, Lohit*, MTL*, Noto*, Padauk*, Roboto*, DDC_Uchen.ttf are all part of the set of fonts we expect to be available as part of the platform. Please do not remove them. See https://github.com/mozilla-b2g/moztt/blob/master/fonts.mk for more details on the exact set of fonts that should be shipped.
In v180, after shallow flash, system partition will remain only 75MB free space to use
The v2.1 system update package need 68.4MB to download in internal storage and decompress to system partition
It will exceed the free space size and encounter OTA fail
But I didn't encounter unusable problem

By the way, v184 cannot do OTA successfully, please do not try this path through it, but shallow flash/full flash are fine.
The reason is v184's device name is changed to "flame-kk" and this causes OTA is directed to a wrong update URL
Flags: needinfo?(mlien)
(In reply to Michael Wu [:mwu] from comment #24)
> Charis*, Droid*, Lohit*, MTL*, Noto*, Padauk*, Roboto*, DDC_Uchen.ttf are
> all part of the set of fonts we expect to be available as part of the
> platform. Please do not remove them. See
> https://github.com/mozilla-b2g/moztt/blob/master/fonts.mk for more details
> on the exact set of fonts that should be shipped.

Good to know, I thought we relied on Fira* exclusively now. I assume this means that Android*, Clockopia.ttf and NanumGothic.ttf can go, right?
A question, How might I be able to get the fonts needed by Firefox OS that were deleted as of comment 9. And would that bring the issue of not being able to do OTA updates back?
(In reply to Rika Pi from comment #27)
> A question, How might I be able to get the fonts needed by Firefox OS that
> were deleted as of comment 9. And would that bring the issue of not being
> able to do OTA updates back?

You'll have to reflash the v180 base image and then shallow-flash gaia & gecko on top. You can also take the fonts from another device using 'adb pull' and push them on your device instead, that's what I did, it's faster and you don't have to backup your profile.
These fonts are not available anywhere else? Eg, Github, repository, etc. Although I am willing to do it, I would rather not go through the hassle of re-flashing my phone.
Hi Keven:

Would you help assign someone to check this issue? Looks like system left space will become smaller&smaller each time user perform OTA, then error occur once the space is not enough.

Mike has repro with v180 (unfortunately can't use v184 to test due to product.device name change) & some findings about "system left size" in comment#25, there is also some observation that "delete font" could be a workaround, but not suggested way.

Since this can be repro. by our OTA I think it makes since to check from our side first. (instead of vendor side, though they made Flame device)

Thank you.
Flags: needinfo?(kkuo)
btw this issue looks like https://bugzilla.mozilla.org/show_bug.cgi?id=1078660 but I'm not 100% sure.
(Following up on my comment 12: I think my mysterious-space-disappearance [with me ending up back at 50MB, several days after deleting media/fonts] might be a filesystem issue, or something else low-level & mysterious.  gsvelto & I compared our /system partitions' contents, and the files & their sizes were all approximately the same, and yet he had 50MB more free space than I did. So, that's weird. Additionally: I deleted and then restored another /system subdirectory [with adb push from a backup copy] [not recommended], and I somehow ended up losing *another* 30 MB in the process. So, at least in my case in comment 12, some of the disk-usage is a bit mysterious.  In any case, I've now reflashed with v180 and up-to-date gecko & gaia and have deleted media & fonts to free up space, and I'm at 100MB free like gsvelto was, and I'm still at 100MB after an OTA update last night, so things are looking good for my phone going forward.)
Duplicate of this bug: 1078660
Duplicate of this bug: 1079786
Summary: [flame] v180 OTA renders phone unusable (/system/b2g/libmozglue.so not found) → [flame] v180 leaves phone with not enough space on /system, which means OTA will fail or might render phone unusable (/system/b2g/libmozglue.so not found)
Hi Kaizhen:

As discussed would you help take a look at this? Thank you.
Flags: needinfo?(kkuo) → needinfo?(kli)
I have a test which is based on v184 and latest pvt build. After OTA update is finished, there will be black screen. But I think this is more likely some error in OTA update.mar and not the issue of OTA update process.

To verify my assumption, I did another test. The steps of this test are:

1. download the ota update.mar from this url
http://update.boot2gecko.org/flame/2.0.0/nightly/b2g_update_20141008000202.mar

2. wrap update.mar to get b2g folder

3. flash b2g (wrap from OTA update.mar) to device -> black screen after restart b2g

4. flash b2g (download from pvt latest) to device -> work properly after restart b2g

5. repeat step 3 and step 4, the results are the same.

So I think we should find someone to check what is wrong in OTA update.mar.
Flags: needinfo?(kli)
I got the answer why OTA update.mar mentioned in comment 36 didn't work in my device. Because my Flame is based on v184 and latest pvt flame-kk build and the OTA update.mar I used is for jb.

This time I try to use OTA update.mar from flame-kk (below url), it is working.
http://update.boot2gecko.org/flame-kk/2.0.0/nightly/b2g_update_20141009000201.mar

So flame jb build should get update from flame OTA; flame-kk build should get update from flame-kk OTA.


Gerry, could you help to verify this? Thank you!
Flags: needinfo?(gchang)
I also used 'adb shell df' to monitor the available space of /system and sdcard. I didn't get the issue of not enough space based on v184 + pvt build. 

//Before OTA
Filesystem               Size     Used     Free   Blksize
/system                354.2M   240.9M   113.2M   4096
/storage/sdcard          3.8G   124.0K     3.8G   4096

//After download OTA
Filesystem               Size     Used     Free   Blksize
/system                354.2M   327.4M    26.7M   4096
/storage/sdcard          3.8G    68.8M     3.8G   4096

//After install OTA
Filesystem               Size     Used     Free   Blksize
/system                354.2M   245.2M   109.0M   4096
/storage/sdcard          3.8G   124.0K     3.8G   4096
Duplicate of this bug: 1081170
I have this issue of not enough space on /system too:
Filesystem               Size     Used     Free   Blksize
/system                354.2M   275.0M    79.2M   4096

My Flame is brand new (got it just 2 days ago), and I followed the MDN Flame's guide https://developer.mozilla.org/en-US/Firefox_OS/Developer_phone_guide/Flame#Updating_your_Flame%27s_software.
So I have base image v180, flashed Gaia and Gecko builds from 2014-10-11-16-02-02-mozilla-b2g32_v2_0-flame-kk/, applied the font fix and set the update channel to nightly-b2g32.
Right after these steps, I get a notification for 2 updates (Marketplace and System Update). The Marketplace update works fine (given its small size) but I could never complete a single System Update.

I tried this update process (from flashing v180) 3 times and always get this behaviour, even after deleting some optional fonts (Android*, Clockopia.ttf, NanumGothic.ttf) and some ringtones.
I can confirm what Ghost of Kendo reported. The only way to be able to do OTA updates on a v180 base image is to do the destructive removal of fonts as is suggested in comment 9
Any news on when a fix for v180 will be released to solve this as it affects all base image installs with v180 (2.0+)
For some reason, v184 has a bit more free space on /system but it still has the unneeded files in esp. /system/media/audio which free up more than 10MB of additional space.

Given that it's our reference device, we also should see that we reduce the fonts on those base images to the minimum required set not just for maximizing space but also for making sure we test with only the fonts installed that we know will be present in target devices.
(In reply to Robert Kaiser (:kairo@mozilla.com) from comment #43)
> For some reason, v184 has a bit more free space on /system but it still has
> the unneeded files in esp. /system/media/audio which free up more than 10MB
> of additional space.
> 
> Given that it's our reference device, we also should see that we reduce the
> fonts on those base images to the minimum required set not just for
> maximizing space but also for making sure we test with only the fonts
> installed that we know will be present in target devices.

Agree, that way any use of older font sets will be more apparent.
(In reply to Kai-Zhen Li [:seinlin] from comment #37)
> I got the answer why OTA update.mar mentioned in comment 36 didn't work in
> my device. Because my Flame is based on v184 and latest pvt flame-kk build
> and the OTA update.mar I used is for jb.
> 
> This time I try to use OTA update.mar from flame-kk (below url), it is
> working.
> http://update.boot2gecko.org/flame-kk/2.0.0/nightly/
> b2g_update_20141009000201.mar
> 
> So flame jb build should get update from flame OTA; flame-kk build should
> get update from flame-kk OTA.
> 
> 
> Gerry, could you help to verify this? Thank you!
Hi Kai-Zhen, 
You are right. 
If the flame is using jb, the url would be https://aus4.mozilla.org/update/3/B2G/32.0/20141008000202/flame/en-US/aurora/Boot2Gecko 2.0.0.0-prerelease (SDK 18)/default/default/update.xml and the package is under http://ftp.mozilla.org/pub/mozilla.org/b2g/nightly/2014/10/2014-10-08-00-02-01-mozilla-aurora-flame/b2g-flame-gecko-update.mar. 

If the flame is using kk, the url become https://aus4.mozilla.org/update/3/B2G/34.0a2/20141012153751/flame/en-US/aurora/Boot2Gecko 2.1.0.0-prerelease (SDK 19)/default/default/update.xml and the package is under http://ftp.mozilla.org/pub/mozilla.org/b2g/nightly/2014/10/2014-10-12-16-02-02-mozilla-aurora-flame-kk/b2g-flame-gecko-update.mar.

The difference between these two URL is the sdk version. The update package of jb is under flame/ and the other is under flame-kk/.
Flags: needinfo?(gchang)
DroidSansFallbackFull from the latest Android SDK download. I couldn't manage to pull it from any of my Flame sys.img files.

I put it on my Flame and renamed it from DroiSansFallbackfull to DroidSansFallback.

Diacritics, accents, and special characters APPEAR to be functional. Let me know if you guys are still missing stuff.
Hi Shawn:

Per all comments above, sounds like we need more system space to enable a smooth OTA, maybe by either

A. reduce unused/unnecessary stuff like some font (comment#24, comment#9+comment#21) and maybe also files under /system/media/audio? (comment 43), OR
B. modified memory map to adjust system partition size (never mentioned in above, not sure if this is recommended)

Would you help assign someone to verify again, if system space is the root cause? (I know Kaizhen has done some but still we see symptom reported)
And if yes can we use solution A or B to fix it? (I assume B should be done by vendor side but not sure about A).

Thank you.
Flags: needinfo?(sku)
Wesly, I think you are right. To solve this issue, we need to increase the free space in /system. Either A or B can meet this goal. From my test I found that the free space of v184 is enough, but if the free space can be increased a bit more could be better. Because the size of OTA package could be bigger in the future.
Flags: needinfo?(sku)
Thanks for Kai-zhen's prompt feedback, as F2F discussed I will get vendor's support to

1. enlarge system partition, target 400MB (currently 354.2M in v184) with space from /data (currently 2G). By doing this we should be able to get ~150MB free in /system, which is around 2 times of current OTA package size (per figures in comment#38).

2. check and ensure only font set mentioned in https://github.com/mozilla-b2g/moztt/blob/master/fonts.mk is used.

@Robert and Frederik, you both mentioned (comment#43 & comment#4) that some file under system/media/audio can be removed to free ~10MB, would you kindly put more detail here so I can get vendor to follow? Thank you.
Flags: needinfo?(kli)
Flags: needinfo?(kairo)
Flags: needinfo?(fbraun)
Flags: needinfo?(kli)
Hi Youlong, pls see comment#49 and I will discuss it with you later in phone, thanks.
Flags: needinfo?(youlong.jiang)
This is not my area of expertise and I would prefer not to be the person who makes the call what is safe to remove and what is not. Please defer this to a gonk engineer.
What I described in earlier comments was nothing but a workaround.
Flags: needinfo?(fbraun)
Thanks Frederik's response, my misunderstanding here.

@Robert: I'm also offline seeking Gonk team's support now, but still much appreciate if you could give us some light here. Thank you.

@Youlong, as phone discussion earlier today, we must secure your FOTA mechanism (which differs from Mozilla's OTA) works if change memory partition layout, so ensure there won't be trouble for normal users to FOTA upgrade from current official JB SW to the coming SW with new memory map. Thanks.
(In reply to Wesly Huang from comment #49)
> Thanks for Kai-zhen's prompt feedback, as F2F discussed I will get vendor's
> support to
> 
> 1. enlarge system partition, target 400MB (currently 354.2M in v184) with
> space from /data (currently 2G). By doing this we should be able to get
> ~150MB free in /system, which is around 2 times of current OTA package size
> (per figures in comment#38).
> 
> 2. check and ensure only font set mentioned in
> https://github.com/mozilla-b2g/moztt/blob/master/fonts.mk is used.
> 
> @Robert and Frederik, you both mentioned (comment#43 & comment#4) that some
> file under system/media/audio can be removed to free ~10MB, would you kindly
> put more detail here so I can get vendor to follow? Thank you.

about partition modify solution, in our opinion, it's not workable for this issue, by the cause of we won't update partition data in our fota upgrade.

we'll try to correct fonts config per link of fonts.mk. then observe effect in next version.

tks.
Flags: needinfo?(youlong.jiang)
Youlong, 

Is it possible to flash a new base(full) image to modify the size of /system partition?

I think you also need have a look into "/system/media", 
per comment 43 some space could be freed from there.
Flags: needinfo?(youlong.jiang)
hi all -

after check latest image version with modify fonts.mk, found some problems here:

1. just confirm base v187 version ,system partition usage

Filesystem               Size     Used     Free   Blksize
/system                354.2M   247.1M   107.1M   4096

so pls help to verify by using v187 + Pvt build

2. modify system fonts config, ui overlap issue exist in dialer and system fonts changed,

modification as follow:

remove:

    external/moztt/FiraMono-2.001/FiraMonoOT-Bold.otf:system/fonts/FiraMonoOT-Bold.otf \
    external/moztt/FiraMono-2.001/FiraMonoOT-Regular.otf:system/fonts/FiraMonoOT-Regular.otf \
    external/moztt/FiraMono-2.001/FiraMono-Bold.otf:system/fonts/FiraMono-Bold.otf \
    external/moztt/FiraMono-2.001/FiraMono-Medium.otf:system/fonts/FiraMono-Medium.otf \
    external/moztt/FiraMono-2.001/FiraMono-Regular.otf:system/fonts/FiraMono-Regular.otf \
    external/moztt/FiraSans-2.001/FiraSansOT-Bold.otf:system/fonts/FiraSansOT-Bold.otf \
    external/moztt/FiraSans-2.001/FiraSansOT-BoldItalic.otf:system/fonts/FiraSansOT-BoldItalic.otf \
    external/moztt/FiraSans-2.001/FiraSansOT-Light.otf:system/fonts/FiraSansOT-Light.otf \
    external/moztt/FiraSans-2.001/FiraSansOT-LightItalic.otf:system/fonts/FiraSansOT-LightItalic.otf \
    external/moztt/FiraSans-2.001/FiraSansOT-Medium.otf:system/fonts/FiraSansOT-Medium.otf \
    external/moztt/FiraSans-2.001/FiraSansOT-MediumItalic.otf:system/fonts/FiraSansOT-MediumItalic.otf \
    external/moztt/FiraSans-2.001/FiraSansOT-Regular.otf:system/fonts/FiraSansOT-Regular.otf \
    external/moztt/FiraSans-2.001/FiraSansOT-RegularItalic.otf:system/fonts/FiraSansOT-RegularItalic.otf \
    external/moztt/FiraSans-2.001/FiraSans-Bold.otf:system/fonts/FiraSans-Bold.otf \
    external/moztt/FiraSans-2.001/FiraSans-BoldItalic.otf:system/fonts/FiraSans-BoldItalic.otf \
    external/moztt/FiraSans-2.001/FiraSans-ExtraBold.otf:system/fonts/FiraSans-ExtraBold.otf \
    external/moztt/FiraSans-2.001/FiraSans-ExtraBoldItalic.otf:system/fonts/FiraSans-ExtraBoldItalic.otf \
    external/moztt/FiraSans-2.001/FiraSans-Hair.otf:system/fonts/FiraSans-Hair.otf \
    external/moztt/FiraSans-2.001/FiraSans-HairItalic.otf:system/fonts/FiraSans-HairItalic.otf \
    external/moztt/FiraSans-2.001/FiraSans-Heavy.otf:system/fonts/FiraSans-Heavy.otf \
    external/moztt/FiraSans-2.001/FiraSans-HeavyItalic.otf:system/fonts/FiraSans-HeavyItalic.otf \
    external/moztt/FiraSans-2.001/FiraSans-Italic.otf:system/fonts/FiraSans-Italic.otf \
    external/moztt/FiraSans-2.001/FiraSans-Light.otf:system/fonts/FiraSans-Light.otf \
    external/moztt/FiraSans-2.001/FiraSans-LightItalic.otf:system/fonts/FiraSans-LightItalic.otf \
    external/moztt/FiraSans-2.001/FiraSans-Medium.otf:system/fonts/FiraSans-Medium.otf \
    external/moztt/FiraSans-2.001/FiraSans-MediumItalic.otf:system/fonts/FiraSans-MediumItalic.otf \
    external/moztt/FiraSans-2.001/FiraSans-Regular.otf:system/fonts/FiraSans-Regular.otf \
    external/moztt/FiraSans-2.001/FiraSans-SemiBold.otf:system/fonts/FiraSans-SemiBold.otf \
    external/moztt/FiraSans-2.001/FiraSans-SemiBoldItalic.otf:system/fonts/FiraSans-SemiBoldItalic.otf \
    external/moztt/FiraSans-2.001/FiraSans-UltraLight.otf:system/fonts/FiraSans-UltraLight.otf \
    external/moztt/FiraSans-2.001/FiraSans-UltraLightItalic.otf:system/fonts/FiraSans-UltraLightItalic.otf \

also we found some fonts in your link not configured into our system:

    external/moztt/FiraMono-3.109/FiraMono-Bold.otf:system/fonts/FiraMono-Bold.otf \
    external/moztt/FiraMono-3.109/FiraMono-Medium.otf:system/fonts/FiraMono-Medium.otf \
    external/moztt/FiraMono-3.109/FiraMono-Regular.otf:system/fonts/FiraMono-Regular.otf \
    external/moztt/FiraSans-3.109/FiraSans-Bold.otf:system/fonts/FiraSans-Bold.otf \
    external/moztt/FiraSans-3.109/FiraSans-BoldItalic.otf:system/fonts/FiraSans-BoldItalic.otf \
    external/moztt/FiraSans-3.109/FiraSans-ExtraBold.otf:system/fonts/FiraSans-ExtraBold.otf \
    external/moztt/FiraSans-3.109/FiraSans-ExtraBoldItalic.otf:system/fonts/FiraSans-ExtraBoldItalic.otf \
    external/moztt/FiraSans-3.109/FiraSans-Hair.otf:system/fonts/FiraSans-Hair.otf \
    external/moztt/FiraSans-3.109/FiraSans-HairItalic.otf:system/fonts/FiraSans-HairItalic.otf \
    external/moztt/FiraSans-3.109/FiraSans-Heavy.otf:system/fonts/FiraSans-Heavy.otf \
    external/moztt/FiraSans-3.109/FiraSans-HeavyItalic.otf:system/fonts/FiraSans-HeavyItalic.otf \
    external/moztt/FiraSans-3.109/FiraSans-Italic.otf:system/fonts/FiraSans-Italic.otf \
    external/moztt/FiraSans-3.109/FiraSans-Light.otf:system/fonts/FiraSans-Light.otf \
    external/moztt/FiraSans-3.109/FiraSans-LightItalic.otf:system/fonts/FiraSans-LightItalic.otf \
    external/moztt/FiraSans-3.109/FiraSans-Medium.otf:system/fonts/FiraSans-Medium.otf \
    external/moztt/FiraSans-3.109/FiraSans-MediumItalic.otf:system/fonts/FiraSans-MediumItalic.otf \
    external/moztt/FiraSans-3.109/FiraSans-Regular.otf:system/fonts/FiraSans-Regular.otf \
    external/moztt/FiraSans-3.109/FiraSans-SemiBold.otf:system/fonts/FiraSans-SemiBold.otf \
    external/moztt/FiraSans-3.109/FiraSans-SemiBoldItalic.otf:system/fonts/FiraSans-SemiBoldItalic.otf \
    external/moztt/FiraSans-3.109/FiraSans-UltraLight.otf:system/fonts/FiraSans-UltraLight.otf \
    external/moztt/FiraSans-3.109/FiraSans-UltraLightItalic.otf:system/fonts/FiraSans-UltraLightItalic.otf \

so pls help to check these results.

tks.
hi all -

for external/moztt/FiraMono-3.109/*.otf resource, not implemented into our source code yet.

about system usage status in v184 as below:

Filesystem               Size     Used     Free   Blksize
/system                354.2M   251.0M   103.2M   4096

it free about 4MB by cutting off used fonts in v187.
Flags: needinfo?(youlong.jiang)
Hi Mike, Michael:

Need your advise here

1. per comment#55 there are 31 fonts to be removed from v184 if follow [1], but which will result in different font in UI like the screenshot of v187 SW I attached above. I thought this font removal is simply for size reduction but won't affect any UI appearance?

2. quickly check those about the 31 to-be-removed fonts, I realized 21 (the other 10 are Fira*OT ones) of them are actually requested before by Mike's bug 1069851, so there is conflict here. Which one we should follow? (try to installed 21 fonts per bug 1069851 but UI looks the same, so the main difference of v184 and v187 comes from those 10 Fira*OT fonts?)

3. per comment#55 there are also 21 fonts currently missed in v184, I assume we don't need to add them back since fonts in v184 seems works well and enough?

[1] https://github.com/mozilla-b2g/moztt/blob/master/fonts.mk
Flags: needinfo?(mwu)
Flags: needinfo?(mlien)
Just found those 31 to-be-removed fonts and 21 missing fonts have high similarity of font name, just different in some "version number" in file path, does that mean some integration issue here?
@Youlong:

Is it same member in your team that made this font change, as the member who worked on bug 1069851 before? Pls read my observation in comment#58 and comment#59 and see if this is something related to integration. (I'm no programmer but feel something strange here), thank you.
Flags: needinfo?(youlong.jiang)
(In reply to Wesly Huang from comment #60)
> @Youlong:
> 
> Is it same member in your team that made this font change, as the member who
> worked on bug 1069851 before? Pls read my observation in comment#58 and
> comment#59 and see if this is something related to integration. (I'm no
> programmer but feel something strange here), thank you.

hi wesly -

you mentioned before that what fonts you need is all just list in your link. so we remove external/moztt/FiraMono-2.001/*. but really external/moztt/FiraSans-3.109/* missing. so I think you should provide resource of this part and we'll integrate into our version. Last come back to the root point of this issue, if add external/moztt/FiraSans-3.109/* into system. I don't think this will save enough space for ota.

tks.
Flags: needinfo?(youlong.jiang)
(In reply to Wesly Huang from comment #58)
> Hi Mike, Michael:
> 
> Need your advise here
> 
> 1. per comment#55 there are 31 fonts to be removed from v184 if follow [1],
> but which will result in different font in UI like the screenshot of v187 SW
> I attached above. I thought this font removal is simply for size reduction
> but won't affect any UI appearance?
> 
> 2. quickly check those about the 31 to-be-removed fonts, I realized 21 (the
> other 10 are Fira*OT ones) of them are actually requested before by Mike's
> bug 1069851, so there is conflict here. Which one we should follow? (try to
> installed 21 fonts per bug 1069851 but UI looks the same, so the main
> difference of v184 and v187 comes from those 10 Fira*OT fonts?)
> 
> 3. per comment#55 there are also 21 fonts currently missed in v184, I assume
> we don't need to add them back since fonts in v184 seems works well and
> enough?
> 
> [1] https://github.com/mozilla-b2g/moztt/blob/master/fonts.mk

Removing fonts is tricky. Part of the problem is that 2.0 and 2.1 use different sets of fonts. If we upgrade to the 2.1 set of fonts, 2.0 will be broken. If we only use the 2.0 set of fonts, users will have to add the 2.1 fonts manually.

In general, I don't recommend removing any fonts except for Clockopia and AndroidClock. If we want to make extra space, removing /system/media/audio is a safe bet. Another thing to check is to make sure our OTA updates aren't too big - non-production builds of gaia often come with many unnecessary apps.
Flags: needinfo?(mwu)
(In reply to Wesly Huang from comment #49)
> @Robert and Frederik, you both mentioned (comment#43 & comment#4) that some
> file under system/media/audio can be removed to free ~10MB, would you kindly
> put more detail here so I can get vendor to follow? Thank you.

I removed the whole /system/media/audio directory as all those are files we do not use in FxOS from what I can tell, they seem to be Android files.
Flags: needinfo?(kairo)
(In reply to Wesly Huang from comment #58)
> Hi Mike, Michael:
> 
> Need your advise here
> 
> 1. per comment#55 there are 31 fonts to be removed from v184 if follow [1],
> but which will result in different font in UI like the screenshot of v187 SW
> I attached above. I thought this font removal is simply for size reduction
> but won't affect any UI appearance?
> 
> 2. quickly check those about the 31 to-be-removed fonts, I realized 21 (the
> other 10 are Fira*OT ones) of them are actually requested before by Mike's
> bug 1069851, so there is conflict here. Which one we should follow? (try to
> installed 21 fonts per bug 1069851 but UI looks the same, so the main
> difference of v184 and v187 comes from those 10 Fira*OT fonts?)
> 
> 3. per comment#55 there are also 21 fonts currently missed in v184, I assume
> we don't need to add them back since fonts in v184 seems works well and
> enough?
> 
> [1] https://github.com/mozilla-b2g/moztt/blob/master/fonts.mk

Just like Michael mentioned, the mainly differences are sets of fonts for different versions, thus I also don't recommend remove fonts to get free space
Flags: needinfo?(mlien)
verify with Flame KK base image v188 and flash gaia/gecko -> OTA can be executed and update system succefully
Thank Michael, Robert, and Mike's reply!

At least now v188 with Gaia/Gecko shallow flash is able to perform OTA, so as a short term we should be able to move with the SW for a while. However for long term I see the remained questions are:

A. make more space for /system? possible solutions including:

1. memory partition adjustment, tech. doable, but not valid for vendor's official SW FOTA mechanism (though user can manually flash SW but this will cause trouble for normal Flame users who rely on FOTA, which put this one as last option so far)
2. remove fonts, experiment fail in v187, not recommended
3. remove whole /system/media/audio directory? worth a try
4. any others?

B. avoid system image grow
1. per previous comments it could be a case that a fail OTA attempt will consume some system space, maybe some mechanism to "clean" those files before perform OTA?
2. Any other suggestion?

My plan is to focus on A.3 first, then see what we can do for A.4. Pls feel free to let me know if any suggestion/comment.
Sorry if this is a stupid question but where the base image v188?
Sorry I meant: Where can we get the base image v188?
I'm composing mail and will send out to b2g-internal soon.
Blocks: 1085230
Since v188 is already had enough space to update from comment 65, mark as WORKSFORME
Nominating another issue bug1085230 for further /system partition tracking
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → WORKSFORME
Where can one get v188?
https://developer.mozilla.org/en-US/Firefox_OS/Developer_phone_guide/Flame#Updating_your_Flame%27s_software has still just a link to v180.
Flags: needinfo?(mlien)
(In reply to Olli Pettay [:smaug] from comment #71)
> Where can one get v188?
> https://developer.mozilla.org/en-US/Firefox_OS/Developer_phone_guide/
> Flame#Updating_your_Flame%27s_software has still just a link to v180.

The 188 build is still in testing. As soon as it's vetted, it'll be posted to the MDN Flame page. If you want to be notified of that, you can subscribe to the Flame News list linked at the top of the MDN page.
Flags: needinfo?(mlien)
Seems fixed using v188 base image and 2.2 with today's OTA update.
Tested with today's OTA update + v188 base image and downloaded/uncompressed and installed successfully.

Looks to be solved.

Like the new updating graphic on the 2.2 build, nice work!
I reflashed with v188. The shallow flashed the aurora channel. After that a couple of OTAs worked, but now it seems to be failing again.
> I reflashed with v188. The shallow flashed the aurora channel. After that a couple of OTAs worked, but now it seems to be failing again.

Actually must have been wrong.. or at least updates are working now.
Still happening on v188
With v188,  2.2, and today's OTA, it works fine
Using base v188, FxOS 2.0, and today's OTA went fine for me too.
Note that after a long time I've just hit this bug once again on my dogfooding Flame. My system partition currently has 88.3MiB free, I'll see if tomorrow update works or not. This is the relevant output from  df:

adb shell df | grep "\(^Filesystem\|^/system\)"

Filesystem               Size     Used     Free   Blksize
/system                354.2M   265.8M    88.3M   4096
:gsvelto: The issue for the v188 base image is being tracked in bug 1093827.
(In reply to Archaeopteryx [:aryx] from comment #81)
> :gsvelto: The issue for the v188 base image is being tracked in bug 1093827.

Thanks for the pointer, I wasn't CC'd there yet.
You need to log in before you can comment on or make changes to this bug.