973046 - [perf][Tarako] Unable to flash Tarako - stuck in fastboot trying to flash/send 'system' (95085 KB)

Status: VERIFIED FIXED

(Firefox OS Graveyard :: General, defect)

Description

[Stephen Donner [:stephend] Not actively reading bugmail]

Raymond and I set up a new Tarako device (properly flashed with the vendor's base build), and attempted to resurrect the b2g.tarako.mozilla-aurora.v1.3.perf job; in doing so, we ran into the same problem, apparently, that caused the 1st device we set up to suddenly fail: when the 1.3 build-flash job runs, it tries to push the "system" app/files, like so:

13:01:02 sending 'system' (95085 KB)...

Both devices are stuck, here, in fastboot mode -- is the image too big for the device, perhaps?  There's a note at the bottom of https://intranet.mozilla.org/B2G_Team/Tarako: "Fastboot can flash system.img < 100MB only; Otherwise the device will be bricked. In case of device is bricked, it can be recovered by using flashing tool to flash a base builds."

Al, are you seeing this too? :-(

Comment 1

[Stephen Donner [:stephend] Not actively reading bugmail]

http://qa-selenium.mv.mozilla.com:8080/view/B2G%20Tarako/job/b2g.tarako.mozilla-aurora.v1.3.perf/112/console is the most recent flash attempt, which fails.

Comment 2

[Al Tsai [:atsai]]

For what I knew, Partner set the partition as 100MB, which should around 110000000 Bytes. The size of system image seems available for flashing. set the NI to Thomas Tsai and Danny to see if they have any suggestion.

Also, you will need to update kernel image for building. You will need to run the command once "./repo kernel update"

Comment 3

[Danny Liang [:dliang]]

Attachment: u-boot.bin

There is a bug in u-boot that caused flash system.ing fail. sprd has fixed this bug and please update u-boot then flash system.img again.
How to update u-boot:
#adb reboot bootloader
#fastboot flash 2ndbl u-boot.bin
#fastboot reboot

With the latest u-boot, the limitation of flash size is around 105M. By my test, I can flash the system.img with size is 101M.

Stephen, could you help to update u-boot then flash system.img again?

Comment 4

[Dave Hunt [:davehunt] [he/him] ⌚BST]

I just tried the suggestion in comment 3 but I believe the device is in a bad state that will first need to be resolved. The fastboot commands are all returning with "< waiting for device >". Can we recover using the flashing tool as mentioned in https://intranet.mozilla.org/B2G_Team/Tarako#Q.26A

Comment 5

[Danny Liang [:dliang]]

(In reply to Dave Hunt (:davehunt) from comment #4)
> I just tried the suggestion in comment 3 but I believe the device is in a
> bad state that will first need to be resolved. The fastboot commands are all
> returning with "< waiting for device >". Can we recover using the flashing
> tool as mentioned in https://intranet.mozilla.org/B2G_Team/Tarako#Q.26A

I think your issue might be another one. May I have some questions to identify your issue?
1. Could you recognize the device by lsusb?
   If it's recognized, it will show something like "Bus 001 Device 073: ID 1782:5d05 Spreadtrum Communications Inc."
2. Do you follow https://intranet.mozilla.org/B2G_Team/Tarako#Q.26A to setup your environment?
3. How is the status of adb connection?
4. Does the screen show "fastboot mode" when you try fastboot command?

Comment 6

[Dave Hunt [:davehunt] [he/him] ⌚BST]

I'll leave :stephend to answer these or delegate as I don't have physical access to the device.

Comment 7

[Stephen Donner [:stephend] Not actively reading bugmail]

Here are, I believe (Raymond can correct me) the udev rules we're using:

webqa@b2g-6:~$ cat /etc/udev/rules.d/51-android.rules 
SUBSYSTEM=="usb", ATTR{idVendor}=="18d1", MODE="0666", GROUP="plugdev"
SUBSYSTEM=="usb", ATTR{idVendor}=="19d2", MODE="0666", GROUP="plugdev"
SUBSYSTEM=="usb", ATTR{idVendor}=="05c6", MODE="0666", GROUP="plugdev"
SUBSYSTEM=="usb", ATTR{idVendor}=="1782", ATTR{idProduct}=="5d06", MODE="0666", GROUP="plugdev"

(In reply to Danny Liang [:dliang] from comment #5)
> I think your issue might be another one. May I have some questions to
> identify your issue?
> 1. Could you recognize the device by lsusb?
>    If it's recognized, it will show something like "Bus 001 Device 073: ID
> 1782:5d05 Spreadtrum Communications Inc."

It's not currently recognized, no.

> 2. Do you follow https://intranet.mozilla.org/B2G_Team/Tarako#Q.26A to setup
> your environment?

Yes, I believe Raymond did.

> 3. How is the status of adb connection?

adb devices doesn't return anything

> 4. Does the screen show "fastboot mode" when you try fastboot command?

Nope :-(

webqa@b2g-6:~$ fastboot mode
< waiting for device >

Comment 8

[Danny Liang [:dliang]]

(In reply to Stephen Donner [:stephend] from comment #7)
> Here are, I believe (Raymond can correct me) the udev rules we're using:
> 
> webqa@b2g-6:~$ cat /etc/udev/rules.d/51-android.rules 
> SUBSYSTEM=="usb", ATTR{idVendor}=="18d1", MODE="0666", GROUP="plugdev"
> SUBSYSTEM=="usb", ATTR{idVendor}=="19d2", MODE="0666", GROUP="plugdev"
> SUBSYSTEM=="usb", ATTR{idVendor}=="05c6", MODE="0666", GROUP="plugdev"
> SUBSYSTEM=="usb", ATTR{idVendor}=="1782", ATTR{idProduct}=="5d06",
> MODE="0666", GROUP="plugdev"
> 
Suggest to add two more lines in /etc/udev/rules.d/51-android.rules
SUBSYSTEM=="usb", ATTR{idVendor}=="1782", ATTR{idProduct}=="5d04", MODE="0666", GROUP="plugdev"
SUBSYSTEM=="usb", ATTR{idVendor}=="1782", ATTR{idProduct}=="5d05", MODE="0666", GROUP="plugdev"

And try to plugin/plugout the cable few times. In my experiment, sometimes device will be disconnected, and sometimes is good.

Comment 9

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to Danny Liang [:dliang] from comment #8)
> (In reply to Stephen Donner [:stephend] from comment #7)
> > Here are, I believe (Raymond can correct me) the udev rules we're using:
> > 
> > webqa@b2g-6:~$ cat /etc/udev/rules.d/51-android.rules 
> > SUBSYSTEM=="usb", ATTR{idVendor}=="18d1", MODE="0666", GROUP="plugdev"
> > SUBSYSTEM=="usb", ATTR{idVendor}=="19d2", MODE="0666", GROUP="plugdev"
> > SUBSYSTEM=="usb", ATTR{idVendor}=="05c6", MODE="0666", GROUP="plugdev"
> > SUBSYSTEM=="usb", ATTR{idVendor}=="1782", ATTR{idProduct}=="5d06",
> > MODE="0666", GROUP="plugdev"
> > 
> Suggest to add two more lines in /etc/udev/rules.d/51-android.rules
> SUBSYSTEM=="usb", ATTR{idVendor}=="1782", ATTR{idProduct}=="5d04",
> MODE="0666", GROUP="plugdev"
> SUBSYSTEM=="usb", ATTR{idVendor}=="1782", ATTR{idProduct}=="5d05",
> MODE="0666", GROUP="plugdev"
> 
> And try to plugin/plugout the cable few times. In my experiment, sometimes
> device will be disconnected, and sometimes is good.

1) Done
2) Done

Still, no dice; the device isn't appearing when issuing lsusb, even (same host machine, same USB cable, etc.).

We even did webqa@b2g-6:~$ sudo udevadm control --reload-rules for good measure :-(

We understand this is pretty important to the Perf team, so perhaps we can make a local, consolidated effort?  cc:ing Tony and Fabrice for potential outreach help.

Comment 10

[Ting-Yu Chou [:ting] (away)]

Not sure if this helps, but I have a "~/.android/adb_usb.ini" with string "0x1782" (idVendor of Spreadtrum) on my Ubuntu.

Comment 11

[thomas tsai]

Sometimes usb device is not found by lsusb, so it can't be solved by any setting.
Sometimes usb device is found by lsusb, but adb devices can not find any devices.
So not the only one condition happens in this case.

Comment 12

[James Zhang (Spreadtrum)]

(In reply to Ting-Yu Chou [:ting] from comment #10)
> Not sure if this helps, but I have a "~/.android/adb_usb.ini" with string
> "0x1782" (idVendor of Spreadtrum) on my Ubuntu.

Please use this method or use spreadtrum download tool in Win7.

Comment 13

[James Zhang (Spreadtrum)]

Thomas, I think you should distribute the spreadtrum research download tool to your guys.

Comment 14

[thomas tsai]

Please refer to internal wiki : https://intranet.mozilla.org/B2G_Team/Tarako
about how to install pac tool and usb driver

Comment 15

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

It turns out that you are in adb mode when this occurs.  You can do a:
adb reboot bootloader

To get into fastboot and then flash a build that has a system.img that is < 100 MB.
For the automation, we should probably have a check to see what the file size of the system.img is before we flash.

For the b2g, perhaps we should have a check to see how big the system.img is before we try to flash for tarako?

You don't necessarily have to use the tool to reset it to the base image.

Comment 16

[James Ho]

Thomas, could you find someone to verify this bug again given Stephen is on PTO?

Comment 17

[thomas tsai]

Hi Stephen, can you update the new build again and test the usb issue?
By https://bugzilla.mozilla.org/show_bug.cgi?id=975232
SPRD has improved the usb compatibilty. The USB compatible issue highly depends on the PC host, so if your adb/fastboot still doesn't work by a new build, please need info me again.

Comment 18

[James Zhang (Spreadtrum)]

Is this bug fixed on my side?

commit 27a4b15ae8c526f538409504acb86643c0cd4daa
Author: lianxiang.zhou <[e-mail]>
Date:   Mon Mar 3 15:25:24 2014 +0800

    Bug #275374 usb tethering can not work
    
    [bug number  ] 275374
    [root cause  ]
    [changes     ]
    [side effects]
    [reviewers   ]
    
    Change-Id: Id490f6450d6f755e94b97b1e5adfa5c1e722af34

Comment 19

[thomas tsai]

James: What do you mean "fixed in my side" ?

Comment 20

[Stephen Donner [:stephend] Not actively reading bugmail]

Comment on attachment 8378090 [details]
u-boot.bin

Once we have a new build from Release Engineering, with the correct size (so it'll fully flash), we'll confirm.

Comment 21

[Jonathan Griffin (:jgriffin)]

Release engineering is producing flashable builds now; can we try to get this running again?

Comment 22

[James Zhang (Spreadtrum)]

(In reply to Jonathan Griffin (:jgriffin) from comment #21)
> Release engineering is producing flashable builds now; can we try to get
> this running again?

What about the system image size? It should be less than 105MB.

Comment 23

[James Zhang (Spreadtrum)]

Add Thomas and Kai-Zhen here.

Comment 24

[thomas tsai]

system.img is now 97MB (user build)  with woff fonts and english, Bengali locales.
Here are the fonts included in the 97MB:
CharisSILCompact-BI.woff        DroidSerif-Italic.woff        Lohit-Kannada.woff
CharisSILCompact-B.woff         DroidSerif-Regular.woff       Lohit-Malayalam.woff
CharisSILCompact-I.woff         FiraMonoOT-Bold.otf           Lohit-Oriya.woff
CharisSILCompact-R.woff         FiraMonoOT-Regular.otf        Lohit-Punjabi.woff
DroidNaskh-Regular.woff         FiraSansOT-BoldItalic.otf     Lohit-Tamil.woff
DroidSansArmenian.woff          FiraSansOT-Bold.otf           Lohit-Telugu.woff
DroidSansEthiopic-Regular.woff  FiraSansOT-LightItalic.otf    MTLc3m.woff
DroidSansFallback.woff          FiraSansOT-Light.otf          MTLmr3m.woff
DroidSansGeorgian.woff          FiraSansOT-MediumItalic.otf   Padauk-bold.woff
DroidSansHebrew-Bold.woff       FiraSansOT-Medium.otf         Padauk.woff
DroidSansHebrew-Regular.woff    FiraSansOT-RegularItalic.otf  Roboto-BoldItalic.ttf
DroidSansMono.woff              FiraSansOT-Regular.otf        Roboto-Bold.ttf
DroidSansThai.woff              Lohit-Bengali.woff            Roboto-Italic.ttf
DroidSerif-BoldItalic.woff      Lohit-Devanagari.woff         Roboto-Regular.ttf
DroidSerif-Bold.woff            Lohit-Gujarati.woff

Comment 25

[Kai-Zhen Li [:seinlin][:kli]]

Hi Stephen, After update u-boot.bin, I think it is able to flash tarako in fastboot with system.img less than 105MB. Can we resolve this bug as fixed with [POVB] in white board?

Comment 26

[James Zhang (Spreadtrum)]

We will provide research download tool on Linux, Sam and our owner is testing it now.

Comment 27

[Stephen Donner [:stephend] Not actively reading bugmail]

Raymond, can you try flashing the Tarako with the base build (1st) and then 1.3T nightly build (2nd) again, today?  Thanks!

Comment 28

[raymond [:retornam] (inactive)]

(In reply to Stephen Donner [:stephend] from comment #27)
> Raymond, can you try flashing the Tarako with the base build (1st) and then
> 1.3T nightly build (2nd) again, today?  Thanks!

I've done both and connected the device to its node in Jenkins.

Comment 29

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to raymond [:retornam] (needinfo? me) from comment #28)
> (In reply to Stephen Donner [:stephend] from comment #27)
> > Raymond, can you try flashing the Tarako with the base build (1st) and then
> > 1.3T nightly build (2nd) again, today?  Thanks!
> 
> I've done both and connected the device to its node in Jenkins.

Thanks, Raymond!  James, Kal-Zhen, can one of you take this bug and resolve it as FIXED, now?  Thanks!

Comment 30

[Stephen Donner [:stephend] Not actively reading bugmail]

Verified; thanks -- this is fine in our CI/Jenkins instance, now.