1182591 - [Aries][Camera] Accessing Share Activities within Camera-Preview images is very slow to show menu

Status: VERIFIED FIXED

(Firefox OS Graveyard :: Gaia::Camera, defect)

Description

[Oliver Nelson [:oliverthor]]

Attachment: logcat_20150710_1132.txt

Description:
When the user has taken pictures in their camera and is viewing them in their preview, attempting to access the share activities for them will result in waiting through a long wait for the menu to appear. This appears to be a device regression from Flame to Aries devices, as Flame appears to respond at an appropriate speed.
May be worth noting, that Aries Share menu has significantly more options (* = flame device supported):
*- Messages
*- Wallpaper
- Facebook
- Studio
- RunWhatsApp(Preview)
*- E-Mail
*- Bluetooth


Repro Steps:
1) Update a Aries to 20150710105517
2) Open the Camera app
3) Take a picture
4) Tap the 'share activity' button
5) Observe wait time

Actual:
Aries devices has a significant wait for the share menu to occur

Expected:
All devices have an appropriately quick wait time for share menu to occur


Environmental Variables:
-------------------------

Device: Aries 2.5
Build ID: 20150710105517
Gaia: ad76c159c641c977d9140c5fedea84aea04e0e60
Gecko: f7e1f596d57d
Gonk: 2916e2368074b5383c80bf5a0fba3fc83ba310bd
Version: 42.0a1 (2.5)
Firmware Version: D5803_23.1.A.1.28_NCB.ftf
User Agent: Mozilla/5.0 (Mobile; rv:42.0) Gecko/42.0 Firefox/42.0
******************

Issue DOES NOT REPRODUCE on master for flame devices
Results: Flame device has an appropriately quick wait time for share menu to occur

Device: Flame 2.5
BuildID: 20150710010203
Gaia: ad76c159c641c977d9140c5fedea84aea04e0e60
Gecko: 2c91d57441fd
Gonk: a4f6f31d1fe213ac935ca8ede7d05e47324101a4
Version: 42.0a1 (2.5) 
Firmware Version: v18D
User Agent: Mozilla/5.0 (Mobile; rv:42.0) Gecko/42.0 Firefox/42.0
-------------------------


Repro frequency: 5/5
See attached: 
video- https://youtu.be/q_mgA4naqHc
logcat

Comment 1

[Peter Bylenga [:PBylenga]]

[Blocking Requested - why for this release]:
performance regression from flame behavior

Comment 2

[Hema Koka [:hema]]

djf: any input on why we are seeing slowness with aries when initiating share from camera?

Comment 3

[David Flanagan [:djf]]

The Aries camera produces images that are not properly rotated and rely on the EXIF orientation flag. EXIF orientation is not well supported on the web, so our design decision (back in the Tarako days) was that only Camera and Gallery would be EXIF-aware, and we would not allow any photos that relied on EXIF to leak out to any of our other apps.  (Otherwise, Contacts and Messages, for example, would have to be modified to be EXIF-aware)

What this means is that before sharing an image we have to decode it, rotate it manually, and re-encode it. Photos on Aries are 20mp, and they decode to 80 megabytes, so decoding, rotating and encoding takes some time.

I believe that this is what the reporter is seeing in this bug. If that is the case, then photos taken in landscape (or upside down landscape) should not have this delay. If that is what we are seeing, it confirms my diagnosis above.  Also, if my diagnosis is correct, then you should see exactly the same issue when sharing single portrait-mode photos from Gallery. (Sharing multiple photos from gallery skips the rotation step, so there should not be a delay in that case.)

I don't know that there is anything we can do about it.  Ideally, we'd have different camera firmware that did the rotation correctly and did not use EXIF.  If we reduced the default image size, that would speed things up, but probably make people unhappy.

There may not be anything we can do but display a spinner while we do the rotation. I think Gallery may already have code for the spinner in this case, but maybe we need to add one to Camera?

Needinfo the bug reporter to check gallery and to check on differences between landscape, upsidedown landscape and portait mode photos.

Comment 4

[Hubert Figuiere [:hub]]

(In reply to David Flanagan [:djf] from comment #3)

> What this means is that before sharing an image we have to decode it, rotate
> it manually, and re-encode it. Photos on Aries are 20mp, and they decode to
> 80 megabytes, so decoding, rotating and encoding takes some time.

There is a JPEG rotation method that does not involve decompression and recompression of the image.
We should use that.
(jpegtran is the name of the command-line tool - it is part of libjpeg)


This should provide all sorts of enhancement to use that method. The code should be emscriptable.

Comment 5

[Oliver Nelson [:oliverthor]]

(In reply to David Flanagan [:djf] from comment #3)
> The Aries camera produces images that are not properly rotated and rely on
> the EXIF orientation flag. EXIF orientation is not well supported on the
> web, so our design decision (back in the Tarako days) was that only Camera
> and Gallery would be EXIF-aware, and we would not allow any photos that
> relied on EXIF to leak out to any of our other apps.  (Otherwise, Contacts
> and Messages, for example, would have to be modified to be EXIF-aware)
> 
> What this means is that before sharing an image we have to decode it, rotate
> it manually, and re-encode it. Photos on Aries are 20mp, and they decode to
> 80 megabytes, so decoding, rotating and encoding takes some time.
> 

This appears to be exactly the case. This creates a very visible issue demonstrated in this video:

* https://youtu.be/q5Wh8gSEMpo

After attempting to share, gesturing to a separate preview, than gesturing back: the portrait orientation image is now skewed with a landscape orientation after going through a rotation re-encoding.


> I believe that this is what the reporter is seeing in this bug. If that is
> the case, then photos taken in landscape (or upside down landscape) should
> not have this delay. If that is what we are seeing, it confirms my diagnosis
> above.  Also, if my diagnosis is correct, then you should see exactly the
> same issue when sharing single portrait-mode photos from Gallery. 
> 

I can create another video if you would like, but this I can confirm this is exactly what's happening. Landscape photos are not experiencing delays in opening share activities in either Camera or Gallery. 

> 
> There may not be anything we can do but display a spinner while we do the
> rotation. I think Gallery may already have code for the spinner in this
> case, but maybe we need to add one to Camera?
> 

Gallery is indeed displaying a spinner due to the load time BUT the load is not as severe as in Camera Preview.

Let me know if there's something else you'd like me to investigate, or potentially log another bug for the visual fault observed just above in the video.

Comment 6

[David Flanagan [:djf]]

Oliver: yes, please file another bug for the issue in https://youtu.be/q5Wh8gSEMpo

Justin or Andrew: could one of you get a spinner to display in camera for this case?  And maybe also work with Oliver on the new bug he files?  I suspect the issue in that other bug is with the rotation metadata that is being passed to MediaFrame in the camera's preview gallery code.

Comment 7

[Andrew Osmond [:aosmond] (he/him)]

I can do it.

Comment 8

[GitHub Autolander Bot]

Attachment: [gaia] aosmond:bug1182591 > mozilla-b2g:master

Comment 9

[Oliver Nelson [:oliverthor]]

(In reply to David Flanagan [:djf] from comment #6)
> Oliver: yes, please file another bug for the issue in
> https://youtu.be/q5Wh8gSEMpo
> 

I had already bugged this issue, the day I had written this as well it seems:
* https://bugzilla.mozilla.org/show_bug.cgi?id=1182557

It was resolved fixed but appears to still be reproducing. I have reopened the bug and included a new video + logcat.

Comment 10

[Justin D'Arcangelo [:justindarc]]

Comment on attachment 8648907 [details] [review]
[gaia] aosmond:bug1182591 > mozilla-b2g:master

LGTM, thanks!

Comment 11

[Ryan VanderMeulen [:RyanVM]]

Master: https://github.com/mozilla-b2g/gaia/commit/a27569599434ab24f60eed550747c77b26ea49df

Comment 12

[Cola, OOO till Oct. 8th]

Hi, Andrew

This bug has been verified as fail on latest build of Aries master by the STR in Comment 0.  
Could you help to check this bug again? thank you very much.

Actual results:Aries devices has a significant wait for the share menu to occur.

See attachments: Aries v2.5.3gp,logcat001.txt

Note:The issue cannot reproduce in landscape mode.

Reproduce rate: 5/5

Device: Aries KK 2.5(Fail)
Build ID               20150823221817
Gaia Revision          cddb9f610cbe03d0ca39d81bbdce46a0fca841ab
Gaia Date              2015-08-23 03:34:38
Gecko Revision         https://hg.mozilla.org/mozilla-central/rev/4ccdd06e51d7209ba429196df7cab97bf66962db
Gecko Version          43.0a1
Device Name            aries
Firmware(Release)      4.4.2
Firmware(Incremental)  eng.worker.20150823.214038
Firmware Date          Sun Aug 23 21:40:46 UTC 2015
Bootloader             s1

Comment 13

[Cola, OOO till Oct. 8th]

Attachment: logcat001.txt

Comment 14

[Cola, OOO till Oct. 8th]

Attachment: Aries v2.5.3gp

Comment 15

[Andrew Osmond [:aosmond] (he/him)]

(In reply to Cola from comment #12)
> Hi, Andrew
> 
> This bug has been verified as fail on latest build of Aries master by the
> STR in Comment 0.  
> Could you help to check this bug again? thank you very much.
> 
> Actual results:Aries devices has a significant wait for the share menu to
> occur.
> 
> See attachments: Aries v2.5.3gp,logcat001.txt
> 
> Note:The issue cannot reproduce in landscape mode.
> 
> Reproduce rate: 5/5
> 
> Device: Aries KK 2.5(Fail)
> Build ID               20150823221817
> Gaia Revision          cddb9f610cbe03d0ca39d81bbdce46a0fca841ab
> Gaia Date              2015-08-23 03:34:38
> Gecko Revision        
> https://hg.mozilla.org/mozilla-central/rev/
> 4ccdd06e51d7209ba429196df7cab97bf66962db
> Gecko Version          43.0a1
> Device Name            aries
> Firmware(Release)      4.4.2
> Firmware(Incremental)  eng.worker.20150823.214038
> Firmware Date          Sun Aug 23 21:40:46 UTC 2015
> Bootloader             s1

Cola, unfortunately speeding it up is difficult, that's why we added the spinner if resizing takes longer than 100ms. Reducing the picture size / quality would be the only other option at this point to my knowledge.

Comment 16

[Shally Li（leave from Mozilla）]

> that's why we added the
> spinner if resizing takes longer than 100ms. Reducing the picture size /
> quality would be the only other option at this point to my knowledge.

Per comment 12 and comment 15, change the status to "verified".