[Camera] Face tracking circle is aligned to the right in Portrait mode

VERIFIED FIXED in 2.2 S10 (17apr)

Status

defect
VERIFIED FIXED
4 years ago
4 years ago

People

(Reporter: anshulj, Assigned: aosmond)

Tracking

({regression})

unspecified
2.2 S10 (17apr)
ARM
Gonk (Firefox OS)
Dependency tree / graph
Bug Flags:
in-testsuite +

Firefox Tracking Flags

(blocking-b2g:2.2+, b2g-v2.1 unaffected, b2g-v2.2 verified, b2g-master verified)

Details

(Whiteboard: [caf priority: p2][CR 808202])

Attachments

(4 attachments)

Reporter

Description

4 years ago
STR

1. Launch the camera(front or back)
2. Focus on a face

Observed: In the portrait mode (vertical) the face tracking circle is aligned to the right of the face.
Expected: The trace tracking circle should be centered on the face

Note that this issue is not seen if the phone is held in a landscape mode.
Reporter

Updated

4 years ago
Whiteboard: [caf priority: p2][CR 808202]

Comment 1

4 years ago
QA,

Please check this on flame. There were no camera UI changes that went into 2.2 that would have changed or broken the face tracking circle alignment. But can we first get this to reproduce on Flame.

NI Diego in the meantime as well.

Thanks
Hema
Flags: needinfo?(dmarcos)
Keywords: qawanted
Reporter

Comment 2

4 years ago
I can reproduce the issue on flame device as well.
I'm uploading a screenshot of this issue.
This issue occurs on Flame 3.0 and Flame 2.2.

Actual Results:  The face is tracked from the right.

Environmental Variables:
Device: Flame 3.0
BuildID: 20150325130912
Gaia: 508b8d48fb5ecf08bf0e5b4fef42bc48b770e7f2
Gecko: f40ee067d081
Version: 39.0a1 (3.0) 
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:39.0) Gecko/39.0 Firefox/39.0

Environmental Variables:
Device: Flame 2.2
BuildID: 20150325031209
Gaia: 48b93b6929c508f016e6f8254ddfa851f9feed59
Gecko: c0927c8ec63d
Version: 37.0 (2.2) 
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:37.0) Gecko/37.0 Firefox/37.0

This issue does NOT occur on Flame 2.1.  

Actual Results: The face is tracked from the center.

Environmental Variables:
Device: Flame 2.1
BuildID: 20150325043108
Gaia: 6f39e4e876152de1dcdcc0e7656197f22f105e4b
Gecko: c411ee193eed
Version: 34.0 (2.1) 
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:34.0) Gecko/34.0 Firefox/34.0
QA Whiteboard: [QAnalyst-Triage?]
Flags: needinfo?(ktucker)
Keywords: qawanted
QA Whiteboard: [QAnalyst-Triage?] → [QAnalyst-Triage+]
Flags: needinfo?(ktucker)
QA Contact: pcheng
The code that renders the face tracked has not changed in a while. Is the driver reporting different coordinates on the latest firmware? Where can I see a firmware change log? If tracked face coordinates have changed what's the rational behind it? Does this bug reproduces on any other device besides flame?
Flags: needinfo?(dmarcos) → needinfo?(jmercado)
b2g-inbound regression window:

Last Working Environmental Variables:
Device: Flame
BuildID: 20141217182050
Gaia: 7a865bd6163e9a0372861d27450b3434875ef5c1
Gecko: 655536bc9ccd
Version: 37.0a1 (2.2 Master)
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:37.0) Gecko/37.0 Firefox/37.0

First Broken Environmental Variables:
Device: Flame
BuildID: 20141217184854
Gaia: 7a865bd6163e9a0372861d27450b3434875ef5c1
Gecko: bea91a0b4dfb
Version: 37.0a1 (2.2 Master)
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:37.0) Gecko/37.0 Firefox/37.0

Gaia is the same so it's a Gecko issue.

Gecko pushlog:
http://hg.mozilla.org/integration/b2g-inbound/pushloghtml?fromchange=655536bc9ccd&tochange=bea91a0b4dfb

Caused by patches for Bug 1104913.
QA Whiteboard: [QAnalyst-Triage+] → [QAnalyst-Triage?]
Flags: needinfo?(ktucker)
It's either a gecko or a firmware issue (v18D-1?). If the problem doesn't reproduce on any other phone than flame is almost sure that it's a driver change.
Flags: needinfo?(pcheng)
I thought it has been established that this occurs on multiple devices since this was originally reported on a non-Flame device.
Flags: needinfo?(pcheng)
Andrew, can you take a look at this please? Looks like the work done on bug 1104913 might be the cause here.
QA Whiteboard: [QAnalyst-Triage?] → [QAnalyst-Triage+]
Depends on: 1104913
Flags: needinfo?(ktucker) → needinfo?(aosmond)
Blocking for regression
Assignee: nobody → aosmond
blocking-b2g: 2.2? → 2.2+
PiWei seems to have covered everything you wanted from me.  Of note, this issue is not likely related to a base since the same base was used to reproduce no 3.0/2.2 but not on 2.1.
Flags: needinfo?(jmercado)
Assignee

Comment 13

4 years ago
I was able to reproduce but it happens to me on all gecko/gaia branches, so I'm not sure if I actually fixed the problem. The calculation for where to put the face detection circle is off in gaia -- the top/left coordinate of the circle needs to be adjusted in order for the centre of that circle to be the same as the bounding rect given by the camera.

Note that the problem is not limited to portrait mode. Due to the rotation, in landscape mode the circle should have a vertical bias. You will therefore see the problem moving the face up and down the viewfinder instead of left and right.

piwei: Can you try out the attached gaia patch and confirm if this fixes the problem (and confirm again on which branches :))? If you are unable to reproduce the issue on 2.1 without this patch, then could you please provide a reproduction video for both 2.1 and 2.2/3.0 so I can see the difference between them? Thanks!
Flags: needinfo?(aosmond) → needinfo?(pcheng)
Video of issue NOT reproducing on v2.1:

https://www.youtube.com/watch?v=Dy87-hAl3dA

This is tested on:
Device: Flame 2.1 (KK, full flash, 319MB mem)
BuildID: 20150327001204
Gaia: 6f39e4e876152de1dcdcc0e7656197f22f105e4b
Gecko: f4d7c0e7292e
Gonk: b83fc73de7b64594cd74b33e498bf08332b5d87b
Version: 34.0 (2.1) 
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:34.0) Gecko/34.0 Firefox/34.0

------

Video of issue reproducing on v3.0:

https://www.youtube.com/watch?v=yHtw2Mlkyy4

This is tested on:
Device: Flame 3.0 Master (KK, full flash, 319MB mem)
BuildID: 20150330010204
Gaia: be25b16efa19bab8d54be08f8fe45dcc93bf93d0
Gecko: dfe60814eda7
Gonk: b83fc73de7b64594cd74b33e498bf08332b5d87b
Version: 39.0a1 (3.0 Master)
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:39.0) Gecko/39.0 Firefox/39.0

------

I'll post my result after applying the patch in the next comment. Keeping NI.
Assignee

Comment 15

4 years ago
With respect to gecko change in bug 1104913, I believe I now may see the connection. If the aspect ratio of the face bounding box given by the camera hardware is controlled by the selected video and/or preview size, it is possible that change exaggerated the flaw in the app rendering (two wrongs make a right!). While I am able to hit the problem in 2.1, different versions of flame support different preview/video sizes, which will affect the degree to which one is able to reproduce the problem.
The patch seems to have NOT fully fixed the issue. Most of the time the face tracking circle seems to put the circle fully centering the face, but sometimes it appears a little off to the left. Attaching a screenshot.

This behavior is observed on both 3.0 and 2.2.
Flags: needinfo?(pcheng)
Assignee

Comment 17

4 years ago
Comment on attachment 8585744 [details] [review]
[gaia] aosmond:bug1147076 > mozilla-b2g:master

dmarcos: I know we discussed changing this to a square, but rather than deal with UX changes for 2.2, I think we should leave it as a circle for now and just fix the positioning.

(In reply to Pi Wei Cheng [:piwei] from comment #16)
> Created attachment 8586911 [details]
> screenshot of issue reproducing after patch
> 
> The patch seems to have NOT fully fixed the issue. Most of the time the face
> tracking circle seems to put the circle fully centering the face, but
> sometimes it appears a little off to the left. Attaching a screenshot.
> 
> This behavior is observed on both 3.0 and 2.2.

Pi Wei, unfortunately, the face detection algorithm is not perfect. That is the right location according to the vendor camera code. From your screenshots and my own testing, it is a marked improvement from before.
Attachment #8585744 - Flags: review?(dmarcos)
(In reply to Pi Wei Cheng [:piwei] from comment #16)
> Created attachment 8586911 [details]
> screenshot of issue reproducing after patch
> 
> The patch seems to have NOT fully fixed the issue. Most of the time the face
> tracking circle seems to put the circle fully centering the face, but
> sometimes it appears a little off to the left. Attaching a screenshot.
> 
> This behavior is observed on both 3.0 and 2.2.

This attachment looks much better compared to the original issue without Andrew's patch. For 2.2, I think we can go with this fix. 

Andrew, since you will be out on jury duty, please work on getting the patch reviewed and getting it in for approval today. Also include Tif for a quick ux review. 

Thanks
Hema
Comment on attachment 8585744 [details] [review]
[gaia] aosmond:bug1147076 > mozilla-b2g:master

It looks good to me. Can we add unit tests for the calculateFaceCircle method?
Attachment #8585744 - Flags: review?(dmarcos) → review+
Assignee

Comment 21

4 years ago
Comment on attachment 8588843 [details] [review]
[gaia] aosmond:bug1147076-v2.2 > mozilla-b2g:v2.2

[Approval Request Comment]
[Bug caused by] (feature/regressing bug #): 1104913
[User impact] if declined: Detected face circles will be poorly placed in take picture mode.
[Testing completed]: Manually ensured face circle is better placed on screen in landscape/portrait modes on the front and back facing cameras for flame. Added unit tests verifying the result of the circle calculation.
[Risk to taking this patch] (and alternatives if risky): Minor; the math choosing the location/diameter of the circle is slightly tweaked, and well understood why it was wrong originally.
[String changes made]: N/A
Attachment #8588843 - Flags: approval-gaia-v2.2?(bbajaj)
Assignee

Comment 22

4 years ago
Comment on attachment 8588843 [details] [review]
[gaia] aosmond:bug1147076-v2.2 > mozilla-b2g:v2.2

Carry r+ from master pull request.
Attachment #8588843 - Flags: review+
Assignee

Comment 23

4 years ago
Set in-testsuite, added gaia unit tests for circle parameter calculations.
Flags: in-testsuite+
waiting for master landing here before acting on branch approval.
Autolander could not locate a review from a user within the suggested reviewer list. Either the patch author or the reviewer should be in the suggested reviewer list.
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
I thought that we had some failures crop up on taskcluster after this landed. I reverted this commit, but was mistaken and ended up re-landing it. Sorry about the churn here, re-landed: https://github.com/mozilla-b2g/gaia/commit/a8bc2fb65d0174fe24385ca650ec02d0f1117903
Attachment #8588843 - Flags: approval-gaia-v2.2?(bbajaj) → approval-gaia-v2.2+
This issue is verified fixed on the latest Nightly 3.0 and 2.2 builds.

Actual Results: Face tracking is centered on the face and not severely off to either side.

Environmental Variables:
Device: Flame 3.0
BuildID: 20150408010203
Gaia: 84cbd4391fb7175d5380fa72c04d68873ce77e6d
Gecko: 078128c2600a
Gonk: b83fc73de7b64594cd74b33e498bf08332b5d87b
Version: 40.0a1 (3.0) 
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:40.0) Gecko/40.0 Firefox/40.0

Environmental Variables:
Device: Flame 3.0
BuildID: 20150408010203
Gaia: 84cbd4391fb7175d5380fa72c04d68873ce77e6d
Gecko: 078128c2600a
Gonk: b83fc73de7b64594cd74b33e498bf08332b5d87b
Version: 40.0a1 (3.0) 
Firmware Version: v18D-1
User Agent: Mozilla/5.0 (Mobile; rv:40.0) Gecko/40.0 Firefox/40.0
Status: RESOLVED → VERIFIED
Flags: needinfo?(ktucker)
Flags: needinfo?(ktucker)
You need to log in before you can comment on or make changes to this bug.