1010638 - [MADAI][Multimedia] During video playback, power consumption was higher than android.

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Performance, defect, P2)

Description

[Jaemin Song]

User Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.137 Safari/537.36

Steps to reproduce:

1. Push the video contents(1080p) in device.
2. play video file.
3. During video playback, measure power consumption.


Actual results:

During video playback, power consumption of FireFoxOS was higher than android.
I have measured power consumption during about 3 minutes.
Result of measurements was like below.

[power consumption]
android: 211.56 mA
FireFox OS: 263.57 mA

Please let me know why power consumption is different in MADAI.


Expected results:

Power consumption of FireFoxOS should be similar with android.

Comment 1

[Gabriele Svelto [:gsvelto]]

Jaemin you should check if the hardware composer was working properly during video playback. We run into these problems in the past so this might account for the difference in power consumption you're seeing.

Comment 2

[Jaemin Song]

(In reply to Gabriele Svelto [:gsvelto] from comment #1)
> Jaemin you should check if the hardware composer was working properly during
> video playback. We run into these problems in the past so this might account
> for the difference in power consumption you're seeing.

If H/W composer is enabled, power consumption was 241.14 mA.
Power consumption is still higher about 30mA than android.
Other environment of development(like kernel and HAL layer) was same with android.
Thanks.

Comment 3

[bhavana bajaj [:bajaj]]

NI: mike lee to weigh in here for blocking decision. How bad is this from our existing power targets.

Comment 4

[Mike Lee [:mlee]]

We don't yet have power consumption targets so blocking 1.4 on this may be premature. Jon (ni'd), can help with additional steps to better understand what's been reported.

As an FYI, we've done initial power use baselining (bug 915714) and are currently building a performance server (bug 997165) and tests (997167) that we'll use to define power consumption targets in Q3. We're also working to integrate power usage data into our profiler (bug 928475) so we can better root cause issues like this.

Comment 5

[Preeti Raghunath(:Preeti)]

Wayne,

Does this block Madai?

Comment 6

[jongsoo.oh]

Battery life test is important topic these days especially internet and video.

Comment 7

[Preeti Raghunath(:Preeti)]

Seems like 25% higher so might be a concern.

Comment 8

[Jon Hylands [:jhylands]]

I have given battery harnesses and ammeters to Jinho Hwang (cc'ed). Since we don't have a MADAI device, we can't reproduce this on our side on that hardware. If you can reproduce this on a Nexus 4, we would be able to at least start the debug process on our end.

I'm not sure what else we can do with this at this point.

Which version of Android were you using to get these numbers?

Comment 9

[Jon Hylands [:jhylands]]

Preeti, given that this is on unsupported hardware, can we remove the 1.4+ flag?

Comment 10

[Preeti Raghunath(:Preeti)]

We should be able to remove. Let me just confirm from Wayne on timelines.

Comment 11

[Jason Smith [:jsmith]]

(In reply to Jon Hylands [:jhylands] from comment #8)
> I have given battery harnesses and ammeters to Jinho Hwang (cc'ed). Since we
> don't have a MADAI device, we can't reproduce this on our side on that
> hardware. If you can reproduce this on a Nexus 4, we would be able to at
> least start the debug process on our end.
> 
> I'm not sure what else we can do with this at this point.
> 
> Which version of Android were you using to get these numbers?

Kitkat.

(In reply to Preeti Raghunath(:Preeti) from comment #10)
> We should be able to remove. Let me just confirm from Wayne on timelines.

We should check to see if we can reproduce on our kitkat reference hardware before doing that.

Comment 12

[Wayne Chang [:wchang]]

1. We do not have madai devices.
2. We do not have an android build for flame for comparison.

If partner believes there's a problem with 1.4 on madai we should investigate, but if we dont see any regression in terms of power consumption during video playback when comparing 1.4 to 1.2/1.3 on the same device then I don't think we should block the 1.4 'release' with this bug.

(In reply to Preeti Raghunath(:Preeti) from comment #10)
> We should be able to remove. Let me just confirm from Wayne on timelines.

Comment 13

[jongsoo.oh]

We will check the memory consumption using Nexus4.
As soon as we take a result, we will update it.

Comment 14

[Jinho Hwang [:jeffhwang]]

Attachment: Android_DOU_test_on_Nexus4.png

Hi Jon, Here is the result 'Power consumption' test on Nexus4 tested by Ammeter you gave me.                                                              
Our Multimedia team and Power team they also do the same test on Nexus4 using our power test equipment.                                                   
The results recorded by Ammeter and our power equipment are similar as I reported you already.                                                            
                                                                                                                                                          
STP                                                                                                                                                       
 1. Airplane mode                                                                                                                                         
 2. Turn Bluetooth, Wifi, GPS and NFC OFF                                                                                                                 
 3. Set Brightness & Volume to maximum                                                                                                                    
 4. Put Full HD(1920*1080) video content to device                                                                                                        
 5. Play the content with full screen mode on Video app.                                                                                                  
 6. Only on Android, we disable H/W composer.                                                                                                             
                                                                                                                                                          
Results                                                                                                                                                   
                                                                                                                                                          
  Nexus4(average of 3times testing)                                                                                                                       
   - FxOS V1.4 : 440.92mA                                                                                                                                 
   - Android V4.3 : 409.14mA                                                                                                                              
                                                                                                                                                          
We found two interesting things on the result.                                                                                                            
                                                                                                                                                          
1. The result shows us that playing Full-HD video content with full screen mode                                                                           
   spent 30mA more than Android.                                                                                                                          
   Without full screen mode, it just spent about 20mA more.                                                                                               
                                                                                                                                                          
2. When we using Gallery app to play the same video on the device spent about 10~15mA more then Video app.                                                
   - The UX for playing a video in Gallery app has a progress bar on bottom of the screen and is always showing it but Video app doesn't have that.       
                                                                                                                                                          
Additionally,                                                                                                                                             
   When we tested with HD or low quality video content, the difference between FxOS and Android wasn't that big(it was just about 5mA~10mA).              
   Android draws more stable graphs than FxOS.                                                                                                            
   Plz, see the attachments.

Comment 15

[Jinho Hwang [:jeffhwang]]

Attachment: FxOS_DOU_test_on_Nexus4.png

Comment 16

[Jon Hylands [:jhylands]]

One thing you need to be aware of, when comparing any two phones - the screen brightness needs to be the same. On a Hamachi, changing from the lowest brightness to the highest brightness consumes an extra 100 mA.

So, you need to be absolutely sure that (a) neither phone is in auto-brightness mode, and (b) both screens are set at the same (or as close as you can get) brightness level.

Comment 17

[Jaemin Song]

(In reply to Jon Hylands [:jhylands] from comment #16)
> One thing you need to be aware of, when comparing any two phones - the
> screen brightness needs to be the same. On a Hamachi, changing from the
> lowest brightness to the highest brightness consumes an extra 100 mA.
> 
> So, you need to be absolutely sure that (a) neither phone is in
> auto-brightness mode, and (b) both screens are set at the same (or as close
> as you can get) brightness level.

To retain same brightness,
when we measure power consumption,
both screen's brightness were set to maximum.
And phone was not in auto-brightness mode.

Comment 18

[bhargavg1]

(In reply to Jaemin Song from comment #17)
> (In reply to Jon Hylands [:jhylands] from comment #16)
> > One thing you need to be aware of, when comparing any two phones - the
> > screen brightness needs to be the same. On a Hamachi, changing from the
> > lowest brightness to the highest brightness consumes an extra 100 mA.
> > 
> > So, you need to be absolutely sure that (a) neither phone is in
> > auto-brightness mode, and (b) both screens are set at the same (or as close
> > as you can get) brightness level.
> 
> To retain same brightness,
> when we measure power consumption,
> both screen's brightness were set to maximum.
> And phone was not in auto-brightness mode.

Another important thing to note is bug 934868

Can we check the value of |adb shell "cat /sys/class/leds/button-backlight/brightness"| if its 1 please re-measure by writing |adb shell "echo 0 > /sys/class/leds/button-backlight/brightness"|

Comment 19

[Jaemin Song]

(In reply to bhargavg1 from comment #18)
> (In reply to Jaemin Song from comment #17)
> > (In reply to Jon Hylands [:jhylands] from comment #16)
> > > One thing you need to be aware of, when comparing any two phones - the
> > > screen brightness needs to be the same. On a Hamachi, changing from the
> > > lowest brightness to the highest brightness consumes an extra 100 mA.
> > > 
> > > So, you need to be absolutely sure that (a) neither phone is in
> > > auto-brightness mode, and (b) both screens are set at the same (or as close
> > > as you can get) brightness level.
> > 
> > To retain same brightness,
> > when we measure power consumption,
> > both screen's brightness were set to maximum.
> > And phone was not in auto-brightness mode.
> 
> Another important thing to note is bug 934868
> 
> Can we check the value of |adb shell "cat
> /sys/class/leds/button-backlight/brightness"| if its 1 please re-measure by
> writing |adb shell "echo 0 > /sys/class/leds/button-backlight/brightness"|

[MADAI]
I can't find button-backlight folder.
And button-backlight of device was off.

[Nexus4]
Nexus4 is using S/W key.
So, there is no button-backlight in the device.

Comment 20

[Mike Lee [:mlee]]

Please see Jaemin's response from comment #19:
> 
> [MADAI]
> I can't find button-backlight folder.
> And button-backlight of device was off.
> 
> [Nexus4]
> Nexus4 is using S/W key.
> So, there is no button-backlight in the device.

Comment 21

[bhargavg1]

(In reply to Mike Lee [:mlee] from comment #20)
> Please see Jaemin's response from comment #19:
> > 
> > [MADAI]
> > I can't find button-backlight folder.
> > And button-backlight of device was off.
> > 
> > [Nexus4]
> > Nexus4 is using S/W key.
> > So, there is no button-backlight in the device.

Can we make sure to check if HWC is being or GPU is being used for rendering.

Comment 22

[Mike Lee [:mlee]]

(In reply to bhargavg1 from comment #21)
> 
> Can we make sure to check if HWC is being or GPU is being used for rendering.

Comment 23

[Jaemin Song]

(In reply to Mike Lee [:mlee] from comment #22)
> (In reply to bhargavg1 from comment #21)
> > 
> > Can we make sure to check if HWC is being or GPU is being used for rendering.

We had wrote HWC status in comment No.14.
Both HWC satus is disable.(GPU is being used for rendering)

Thanks.

Comment 24

[Jon Hylands [:jhylands]]

So, after a lot of messing around with my newly made Nexus 4 harness, it appears the problems I'm having are in the actual battery connector (the one on the battery). I've made a bunch of harnesses with this battery, and it appears that one of the connectors has an intermittent break somewhere between the battery and the connector.

I'm going to try and order a new battery off ebay.

Comment 25

[Sotaro Ikeda [:sotaro]]

On nexus-4 b2g, power consumption might be increase by the following reason.
- gfx layers are complex than android one.
- overhead of video file read in app process is bigger than android.
   + android read file data directly from kernel.
   + b2g read file data via b2g process.
- soft button nexus-4 add more gfx layers.

Comment 26

[Sotaro Ikeda [:sotaro]]

To compare the power consumption, it seems better that b2g's app reads video data same way as android.

Comment 27

[Sotaro Ikeda [:sotaro]]

(In reply to Sotaro Ikeda [:sotaro] from comment #26)
> To compare the power consumption, it seems better that b2g's app reads video
> data same way as android.

I am going to create a patch for it.

Comment 28

[Sotaro Ikeda [:sotaro]]

Attachment: patch for b2g v1.4 - Directly read fixed movie file from content

This is an ugly temporary patch. It directly read a fixed movie file from content. It read the file same way as android. It seems easier to compare/ nail down the power consumption difference. The patch hard coded the file name to "//storage//sdcard1//Movies//sample.mp4". I tested the patch on v1.4 flame. The file name needs to be changed depends on devices and files.

Comment 29

[bhargavg1]

(In reply to Sotaro Ikeda [:sotaro] from comment #25)
> On nexus-4 b2g, power consumption might be increase by the following reason.
> - gfx layers are complex than android one.
> - overhead of video file read in app process is bigger than android.
>    + android read file data directly from kernel.
>    + b2g read file data via b2g process.
> - soft button nexus-4 add more gfx layers.

Also can we confirm if Nexus5 is using tunnel mode audio even for video playback. the property to check is |av.offload.enable| 

If that is set to true and we see a logcat somthing similar to
D/audio_hw_primary(  198): out_set_parameters: enter: usecase(3: compress-offload-playback) kvpairs: routing=2

Its confirmed we are using TM playback for video as well

Comment 30

[Sotaro Ikeda [:sotaro]]

(In reply to Sotaro Ikeda [:sotaro] from comment #25)
> On nexus-4 b2g, power consumption might be increase by the following reason.
> - gfx layers are complex than android one.
> - overhead of video file read in app process is bigger than android.
>    + android read file data directly from kernel.
>    + b2g read file data via b2g process.
> - soft button nexus-4 add more gfx layers.

Another possibility is that MediaDecoderStateMachine and audio related threads run often.

Comment 31

[Jon Hylands [:jhylands]]

So, I got the patch installed and flashed onto the phone - I know its working because it will only show the video if the name is "correct".

It appears to consume, on average, slightly less power than before, but we're talking ~8-10 mA at most.

Jinho, can you test this patch, and see what results you get?

Comment 32

[Jinho Hwang [:jeffhwang]]

Hi Jon,

Sure, I'll test the patch with Jaemin Song. 
And then I'll let you know the result asap.

Comment 33

[Sotaro Ikeda [:sotaro]]

(In reply to Sotaro Ikeda [:sotaro] from comment #30)
> 
> Another possibility is that MediaDecoderStateMachine and audio related
> threads run often.

On android media framework, all function calls are basically based on callback or event. But in gecko media framworks, some parts are still doing polling.

Comment 34

[Jaemin Song]

(In reply to Jon Hylands [:jhylands] from comment #31)
> So, I got the patch installed and flashed onto the phone - I know its
> working because it will only show the video if the name is "correct".
> 
> It appears to consume, on average, slightly less power than before, but
> we're talking ~8-10 mA at most.
> 
> Jinho, can you test this patch, and see what results you get?

We have checked power consumption with attached patch.
But result of measure, patched version's result was similar with original version's result.
Both results were almost same.

Comment 35

[Jon Hylands [:jhylands]]

Can someone with the correct equipment make a 1080p video without an audio track? Or is this even possible? If it is, playing it on both B2G and Android will at least tell us if the power usage difference is caused by the audio side or the video side of things...

Comment 36

[Sotaro Ikeda [:sotaro]]

(In reply to Jon Hylands [:jhylands] from comment #35)
> Can someone with the correct equipment make a 1080p video without an audio
> track? Or is this even possible? If it is, playing it on both B2G and
> Android will at least tell us if the power usage difference is caused by the
> audio side or the video side of things...

It should be possible. I am going to create a patch for b2g.

Comment 37

[Sotaro Ikeda [:sotaro]]

Attachment: patch for b2g v1.4 - Directly read fixed movie file from content and playback without audio

Video playback without audio and directly a read file like android.

Comment 38

[Sotaro Ikeda [:sotaro]]

(In reply to Sotaro Ikeda [:sotaro] from comment #37)
> Created attachment 8444458 [details] [diff] [review]
> temporary patch - Directly read fixed movie file from content and playback
> without audio
> 
> Video playback without audio and directly a read file like android.

Same thing could be easily done also on android.

Comment 39

[Jon Hylands [:jhylands]]

So with this  patch installed, my power usage is down roughly 40 mA. We need to figure out a way to do the same thing under android. It would be interesting to see if an audio-less video file would produce the same power profile as this patch...

Comment 40

[bhargavg1]

(In reply to Sotaro Ikeda [:sotaro] from comment #37)
> Created attachment 8444458 [details] [diff] [review]
> patch for b2g v1.4 - Directly read fixed movie file from content and
> playback without audio
> 
> Video playback without audio and directly a read file like android.

Can we not use the ffmpeg to just strip out the audio stream, any queries related to audio would be just null

From, http://superuser.com/questions/268985/remove-audio-from-video-file-with-ffmpeg

|fmpeg -i [input_file] -vcodec copy -an [output_file]|

Comment 41

[Jon Hylands [:jhylands]]

Yes, that does in fact work. I'm just rebuilding now, to see if the power profile is the same with stock b2g and an audio-less file, versus a normal file and your audio-less patch.

Comment 42

[Jon Hylands [:jhylands]]

So, interestingly, the B2G video player won't even display the file in the list of files to be played if it doesn't have the audio track. I'm going to re-apply your original patch (comment 28) and see what the power profile looks like, since we know that patch doesn't have a lot of effect on power, but will allow me to play the video.

Comment 43

[Jon Hylands [:jhylands]]

After applying the original patch from comment 28, and copying over the video file that has audio stripped from it, I get about 30 mA less power usage than playing the same file without the audio stripped.

Jinho, can you take whatever video you're using, run:

ffmpeg -i [input_file] -vcodec copy -an [output_file]

and try playing that on both Android and B2G? I'd specifically like to see what the numbers look like compared to the results in comment 14. This will let us know what amount of the difference is due to audio and what amount is due to video.

Comment 44

[Jinho Hwang [:jeffhwang]]

(In reply to Jon Hylands [:jhylands] from comment #43)

> Jinho, can you take whatever video you're using, run:
> 
> ffmpeg -i [input_file] -vcodec copy -an [output_file]
> 
> and try playing that on both Android and B2G? I'd specifically like to see
> what the numbers look like compared to the results in comment 14.

Hi, Jon.

I tested according to your guide here strip audio from the video content using ffmpeg.
Here is the result.

------------------------------------------
  No.|    version    |     Result (Difference between original and audio stripped content)
------------------------------------------
   1 |    Android    |  about -30~35 mA
   2 |  B2G Patched  |  about -35~45 mA 
------------------------------------------
Playing audio stripped content spent more less power on both Android and B2G as expected.

After I applied the patch on comment 28, there was no big changes on the result.
But with the patch on comment 37, the testing result was same with the testing result with audio stripped content.

Let me know if you have any questions.
Thanks.

Comment 45

[Jon Hylands [:jhylands]]

So, it appears the difference in power consumption between Android and B2G is on the video side, and not the audio side. One difference I've seen is under B2G, the home button is a soft button, and always visible. I don't know if the GPU is optimized to display full-screen video from the hardware decoder versus almost full screen, so that may be one place to look. The other is what Sotaro said originally, which is that the B2G video player uses more or more complex layers than the Android one.

Anyone have any ideas on how we can test these out?

Comment 46

[Sushil]

In full screen 1080p video playback, there are unnecessary layers present in layer tree which are entirely hidden behind full screen Video layer, so they should never be painted and composed. So if you can fix this bug in layer tree formation, this will help in improving power numbers in GPU Composition (as compared to Android). Here are the layers which I see during "full screen" 1080p video playback in landscape mode:

Layer[0]::ThebesLayerComposite: visibleRect=[0 0 1280 720] dst=[0 0 720 1280] src=[0 0 1280 720]
Layer[1]::ThebesLayerComposite: visibleRect=[0 0 1280 720] dst=[0 0 672 1280] src=[0 0 1280 672]
Layer[2]::ColorLayerComposite: visibleRect=[0 0 1280 720] dst=[0 0 720 1280] src=[0 0 1280 720]
Layer[3]::ThebesLayerComposite: visibleRect=[0 0 1280 720] dst=[0 0 720 1280] src=[0 0 1280 720]
Layer[4]::ImageLayerComposite: visibleRect=[0 0 1920 1080] dst=[0 0 720 1280] src=[0 0 1920 1080]

Ideally, Layer[4] should be the "only" layer which should be painted and composed on screen in full screen Video playback and that should be happening on Android, please check by "adb shell dumpsys SurfaceFlinger". I am dropping there layers in the H/W Composition path as per the bug: https://bugzilla.mozilla.org/show_bug.cgi?id=981732 but still these layers should never be present in the layer tree itself, which is the root cause. This will help to reduce unnecessary processing through out the display pipeline, hence will improve the power numbers.

Comment 47

[Mason Chang [Inactive] [:mchang]]

Attachment: Layer Tree

The layer tree attached.

This looks fairly bad. We have a large thebes layer that we composite but shouldn't be here. Also, since this is Nexus 4, we have a transparent overlay for the home button which contributes to extra compositing.

Comment 48

[Sotaro Ikeda [:sotaro]]

(In reply to Mason Chang [:mchang] from comment #47)
> Created attachment 8445402 [details]
> Layer Tree
> 
> The layer tree attached.
> 
> This looks fairly bad. We have a large thebes layer that we composite but
> shouldn't be here. Also, since this is Nexus 4, we have a transparent
> overlay for the home button which contributes to extra compositing.

Some layer's problem is fixed and is going to be fixed under Bug 1022164 on master.

Comment 49

[Jon Hylands [:jhylands]]

So bug 1022164 is currently backlogged, and appears to be set up as a feature for 2.1.

Are we satisfied with this timeline, given that this does not appear to be a regression?

If we are, then I suggest removing 1.4+ from this bug, and setting it to be nom'ed for 2.1 when we get to that point.

Comment 50

[Mike Lee [:mlee]]

Pushing this to 2.0 since Madai has switched to 2.0 instead of 1.4.

Comment 51

[Jon Hylands [:jhylands]]

Jaemin, given that this bug is dependent on another bug that is currently slated for 2.1, we need to either (a) change this bug to be a 2.1 blocker, or (b) change bug 1022164 to be a 2.0 blocker. 

Benwa, what is stopping bug 1022164 from being resolved in 2.0?

Comment 52

[Benoit Girard (:BenWa)]

We're waiting on the dependencies of bug 1022164 to land.

Comment 53

[Jon Hylands [:jhylands]]

Sure, but both bugs it depends on are 2.0 blockers. Is it strictly a time thing, that 1022164 will not be fixed in time for 2.0? We need to determine where this bug will fall, 2.0 or 2.1, since it depends on bug 1022164.

Comment 54

[Benoit Girard (:BenWa)]

No, bug 1022164 is a tracking bug at this point so it could/should be 2.0 (since it's a major regression). If none of the dependencies slip then bug 1022164 will be fixed for 2.0 as a result. Feel free to request a change.

Comment 55

[Jaemin Song]

(In reply to Jon Hylands [:jhylands] from comment #51)
> Jaemin, given that this bug is dependent on another bug that is currently
> slated for 2.1, we need to either (a) change this bug to be a 2.1 blocker,
> or (b) change bug 1022164 to be a 2.0 blocker. 

(b).
Because MADAI's version is 2.0.
This issue should be fixed for MADAI's power consumption problem.

Thanks.

Comment 56

[Jon Hylands [:jhylands]]

This bug is actively being worked on in bug 1022164 - since the fix for that bug has landed in master, I'll be testing tomorrow on my nexus 4 to see if the power results have changed.

Comment 57

[Jon Hylands [:jhylands]]

I don't see any real improvement between 2.0 without this change, and current master (which is supposed to have it). Of course, that's not to say it doesn't have an effect - we really can't compare 2.0 and 2.1 directly, at least in terms of finding out if this fix helps.

Jeff, can you run the test on your end, and see what results you get?

Benoit, can you verify that current master/m-c has the fix from bug 1022164? When can we expect this to land on 2.0?

Comment 58

[Benoit Girard (:BenWa)]

This fixes are on 2.0. I don't have a nexus 4. Can you get me a layer tree dump so we can look at the complexity of the layer tree in your test case?

Flip layers.dump:true, reboot and capture one of the layers dump that is spewing during playback. They should all be identical if you're just flipping frame.

Turn on the FPS counter. If you're using HardwareComposer then the FPS counter will disappear (since it doesn't support drawing it). At the same time the FPS rate should be outputed to adb lolcat as a replacement. It would be worth checking if it matches the frame rate of the video being played.

Comment 59

[Jon Hylands [:jhylands]]

Attachment: Post Fix Layer Tree

Here's the layer tree, from current master.

Comment 60

[Jon Hylands [:jhylands]]

Benoit, it appears the layer tree is actually more complex now than it was in the one Mason attached (the attached "Post Fixed Layer Tree" is from current master). I'll reflash with current 2.0 and see what that looks like.

Comment 61

[Jon Hylands [:jhylands]]

Attachment: Post Fix Layer Tree (v2.0)

Here's the layer tree playing video in landscape mode, with the device (nexus 4) flashed to latest v2.0.

Comment 62

[Jon Hylands [:jhylands]]

I turned on FPS, and it displayed while I was playing the video in portrait mode, but as soon as I switch to landscape it switched off. However, I did not see any FPS measurements showing up in the logcat.

Comment 63

[Benoit Girard (:BenWa)]

The 'complexity' of the layer tree can be misleading:
- Container layers tend to be free, unless their have opacity in which case they are VERY expensive.
- Color layers are super cheap
- Image layers are somewhat cheap
- Thebes layer consume a lot of memory + bandwidth proportional to their valid area.

So we have when you removed the positioning data:
(1) Thebes layer sized (x=0, y=668, w=1280, h=100)
(2) Color layer sized (x=0, y=0, w=1280, h=668) 
(3) Color layer sized (x=0, y=0, w=1280, h=668) *differs in transform
(4) Hidden thebes layer but still consuming memory sized (x=0, y=0, w=1280, h=668)
(5) Image layer sized (x=0, y=0, w=1920, h=1080)

- (2) + (3) I'm not sure why we get two color layers. They might not overlap the same area but we still filling a lot of redundant area for nothing.
- (4) is wasting memory but I'd image the HWC calculation are smart enough to deal with a hidden layer.
- (5) The video is larger then the screen so we have to downscale it which could put us over the bandwidth budget for HWC.
- (1) I think this is for the status bar but I think 100 pixels is a bit much for it. This depends on your device I guess.

I'd start removing the duplicate color layer. I'd be interesting hiding the status bar during video playback to also remove that layer. Can you confirm that the FPS matches the video exactly?

Comment 64

[Jon Hylands [:jhylands]]

No, I think (1) is for the home button (y value is 668, so its at the bottom of the screen).

I can't confirm anything about FPS, because it doesn't show up in logcat, and the display disappears when in landscape mode.

Comment 65

[Benoit Girard (:BenWa)]

Ahh, yes, that would be the home button.

But when it does display what does it display at in non landscape mode?

Comment 66

[Jon Hylands [:jhylands]]

So when it displays in portrait mode, it displays a very solid 30 FPS, which is the rate the video I'm playing is recorded at.

Comment 67

[Jon Hylands [:jhylands]]

Sotaro, how hard would it be to do a patch based on current master that does the decoding of the video, but doesn't display anything? We're trying to see where in the pipeline the power difference is. We've already determined that the audio processing does not appear to make any difference on the two platforms.

Once we have a patch like that for FxOS, we will need to create something similar for Android, and I have no idea how we'll do that...

Comment 68

[Sotaro Ikeda [:sotaro]]

(In reply to Jon Hylands [:jhylands] from comment #67)
> Sotaro, how hard would it be to do a patch based on current master that does
> the decoding of the video, but doesn't display anything?

Does it mention only video frames? If it is just not display video frames, it seems easier.

Comment 69

[Sotaro Ikeda [:sotaro]]

Attachment: patch pattern 1 - Do not render video frames

By applying the patch, video frames are not rendered at all. The video frames are not rendered to display nor canvas(thumbnail generation). And layout code triggered by video frame update does not run.

Comment 70

[Sotaro Ikeda [:sotaro]]

Attachment: patch pattern 2 - Do not render video frames

By applying the patch, video frames are not rendered to display, but they are rendered to Canvas(for thumbnail generation). And layout related code run by triggered by video frame update.

Comment 71

[Sotaro Ikeda [:sotaro]]

To check video rendering from VideoFrameContainer, the following diagram might help.

https://github.com/sotaroikeda/firefox-diagrams/blob/master/gfx/gfx_ImageBridge_FirefoxOS_2_1.pdf

Comment 72

[Jon Hylands [:jhylands]]

sotaro, the first patch doesn't seem to work - I don't see any videos when I apply it, even though (after flashing) I pushed the same video to the sdcard.

The second one works, and gives me an almost exactly 20% reduction in power usage over playing the video straight.

Comment 73

[Jon Hylands [:jhylands]]

Michael, in order to really figure this problem out, we really need to play video on the Nexus 4 under Android with a similar patch to comment 70. Is this doable?

Comment 74

[Michael Wu [:mwu]]

We can build with a custom patch, but I have no idea what an equivalent patch for Android would be.

Comment 75

[Jon Hylands [:jhylands]]

Attachment: Profile of Video App

Looking at profiles taken from while I was running the Video app, it spends about 90% of the time idling. I'm attaching the profile I got in the hope that someone can tell something useful from it.

I created this profile with the following command line:

./profile.sh start -p b2g -i 10 && ./profile.sh start -p Video -i 10 -e 500000 -f stackwalk.js
(play the video, ~30 seconds)
./profile.sh capture

Benwa, can you have a quick look and tell me what you see?

Comment 76

[Benoit Girard (:BenWa)]

Most of the video thread doesn't happen on the main thread but that's what you're profiling. So you don't see anything. You need to make sure the thread that are doing work are register for profiling, see GeckoProfiler.h and use -t <name>[,<name2>].

Comment 77

[Jon Hylands [:jhylands]]

David, can you comment here with respect to comment 76? Specifically, does the video app spawn threads, and if so what are they called?

Comment 78

[David Flanagan [:djf]]

Jon,

We don't have threads in JavaScript, so the Video app itself wouldn't be doing that. I would guess that Benoit means threads spawned by gecko to display the video. I don't know anything about how that works.

Benoit: could you clarify where video playback threads are spawned?

Jon: if Benoit can't clarify, perhaps Sotaro can help you with this.

Comment 79

[Jon Hylands [:jhylands]]

I'll wait until I hear from Benoit, since Sotaro is PTO until Monday.

Comment 80

[Benoit Girard (:BenWa)]

I'm not familiar with video playback, all I know is it happens off the main thread and the decoded frames are put into Gralloc buffers and pass to the compositor thread directly via the ImageBridge. I'm not sure how the decoding works. Sotaro will be back on Tuesday and can provide more details.

Comment 81

[Jon Hylands [:jhylands]]

Sotaro, once you're back can you comment on comment 76 above? Specifically, does the Video app use threads in gecko, and if so what are they called so we can profile them?

Comment 82

[Chris Pearce [:cpearce (Not reading bugmail)]]

(In reply to Jon Hylands [:jhylands] from comment #81)
> Sotaro, once you're back can you comment on comment 76 above? Specifically,
> does the Video app use threads in gecko, and if so what are they called so
> we can profile them?

The gecko media stack uses threads. Normally named "Media Decode", but not always if the platform's decoder creates its own threads, which we have limited control over.

Comment 83

[Sotaro Ikeda [:sotaro]]

(In reply to Chris Pearce (:cpearce) from comment #82)
> 
> The gecko media stack uses threads. Normally named "Media Decode", but not
> always if the platform's decoder creates its own threads, which we have
> limited control over.

"Media Decode" name is the following.
  http://mxr.mozilla.org/mozilla-central/source/content/media/VideoUtils.cpp#197

And it is used here.
  http://mxr.mozilla.org/mozilla-central/source/content/media/MediaDecoderStateMachine.cpp#1059

Comment 84

[Jon Hylands [:jhylands]]

So it appears that "Media Decode" isn't one of the threads that the profiler is set up to profile.

I finally got set up to flash android on my device, and the idle screen full-brightness numbers are almost identical between Android 4.4.4 and FxOS 2.0.

Investigation is continuing...

Comment 85

[Jon Hylands [:jhylands]]

To summarize where we are now:

We don't know why FxOS uses more power playing fullscreen video. We know that FxOS 2.0 and Android 4.3 Jelly Bean take about the same amount of power when idle on a Nexus 4, with the screen fully bright (~260 mA)

We know that in FxOS on my Nexus 4, the screen takes about 260 mA, decoding takes about 110 mA, and composition takes about 80 mA.

On Android on my Nexus 4, the screen takes 260 mA, and decoding and composition take about 140 mA in total. We don't know what the split is between decoding and composition.

Comment 86

[[:SalvadorR]]

No STR is present to create test case to address bug.

Comment 87

[Michael:mtreese]

per Jon's email:

I've taken 1010638 about as far as I can without dedicating a week or two to it. What it needs now is for someone to write a simple Java application for Android that plays full screen video, determine if the power consumption is on par with the official video player app, and then do the same breakdown we did with FxOS in order to determine the split between composition and decoding.

Once we have that, we can compare relative percentages between the two breakdowns, and determine where (and eventually why) FxOS is using up more power.

- Jon

Comment 88

[Hema Koka [:hema]]

Per Wayne's most recent email update, Madai folks are re-testing with possibly including fix in 1054105.

Wayne, I am leaving NI on you to provide an update and see if this is still considered a 2.0 blocker given the update from Jon

Comment 89

[Wayne Chang [:wchang]]

Redirecting this to Jongsoo.

Remove blocking on this since there should be enough improvement on 2.0 and that we have not heard back from our partner re test outcome for over a week.

Jongsoo, please see above comment. At the moment applying the patch from 1054105 should introduce an improvement. We do not have any more to add further than this for now. Can you share your test results with the patch applied?

Hema, being a non-blocker, how do you want this bug to proceed?

Comment 90

[Wayne Chang [:wchang]]

Received update from partner, OK to remove this as a blocker.

Comment 91

[Gabriele Svelto [:gsvelto]]

It would be interesting to check if this is still a problem on the Aries which also uses the software home button and close it in case the problem went away. Alexandre do you know if we have HWC support on the Aries already?

(clearing the old NI as it's not relevant anymore)

Comment 92

[:gerard-majax]

(In reply to Gabriele Svelto [:gsvelto] from comment #91)
> It would be interesting to check if this is still a problem on the Aries
> which also uses the software home button and close it in case the problem
> went away. Alexandre do you know if we have HWC support on the Aries already?
> 
> (clearing the old NI as it's not relevant anymore)

Yes, HWComposer should work well now, if you build by yourself.

Comment 93

[Gabriele Svelto [:gsvelto]]

Thanks, NI'ing myself to check if the HWC works correctly. I've received the power harness so I might even be able to do an actual power check.

Comment 94

[Bobby Chien]

[Tracking Requested - why for this release]: