891274 - [fugu][A/V]It can not generate thumbnail list when first entry video app

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: General, defect)

Description

[cheng.luo]

User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.19 (KHTML, like Gecko) Ubuntu/11.04 Chromium/18.0.1025.151 Chrome/18.0.1025.151 Safari/535.19

Steps to reproduce:

Apply the patch of  bug889238 and found another issue happen: 
1、apply the patch bug889238;
2、rm -r /data/local/indexedDB/*video*;
3、open the video app;


Actual results:

no thumbnail list to generate and show


Expected results:

could generate video thumbnail list and show

Comment 1

[cheng.luo]

I found lots of JS error prompt in the log.

E/GeckoConsole(  348): Content JS ERROR at app://video.gaiamobile.org/js/metadata.js:314 in captureFrame: Exception in captureFrame: [Exception... "Component is not available"  nsresult: "0x80040111 (NS_ERROR_NOT_AVAILABLE)"  location: "JS frame :: app://video.gaiamobile.org/js/metadata.js :: captureFrame :: line 308"  data: no]

Comment 2

[cheng.luo]

after camera recording,there is no

Comment 3

[cheng.luo]

After camera recording,there is no thumbnail too.
Error in GeckoConsole,maybe like Bug 764066:
[JavaScript Error: "NS_ERROR_NOT_AVAILABLE: " {file: "app://camera.gaiamobile.org/js/camera.js" line: 777}]

Stack as follow:
#0  mozilla::dom::CanvasRenderingContext2D::DrawImage (this=0x44511800, image=..., sx=-9.1818510554730892e-06, 
    sy=0, sw=0, sh=0, dx=0, dy=0, dw=0, dh=0, optional_argc=0 '\000', error=...)
    at /home/cheng/Mozilla/7710gonk/B2G/gecko/content/canvas/src/CanvasRenderingContext2D.cpp:2919
#1  0x40f07332 in mozilla::dom::CanvasRenderingContext2D::DrawImage (cx=0x43a745b0, obj=<value optimized out>, 
    self=0x44511800, args=...) at ../../dist/include/mozilla/dom/CanvasRenderingContext2D.h:185
#2  drawImage (cx=0x43a745b0, obj=<value optimized out>, self=0x44511800, args=...)
    at /home/cheng/Mozilla/7710gonk/B2G/objdir-gecko/dom/bindings/CanvasRenderingContext2DBinding.cpp:2273
#3  0x40f05334 in genericMethod (cx=0x43a745b0, argc=<value optimized out>, vp=<value optimized out>)
    at /home/cheng/Mozilla/7710gonk/B2G/objdir-gecko/dom/bindings/CanvasRenderingContext2DBinding.cpp:4177
#4  0x414a3632 in CallJSNative (cx=0x43a745b0, args=..., construct=<value optimized out>)
    at /home/cheng/Mozilla/7710gonk/B2G/gecko/js/src/jscntxtinlines.h:218
#5  Invoke (cx=0x43a745b0, args=..., construct=<value optimized out>)
    at /home/cheng/Mozilla/7710gonk/B2G/gecko/js/src/vm/Interpreter.cpp:478
#6  0x414a277e in Interpret (cx=0x43a745b0, state=<value optimized out>)
    at /home/cheng/Mozilla/7710gonk/B2G/gecko/js/src/vm/Interpreter.cpp:2454
#7  0x414a44e8 in RunScript (cx=0x43a745b0, thisv=<value optimized out>, fval=..., argc=<value optimized out>, 
    argv=0xbee348b8, rval=...) at /home/cheng/Mozilla/7710gonk/B2G/gecko/js/src/vm/Interpreter.cpp:435
#8  Invoke (cx=0x43a745b0, thisv=<value optimized out>, fval=..., argc=<value optimized out>, argv=0xbee348b8, 
    rval=...) at /home/cheng/Mozilla/7710gonk/B2G/gecko/js/src/vm/Interpreter.cpp:497

Comment 4

[ying.xu]

https://bugzilla.mozilla.org/show_bug.cgi?id=910498#c38

Some people have encountered the same question.

Comment 5

[ying.xu]

And he has supplied a patch.
Shall we check it ?

https://bugzilla.mozilla.org/show_bug.cgi?id=910498#c40

Comment 6

[ying.xu]

https://bugzilla.mozilla.org/show_bug.cgi?id=832653#c30

832745 832650

Comment 7

[Marco Chen [:mchen]]

(In reply to ying.xu from comment #5)
> And he has supplied a patch.
> Shall we check it ?
> 
> https://bugzilla.mozilla.org/show_bug.cgi?id=910498#c40

Could that patch solve your issue here? Thanks.

Comment 8

[cheng.luo]

(In reply to Marco Chen [:mchen] (Summit 10/03 ~10/08) from comment #7)
> (In reply to ying.xu from comment #5)
> > And he has supplied a patch.
> > Shall we check it ?
> > 
> > https://bugzilla.mozilla.org/show_bug.cgi?id=910498#c40
> 
> Could that patch solve your issue here? Thanks.

I think it does not work. the error log is the same:
10-18 14:20:39.260 D/libEGL  (  856): loaded /system/lib/egl/libGLESv1_CM_mali.so
10-18 14:20:39.270 D/libEGL  (  856): loaded /system/lib/egl/libGLESv2_mali.so
10-18 14:20:39.280 I/Gecko   (  856): OpenGL version detected: 200
10-18 14:20:39.280 I/Gecko   (  856): SharedSurface_Gralloc::Create -------
10-18 14:20:39.280 D/gralloc.sc7710(  527): alloc_device_alloc w:352, h:288, format:1 usage:768 start
10-18 14:20:39.280 D/gralloc.sc7710(  527): alloc_device_alloc handle:0x45a2c600 end
10-18 14:20:39.290 I/Gecko   (  856): SharedSurface_Gralloc::Create: success -- surface 0x432d4040, GraphicBuffer 0x441ed600.
10-18 14:20:39.310 E/GeckoConsole(  856): [JavaScript Error: "NS_ERROR_NOT_AVAILABLE: " {file: "app://camera.gaiamobile.org/js/camera.js" line: 777}]

Comment 9

[Sotaro Ikeda [:sotaro]]

(In reply to Marco Chen [:mchen] from comment #7)
> (In reply to ying.xu from comment #5)
> > And he has supplied a patch.
> > Shall we check it ?
> > 
> > https://bugzilla.mozilla.org/show_bug.cgi?id=910498#c40
> 
> Could that patch solve your issue here? Thanks.

It is unrelated patch. HAL_PIXEL_FORMAT_YCbCr_420_SP_TILED is handled at 

https://github.com/mozilla-b2g/platform_frameworks_av/blob/b2g-4.3_r2.1/media/libstagefright/colorconversion/ColorConverter.cpp#L136

Comment 10

[Sotaro Ikeda [:sotaro]]

cheng.luo, in which hardware are you seeing the problem?

Comment 11

[Sotaro Ikeda [:sotaro]]

(In reply to Sotaro Ikeda [:sotaro] from comment #10)
> cheng.luo, in which hardware are you seeing the problem?

From the title, it seems to happen on 'fugu' device. Does the problem happen on other devices?

Comment 12

[James Zhang (Spreadtrum)]

(In reply to Sotaro Ikeda [:sotaro] from comment #11)
> (In reply to Sotaro Ikeda [:sotaro] from comment #10)
> > cheng.luo, in which hardware are you seeing the problem?
> 
> From the title, it seems to happen on 'fugu' device. Does the problem happen
> on other devices?

Tara and unagi also have this issue. We will try hamachi next week.

Comment 13

[ying.xu]

We found this function run fault.

CanvasRenderingContext2D::DrawImage

 if (!srcSurf) {
    // The canvas spec says that drawImage should draw the first frame
    // of animated images
    uint32_t sfeFlags = nsLayoutUtils::SFE_WANT_FIRST_FRAME;

//here, both mSurface and mIsStillLoading of res were null,hence return a error
    nsLayoutUtils::SurfaceFromElementResult res =
      nsLayoutUtils::SurfaceFromElement(element, sfeFlags);

    if (!res.mSurface) {
      // Spec says to silently do nothing if the element is still loading.
      if (!res.mIsStillLoading) {
        error.Throw(NS_ERROR_NOT_AVAILABLE);
      }
      return;
    }


And there was an unhandled format in this funcion where format = 33 HAL_PIXEL_FORMAT_BLOB
already_AddRefed<gfxASurface>
GrallocImage::GetAsSurface()

  uint32_t format = graphicBuffer->getPixelFormat();
  uint32_t omxFormat = 0;

  for (int i = 0; sColorIdMap[i]; i += 2) {
    if (sColorIdMap[i] == format) {
      omxFormat = sColorIdMap[i + 1];
      break;
    }
  }

  DEBUG_PRINT("GrallocImage::GetAsSurface , format== %d omxFormat == %d\n",  format ,omxFormat);

//here. omxFormat==0, return directly
  if (!omxFormat) {
    NS_ERROR("Unknown color format");
    return nullptr;
  }

Comment 14

[Marco Chen [:mchen]]

(In reply to ying.xu from comment #13)
> And there was an unhandled format in this funcion where format = 33
> HAL_PIXEL_FORMAT_BLOB
> already_AddRefed<gfxASurface>
> GrallocImage::GetAsSurface()

Hi Sotaro,

It seems that this issue is similar to what I post on Comment 7.
I met such kind of issue on porting different platforms.
Different platforms may have their own color format and normally they added into their own Android base. So once FxOS missed it the surface can't be displayed.

Comment 15

[Bruce Sun [:brsun]]

(In reply to ying.xu from comment #13)
> And there was an unhandled format in this funcion where format = 33
> HAL_PIXEL_FORMAT_BLOB

Hi Ying,

I don't have much information about how to convert the buffer of GraphicBuffer in HAL_PIXEL_FORMAT_BLOB format into the data of gfxImageSurface in OMX_COLOR_Format16bitRGB565 format. Do you know how this conversion could be achieved?

Comment 16

[ying.xu]

(In reply to Bruce Sun [:brsun] from comment #15)
> (In reply to ying.xu from comment #13)
> > And there was an unhandled format in this funcion where format = 33
> > HAL_PIXEL_FORMAT_BLOB

format = 33 means HAL_PIXEL_FORMAT_YCbCr_420_SP in sprd platform at android4.0 version.
HAL_PIXEL_FORMAT_BLOB doesn't exist on android 4.0,sorry for this.

Problem is that the same format HAL_PIXEL_FORMAT_YCbCr_420_SP has two different enum values on qcom/sprd platforms.

We have to change the value of HAL_PIXEL_FORMAT_YCbCr_420_SP  in GrallocImage.h to match the definition in our HAL module.
Other vendor may need change this either. It's the matter that you people should improve.

 
> Hi Ying,
> 
> I don't have much information about how to convert the buffer of
> GraphicBuffer in HAL_PIXEL_FORMAT_BLOB format into the data of
> gfxImageSurface in OMX_COLOR_Format16bitRGB565 format. Do you know how this
> conversion could be achieved?

Comment 17

[Bruce Sun [:brsun]]

Attachment: bug891274_hal_pixel_format_sprd.patch

Add HAL_PIXEL_FORMAT_YCbCr_420_SP_SPRD enum and map it to OMX_COLOR_FormatYUV420SemiPlanar before doing color conversion.

Comment 18

[ying.xu]

Thanks for the patch,it should work.

I have fixed this bug by modifying our HAL code that is vendor should supply.

I think it's better to match our code with ffos's code.

You can close this bug now.

(In reply to Bruce Sun [:brsun] from comment #17)
> Created attachment 822170 [details] [diff] [review]
> bug891274_hal_pixel_format_sprd.patch
> 
> Add HAL_PIXEL_FORMAT_YCbCr_420_SP_SPRD enum and map it to
> OMX_COLOR_FormatYUV420SemiPlanar before doing color conversion.

Comment 19

[Bruce Sun [:brsun]]

(In reply to ying.xu from comment #18)
> Thanks for the patch,it should work.
> I have fixed this bug by modifying our HAL code that is vendor should supply.
> I think it's better to match our code with ffos's code.
> You can close this bug now.

Hi Ying,

In order to keep gecko as general as possible, I think all these vendors' customized enum should be moved out from gecko to some other places in the future for long term solution. If this issue has been fixed in your codes, I would suggest to cancel my patch to ease future work. What do you think?
And then the next step we should work out is to provide a solution on how different vendors can use gecko more seamless without breaking the generality of gecko code base.

Comment 20

[ying.xu]

(In reply to Bruce Sun [:brsun] from comment #19)
> (In reply to ying.xu from comment #18)
> > Thanks for the patch,it should work.
> > I have fixed this bug by modifying our HAL code that is vendor should supply.
> > I think it's better to match our code with ffos's code.
> > You can close this bug now.
> 
> Hi Ying,
> 
> In order to keep gecko as general as possible, I think all these vendors'
> customized enum should be moved out from gecko to some other places in the
> future for long term solution. If this issue has been fixed in your codes, I
> would suggest to cancel my patch to ease future work. What do you think?

That's OK.


> And then the next step we should work out is to provide a solution on how
> different vendors can use gecko more seamless without breaking the
> generality of gecko code base.

Comment 21

[Bruce Sun [:brsun]]

(In reply to ying.xu from comment #20)
> That's OK.

Thanks for your kind understanding.

Comment 22

[Bruce Sun [:brsun]]

I've created a followup bug for a new porting methodology: bug931733.

Comment 23

[cheng.luo]

Attachment #822170 [details] [diff] work well

Comment 24

[Sotaro Ikeda [:sotaro]]

Attachment 822170 [details] [diff], overwrite the already android defined format in the following.
  http://androidxref.com/4.3_r2.1/xref/system/core/include/system/graphics.h#162

Comment 25

[Marco Chen [:mchen]]

(In reply to Sotaro Ikeda [:sotaro] from comment #24)
> Attachment 822170 [details] [diff], overwrite the already android defined
> format in the following.
>  
> http://androidxref.com/4.3_r2.1/xref/system/core/include/system/graphics.
> h#162

Hi Sotaro,

Fugu is based on Gonk-ICS version and ICS didn't define "HAL_PIXEL_FORMAT_BLOB = 0x21".
http://androidxref.com/4.0.4/xref/system/core/include/system/graphics.h#92

So I think even Fugu used 0x21 is not a problem on ICS version.

Comment 26

[Marco Chen [:mchen]]

1. Close this bug and set to wontfix because partner fixed this issue on their side already.

2. To discuss the different definition of HAL Pixel format, please go to bug 931733.

Comment 27

[Sotaro Ikeda [:sotaro]]

> Hi Sotaro,
> 
> Fugu is based on Gonk-ICS version and ICS didn't define
> "HAL_PIXEL_FORMAT_BLOB = 0x21".
> http://androidxref.com/4.0.4/xref/system/core/include/system/graphics.h#92
> 
> So I think even Fugu used 0x21 is not a problem on ICS version.

If it is used only on spreadtrum specific branch, there is no problem. I do not like that such definition is present on mozilla's gecko code even if it is for ICS.