1673635 - CanvasRenderingContext2D.drawImage not called fast enough / not correctly

Status: UNCONFIRMED

(Core :: Graphics: Canvas2D, defect)

Description

[pascal.galle]

User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.75 Safari/537.36

Steps to reproduce:

I call the function CanvasRenderingContext2D.drawImage() in fast succession (wrapped in promises, which are essentially waiting for a video to end seeking on the next timestamp).
I can work around the issue when using a wait function:

await new Promise(r => setTimeout(r, 200))

in between my draw function. So it seems firefox, in contrast to chrome is too slow in drawing the video frame on the canvas.

Actual results:

The second drawImage call doesn't do anything, even though it's being called correctly.

Expected results:

The drawImage() function always draws the image or throws an error.

Comment 1

[Timothy Nikkel (:tnikkel)]

Can you provide a testcase?

Comment 2

[pascal.galle]

I can show you the code I use. It's not copyrighted:

private createThumbnailfromVideo = async (): Promise<Array<CanvasRenderingContext2D>> => {
      return new Promise<Array<CanvasRenderingContext2D>>((resolve, reject) => {
         const video = this.renderPlainVideo()

         if (!video) {
            reject()
         }
         video.style.display = 'none'

         video.addEventListener('loadedmetadata', async () => {
            const thumbnailInterval = this.props.options?.thumbnail?.interval || 1 // 1 thumbnail per second
            const thumbnailWidth = this.props.options?.thumbnail?.width || constants.thumbnail.width
            const thumbnailHeight = this.props.options?.thumbnail?.height || constants.thumbnail.height
            const thumbnailHorizontalImageAmount = this.props.options?.thumbnail?.horizontalImageAmount || constants.thumbnail.horizontalImageAmount
            const thumbnailVerticalImageAmount = this.props.options?.thumbnail?.verticalImageAmount || constants.thumbnail.verticalImageAmount

            const contexts: Array<CanvasRenderingContext2D> = []
            const neccessaryImages = Math.ceil(video.duration / (thumbnailHorizontalImageAmount * thumbnailVerticalImageAmount))

            for (let i = 0; i < neccessaryImages; i++) {
               const canvas = document.createElement('canvas')
               canvas.width = thumbnailHorizontalImageAmount * thumbnailWidth
               canvas.height = thumbnailVerticalImageAmount * thumbnailHeight
               const ctx = canvas.getContext('2d')

               if (!ctx) {
                  return
               }

               contexts.push(ctx)

               resolve(contexts)
            }

            try {
               // here we draw the images on the canvas
               const drawImageOnSeeked = (
                  context: CanvasRenderingContext2D,
                  currentTime: number,
                  dx = 0,
                  dy = 0,
               ): Promise<void> => {
                  return new Promise((resolve, reject) => {
                     video.currentTime = currentTime

                     const draw = async () => {
                        // TODO: this is here because firefox doesn't render the second drawImage. Bug Report: https://bugzilla.mozilla.org/show_bug.cgi?id=1673635
                        await new Promise(r => setTimeout(r, 100))

                        try {
                           context.drawImage(
                              video, // image source
                              dx * thumbnailWidth, // top left position of
                              dy * thumbnailHeight, // drawn thumbnail
                              thumbnailWidth, // dimensions of the
                              thumbnailHeight, // drawn thumbnail
                           )

                           video.removeEventListener('seeked', draw)
                           resolve()
                        } catch (error) {
                           reject(error)
                        }
                     }
                     video.addEventListener('seeked', draw)
                  })
               }

               // this recursive function triggers the drawing on the canvas
               const triggerDrawingOnCanvas = async (
                  context: CanvasRenderingContext2D,
                  contextIndex: number,
                  thumbnailCount = 0,
                  currentTime = 0,
                  dx = 0,
                  dy = 0,
               ): Promise<void> => {
                  if (
                     thumbnailCount !== thumbnailHorizontalImageAmount * thumbnailVerticalImageAmount &&
                     thumbnailCount * thumbnailInterval !== Math.floor(video.duration)
                  ) {
                     if (dx >= thumbnailHorizontalImageAmount) {
                        dx = 0
                     }
                     dy = Math.floor(thumbnailCount / thumbnailHorizontalImageAmount)

                     await drawImageOnSeeked(
                        context,
                        currentTime + contextIndex * thumbnailHorizontalImageAmount * thumbnailVerticalImageAmount,
                        dx,
                        dy,
                     )
                     dx++

                     return await triggerDrawingOnCanvas(
                        context,
                        thumbnailCount + 1 > thumbnailHorizontalImageAmount * thumbnailVerticalImageAmount ? contextIndex + 1 : contextIndex,
                        thumbnailCount + 1,
                        currentTime = currentTime + thumbnailInterval,
                        dx,
                        dy,
                     )
                  }
               }

               for await (const [index, context] of contexts.entries()) {
                  await triggerDrawingOnCanvas(context, index)
               }
               document.body.removeChild(video)
            } catch (error) {
               console.error(error)
            }
         })

         document.body.appendChild(video)
      })
   }

I hope it's understandable. I wrote it in typescript.

regards

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:lsalzman, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 4

[Jason Orendorff [:jorendorff]]

@pascal.galle: Thank you for the bug report. If you were to include a complete, working testcase and steps to reproduce, we would look into this. The code you've attached seems to be most of what we need—maybe a few more lines of code would do it?

Sending this back to the "Canvas: 2D" component. If this is a real bug, I think it has to be a problem with the drawImage method or else the current state of the video element (considered as a CanvasImageSource) when the seeked event fires. I don't see how it could be an issue with Promises or async functions.

Comment 5

[pascal.galle]

Here I created a simple repository for you. Not perfect code, but the misbehavior is shown: https://github.com/pgalle/CanvasRenderingErrorFirefox

First time I share a repository of mine. So any advice on how to improve this process is appreciated.